concatenation of two unicode srings?

Question

#!/usr/bin/python
# -*- coding: utf-8 -*-

import re
separators = [u"।", u",", u"."]
dat=open(r"C:\Users\User\Desktop\text4.txt",'r').read()
text=dat.decode("utf-8")
wros=text.split()
out=""
import string
space=" "
counter=0;
for word in wros:
        out=u" ".join(word)

writ=open("C:\\Users\\User\\Desktop\\text5.txt",'w')
writ.write(out.encode('utf-8'))
writ.close()

text4.txt contains भारत का इतिहास काफी समृद्ध एवं विस्तृत है।

text5.txt outputs as ह ै ।

desired output is भारत का इतिहास काफी समृद्ध एवं विस्तृत है।

please tell me what i am doing is wrong ? HElp required ! thanks in advance

Answer 1

I don't know what you have to do with word but I would do it this way:

text = open('text4.txt').read()

text = text.decode("utf-8")

# split one string into list of (old) words
words = text.split()

# list for new words
out = []

# modify words
for word in words:
    # here - do something with `word`
    out.append(word)

# concatenate all new words to one string 
result = u' '.join(out)

result = result.encode('utf-8')

writ = open('text5.txt', 'w')
writ.write(result)
writ.close()

Answer 2

Here's a template:

# no need for coding:utf8 unless *source code* has non-ASCII.

import io # Contains modern version of open compatible with Python 3.

# Use with and file is automatically closed when block is exited.
with io.open(r'c:\users\user\desktop\text4.txt',encoding='utf8') as dat:
    text = dat.read()

words = text.split()

#edit words

text = u' '.join(words)

with io.open(r'c:\users\user\desktop\text5.txt','w',encoding='utf8') as writ:
    writ.write(text)