Python NLTK multi threading

守給你的承諾、 提交于 2020-01-14 12:43:29

问题


I am writing an algorithm which identifies sentences in given text, split each sentence into words & return these words after some validations.

I want to implement the same with the help of multi threading.

I'm calling my function which deals with each sentence in threading.thread() for which it throws an error:

AttributeError: 'WordListCorpusReader' object has no attribute '_LazyCorpusLoader__args'  

However, there are few blogs which suggest to use "wn.ensure_loaded()" function.

But python throws an error saying ensure_loaded() is not defined.

Can someone help me solve this.

EDIT:

text = "This is my sample text. I want to break it into sentences"  
sentences=(re.split(r"(?<!\w\.\w.)(?<![A-Z][a-z]\.)(?<=\.|\?)\s",text))  
wn.ensure_loaded()      
co = CoOccurence()     #CoOccurence is a class in my package which will work on the sentences  
for sentence in sentences:  
    t = Thread(target=co.__prepareHash__, args=(sentence,)) #co.__prepareHash__   is an other method which works around the sentences  
    threads.append(t)  
    t.start()  

flag =1  
while (flag):  
    flag = __isThreadAlive__()  

This throws me an error: Attribute error: 'WordNetCorpusReader' object has no attribute '_LazyCorpusLoader__args'
When I try to check wn.ensure_loaded(), it throws AttributeError: 'module' object has no attribute 'ensure_loaded'

with ref. to : this SO question

Thank You

来源:https://stackoverflow.com/questions/33076532/python-nltk-multi-threading

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!