Stanford CoreNLP Annotators Thread safe?

北城余情 提交于 2019-12-07 02:22:17

问题


the website of Stanford CoreNLP

http://nlp.stanford.edu/software/corenlp.shtml

lists dozens of Annotators which work like a charm. I would like to use instances of the Annotators for the common tasks (lemmatization, tagging, parsing) by multiple threads. For example to split up the processing of a massively large (GBs of Text) into threads or to provide web services.

There has been some discussion in the past referring to LocalThreads which, by my understanding, use one instance of an Annotator per Thread (thus avoiding problems regarding thread-safety). This is an option but that way all model files and resources have to be loaded n times as well.

Are the Annotators (or some of them) thread-safe to use? I couldn't find anything conclusive/official in the discussions, docs or faqs.


回答1:


Yes, the annotators are intended to be thread-safe. You can create a new AnnotationPipeline (e.g., a new StanfordCoreNLP object), and then many threads can pass annotations into this pipeline without reloading the model for each thread.



来源:https://stackoverflow.com/questions/30047133/stanford-corenlp-annotators-thread-safe

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!