SOLR and Natural Language Parsing - Can I use it?

若如初见. 提交于 2021-02-15 08:16:45

问题


Requirements

Word frequency algorithm for natural language processing

Using Solr

While the answer for that question is excellent, I was wondering if I could make use of all the time I spent getting to know SOLR for my NLP.

I thought of SOLR because:

  1. It's got a bunch of tokenizers and performs a lot of NLP.
  2. It's pretty use to use out of the box.
  3. It's restful distributed app, so it's easy to hook up
  4. I've spent some time with it, so using could save me time.

Can I use Solr?

Although the above reasons are good, I don't know SOLR THAT well, so I need to know if it would be appropriate for my requirements.

Ideal Usage

Ideally, I'd like to configure SOLR, and then be able to send SOLR some text, and retrieve the indexed tonkenized content.

Context

I'm working on a small component of a bigger recommendation engine.


回答1:


I guess you can use Solr and combine it with other tools. Tokenization, stop word removal, stemming, and even synonyms come out of the box with Solr. If you need named entity recognition or base noun-phrase extraction, you need to use OpenNLP or an equivalent tool as a pre-processing stage. You will probably need term vectors for your retrieval purposes. Integrating Apache Mahout with Apache Lucene and Solr may be useful as it discusses Lucene and Solr integration with a machine learning (including recommendation) engine. Other then that, feel free to ask further more specific questions.




回答2:


You can actually configure Solr to use NLP algorithms both when indexing documents and at search time. The first phase (indexing time) can be done using/writing Solr UpdateRequestProcessor plugins for analyzing fields texts while the second phase can be implemented writing a custom QParserPlugin which analyzes the query hit by the user. I've presented an approach for implementing natural language search in Solr at Lucene Eurocon 2011 which takes advantage of Apache UIMA for running (open source) NLP algorithms. You can take a look at the slides and at the video of the talk. Hope this helps. Tommaso




回答3:


There is a special request handler designed to apply parsing to filter our less relevant search results. It is based on machine learning of constituency parse trees obtained by OpenNLP.

Please see the blog http://search-engineering.blogspot.com

and the paper http://dx.doi.org/10.1016/j.datak.2012.07.003

This SOLR search request handler will be available as a part of OpenNLP Similarity component




回答4:


In this Google code project

http://code.google.com/p/relevance-based-on-parse-trees

you can use the linguistic-based request handler in the package opennlp.tools.similarity.apps.solr public class SyntGenRequestHandler extends SearchHandler

where the search results obtained by SearchHandler are re-ranked based on similarity of parse trees.



来源:https://stackoverflow.com/questions/2954814/solr-and-natural-language-parsing-can-i-use-it

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!