Named Entity Recognition with Regular Expression: NLTK

后端 未结 3 1592
-上瘾入骨i
-上瘾入骨i 2020-12-16 19:27

I have been playing with NLTK toolkit. I come across this problem a lot and searched for solution online but nowhere I got a satisfying answer. So I am putting my query here

3条回答
  •  情话喂你
    2020-12-16 20:10

    There is a bug in @alvas's answer. Fencepost error. Make sure to run that elif check outside of the loop as well so that you don't leave off a NE that occurs at the end of the sentence. So:

    from nltk import ne_chunk, pos_tag, word_tokenize
    from nltk.tree import Tree
    
    def get_continuous_chunks(text):
        chunked = ne_chunk(pos_tag(word_tokenize(text)))
        prev = None
        continuous_chunk = []
        current_chunk = []
    
        for i in chunked:
            if type(i) == Tree:
                current_chunk.append(" ".join([token for token, pos in i.leaves()]))
            elif current_chunk:
                named_entity = " ".join(current_chunk)
                if named_entity not in continuous_chunk:
                    continuous_chunk.append(named_entity)
                    current_chunk = []
            else:
                continue
        if current_chunk:
            named_entity = " ".join(current_chunk)
            if named_entity not in continuous_chunk:
                continuous_chunk.append(named_entity)
                current_chunk = []
        return continuous_chunk
    
    txt = "Barack Obama is a great person and so is Michelle Obama." 
    print get_continuous_chunks(txt)
    

提交回复
热议问题