Fast/Optimize N-gram implementations in python

后端 未结 3 1841
情深已故
情深已故 2020-11-29 08:48

Which ngram implementation is fastest in python?

I\'ve tried to profile nltk\'s vs scott\'s zip (http://locallyoptimal.com/blog/2013/01/20/elegant-n-gram-generation-

3条回答
  •  北海茫月
    2020-11-29 09:26

    Extending M4rtini's Code

    Using Python3.6.5, nltk == 3.3

    from nltk import ngrams
    def get_n_gramlist(text,n=2):        
        nngramlist=[]
        for s in ngrams(text.split(),n=n):        
            nngramlist.append(s)                
        return nngramlist
    

    Timeit results

提交回复
热议问题