Python Gensim: how to calculate document similarity using the LDA model?

后端 未结 3 934
遇见更好的自我
遇见更好的自我 2020-12-12 21:22

I\'ve got a trained LDA model and I want to calculate the similarity score between two documents from the corpus I trained my model on. After studying all the Gensim tutoria

3条回答
  •  北荒
    北荒 (楼主)
    2020-12-12 21:32

    Don't know if this'll help but, I managed to attain successful results on document matching and similarities when using the actual document as a query.

    dictionary = corpora.Dictionary.load('dictionary.dict')
    corpus = corpora.MmCorpus("corpus.mm")
    lda = models.LdaModel.load("model.lda") #result from running online lda (training)
    
    index = similarities.MatrixSimilarity(lda[corpus])
    index.save("simIndex.index")
    
    docname = "docs/the_doc.txt"
    doc = open(docname, 'r').read()
    vec_bow = dictionary.doc2bow(doc.lower().split())
    vec_lda = lda[vec_bow]
    
    sims = index[vec_lda]
    sims = sorted(enumerate(sims), key=lambda item: -item[1])
    print sims
    

    Your similarity score between all documents residing in the corpus and the document that was used as a query will be the second index of every sim for sims.

提交回复
热议问题