tf-idf feature weights using sklearn.feature_extraction.text.TfidfVectorizer

后端 未结 2 1243
粉色の甜心
粉色の甜心 2020-12-07 15:37

this page: http://scikit-learn.org/stable/modules/feature_extraction.html mentions:

As tf–idf is a very often used for text features, there is also an

2条回答
  •  既然无缘
    2020-12-07 16:35

    Since version 0.15, the tf-idf score of each feature can be retrieved via the attribute idf_ of the TfidfVectorizer object:

    from sklearn.feature_extraction.text import TfidfVectorizer
    corpus = ["This is very strange",
              "This is very nice"]
    vectorizer = TfidfVectorizer(min_df=1)
    X = vectorizer.fit_transform(corpus)
    idf = vectorizer.idf_
    print dict(zip(vectorizer.get_feature_names(), idf))
    

    Output:

    {u'is': 1.0,
     u'nice': 1.4054651081081644,
     u'strange': 1.4054651081081644,
     u'this': 1.0,
     u'very': 1.0}
    

    As discussed in the comments, prior to version 0.15, a workaround is to access the attribute idf_ via the supposedly hidden _tfidf (an instance of TfidfTransformer) of the vectorizer:

    idf = vectorizer._tfidf.idf_
    print dict(zip(vectorizer.get_feature_names(), idf))
    

    which should give the same output as above.

提交回复
热议问题