nlp

来源： https://stackoverflow.com/questions/59105346/longest-match-only-with-spacy-phrasematcher

来源： https://stackoverflow.com/questions/59105346/longest-match-only-with-spacy-phrasematcher

问题 I'm working with RNN and using Pytorch & Torchtext. I've got a problem with building vocab in my RNN. My code is as follows: TEXT = Field(tokenize=tokenizer, lower=True) LABEL = LabelField(dtype=torch.float) trainds = TabularDataset( path='drive/{}'.format(TRAIN_PATH), format='tsv', fields=[ ('label_start', LABEL), ('label_end', None), ('title', None), ('symbol', None), ('text_content', TEXT), ]) testds = TabularDataset( path='drive/{}'.format(TEST_PATH), format='tsv', fields=[ ('text_content

问题 I'm working with RNN and using Pytorch & Torchtext. I've got a problem with building vocab in my RNN. My code is as follows: TEXT = Field(tokenize=tokenizer, lower=True) LABEL = LabelField(dtype=torch.float) trainds = TabularDataset( path='drive/{}'.format(TRAIN_PATH), format='tsv', fields=[ ('label_start', LABEL), ('label_end', None), ('title', None), ('symbol', None), ('text_content', TEXT), ]) testds = TabularDataset( path='drive/{}'.format(TEST_PATH), format='tsv', fields=[ ('text_content

问题 I'm working with written text (paragraphs of articles and books) that includes both locations and dates. I want to extract from the texts pairs that contain locations and dates that are associated with one another. For example, given the following phrase: The man left Amsterdam on January and reached Nepal on October 21st I would have an output such as this: >>>[(Amsterdam, January), (Nepal, October 21st)] I tried splitting the text through "connecting words" (such as "and" for example) and

问题 I'm working with written text (paragraphs of articles and books) that includes both locations and dates. I want to extract from the texts pairs that contain locations and dates that are associated with one another. For example, given the following phrase: The man left Amsterdam on January and reached Nepal on October 21st I would have an output such as this: >>>[(Amsterdam, January), (Nepal, October 21st)] I tried splitting the text through "connecting words" (such as "and" for example) and

问题 I am using Keras model.predict to get label for new sentences not present in the dataset. But the prediction is always giving the same value irrespective of the sentence. Here is my prediction code from sklearn.preprocessing import LabelEncoder maxlen = 300 ### PREDICT NEW UNSEEN DATA ### tokenizer = Tokenizer() label_enc = LabelEncoder() label_enc.fit(tar_list) X_test = ['asdsadav dawd','this is boring', 'wow i like this you did a great job', 'ima cry tht was mean','1 nov 1968 george

问题 I am using Huggingface's transformers library and want to perform NER using BERT. I tried to find an explicit example of how to properly format the data for NER using BERT. It is not entirely clear to me from the paper and the comments I've found. Let's say we have a following sentence and labels: sent = "John Johanson lives in Ramat Gan." labels = ['B-PER', 'I-PER', 'O', 'O', 'B-LOC', 'I-LOC'] Would data that we input to the model be something like this: sent = ['[CLS]', 'john', 'johan', '#

问题 Is that possible to generate texts from OpenAI GPT-2 using TensorFlowJS? If not what is the limitation, like model format or ...? 回答1: I don't see any reason as to why not, other than maybe some operation that is in gpt-2 that is not supported by tensorflowjs. I don't know how to do it, but here's a nice starting point: install.sh python3 -m pip install -q git+https://github.com/huggingface/transformers.git python3 -m pip install tensorflow tensorflowjs save.py from transformers import

问题 Is that possible to generate texts from OpenAI GPT-2 using TensorFlowJS? If not what is the limitation, like model format or ...? 回答1: I don't see any reason as to why not, other than maybe some operation that is in gpt-2 that is not supported by tensorflowjs. I don't know how to do it, but here's a nice starting point: install.sh python3 -m pip install -q git+https://github.com/huggingface/transformers.git python3 -m pip install tensorflow tensorflowjs save.py from transformers import