Regular expression SpaCy

问题

I am creating a spaCy regular expression matches for matching number and extracting it pandas data frame.

Question: Panda picks up from number but overwrites value instead of appending. How to solve it?

(original code credit: yarongon)

from __future__ import unicode_literals
import spacy
import re
import pandas as pd
from datetime import date
nlp = spacy.load('en_core_web_sm', disable=['parser', 'tagger', 'ner'])
doc = nlp("This is a sample number: 11. This is second sample number: 1145.")
NUM_PATTERN = re.compile(r"\d+")
for match in re.finditer(NUM_PATTERN, doc.text):
    start, end = match.span()
    Number = doc.char_span(start, end)
    print Number
pandas_attributes = [Number,]
df = pd.DataFrame(pandas_attributes,
                  columns=['Number'])
print df

Output:

11
1145
  Number
0   1145

Expected output:
      Number
o      11 
1      1145

Edit 1: I am trying multiple pattern match on single text.

from __future__ import unicode_literals
import spacy
import re
import pandas as pd
from datetime import date
nlp = spacy.load('en_core_web_sm', disable=['parser', 'tagger', 'ner'])
doc = nlp("This is a sample-number: 11. This is second sample number: 1145.")
NUM_PATTERN = re.compile(r"\d+")
HYPH_PATTERN = re.compile('\w+(?:-)\w+')

for match in re.finditer(NUM_PATTERN, doc.text):
    start, end = match.span()
    Number = doc.char_span(start, end)
    print Number

for match in re.finditer(HYPH_PATTERN, doc.text):
    start, end = match.span()
    Hyph_word = doc.char_span(start, end)
    print Hyph_word

pandas_attributes = [Number,Hyph_word]
df = pd.DataFrame(pandas_attributes,
                  columns=['Number','Hyphenword'])
print df

Current output.

Output:
11
1145
sample-number

AssertionError: 2 columns passed, passed data had 3 columns

Expected output:
Number  Hyphen_word
11      sample-number
1145

edit 2: output

                Number Hyphenword
0                 (11)     (1145)
1  (sample, -, number)       Non

Expected output:

    Number   Hyphenword
0        11   sample-word
1      1145   Non

回答1:

You need append values to list in loop:

L = []
for match in re.finditer(NUM_PATTERN, doc.text):
    start, end = match.span()
    L.append(doc.char_span(start, end))

and then use DataFrame constructor:

df = pd.DataFrame(L,columns=['Number'])

You can also append tuples with multiple values:

Sample:

L = []
for x in range(3):
    Number = x + 1
    Val = x + 4
    L.append((Number, Val))

print (L)
[(1, 4), (2, 5), (3, 6)]

df = pd.DataFrame(L,columns=['Number', 'Val'])
print (df)
   Number  Val
0       1    4
1       2    5
2       3    6

I believe you can use double append:

PATTERNS = [NUM_PATTERN, HYPH_PATTERN]

pandas_attributes = []
for pat in PATTERNS:
    L = []
    for match in re.finditer(pat, doc.text):
        start, end = match.span()
        L.append(doc.char_span(start, end))
    pandas_attributes.append(L) 

df = pd.DataFrame(pandas_attributes,
                  index=['Number','Hyphenword']).T

来源：https://stackoverflow.com/questions/48222307/regular-expression-spacy

标签

regex

pandas

nltk

spacy