Modify large file containing multiple XML files to create small file depending on condition

问题

I have a large file that contains multiple XMLs in different lines. I want to create a new file with lines (or XMLs) depending on a condition where multiple tags match columns of spreadsheet. For example, I have a large XML file.

<?xml version="1.0" encoding="UTF-8"?><data><student><result><grade>A</grade></result><details><name>John</name><house>Red</house><id>100</id><age>16</age><email>john@mail.com</email></details></student></data>
<?xml version="1.0" encoding="UTF-8"?><data><student><result><grade>B</grade></result><details><name>Alice</name><house>Blue</house><id>101</id><age>17</age><email>alice@mail.com</email></details></student></data>
<?xml version="1.0" encoding="UTF-8"?><data><student><result><grade>F</grade></result><details><name>Bob</name><house>Blue</house><id>100</id><age>16</age><email>bob@mail.com</email></details></student></data>
<?xml version="1.0" encoding="UTF-8"?><data><student><result><grade>A</grade></result><details><name>Hannah</name><house>Blue</house><id>103</id><age>17</age><email>hannah@mail.com</email></details></student></data>
<?xml version="1.0" encoding="UTF-8"?><data><student><result><grade>C</grade></result><details><name>James</name><house>Red</house><id>101</id><age>18</age><email>james@mail.com</email></details></student></data>

I need to create a file where the house and id are picked from a xlsx file like below:

and create a new file like below:

<?xml version="1.0" encoding="UTF-8"?><data><student><result><grade>F</grade></result><details><name>Bob</name><house>Blue</house><id>100</id><age>16</age><email>bob@mail.com</email></details></student></data>
<?xml version="1.0" encoding="UTF-8"?><data><student><result><grade>A</grade></result><details><name>Hannah</name><house>Blue</house><id>103</id><age>17</age><email>hannah@mail.com</email></details></student></data>

What I have tried:

from lxml import etree as ET
import pandas as pd

df = pd.read_excel(open('Student_data.xlsx','rb'),sheet_name="Sheet2")
df['House_Id']=df['House'].map(str)+'-'+df['Id'].map(str)
required_ids = df['House_Id'].tolist()
required_ids = [str(i) for i in required_ids]
for event, element in ET.iterparse('new_student.xml'):
    if element.tag == 'xml' and not(element.xpath('.//id/text()')[0] in required_ids):
        element.clear()
        element.getparent().remove(element)
    if element.tag == 'data':
        tree = ET.ElementTree(element)
        tree.write('student_output.xml')

I am able to create the required id using the 2 variables from the xlsx file (i.e. ['Blue-100', 'Blue-103']) but don't know how to:

Create a similar "pair-id" using the XMLs
Navigate to look for the "pair-id" and create a new file that contains only the required lines

Please let me know a way to do this. Thanks in advance.

回答1:

I think, if you know the encoding given in the various XML declarations on each line is always UTF-8, then you can process the file with several lines where each line represents an XML document with file based IO and lxml etree as follows:

from lxml import etree as ET

required_ids = ['100','103']
required_houses = ['Blue', 'Blue']

with open('input.txt', 'r') as f, open('output.txt', 'w') as w:
    for line in f.readlines():
        root = ET.fromstring(bytes(line, encoding = 'UTF-8'))
        if root.xpath('.//id/text()')[0] in required_ids and root.xpath('.//house/text()')[0] in required_houses:
            #ET.dump(root)
            print(ET.tostring(root, encoding = 'unicode'), file = w, end = '\n')

来源：https://stackoverflow.com/questions/57966419/modify-large-file-containing-multiple-xml-files-to-create-small-file-depending-o

标签

python-3.x

lxml

elementtree