1. 技术文章
  2. R-XML pulling nodes into matrix/DF accounting for missing nodes

R-XML pulling nodes into matrix/DF accounting for missing nodes

由 倖福魔咒の 提交于 2019-12-06 08:11:43

If your xpath queries return too many or few results, I think it's easier to work with the nodes

doc <- xmlParse( '<your xml here>')
nodes<- getNodeSet(doc, "//iati-activity")

#Compare
xpathSApply(doc, "//budget/value", xmlValue)
xpathSApply(doc, "//participating-org[@role='Funding']", xmlValue)

sapply(nodes, function(x) xpathSApply(x, "./budget/value", xmlValue))
sapply(nodes, function(x) xpathSApply(x, "./participating-org[@role='Funding']", xmlValue))

Add a function to handle missing or multiple nodes and then create the data.frame

xpath2 <-function(x, path, fun = xmlValue, ...){
   y <- xpathSApply(x, path, fun, ...)
   ifelse(length(y) == 0, NA,
    ifelse(length(y) > 1, paste(unlist(y), collapse=", "), y))
}

GrantAmount <- sapply(nodes, xpath2, "./budget/value")
UniqueID    <- sapply(nodes, xpath2, "./iati-identifier")
GrantTitle  <- sapply(nodes, xpath2, "./title")
Recipient   <-  sapply(nodes, xpath2, "./participating-org[@role='Implementing']")
## updated xpath2 so xmlGetAttr will also work
Funding_ref  <- sapply(nodes, xpath2, "./participating-org[@role='Funding']", xmlGetAttr, "ref")
Budget_start <- sapply(nodes, xpath2, ".//period-start", xmlGetAttr, "iso-date")

data.frame(UniqueID, GrantTitle, GrantAmount, Recipient)
                UniqueID                               GrantTitle GrantAmount             Recipient
1 NL-KVK-41160054-100530         Improvement of basic health care        <NA>           CORDAID RCA
2 NL-KVK-41160054-100625                         Pigs for Pencils       12500 PREDA Foundation Inc.
3 NL-KVK-41160054-100815 Job and housing opportunities for women       227000                  WISE

Consider using an XSLT transformation file. As information, XSLT is a declarative special-purpose programming language used to transform, style, or re-format XML files in various structures. Sadly this forgotten language in the web age is still very useful for every day needs.

Specifically for you, you need a simpler structure for R to easily import. The one caveat is R does not have a dedicated, reliable XSLT library. Therefore, you need to find an external XSLT processor. Fortunately, several open-source solutions are available which R can leverage.

XSLT Stylesheet

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output method="xml" indent="yes" omit-xml-declaration="no" />

<xsl:template match="iati-activities">
<GrantData>

    <xsl:for-each select="iati-activity">
    <iati-activity>
        <UniqueID><xsl:value-of select="iati-identifier"/></UniqueID>
        <GrantTitle><xsl:value-of select="title"/></GrantTitle>
        <GrantAmount><xsl:value-of select="budget/value"/></GrantAmount>
        <Recipient><xsl:value-of select="participating-org[@role='Implementing']"/></Recipient>
    </iati-activity>
    </xsl:for-each>

</GrantData>

</xsl:template>

</xsl:stylesheet>

Python (using its lxml library)

#!/usr/bin/python
import lxml.etree as ET

dom = ET.parse('C:\Path\To\RawXMLFile.xml')
xslt = ET.parse('C:\Path\To\RawXSLTFile.xsl')
transform = ET.XSLT(xslt)
newdom = transform(dom)

tree_out = ET.tostring(newdom, encoding='UTF-8', pretty_print=True,  xml_declaration=True)
print(tree_out)

xmlfile = open('C:\Path\To\OutputXMLFile.xml','ab')
xmlfile.write(tree_out)
xmlfile.close()

PHP (using its xsl extension)

This is for those who think PHP is not a general-purpose language but only restricted to the web:

<?php

// Load the XML source and XSLT file
$cd = dirname(__FILE__);

$xml = new DOMDocument;
$xml->load('C:\Path\To\RawXMLFile.xml');
$xsl = new DOMDocument;
$xsl->load('C:\Path\To\RawXSLTFile.xsl');    

// Configure the transformer
$proc = new XSLTProcessor;
$proc->importStyleSheet($xsl); // attach the xsl rules    

// Transform XML source
$newXml = $proc->transformToXML($xml);

// Save output to file
$xmlfile = 'C:\Path\To\OutputXMLFile.xml.xml';
file_put_contents($xmlfile, $newXml);  

?>

Both of which produces XML with a simpler structure for your R data frame needs (notice the empty node for GrantAmount).

<?xml version="1.0"?>

<GrantData>
  <iati-activity>
    <UniqueID>NL-KVK-41160054-100530</UniqueID>
    <GrantTitle>Improvement of basic health care</GrantTitle>
    <GrantAmount/>
    <Recipient>CORDAID RCA</Recipient>
  </iati-activity>
  <iati-activity>
    <UniqueID>NL-KVK-41160054-100625</UniqueID>
    <GrantTitle>Pigs for Pencils</GrantTitle>
    <GrantAmount>12500</GrantAmount>
    <Recipient>PREDA Foundation Inc.</Recipient>
  </iati-activity>
  <iati-activity>
    <UniqueID>NL-KVK-41160054-100815</UniqueID>
    <GrantTitle>Job and housing opportunities for women </GrantTitle>
    <GrantAmount>227000</GrantAmount>
    <Recipient>WISE</Recipient>
  </iati-activity>
</GrantData>

Once the simpler XML is generated, simply import into R:

library(XML)

# LOADING TRANSFORMED XML INTO R DATA FRAME
doc<-xmlParse("C:\\Path\\To\\OutputXMLFile.xml")
xmldf <- xmlToDataFrame(nodes = getNodeSet(doc, "//iati-activity"))

VBA (using the MSXML object)

Finally, there is a VBA solution to be run in MS Excel or MS Access. Fortunately, for this last solution you can connect the VBA with R using the R's RDCOMClient package (only a PC solution):

library(XML)
library(RDCOMClient)

xmlfile = COMCreate("MSXML2.DOMDocument")
xslfile = COMCreate("MSXML2.DOMDocument")
newxmlfile = COMCreate("MSXML2.DOMDocument")

xmlstr = "C:\\Path\\To\\RawXMLFile.xml"
xslstr = "C:\\Path\\To\\RawXSLTFile.xsl"
newxmlstr = "C:\\Path\\To\\OutputXMLFile.xml"

# LOADING XML & XSLT FILES
xmlfile.async = FALSE
xmlfile$Load(xmlstr)

xslfile.async = FALSE
xslfile$Load(xslstr)

# TRANSFORMING XML FILE USING XLST INTO NEW FILE
xmlfile$transformNodeToObject(xslfile, newxmlfile)
newxmlfile$Save(newxmlstr)

# LOADING TRANSFORMED XML INTO R DATA FRAME
doc<-xmlParse("C:\\Path\\To\\OutputXMLFile.xml")
xmldf <- xmlToDataFrame(nodes = getNodeSet(doc, "//iati-activity"))
View(xmldf)

xmlfile <- NULL
xslfile <- NULL
newxmlfile <- NULL

Data frame results with missing node as NaN:

    UniqueID                GrantTitle                          GrantAmount    Recipient
1   NL-KVK-41160054-100530  Improvement of basic health care                   CORDAID RCA
2   NL-KVK-41160054-100625  Pigs for Pencils                           12500   PREDA Foundation Inc.
3   NL-KVK-41160054-100815  Job and housing opportunities for women   227000   WISE
标签
xml
r
xpath
xml-parsing
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!