1. 技术文章
  2. SQL Server XML Processing: Join different Nodes based on ID

SQL Server XML Processing: Join different Nodes based on ID

由 点点圈 提交于 2020-07-22 05:14:31

问题


I am trying to query XML with SQL. Suppose I have the following XML.

<xml>
    <dataSetData>
        <text>ABC</text>
    </dataSetData>
    <generalData>
        <id>123</id>
        <text>text data</text>
    </generalData>
    <generalData>
        <id>456</id>
        <text>text data 2</text>
    </generalData>
    <specialData>
        <id>123</id>
        <text>special data text</text>
    </specialData>
    <specialData>
        <id>456</id>
        <text>special data text 2</text>
    </specialData>
</xml>

I want to write a SELECT query that returns 2 rows as follows:

DataSetData | GeneralDataID | GeneralDataText | SpecialDataTest
ABC         | 123           | text data       | special data text
ABC         | 456           | text data  2    | special data text 2

My current approach is as follows:

SELECT 
    dataset.nodes.value('(dataSetData/text)[1]', 'nvarchar(500)'),
    general.nodes.value('(generalData/text)[1]', 'nvarchar(500)'),
    special.nodes.value('(specialData/text)[1]', 'nvarchar(500)'),
FROM @MyXML.nodes('xml') AS dataset(nodes)
   OUTER APPLY @MyXML.nodes('xml/generalData') AS general(nodes)
   OUTER APPLY @MyXML.nodes('xml/specialData') AS special(nodes)
WHERE 
    general.nodes.value('(generalData/text/id)[1]', 'nvarchar(500)') = special.nodes.value('(specialData/text/id)[1]', 'nvarchar(500)')

What I do not like here is that I have to use OUTER APPLY twice and that I have to use the WHERE clause to JOIN the correct elements.

My question therefore is: Is it possible to construct the query in a way where I do not have to use the WHERE clause in such a way, because I am pretty sure that this affects performance very negatively if files become larger.

Shouldn't it be possible to JOIN the correct nodes (that is, the corresponding generalData and specialData nodes) with some XPATH statement?


回答1:


Your XPath expressions are completely off.

Please try the following. It is pretty efficient. You can test its performance with a large XML.

SQL

-- DDL and sample data population, start
DECLARE @xml XML = 
N'<xml>
    <dataSetData>
        <text>ABC</text>
    </dataSetData>
    <generalData>
        <id>123</id>
        <text>text data</text>
    </generalData>
    <generalData>
        <id>456</id>
        <text>text data 2</text>
    </generalData>
    <specialData>
        <id>123</id>
        <text>special data text</text>
    </specialData>
    <specialData>
        <id>456</id>
        <text>special data text 2</text>
    </specialData>
</xml>';
-- DDL and sample data population, end

SELECT c.value('(dataSetData/text/text())[1]', 'VARCHAR(20)') AS DataSetData
    , g.value('(id/text())[1]', 'INT') AS GeneralDataID 
    , g.value('(text/text())[1]', 'VARCHAR(30)') AS GeneralDataText
    , sp.value('(id/text())[1]', 'INT') AS SpecialDataID 
    , sp.value('(text/text())[1]', 'VARCHAR(30)') AS SpecialDataTest
FROM @xml.nodes('/xml') AS t(c)
    OUTER APPLY c.nodes('generalData') AS general(g)
    OUTER APPLY c.nodes('specialData') AS special(sp)
WHERE g.value('(id/text())[1]', 'INT') = sp.value('(id/text())[1]', 'INT');

Output

+-------------+---------------+-----------------+---------------+---------------------+
| DataSetData | GeneralDataID | GeneralDataText | SpecialDataID |   SpecialDataTest   |
+-------------+---------------+-----------------+---------------+---------------------+
| ABC         |           123 | text data       |           123 | special data text   |
| ABC         |           456 | text data 2     |           456 | special data text 2 |
+-------------+---------------+-----------------+---------------+---------------------+



回答2:


I want to suggest one more solution:

DECLARE @xml XML=
N'<xml>
    <dataSetData>
        <text>ABC</text>
    </dataSetData>
    <generalData>
        <id>123</id>
        <text>text data</text>
    </generalData>
    <generalData>
        <id>456</id>
        <text>text data 2</text>
    </generalData>
    <specialData>
        <id>123</id>
        <text>special data text</text>
    </specialData>
    <specialData>
        <id>456</id>
        <text>special data text 2</text>
    </specialData>
</xml>';

--The query

SELECT @xml.value('(/xml/dataSetData/text/text())[1]','varchar(100)')
      ,B.*
      ,@xml.value('(/xml/specialData[(id/text())[1] cast as xs:int? = sql:column("B.General_Id")]/text/text())[1]','varchar(100)') AS Special_Text
FROM @xml.nodes('/xml/generalData') A(gd)
CROSS APPLY(SELECT A.gd.value('(id/text())[1]','int') AS General_Id
                  ,A.gd.value('(text/text())[1]','varchar(100)') AS General_Text) B;

The idea in short:

  • We can read the <dataSetData>, as it is not repeating, directly from the variable.
  • We can use .nodes() to get a derived set of all <generalData> entries.
  • Now the magic trick: I use APPLY to get the values from the XML as regular columns into the result set.
  • This trick allows now to use sql:column() in order to build a XQuery predicate to find the corresponding <specialData>.

One more approach with FLWOR

You might try this:

SELECT @xml.query
('
    <xml>
    {
    for $i in distinct-values(/xml/generalData/id/text())
    return
    <combined dsd="{/xml/dataSetData/text/text()}"
              id="{$i}"
              gd="{/xml/generalData[id=$i]/text/text()}"
              sd="{/xml/specialData[id=$i]/text/text()}"/>
    }
    </xml>
');

The result

<xml>
  <combined dsd="ABC" id="123" gd="text data" sd="special data text" />
  <combined dsd="ABC" id="456" gd="text data 2" sd="special data text 2" />
</xml>

The idea in short:

  • With the help of distinct-values() we get a list of all id values in your XML
  • we can iterate this and pick the corresponding values
  • We return the result as a re-structured XML

Now you can use .nodes('/xml/combined') against this new XML and retrieve all values easily.

Performance test

I just want to add a performance test:

CREATE TABLE dbo.TestXml(TheXml XML);
INSERT INTO dbo.TestXml VALUES
(
  (
    SELECT 'blah1' AS [dataSetData/text]
          ,(SELECT o.[object_id] AS [id]
                  ,o.[name]      AS [text] 
            FROM sys.objects o
            FOR XML PATH('generalData'),TYPE)
          ,(SELECT o.[object_id] AS [id]
                  ,o.create_date AS [text] 
            FROM sys.objects o
            FOR XML PATH('specialData'),TYPE)
    FOR XML PATH('xml'),TYPE
  )
)
,(
  (
    SELECT 'blah2' AS [dataSetData/text]
          ,(SELECT o.[object_id] AS [id]
                  ,o.[name]      AS [text] 
            FROM sys.objects o
            FOR XML PATH('generalData'),TYPE)
          ,(SELECT o.[object_id] AS [id]
                  ,o.create_date AS [text] 
            FROM sys.objects o
            FOR XML PATH('specialData'),TYPE)
    FOR XML PATH('xml'),TYPE
  )
)
,(
  (
    SELECT 'blah3' AS [dataSetData/text]
          ,(SELECT o.[object_id] AS [id]
                  ,o.[name]      AS [text] 
            FROM sys.objects o
            FOR XML PATH('generalData'),TYPE)
          ,(SELECT o.[object_id] AS [id]
                  ,o.create_date AS [text] 
            FROM sys.objects o
            FOR XML PATH('specialData'),TYPE)
    FOR XML PATH('xml'),TYPE
  )
);
GO
--just a dummy call to avoid *first call bias*
SELECT x.query('.') FROM dbo.TestXml
                    CROSS APPLY TheXml.nodes('/xml//*') A(x)
GO

DECLARE @t DATETIME2=SYSUTCDATETIME();
--My first approach
SELECT TheXml.value('(/xml/dataSetData/text/text())[1]','varchar(100)') AS DataSetValue
      ,B.*
      ,TheXml.value('(/xml/specialData[(id/text())[1] cast as xs:int? = sql:column("B.General_Id")]/text/text())[1]','varchar(100)') AS Special_Text
INTO dbo.testResult1
FROM dbo.TestXml
CROSS APPLY TheXml.nodes('/xml/generalData') A(gd)
CROSS APPLY(SELECT A.gd.value('(id/text())[1]','int') AS General_Id
                  ,A.gd.value('(text/text())[1]','varchar(100)') AS General_Text) B;
SELECT DATEDIFF(MILLISECOND,@t,SYSUTCDATETIME());
GO
              
              
DECLARE @t DATETIME2=SYSUTCDATETIME();
--My second approach
SELECT B.c.value('@dsd','varchar(100)') AS dsd
      ,B.c.value('@id','int') AS id
      ,B.c.value('@gd','varchar(100)') AS gd
      ,B.c.value('@sd','varchar(100)') AS sd
INTO dbo.TestResult2
FROM dbo.TestXml
CROSS APPLY (SELECT TheXml.query
('
    <xml>
    {
    for $i in distinct-values(/xml/generalData/id/text())
    return
    <combined dsd="{/xml/dataSetData/text/text()}"
              id="{$i}"
              gd="{/xml/generalData[id=$i]/text/text()}"
              sd="{/xml/specialData[id=$i]/text/text()}"/>
    }
    </xml>
') AS ResultXml) A
CROSS APPLY A.ResultXml.nodes('/xml/combined') B(c) 

SELECT DATEDIFF(MILLISECOND,@t,SYSUTCDATETIME());
GO

DECLARE @t DATETIME2=SYSUTCDATETIME();
--Yitzhak'S approach
SELECT c.value('(dataSetData/text/text())[1]', 'VARCHAR(20)') AS DataSetData
    , g.value('(id/text())[1]', 'INT') AS GeneralDataID 
    , g.value('(text/text())[1]', 'VARCHAR(30)') AS GeneralDataText
    , sp.value('(id/text())[1]', 'INT') AS SpecialDataID 
    , sp.value('(text/text())[1]', 'VARCHAR(30)') AS SpecialDataTest
INTO dbo.TestResult3
FROM dbo.TestXml
CROSS APPLY TheXml.nodes('/xml') AS t(c)
    OUTER APPLY c.nodes('generalData') AS general(g)
    OUTER APPLY c.nodes('specialData') AS special(sp)
WHERE g.value('(id/text())[1]', 'INT') = sp.value('(id/text())[1]', 'INT');

SELECT DATEDIFF(MILLISECOND,@t,SYSUTCDATETIME());
GO

SELECT * FROM TestResult1;
SELECT * FROM TestResult2;
SELECT * FROM TestResult3;
GO
--careful with real data!
DROP TABLE testResult1
DROP TABLE testResult2
DROP TABLE testResult3
DROP TABLE dbo.TestXml;

The result is clearly pointing against XQuery. (Someone might say so sad! now :-) ).

The predicate approach is by far the slowest (4700ms). The FLWOR approach is on rank 2 (1200ms) and the winner is - tatatataaaaa - Yitzhak's approach (400ms, by factor ~10!).

Which solution is best for you, will depend on the actual data (count of elements per XML, count of XMLs and so on). But the visual elegance is - regrettfully - not the only parameter for this choice :-)




回答3:


Sorry to add this as another answer, but I don't want to add to the other answer. It's big enough already :-)

A combination of Yitzhak's and mine is even faster:

--This is the additional code to be placed into the performance comparison

DECLARE @t DATETIME2=SYSUTCDATETIME();

SELECT TheXml.value('(/xml/dataSetData/text/text())[1]', 'VARCHAR(20)') AS DataSetData
     ,B.*
    , sp.value('(id/text())[1]', 'INT') AS SpecialDataID 
    , sp.value('(text/text())[1]', 'VARCHAR(30)') AS SpecialDataTest
INTO dbo.TestResult4
FROM dbo.TestXml
CROSS APPLY TheXml.nodes('/xml/generalData') AS A(g)
CROSS APPLY(SELECT g.value('(id/text())[1]', 'INT') AS GeneralDataID 
                 , g.value('(text/text())[1]', 'VARCHAR(30)') AS GeneralDataText) B
OUTER APPLY TheXml.nodes('/xml/specialData[id=sql:column("B.GeneralDataID")]') AS special(sp);

SELECT DATEDIFF(MILLISECOND,@t,SYSUTCDATETIME());

The idea in short:

  • We read the <dataSetData> directly (no repetition)
  • We use APPLY .nodes() to get all <generalData>
  • We use APPLY SELECT to fetch the values of <generalData> elements as real columns.
  • We use another APPLY .nodes() to fetch the corresponding <specialData> elements

One advantage of this solution: If there might be more than one special-data entry per general-data element, this would work too.

This is now the fastest in my test (~300ms).



来源：https://stackoverflow.com/questions/60511464/sql-server-xml-processing-join-different-nodes-based-on-id

标签
sql-server
xml
tsql
xpath
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!