How to remove duplicates from a string in SQL

问题

I got a column in my SQL table where some values are seperated bij ". Some values in this string are duplicated which I want to remove. Here is an example of my data:

---------------
| qw"qw"er"er |
---------------
| q"w"w"q     |
---------------
| f"k"s"g     |
---------------

Now The result should replace any duplicates:

---------------
| qw"er       |
---------------
| q"w"        |
---------------
| f"k"s"g     |
---------------

So first I want to split the string and then remove duplicates. Could anyone help me with this issue?

回答1:

Option 1 with a Parse Function

Declare @YourTable table (ID int,YourCol varchar(50))
Insert Into @YourTable values
(1,'qw"qw"er"er'),
(2,'q"w"w"q'),
(3,'f"k"s"g')

Select A.ID
      ,A.YourCol
      ,DeDuped   = Stuff((Select '"' + RetVal 
                           From (Select RetSeq=Min(RetSeq),RetVal 
                                  From  [dbo].[udf-Str-Parse](A.YourCol,'"') 
                                  Group By RetVal) P  
                            Order by 1 For XML Path('')),1,1,'') 
 From  @YourTable A

Returns

ID  YourCol      DeDuped
1   qw"qw"er"er  qw"er
2   q"w"w"q      q"w
3   f"k"s"g      f"k"s"g

Option 2: Without a Parse Function

Declare @YourTable table (ID int,YourCol varchar(50))
Insert Into @YourTable values
(1,'qw"qw"er"er'),
(2,'q"w"w"q'),
(3,'f"k"s"g')

Select A.ID
      ,A.YourCol
      ,DeDuped   = Stuff((Select '"' + RetVal 
                           From (Select RetSeq=Min(RetSeq),RetVal 
                                  From  (
                                            Select RetSeq = Row_Number() over (Order By (Select null))
                                                  ,RetVal = LTrim(RTrim(B.i.value('(./text())[1]', 'varchar(max)')))
                                            From  (Select x = Cast('<x>' + replace((Select replace(A.YourCol,'"','§§Split§§') as [*] For XML Path('')),'§§Split§§','</x><x>')+'</x>' as xml).query('.')) as A 
                                            Cross Apply x.nodes('x') AS B(i)
                                        ) P1
                                  Group By RetVal) P  
                            Order by RetSeq 
                            For XML Path('')),1,1,'') 
 From  @YourTable A

Returns

ID  YourCol      DeDuped
1   qw"qw"er"er  qw"er
2   q"w"w"q      q"w
3   f"k"s"g      f"k"s"g

The UDF if Interested

CREATE FUNCTION [dbo].[udf-Str-Parse] (@String varchar(max),@Delimiter varchar(10))
Returns Table 
As
Return (  
    Select RetSeq = Row_Number() over (Order By (Select null))
          ,RetVal = LTrim(RTrim(B.i.value('(./text())[1]', 'varchar(max)')))
    From  (Select x = Cast('<x>' + replace((Select replace(@String,@Delimiter,'§§Split§§') as [*] For XML Path('')),'§§Split§§','</x><x>')+'</x>' as xml).query('.')) as A 
    Cross Apply x.nodes('x') AS B(i)
);
--Thanks Shnugo for making this XML safe
--Select * from [dbo].[udf-Str-Parse]('Dog,Cat,House,Car',',')
--Select * from [dbo].[udf-Str-Parse]('John Cappelletti was here',' ')
--Select * from [dbo].[udf-Str-Parse]('this,is,<test>,for,< & >',',')
--Performance On a 5,000 random sample -8K 77.8ms, -1M 79ms (+1.16), -- 91.66ms (+13.8)

来源：https://stackoverflow.com/questions/42738165/how-to-remove-duplicates-from-a-string-in-sql

标签

sql

sql-server

split

sql-server-2012

duplicates