Remove duplicate values from a string in java

Question

Can anyone please let me know how to remove duplicate values from

String s="Bangalore-Chennai-NewYork-Bangalore-Chennai"; 

and output should be like

String s="Bangalore-Chennai-NewYork-";

using Java..

Any help would be appreciated.

Answer 1

This does it in one line:

public String deDup(String s) {
    return new LinkedHashSet<String>(Arrays.asList(s.split("-"))).toString().replaceAll("(^\\[|\\]$)", "").replace(", ", "-");
}

public static void main(String[] args) {
    System.out.println(deDup("Bangalore-Chennai-NewYork-Bangalore-Chennai"));
}

Output:

Bangalore-Chennai-NewYork

Notice that the order is preserved :)

Key points are:

• split("-") gives us the different values as an array
• Arrays.asList() turns the array into a List
• LinkedHashSet preserves uniqueness and insertion order - it does all the work of giving us the unique values, which are passed via the constructor
• the toString() of a List is [element1, element2, ...]
• the final replace commands remove the "punctuation" from the toString()

This solution requires the values to not contain the character sequence ", " - a reasonable requirement for such terse code.

Java 8 Update!

Of course it's 1 line:

public String deDup(String s) {
    return Arrays.stream(s.split("-")).distinct().collect(Collectors.joining("-"));
}

Regex update!

If you don't care about preserving order (ie it's OK to delete the first occurrence of a duplicate):

public String deDup(String s) {
    return s.replaceAll("(\\b\\w+\\b)-(?=.*\\b\\1\\b)", "");
}

Answer 2

public static String removeDuplicates(String txt, String splitterRegex)
{
    List<String> values = new ArrayList<String>();
    String[] splitted = txt.split(splitterRegex);
    StringBuilder sb = new StringBuilder();
    for (int i = 0; i < splitted.length; ++i)
    {
        if (!values.contains(splitted[i]))
        {
            values.add(splitted[i]);
            sb.append('-');
            sb.append(splitted[i]);
        }
    }
    return sb.substring(1);

}

Usage:

String s = "Bangalore-Chennai-NewYork-Bangalore-Chennai";
s = removeDuplicates(s, "\\-");
System.out.println(s);

Prints:

Bangalore-Chennai-NewYork

Answer 3

You could add your strings to a HashSet.

1. Split the strings on a "-".
2. Store the individual words in Array. i.e arr[]

Sinppet :

Set<String> set = new HashSet<String>();

    for(int i=0; i < arr.length; i++){
      if(set.contains(arr[i])){
        System.out.println("Duplicate string found at index " + i);
      } else {
        set.add(arr[i]);
      }

Answer 4

Create array of string by spliting by - and then create a hashSet from it.

String s="Bangalore-Chennai-NewYork-Bangalore-Chennai"; 
String[] strArr = s.split("-");
Set<String> set = new HashSet<String>(Arrays.asList(strArr));

If you want back it as string array then do following:

String[] result = new String[set.size()];
set.toArray(result);

Here is a sample code to do this:

String s="Bangalore-Chennai-NewYork-Bangalore-Chennai"; 
String[] strArr = s.split("-");
Set<String> set = new LinkedHashSet<String>(Arrays.asList(strArr));
String[] result = new String[set.size()];
set.toArray(result);
StringBuilder res = new StringBuilder();
for (int i = 0; i < result.length; i++) {
    String string = result[i];
    if(i==result.length-1)
        res.append(string);
    else
        res.append(string).append("-");
}
System.out.println(res.toString());

Output:-

Bangalore-Chennai-NewYork

Answer 5

Just the idea:

1. parse the string and split the tokens using separator "-"
2. load the tokens into a Collection
3. iterate the Collection and erase duplicates
4. use the result Collection to build the new string

The most tricky part should be 3, but not impossible. If you use a Set, you can skip this step.

EDIT maybe you can substitute 2&3 with a presence check before adding the element

Answer 6

static String RemoveDuplicateCharInString(String s){
    for (int i = 0; i < s.length(); i++) {
        if((s.substring(i+1)).indexOf(s.charAt(i))!=-1){
            s=s.substring(0,i+1)+(s.substring(i+1)).replaceAll(""+s.charAt(i),"");
        }
    }
    return s;
}

Answer 7

wordsArray = s.split("-");
List<String> wordsList = new Arrays.asList(wordsArray);
Set<String>  wordsSet  = new LinkedHashSet<String>(wordsList);

String[] noDuplicates = new String[wordsSet.size()];
wordsSet.toArray(noDuplicates);

Answer 8

I'd prefer this which is simpler than all of the above.

public void removeDuplicates() {
  String myString = "Bangalore-Chennai-NewYork-Bangalore-Chennai";

  String[] array = myString.split("-");

  Set<String> hashSet = new HashSet<String>(Arrays.asList(array));

  String newString = StringUtils.join(hashSet, "-");        
}

Answer 9

public static void main(String[] args) {
    String str="Bangalore-Chennai-Newyork-Bangalore-Chennai";
    String output="";
    String [] arr=str.split("-");

    LinkedHashSet<String> lhs=new LinkedHashSet<String>();
    for (int i = 0; i < arr.length; i++) {
        lhs.add(arr[i]);
    }
    for(String s:lhs){
        output=output+s+"-";
    }

    System.out.println(output);
}

Answer 10

A little late to the game, but I would simply use a HashMap. It's easy to understand and has quick lookups on the keys, might not be the best way but it's still a good answer IMO. I use it all the time when I need to format quick and dirty:

                    String reason = "Word1 , Word2 , Word3";
                    HashMap<String,String> temp_hash = new HashMap<String,String>();
                    StringBuilder reason_fixed = new StringBuilder();
                    //in:
                    for(String word : reason.split(",")){
                        temp_hash.put(word,word);
                    }
                    //out:
                    for(String words_fixed : temp_hash.keySet()){
                        reason_fixed.append(words_fixed + " , ");
                    }
                    //print:
                    System.out.println(reason_fixed.toString());

Answer 11

public class RemDuplicateWordFromString {
public static void main(String[] args) {
    String s1 = "Hello India Hello India Hello India Hello India";
    countWords(s1);
}
public static void countWords(String s1) {
    String[] s2 = s1.split(" ");
    for (int i = 0; i < s2.length; i++) {
        for (int j = i + 1; j < s2.length; j++) {
            if (s2[i].equals(s2[j])) {
                if (i != j) {
                    s2[i] = "";
                }
            }
        }
    }
    for (int i = 0; i < s2.length; i++) {
        if (s2[i] != "") {
            System.out.print(s2[i] + " ");
        }

    }

}

}

Answer 12

StringBuilder builderWord = new StringBuilder(word);
for(int index=0; index < builderWord.length(); index++) {
    for(int reverseIndex=builderWord.length()-1; reverseIndex > index;reverseIndex--) {
        if (builderWord.charAt(reverseIndex) == builderWord.charAt(index)) {
            builderWord.deleteCharAt(reverseIndex);
        }
    }
}
return builderWord.toString();

Answer 13

import java.util.HashSet;

public class SplitString {
    public static void main(String[] args) {
        String st = new String("New Delhi-Chennai-New York-Bangalore-Chennai-New Delhi-Chennai-New York");
        StringBuffer stb = new StringBuffer();

        HashSet<String> hashset = new HashSet<String>();
        for (String a : st.split("-"))
            hashset.add(a);

        Object[] str = (Object[]) hashset.toArray();

        for (int i = 0; i < str.length; i++) {
            stb.append(str[i]);

            if (i < str.length - 1)
                stb.append("-");

        }

        System.out.println(stb);
    }
}

Answer 14

import java.util.*;

public class RemoveDuplicateWord {

    public static void main(String[] args) {
        String str = "Hai hello Hai how hello are how you";
        removeDupWord(str);
    }

    public static void removeDupWord(String input) {
        List<String> list = Arrays.asList(input.split(" "));
        LinkedHashSet<String> lhs = new LinkedHashSet<String>(list);
        for(String s : lhs) {
            System.out.print(s+" ");
        }                   
    }
}