php: check if an array has duplicates

Question

I'm sure this is an extremely obvious question, and that there's a function that does exactly this, but I can't seem to find it. In PHP, I'd like to know if my array has duplicates in it, as efficiently as possible. I don't want to remove them like array_unique does, and I don't particularly want to run array_unique and compare it to the original array to see if they're the same, as this seems very inefficient. As far as performance is concerned, the "expected condition" is that the array has no duplicates.

I'd just like to be able to do something like

if (no_dupes($array))
    // this deals with arrays without duplicates
else
    // this deals with arrays with duplicates

Is there any obvious function I'm not thinking of?
How to detect duplicate values in PHP array?
has the right title, and is a very similar question, however if you actually read the question, he's looking for array_count_values.

Answer 1

You can do:

function has_dupes($array) {
    $dupe_array = array();
    foreach ($array as $val) {
        if (++$dupe_array[$val] > 1) {
            return true;
        }
    }
    return false;
}

Answer 2

I know you are not after array_unique(). However, you will not find a magical obvious function nor will writing one be faster than making use of the native functions.

I propose:

function array_has_dupes($array) {
   // streamline per @Felix
   return count($array) !== count(array_unique($array));
}

Adjust the second parameter of array_unique() to meet your comparison needs.

Answer 3

⚡ PERFORMANCE SOLUTION ⚡

If you care about performance and micro-optimizations check this one-liner:

function no_dupes(array $input_array) {
    return count($input_array) === count(array_flip($input_array));
}

Description:

Function compares number of array elements in $input_array with array_flip'ed elements. Values become keys and guess what - keys must be unique in associative arrays so not unique values are lost and final number of elements is lower than original.

As said in manual array keys can be only type of int or string so this is what you can have in original array values to compare, otherwise PHP will start casting with unexpected results.

PROOF FOR 10M RECORDS ARRAY

• Most voted solution: 14.187316179276s 🐌🐌🐌🐌🐌🐌🐌🐌🐌🐌🐌🐌🐌🐌
• Accepted solution: 2.0736091136932s 🐌🐌
• This answer solution: 0.14155888557434s 🐌/10

Test case:

<?php

$elements = array_merge(range(1,10000000),[1]);

$time = microtime(true);
accepted_solution($elements);
echo 'Accepted solution: ', (microtime(true) - $time), 's', PHP_EOL;

$time = microtime(true);
most_voted_solution($elements);
echo 'Most voted solution: ', (microtime(true) - $time), 's', PHP_EOL;

$time = microtime(true);
this_answer_solution($elements);
echo 'This answer solution: ', (microtime(true) - $time), 's', PHP_EOL;

function accepted_solution($array){
 $dupe_array = array();
 foreach($array as $val){
  // sorry, but I had to add below line to remove millions of notices
  if(!isset($dupe_array[$val])){$dupe_array[$val]=0;}
  if(++$dupe_array[$val] > 1){
   return true;
  }
 }
 return false;
}

function most_voted_solution($array) {
   return count($array) !== count(array_unique($array));
}

function this_answer_solution(array $input_array) {
    return count($input_array) === count(array_flip($input_array));
}

Notice that accepted solution might be faster in certain condition when not unique values are near the beginning of huge array.

Answer 4

$duplicate = false;

 if(count(array) != count(array_unique(array))){
   $duplicate = true;
}

Answer 5

Here's my take on this… after some benchmarking, I found this to be the fastest method for this.

function has_duplicates( $array ) {
    return count( array_keys( array_flip( $array ) ) ) !== count( $array );
}

…or depending on circumstances this could be marginally faster.

function has_duplicates( $array ) {
    $array = array_count_values( $array );
    rsort( $array );
    return $array[0] > 1;
}

Answer 6

count($array) > count(array_unique($array)); 

Will be false if duplicates, or true if no duplicates.

Answer 7

Keep it simple, silly! ;)

Simple OR logic...

function checkDuplicatesInArray($array){
    $duplicates=FALSE;
    foreach($array as $k=>$i){
        if(!isset($value_{$i})){
            $value_{$i}=TRUE;
        }
        else{
            $duplicates|=TRUE;          
        }
    }
    return ($duplicates);
}

Regards!

Answer 8

Find this useful solution

function get_duplicates( $array ) {
    return array_unique( array_diff_assoc( $array, array_unique( $array ) ) );
}

After that count result if greater than 0 than duplicates else unique.

Answer 9

Two ways to do it efficiently that I can think of:

1. inserting all the values into some sort of hashtable and checking whether the value you're inserting is already in it(expected O(n) time and O(n) space)

2. sorting the array and then checking whether adjacent cells are equal( O(nlogn) time and O(1) or O(n) space depending on the sorting algorithm)

stormdrain's solution would probably be O(n^2), as would any solution which involves scanning the array for each element searching for a duplicate

Answer 10

As you specifically said you didn't want to use array_unique I'm going to ignore the other answers despite the fact they're probably better.

Why don't you use array_count_values() and then check if the resulting array has any value greater than 1?

Answer 11

Php has an function to count the occurrences in the array http://www.php.net/manual/en/function.array-count-values.php

Answer 12

I'm using this:

if(count($array)==count(array_count_values($array))){
    echo("all values are unique");
}else{
    echo("there's dupe values");
}

I don't know if it's the fastest but works pretty good so far

Answer 13

You can do it like that way also: This will return true if unique else return false.

$nofollow = (count($modelIdArr) !== count(array_unique($modelIdArr))) ? true : false;

Answer 14

The simple solution but quite faster.

$elements = array_merge(range(1,10000000),[1]);

function unique_val_inArray($arr) {
    $count = count($arr);
    foreach ($arr as $i_1 => $value) {
        for($i_2 = $i_1 + 1; $i_2 < $count; $i_2++) {
            if($arr[$i_2] === $arr[$i_1]){
                return false;
            }
        }
    }
    return true;
}

$time = microtime(true);
unique_val_inArray($elements);
echo 'This solution: ', (microtime(true) - $time), 's', PHP_EOL;

Speed - [0.71]!