Performance tips for finding unique permutation

Question

TLDR: how to find multidimensional array permutation in php and how to optimize for bigger arrays?

This is continuation of this question: how to fin

Answer 1

Based on the answer to the previous question you supplied this can be solved ( in that case ) way more elegantly using a few built-in functions PHP has for array support. Which is probably the best of any language.

function solve($matrix){
    $master = [];
    $_matrix = [];
    foreach($matrix as $key => $array){
        $_matrix[$key] = array_combine($array,$array);
        $master += $_matrix[$key];
    }
    $default = array_fill_keys($master, ''); 
    $result = [];
    foreach($_matrix as $array){
        $result[] = array_values(array_merge($default, $array));
    }
    print_r($result);
}

Using their same tests

$tests = [
    [ ['X'], ['X'] ],
    [ ['X'], ['X'], ['X'] ],
    [ [ 'X', '' ], [ '', 'X' ] ],
    [ ['X', 'Y', 'Z'], ['X', 'Y'], ['X']],
    [ ['X', 'Y'], ['X', 'Y'], ['X', 'Y'] ],
    [ ['X', 'Y', 'Z'], ['X', 'Y', 'Z'], ['X', 'Y', 'Z'] ],
    [ ['X', 'Y', 'Z', 'I', 'J'], ['X', 'Y', 'Z', 'I'], ['X', 'Y', 'Z', 'I'], ['X', 'Y', 'Z', 'I'], ['X', 'Y', 'Z'], ['X', 'Y', 'Z'] ],
];
array_map(function ($matrix) {
    solve($matrix);
}, $tests);

This is what I get in comparison

[
  0 => ['X', 'Y', 'Z', 'I', 'J'] //<- contains all unique values
  1 => ['X', 'Y', 'Z', 'I']
  2 => ['X', 'Y', 'Z', 'I']
  3 => ['X', 'Y', 'Z', 'I']
  4 => ['X', 'Y', 'Z']
  5 => ['X', 'Y', 'Z']
]
Their Result:
[
  0 => ['', 'X', 'Y', 'Z', 'I', 'J'] //<- contains an extra '' empty value
  1 => ['', '', 'X', 'Y', 'Z', 'I']
  2 => ['I', '', '', 'X', 'Y', 'Z']
  3 => ['Z', 'I', '', '', 'X', 'Y']
  4 => ['Y', 'Z', '', '', '', 'X']
  5 => ['X', 'Y', 'Z', '', '', '']
]
My Result
[
  0 => ['X', 'Y', 'Z', 'I', 'J']
  1 => ['X', 'Y', 'Z', 'I', '']
  2 => ['X', 'Y', 'Z', 'I', '']
  3 => ['X', 'Y', 'Z', 'I', '']
  4 => ['X', 'Y', 'Z','','']
  5 => ['X', 'Y', 'Z','','']
]

You can test it here.

http://sandbox.onlinephpfunctions.com/code/86d0b4332963f95449df2e7d4d47fcd8224fe45d

I even timed it using microtime

mine 0.00013017654418945 milliseconds

theirs 0.10895299911499 milliseconds

Which is not really a surprise, as theirs is around 60 lines of code and 7 function calls. Mine is only 1 function 14 lines of code.

That said I don't know if the position of the values is important in the output. Nor exactly what you expect as the output extending that question.

The fact is they also lose the index position, just look at the second array in their result, 2 => ['I', '', '', 'X', 'Y', 'Z'] compared to the input 2 => ['X', 'Y', 'Z', 'I']. And I won't mention the extra '' in the output that probably doesn't belong there.

Maybe I'm missing something, lol, I don't typically do these math-y type things.

UPDATE if you want an explanation of how this works,

• array_combine($array,$array); creates an array with matched key => value, we abuse the fact that array keys are unique by nature. Like so ['X'=>'X','Y'=>'Y'...]
• then we build a "master" array with all the values in it and matched keys, one array to rule them all. The master array is limited in size to the max number or unique values because we are using the keys to eliminate duplicates.
• then we use array_fill_keys($master, ''); to sort of create a template of all the values. The keys of "master" are all the unique values in all of the inner arrays, so we fill it with our "wildcard" placeholder. In this case it looks like this ['X'=>'', 'Y'=>'', 'Z'=>'', 'I'=>'', 'J'=>'']
• then we merge the "modified" original array, also abusing the array keys for our advantage, by replacing the placeholders in the "templated" master array with the "modified" original array, because the keys match.
• last we strip the keys from the array using array_values

And we are left with each inner array "templated" by the master array but with the original values filled in and the missing ones empty.

Answer 2

what you should try to use is called a Power set which is:

from wikipedia in mathematics, the power set (or powerset) of any set S is the set of all subsets of S, including the empty set and S itself, variously denoted as P(S),

Answer 3

The soluton is quite simple actually. You check the number of unique chars and that's the number of values in the output array. Below code will do what you want almost instantly.

The hard part is removing the wildcards. This is something you can only do with bruteforce if you want 100% certainty. The solution below will try it's best to remove all wildcards by switching positions several times in order.

This is similar to the way google handles the Traveling Salesman Problem in it's OR-tools. You need to find the best mix between accuracy and speed. By setting the loop count higher in the function below, chances of success increase. But it will be slower.

/* HELPERS */

function ShowNice($output) {
  //nice output:
  echo '<pre>';
  foreach($output as $key=>$val) {
    echo '<br />' . str_pad($key,2," ",STR_PAD_LEFT) . ' => [';
    $first = true;
    foreach($val as $char) {
      if (!$first) {
        echo ', ';
      }
      echo "'".$char."'";
      $first = false;
    }
    echo ']';
  }
  echo '</pre>';
}

function TestValid($output, $nullchar) {
  $keys = count($output[0]);
  for ($i=0;$i<$keys;$i++) {
    $found = [];
    foreach($output as $key=>$val) {
      $char = $val[$i];
      if ($char==$nullchar) {
        continue;
      }
      if (array_key_exists($char, $found)) {
        return false; //this char was found before
      }
      $found[$char] = true;
    }
  }

  return true;
}


$input = [
  0 => ['X', 'Y', 'Z', 'I', 'J'],
  1 => ['X', 'Y', 'Z', 'I'],
  2 => ['X', 'Y', 'Z', 'I'],
  3 => ['X', 'Y', 'Z', 'I'],
  4 => ['X', 'Y', 'Z'],
  5 => ['X', 'Y', 'Z']
];

//generate large table
$genLength = 30; //max double alphabet
$innerLength = $genLength;
$input2 = [];
for($i=0;$i<$genLength;$i++) {
  $inner = [];

  if (rand(0, 1)==1) {
    $innerLength--;
  }

  for($c=0;$c<$innerLength;$c++) {
    $ascii = 65 + $c; //upper case
    if ($ascii>90) {
      $ascii += 6; //lower case
    }
    $inner[] = chr($ascii);
  }
  $input2[] = $inner;
}


//generate large table with different keys
$genLength = 10; //max double alphabet
$innerLength = $genLength;
$input3 = [];
for($i=0;$i<$genLength;$i++) {
  $inner = [];

  if (rand(0, 1)==1) {
    //comment o make same length inner arrays, but perhaps more distinct values
    //$innerLength--;
  }

  $nr = 0;
  for($c=0;$c<$innerLength;$c++) {
    $ascii = 65 + $c + $nr; //upper case
    if ($ascii>90) {
      $ascii += 6; //lower case
    }
    //$inner[] = chr($ascii);
    $inner[] = $c+$nr+1;

    //increase nr?
    if (rand(0, 2)==1) {
      $nr++;
    }

  }
  $input3[] = $inner;
}


//generate table with numeric values, to show what happens
$genLength = 10; //max double alphabet
$innerLength = $genLength;
$input4 = [];
for($i=0;$i<$genLength;$i++) {
  $inner = [];

  for($c=0;$c<$innerLength;$c++) {
    $inner[] = $c+1;
  }
  $input4[] = $inner;
}


$input5 = [
  0 => ['X', 'Y'],
  1 => ['X', 'Y'],
  2 => ['X', 'Y'],
];

$input6 = [
  0 => ['X', 'Y', 'Z', 'I', 'J'],
  1 => ['X', 'Y', 'Z', 'I'],
  2 => ['X', 'Y', 'Z', 'I'],
  3 => ['X', 'Y', 'Z', 'I'],
  4 => ['X', 'Y', 'Z']
];

$input7 = [
  ['X', 'Y', 'A', 'B'],
  ['X', 'Y', 'A', 'C']
];

$input8 = [
  ['X', 'Y', 'A'],
  ['X', 'Y', 'B'],
  ['X', 'Y', 'C']
];

$input9 = [
  ['X', 'Z', 'Y', 'A', 'E', 'D'],
  ['X', 'Z', 'Y', 'A', 'B'],
  ['X', 'Z', 'Y', 'A', 'C'],
  ['X', 'Z', 'Y', 'A', 'D'],
  ['X', 'Z', 'Y', 'A', 'D'],
  ['X', 'Z', 'Y', 'A', 'D']
];

/* ACTUAL CODE */

CreateOutput($input, 1);

function CreateOutput($input, $loops=0) {


  echo '<h2>Input</h2>';
  ShowNice($input);


  //find all distinct chars
  //find maxlength for any inner array

  $distinct = [];
  $maxLength = 0;
  $minLength = -1;
  $rowCount = count($input);
  $flipped = [];
  $i = 1;
  foreach($input as $key=>$val) {
    if ($maxLength<count($val)) {
      $maxLength = count($val);
    }
    if ($minLength>count($val) || $minLength==-1) {
      $minLength = count($val);
    }
    foreach($val as $char) {
      if (!array_key_exists($char, $distinct)) {
        $distinct[$char] = $i;
        $i++;
      }
    }

    $flipped[$key] = array_flip($val);
  }

  //keep track of the count for actual chars
  $actualChars = $i-1;
  $nullchar = '_';
  //add null values to distinct
  if ($minLength!=$maxLength && count($distinct)>$maxLength) {
    $char = '#'.$i.'#';
    $distinct[$nullchar] = $i; //now it only gets add when a key is smaller, not if all are the same size
    $i++;
  }

  //if $distinct count is small then rowcount, we need more distinct
  $addForRowcount = (count($distinct)<$rowCount);
  while (count($distinct)<$rowCount) {
    $char = '#'.$i.'#';
    $distinct[$char] = $i;
    $i++;
  }


  //flip the distinct array to make the index the keys
  $distinct = array_flip($distinct);

  $keys = count($distinct);

  //create output
  $output = [];
  $start = 0;
  foreach($input as $key=>$val) {
    $inner = [];
    for ($i=1;$i<=$keys;$i++) {
      $index = $start + $i;
      if ($index>$keys) {
          $index -= $keys;
      }

      if ($index>$actualChars) {
        //just add the null char
        $inner[] = $nullchar;
      } else {
        $char = $distinct[$index];

        //check if the inner contains the char
        if (!array_key_exists($char, $flipped[$key])) {
          $char = $nullchar;
        }

        $inner[] = $char;
      }

    }
    $output[] = $inner;
    $start++;
  }

  echo '<h2>First output, unchecked</h2>';
  ShowNice($output);

  $newOutput = $output;
  for ($x=0;$x<=$loops;$x++) {
    $newOutput = MoveLeft($newOutput, $nullchar);
    $newOutput = MoveLeft($newOutput, $nullchar, true);
    $newOutput = SwitchChar($newOutput, $nullchar);
  }

  echo '<h2>New output</h2>';
  ShowNice($newOutput);
  //in $newoutput we moved all the invalid wildcards to the end
  //now we need to test if the last row has wildcards

  if (count($newOutput[0])<count($output[0])) {
    $output = $newOutput;
  }


  echo '<h2>Best result ('.(TestValid($output, $nullchar)?'VALID':'INVALID').')</h2>';
  ShowNice($output);

  return $output;
}

function MoveLeft($newOutput, $nullchar, $reverse=false) {
  //see if we can remove more wildcards
  $lastkey = count($newOutput[0])-1;
  $testing = true;
  while ($testing) {
    $testing = false; //we decide if we go another round ob_deflatehandler
    $test = $newOutput;

    $lastkey = count($newOutput[0])-1;

    $start = 0;
    $end = count($test);
    if ($reverse) {
      $start = count($test)-1;
      $end = -1;
    }

    for($key = $start;$key!=$end;$key += ($reverse?-1:1) ) {
      $val = $test[$key];
      $org = array_values($val);
      foreach($val as $i=>$char) {
        if ($char!=$nullchar) {
          continue; //we only test wildcards
        }


        $wildcardAtEnd=true;
        for($x=$i+1;$x<=$lastkey;$x++) {
          $nextChar = $val[$x];
          if ($nextChar!=$nullchar) {
            $wildcardAtEnd = false;
            break;
          }
        }


        if ($wildcardAtEnd===true) {
          continue; //the char next to it must not be wildcard
        }

        //remove the wildcard and add it to the base64_encode
        unset($val[$i]);
        $val[] = $nullchar;
        $test[$key] = array_values($val); //correct order

        if (TestValid($test, $nullchar)) {
          //we can keep the new one
          $newOutput = $test;
          $testing = true; //keep testing, but start over to make sure we dont miss anything
          break 2; //break both foreach, not while
        }

        $test[$key] = $org; //reset original values before remove for next test
      }
    }
  }

  $allWildCards = true;
  while ($allWildCards) {
    $lastkey = count($newOutput[0])-1;
    foreach($newOutput as $key=>$val) {
      if ($val[$lastkey]!=$nullchar)  {
        $allWildCards = false;
        break;
      }
    }
    if ($allWildCards) {
      foreach($newOutput as $key=>$val) {
        unset($val[$lastkey]);
        $newOutput[$key] = array_values($val);
      }
      $output = $newOutput;
    }
  }

  return $newOutput;
}

function SwitchChar($newOutput, $nullchar) {
  $switching = true;
  $switched = [];
  while($switching) {
    $switching = false;

    $test = $newOutput;
    $lastkey = count($newOutput[0])-1;

    foreach($test as $key=> $val) {
      foreach($val as $index=>$char) {
        $switched[$key][$index][$char] = true;//store the switches we make

        //see if can move the char somewhere else
        for($i=0;$i<=$lastkey;$i++)
        {
          if ($i==$index) {
            continue;//current pos
          }
          if (isset($switched[$key][$i][$char])) {
            continue; //been here before
          }

          $org = array_values($val);
          $switched[$key][$i][$char] = true;
          $t = $val[$i];
          $val[$index] = $t;
          $val[$i] = $char;
          $test[$key] = array_values($val);

          if (TestValid($test, $nullchar)) {
            //echo '<br />VALID: ' . $key . ' - ' . $index . ' - ' . $i . ' - ' . $t . ' - ' . $char;
            $newOutput = MoveLeft($test, $nullchar);
            $switching = true;
            break 3;//for and two foreach
          }

          //echo '<br />INVALID: ' . $key . ' - ' . $index . ' - ' . $i . ' - ' . $t . ' - ' . $char;
          $val = $org;
          $test[$key] = $org;
        }
      }
    }
  }

  return $newOutput;
}

Result:

Input

   0 => ['X', 'Y', 'A']
   1 => ['X', 'Y', 'B']
   2 => ['X', 'Y', 'C']

   First output, unchecked

   0 => ['X', 'Y', 'A', '_', '_']
   1 => ['Y', '_', 'B', '_', 'X']
   2 => ['_', '_', 'C', 'X', 'Y']

   New output

   0 => ['X', 'Y', 'A', '_', '_']
   1 => ['Y', 'B', 'X', '_', '_']
   2 => ['C', 'X', 'Y', '_', '_']

   Best result (VALID)

   0 => ['X', 'Y', 'A']
   1 => ['Y', 'B', 'X']
   2 => ['C', 'X', 'Y']

Answer 4

hope this is the solution we looking for, I have request for everybody to check my solution

the code:

<?php
$input = [
    [ 'F', 'I', 'J', 'Z' ],
    [ 'F', 'R', 'U', 'V' ],
    [ 'I', 'R', 'U', 'V' ],
    [ 'M', 'P', 'U', 'V' ],
];
do {
    $result = calculate($input);
} while(!TestValid($input, $result, '_'));


echo (TestValid($input, $result, '_')) ? 'VALID' : 'INVALID';

ShowNice($result);

function TestValid($input, $output, $nullchar) {

    foreach($output as $k => $o) {
        $test = array_filter($o, function($v) use ($nullchar) {
            return $v != $nullchar;
        });

        if (count($test) != count($input[$k])) {
            return false;
        }
    }
    return true;
}

function ShowNice($output) {
    $height = getHeight($output);

  foreach($output as $k => $v ) {
        for($i = 0;$i < $height;$i ++) {

            if (!isset($output[$k][$i])) {
                $output[$k][$i] = '_';
            }
        }
    }

  echo '<pre>';
  foreach($output as $key=>$val) {
    echo '<br />' . str_pad($key,2," ",STR_PAD_LEFT) . ' => [';
    ksort($val);
    echo join(', ', $val);
    echo ']';
  }
  echo '</pre>';
}

function calculate($array) {

    echo "<pre>";
    $full = getFullList($array);

    foreach($full as $f) {
        $frequency[$f] = getFrequency($array, $f);
    }

    // uksort($frequency, function($i, $j) use ($frequency) { 
    //     return $frequency[$j] <=> $frequency[$i];
    // });

    $frequency = array_keys($frequency);

    shuffle($frequency);


    $height = getHeight($array);

    foreach($array as $k => $v ) {
        for($i = 0;$i < $height;$i ++) {
            if (!isset($array[$k][$i]))
                $array[$k][$i] = '_';
        }
    }

    foreach($array as $key => $value ) {
        $output[$key] = [];
        $used[$key] = [];
    }

    foreach($array as $key => $value ) {
        foreach($frequency as $k => $v) {

            $j = 0;

            foreach($array as $kk => $col) {
                if (in_array($v, $col)) {
                    for ($h = 0; $h <= $height; $h++) {

                        if (!isset($_h[$v][$kk])) {
                            $_h[$v][$kk] = 0;
                        }

                        if ($h + $_h[$v][$kk] >= $height) {
                            $hh = ($h + $_h[$v][$kk]) - $height;
                        } else {
                            $hh = $h + $_h[$v][$kk];
                        }
                        $row = getRow($output, $hh);
                        if (!in_array($v, $row) && !in_array($v, $used[$kk])) {
                            if (!isset($output[$kk][$hh]) || $output[$kk][$hh] == '_') {
                                $output[$kk][$hh] = $v;
                                $used[$kk][] = $v;

                                $keys = array_keys($array);
                                foreach($keys as $i => $ke) {
                                    if ($ke == $kk) {
                                        if(isset($keys[$i+1])) {
                                            $_h[$v][$keys[$i + 1]] = $hh;
                                        } else {
                                            $_h[$v][$keys[0]] = $hh;
                                        }
                                    }
                                }
                               $unused[$kk] = array_diff($col, $used[$kk]);
                                $j++;
                                break;
                            }
                        }
                    }
                }
            }

            // ShowNice($output);
        }

    }
    foreach($output as $k => $v ) {
        for($i = 0;$i < $height;$i ++) {
            if (!isset($output[$k][$i]))
                $output[$k][$i] = '_';
        }
    }

    foreach($output as $k => $o) {
        ksort($output[$k]);
    }

    return $output;
}

function getHeight($array) {
    $heights = [];
    $max3 = count($array);
    $max2 = 0;
    foreach($array as $v) {
        if ($max2 < count($v)) {
            $max2 = count($v);
        }

        foreach ($v as $e) {
            $heights[$e] = (isset($heights[$e])) ? $heights[$e] + 1 : 1;
        }

    }
    $max = 0;
    foreach ($heights as $h ) {
        if ($h > $max) {
            $max = $h;
        }
    }

    return max($max, $max2, $max3);
}

function getRow($array, $row) {
    $res = [];
    foreach ($array as $a) {
        if (is_array($a)) {
            foreach($a as $k =>$b) {
                if ($row == $k) {
                    $res[] = $b;
                }
            }
        }
    }

    return $res;
}

function getFrequency($array, $value) {
    $c=0;
    foreach ($array as $key => $v) {
        foreach ($v as $e) {
            if ($e == $value) 
                $c++;
        }
    }
    return $c;
}

function getFullList($array) {

    $m = [];
    foreach($array as $a) {
        $m = array_merge($m, $a);
    }

    return array_unique($m);
}

UPDATED

This may be the final, check please: playground: https://eval.in/906355

Answer 5

After quite some fiddling, I came up with the following code.

The idea is to identify conflicting elements and swap them to a column where they are no longer a problem. For cases where this is not applicable a random selection is done. The code works recursive and thus there are edge-cases where it takes very long to complete.

An extreme edge-case is an input where all rows consist of exactly the same values.

<?php
declare(strict_types=1);

final class SwapSolver
{
    /**
     * @param array $input
     *
     * @return array
     */
    public function solve(array $input): array
    {
        $input = array_values($input);

        return $this->swapDuplicates($this->prepare($input, $this->getMinRowLength($input)));
    }

    /**
     * @param array $input
     *
     * @return array
     */
    private function swapDuplicates(array $input): array
    {
        $unswappable = [];

        foreach ($this->duplicates($input) as $position) {
            list($r, $a) = $position;

            $swapped = false;
            foreach ($this->swapCandidates($input, $r, $a, true) as $b) {
                $input[$r] = $this->swap($input[$r], $a, $b);
                $swapped = true;
                break;
            }

            if (!$swapped) {
                $unswappable[] = $position;
            }
        }

        // still unswappable
        $unswappable = array_values(array_filter($unswappable, function (array $position) use ($input): bool {
            return $this->isDuplicate($input, ...$position);
        }));

        // tie breaker
        if (count($unswappable) > 0) {
            list($r, $a) = $unswappable[mt_rand(0, count($unswappable) - 1)];

            $candidates = [];
            foreach ($this->swapCandidates($input, $r, $a, false) as $b) {
                $candidates[] = $b;
            }

            $input[$r] = $this->swap($input[$r], $a, $candidates[mt_rand(0, count($candidates) - 1)]);

            return $this->swapDuplicates($input);
        }

        return $input;
    }

    /**
     * @param array $input
     *
     * @return \Generator
     */
    private function duplicates(array &$input): \Generator
    {
        foreach ($input as $r => $row) {
            foreach ($row as $c => $value) {
                if ($this->isDuplicate($input, $r, $c)) {
                    yield [$r, $c];
                }
            }
        }
    }

    /**
     * @param array $input
     * @param int   $row
     * @param int   $column
     *
     * @return bool
     */
    private function isDuplicate(array $input, int $row, int $column): bool
    {
        $candidate = $input[$row][$column];

        if (is_null($candidate)) {
            return false;
        }

        foreach (array_column($input, $column) as $r => $value) {
            if ($r !== $row && $value === $candidate) {
                return true;
            }
        }

        return false;
    }

    /**
     * @param array $input
     * @param int   $row
     * @param int   $column
     * @param bool  $strict
     *
     * @return \Generator
     */
    private function swapCandidates(array &$input, int $row, int $column, bool $strict): \Generator
    {
        foreach ($input[$row] as $c => $dst) {
            if ((!$strict || !in_array($input[$row][$column], array_column($input, $c), true))
                && (is_null($dst) || !in_array($dst, array_column($input, $column), true))
            ) {
                yield $c;
            }
        }
    }

    /**
     * @param array $row
     * @param int   $from
     * @param int   $to
     *
     * @return array
     */
    private function swap(array $row, int $from, int $to): array
    {
        $tmp = $row[$to];
        $row[$to] = $row[$from];
        $row[$from] = $tmp;

        return $row;
    }

    /**
     * @param array $input
     * @param int   $padSize
     *
     * @return array
     */
    private function prepare(array $input, int $padSize): array
    {
        return array_map(function (array $row) use ($padSize): array {
            $row = array_pad($row, $padSize, null);
            shuffle($row);

            return $row;
        }, $input);
    }

    /**
     * @param array $input
     *
     * @return int
     */
    private function getMinRowLength(array $input): int
    {
        return max(
            ...array_values(array_count_values(array_merge(...$input))),
            ...array_map('count', $input)
        );
    }
}

Usage:

<?php
$solver = new SwapSolver();
$solution = $solver->solve($input);

---

More code at: https://github.com/Yoshix/so-47261385