How to take all the words in one normal array from the database in the Laravel framework with a minimum waste of time?

Question

I want to take all the words from the database into default array. I have more than 50,000 words in the database, and most likely this number is up to a million. Therefore, I want this operation did not take much time. I tried such ways in which not a word is put into the usual array. That is, the words are passed to the associative array:

$words = DB::table('words')->pluck('word');
dump($words);

Result:

Collection {#197 ▼
  #items: array:12 [▼
    0 => "тоҷик"
    1 => "ӯзбек"
    2 => "қирғиз"
    3 => "эрон"
    4 => "япон"
    5 => "англис"
    6 => "тоҷик"
    7 => "ӯзбек"
    8 => "қирғиз"
    9 => "эрон"
    10 => "япон"
    11 => "англис"
  ]
}

Second method:

$words = DB::select("SELECT `word` FROM `words`");
dump($words);

Result:

array:12 [▼
  0 => {#210 ▼
    +"word": "тоҷик"
  }
  1 => {#207 ▼
    +"word": "ӯзбек"
  }
  2 => {#209 ▼
    +"word": "қирғиз"
  }
  3 => {#206 ▼
    +"word": "эрон"
  }
  4 => {#208 ▼
    +"word": "япон"
  }
  5 => {#205 ▼
    +"word": "англис"
  }
  6 => {#204 ▼
    +"word": "тоҷик"
  }
  7 => {#203 ▼
    +"word": "ӯзбек"
  }
  8 => {#202 ▼
    +"word": "қирғиз"
  }
  9 => {#200 ▼
    +"word": "эрон"
  }
  10 => {#213 ▼
    +"word": "япон"
  }
  11 => {#214 ▼
    +"word": "англис"
  }
]

And I want to take all the words in this form on the usual array:

array:12 [▼
  0 => "тоҷик"
  1 => "ӯзбек"
  2 => "қирғиз"
  3 => "эрон"
  4 => "япон"
  5 => "англис"
  6 => "тоҷик"
  7 => "ӯзбек"
  8 => "қирғиз"
  9 => "эрон"
  10 => "япон"
  11 => "англис"
]

I need this to get the difference between the two arrays. The input array is an upbeat array so I need to take all the words from the bass to add to the copious array. In general, is there any function like array_diff() in the Laravel framework itself?

$diff = array_diff($input_array, $words_from_db);

And add the difference to the database:

foreach ($diff as $value) {
    DB::table('words')->insert(['word'=>$value]);
}

How correct is my method of adding a large number of words (data) to the database? Can I need to optimize if of course it's possible?

In the general i found this function from Laravel:

public function diff($items)
{
    return new static(array_diff($this->items, $this->getArrayableItems($items)));
}

But how I can use it for get diff 2 arrays? Input array will be default array and second array words from db

Thank you all in advance for answering and forgive me for apologizing for grammatical errors in the question.

Answer 1

You can accomplish inserting unique words into the database entirely inside of MySQL, this should always be faster than pulling all the words into memory, diffing them and sending in the unique values.

To do this you need to add a unique index on your words column. (ref: https://dev.mysql.com/doc/refman/5.7/en/create-index.html)

CREATE UNIQUE INDEX `idx_word` ON `db`.`words` (word);

Then on your insert you need to add ON DUPLICATE KEY (Laravel query builder does not support on duplicate key - can use raw to insert) ref: https://dev.mysql.com/doc/refman/5.7/en/insert-on-duplicate.html, https://laravel.com/docs/5.5/queries#raw-expressions

INSERT INTO word VALUES ('foo'),('bar') ON DUPLICATE KEY word = VALUES(word);

I would use on duplicate key rather than insert ignore. I think insert ignore may hide other potential issues unrelated to duplicate key.

Answer 2

You can get diff two array in Laravel 5.5 so:

$words = DB::table('words')->pluck('word')->toArray(); // Collection converted to simple array

And you can use default function in php for get diff arrays:

$result = array_diff($input_default_array, $words);