How to check if a value already exists to avoid duplicates?

Question

I\'ve got a table of URLs and I don\'t want any duplicate URLs. How do I check to see if a given URL is already in the table using PHP/MySQL?

Answer 1

The simple SQL solutions require a unique field; the logic solutions do not.

You should normalize your urls to ensure there is no duplication. Functions in PHP such as strtolower() and urldecode() or rawurldecode().

Assumptions: Your table name is 'websites', the column name for your url is 'url', and the arbitrary data to be associated with the url is in the column 'data'.

Logic Solutions

SELECT COUNT(*) AS UrlResults FROM websites WHERE url='http://www.domain.com'

Test the previous query with if statements in SQL or PHP to ensure that it is 0 before you continue with an INSERT statement.

Simple SQL Statements

Scenario 1: Your db is a first come first serve table and you have no desire to have duplicate entries in the future.

ALTER TABLE websites ADD UNIQUE (url)

This will prevent any entries from being able to be entered in to the database if the url value already exists in that column.

Scenario 2: You want the most up to date information for each url and don't want to duplicate content. There are two solutions for this scenario. (These solutions also require 'url' to be unique so the solution in Scenario 1 will also need to be carried out.)

REPLACE INTO websites (url, data) VALUES ('http://www.domain.com', 'random data')

This will trigger a DELETE action if a row exists followed by an INSERT in all cases, so be careful with ON DELETE declarations.

INSERT INTO websites (url, data) VALUES ('http://www.domain.com', 'random data')
ON DUPLICATE KEY UPDATE data='random data'

This will trigger an UPDATE action if a row exists and an INSERT if it does not.