duplicate-data

A query that is used to loop through 17 millions records to remove duplicates has been running now for about 16 hours and I wanted to know if the query is stopped right now if it will finalize the delete statements or if it has been deleting while running this query? Indeed, if I do stop it, does it finalize the deletes or rolls back? I have found that when I do a select count(*) from myTable That the rows that it returns (while doing this query) is about 5 less than what the starting row count was. Obviously the server resources are extremely poor, so does that mean that this process has

I've been thinking this problem for a whole night: here is my matrix: 'a' '#' 3 '#' 'a' 3 0 'I am' 2 'I am' 0 2 ..... I want to treat the rows like the first two rows are the same, because it's just different order of 'a' and '#'. In my case, I want to delete such kind of rows. The toy example is simple, the first two are the same, the third and the forth are the same. but in my data set, I don't know where is the 'same' row. I'm writing in R. Thanks. Perhaps something like this would work for you. It is not clear what your desired output is though. x <- structure(c("a", "#", "0", "I am", "#",

How do you count duplicates in a ruby array? For example, if my array had three a's, how could I count that miku This will yield the duplicate elements as a hash with the number of occurences for each duplicate item. Let the code speak: #!/usr/bin/env ruby class Array # monkey-patched version def dup_hash inject(Hash.new(0)) { |h,e| h[e] += 1; h }.select { |k,v| v > 1 }.inject({}) { |r, e| r[e.first] = e.last; r } end end # unmonkeey'd def dup_hash(ary) ary.inject(Hash.new(0)) { |h,e| h[e] += 1; h }.select { |_k,v| v > 1 }.inject({}) { |r, e| r[e.first] = e.last; r } end p dup_hash([1, 2, "a",

I'm attempting to clean up a database that, over the years, had acquired many duplicate records, with slightly different names. For example, in the companies table, there are names like "Some Company Limited" and "SOME COMPANY LTD!". My plan was to export the offending tables into R, convert names to lower case, replace common synonyms (like "limited" -> "ltd"), strip out non-alphabetic characters and then use agrep to see what looks similar. My first problem is that agrep only accepts a single pattern to match, and looping over every company name to match against the others is slow. (Some

What's the simplest SQL statement that will return the duplicate values for a given column and the count of their occurrences in an Oracle database table? For example: I have a JOBS table with the column JOB_NUMBER . How can I find out if I have any duplicate JOB_NUMBER s, and how many times they're duplicated? Bill the Lizard SELECT column_name, COUNT(column_name) FROM table_name GROUP BY column_name HAVING COUNT(column_name) > 1; Grrey Another way: SELECT * FROM TABLE A WHERE EXISTS ( SELECT 1 FROM TABLE WHERE COLUMN_NAME = A.COLUMN_NAME AND ROWID < A.ROWID ) Works fine (quick enough) when