How to find special characters in DB2?

Question

I have a DB2 database containing millions of records. I found that some char() or varchar() fields contain special characters which shouldn\'t be stored. I guess application

Answer 1

You can use a regular expression in order to retrieve the invalid characters. However this process is very expensive, because you have to read all the data, and then process it.

In order to use regular expressions in DB2, you have to adapt the environement, because this functionality is not available for SQL in the installation. You have these three options:

• Use Regular expressions with xQuery instead of normal SQL. http://publib.boulder.ibm.com/infocenter/db2luw/v10r1/topic/com.ibm.db2.luw.xml.doc/doc/xqrregexp.html
• Define external C stored procedures as described in this article: http://www.ibm.com/developerworks/data/library/techarticle/0301stolze/0301stolze.html
• If you undestand Japannese, here you have a good article explaining how to use RegEx in xQuery https://www.ibm.com/developerworks/jp/data/library/db2/j_d-regularexpression/ You could only download the sources and install them. With the few examples in latin characters, I think you could understand how to use this.

Once you have defined a regular expression to ignore the valid characters (something like /[^a-zA-Z0-9]/ ), then you could executed in the database. Remember to retrieve other column where you can detect the row (for example a column ID) and then perfom updates or delete to prune the invalid characters.

If you do not know how to use regular expression, here you have a good source of information: http://www.regular-expressions.info/ Specially http://www.regular-expressions.info/charclass.html

There is a related question about regular expression: Regular Expressions in DB2 SQL