NSString - Convert to pure alphabet only (i.e. remove accents+punctuation)

Question

I\'m trying to compare names without any punctuation, spaces, accents etc. At the moment I am doing the following:

-(NSString*) prepareString:(NSString*)a {
         


        

Answer 1

@interface NSString (Filtering)
    - (NSString*)stringByFilteringCharacters:(NSCharacterSet*)charSet;
@end

@implementation NSString (Filtering)
    - (NSString*)stringByFilteringCharacters:(NSCharacterSet*)charSet {
      NSMutableString * mutString = [NSMutableString stringWithCapacity:[self length]];
      for (int i = 0; i < [self length]; i++){
        char c = [self characterAtIndex:i];
        if(![charSet characterIsMember:c]) [mutString appendFormat:@"%c", c];
      }
      return [NSString stringWithString:mutString];
    }
@end

Answer 2

On a similar question, Ole Begemann suggests using stringByFoldingWithOptions: and I believe this is the best solution here:

NSString *accentedString = @"ÁlgeBra";
NSString *unaccentedString = [accentedString stringByFoldingWithOptions:NSDiacriticInsensitiveSearch locale:[NSLocale currentLocale]];

Depending on the nature of the strings you want to convert, you might want to set a fixed locale (e.g. English) instead of using the user's current locale. That way, you can be sure to get the same results on every machine.

Answer 3

Before using any of these solutions, don't forget to use decomposedStringWithCanonicalMapping to decompose any accented letters. This will turn, for example, é (U+00E9) into e ‌́ (U+0065 U+0301). Then, when you strip out the non-alphanumeric characters, the unaccented letters will remain.

The reason why this is important is that you probably don't want, say, “dän” and “dün”* to be treated as the same. If you stripped out all accented letters, as some of these solutions may do, you'll end up with “dn”, so those strings will compare as equal.

So, you should decompose them first, so that you can strip the accents and leave the letters.

*Example from German. Thanks to Joris Weimar for providing it.

Answer 4

If you are trying to compare strings, use one of these methods. Don't try to change data.

- (NSComparisonResult)localizedCompare:(NSString *)aString
- (NSComparisonResult)localizedCaseInsensitiveCompare:(NSString *)aString
- (NSComparisonResult)compare:(NSString *)aString options:(NSStringCompareOptions)mask range:(NSRange)range locale:(id)locale

You NEED to consider user locale to do things write with strings, particularly things like names. In most languages, characters like ä and å are not the same other than they look similar. They are inherently distinct characters with meaning distinct from others, but the actual rules and semantics are distinct to each locale.

The correct way to compare and sort strings is by considering the user's locale. Anything else is naive, wrong and very 1990's. Stop doing it.

If you are trying to pass data to a system that cannot support non-ASCII, well, this is just a wrong thing to do. Pass it as data blobs.

https://developer.apple.com/library/ios/documentation/cocoa/Conceptual/Strings/Articles/SearchingStrings.html

Plus normalizing your strings first (see Peter Hosey's post) precomposing or decomposing, basically pick a normalized form.

- (NSString *)decomposedStringWithCanonicalMapping
- (NSString *)decomposedStringWithCompatibilityMapping
- (NSString *)precomposedStringWithCanonicalMapping
- (NSString *)precomposedStringWithCompatibilityMapping

No, it's not nearly as simple and easy as we tend to think. Yes, it requires informed and careful decision making. (and a bit of non-English language experience helps)

Answer 5

Consider using NSScanner, and specifically the methods -setCharactersToBeSkipped: (which accepts an NSCharacterSet) and -scanString:intoString: (which accepts a string and returns the scanned string by reference).

You may also want to couple this with -[NSString localizedCompare:], or perhaps -[NSString compare:options:] with the NSDiacriticInsensitiveSearch option. That could simplify having to remove/replace accents, so you can focus on removing puncuation, whitespace, etc.

If you must use an approach like you presented in your question, at least use an NSMutableString and replaceOccurrencesOfString:withString:options:range: — that will be much more efficient than creating tons of nearly-identical autoreleased strings. It could be that just reducing the number of allocations will boost performance "enough" for the time being.

Answer 6

Peter's Solution in Swift:

let newString = oldString.componentsSeparatedByCharactersInSet(NSCharacterSet.letterCharacterSet().invertedSet).joinWithSeparator("")

Example:

let oldString = "Jo_ - h !. nn y"
// "Jo_ - h !. nn y"
oldString.componentsSeparatedByCharactersInSet(NSCharacterSet.letterCharacterSet().invertedSet)
// ["Jo", "h", "nn", "y"]
oldString.componentsSeparatedByCharactersInSet(NSCharacterSet.letterCharacterSet().invertedSet).joinWithSeparator("")
// "Johnny"