What's the best approach for parsing XML/'screen scraping' in iOS? UIWebview or NSXMLParser?

Question

I am creating an iOS app that needs to get some data from a web page. My first though was to use NSXMLParser initWithContentsOfURL: and parse the HTML with the

Answer 1

I've done this a few times. The best approach I've found is to use libxml2 which has a mode for HTML. Then you can use XPath to query the document.

Working with the libxml2 API is not the most enjoyable. So, I usually bring over the XPathQuery.h/.m files documented on this page:

http://cocoawithlove.com/2008/10/using-libxml2-for-parsing-and-xpath.html

Then I fetch the data using a NSConnection and query the data with something like this:

NSArray *tdNodes = PerformHTMLXPathQuery(self.receivedData, @"//td[@class='col-name']/a/span");

Summary:

1. Add libxml2 to your project, here are some quick instructions for XCode4: http://cmar.me/2011/04/20/adding-libxml2-to-an-xcode-4-project/

2. Get the XPathQuery.h/.m

3. Use an XPath statement to query the html document.

Answer 2

Parsing HTML with an XML parser usually does not work anyway because many sites have incorrect HTML, which a web browser will deal with, but a strict XML parser like NSXMLParser will totally fail on.

For many scripting languages there are great scraping libraries that are more merciful. Like Python's Beautiful Soup module. Unfortunately I do not know of such modules for Objective-C.

Loading stuff into a UIWebView might be the simplest way to go here. Note that you do not have to put the UIWebView on screen. You can create a separate UIWindow and add the UIWebView to it, so that you do full off-screen rendering. There was a WWDC2009 video about this I think. As you already mention, it will not be lightweight though.

Depending on the data that you want and the complexity of the pages that you need to parse, you might also be able to parse it by using regular expressions or even a hand written parser. I have done this many times, and for simple data this works well.