NSURLRequest, why does craigslist return a 404?

Question

I'm building my first iPhone app and I'm stymied.

I'm trying to build an RSS reader, and I'm trying to use a feed from craigslist. This code, using stackoverflow, returns "Status code: 200":

- (void)parseRSSFeed:(NSString *)feed withDelegate:(id)delegate
{
 responseData = [[NSMutableData data] retain];
 feed = @"http://stackoverflow.com";
 NSURLRequest *request = [NSURLRequest requestWithURL:[NSURL URLWithString:feed]];
 [[NSURLConnection alloc] initWithRequest:request delegate:self]; 
}

- (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response {
 NSLog(@"status code: %d", [((NSHTTPURLResponse*) response) statusCode]);
 [responseData setLength:0];
}

All is well. But if I change feed to, say, "http://portland.craigslist.org/muc/", I get a status code of 404.

Is there anything I'm missing? Does Craigslist disallow iPhone access to its website? Is there some escaping I need to do on the URL?

The code, with the craigslist URL, is here. This is exactly as I'm using it, and it returns a 404:

- (void)parseRSSFeed:(NSString *)feed withDelegate:(id)delegate
{
 responseData = [[NSMutableData data] retain];
 feed = @"http://portland.craigslist.org/muc/";
 NSURLRequest *request = [NSURLRequest requestWithURL:[NSURL URLWithString:feed]];
 [[NSURLConnection alloc] initWithRequest:request delegate:self]; 
}

Answer 1

Ok i figured it out. Craigslist is blocking this request. The same code works when you change the name of the app. For example when the app name contained the string "craig" the response code that was returned was 404. When i changed the app name to not contain the name "craig" then the response is 200.

Answer 2

Craigslist blocks URL requests if the User-Agent header contains the string "craig". Apple, by default, includes your app name in the User-Agent header of your URL requests. To get around the block, use NSMutableURLRequest instead of NSURLRequest and fake out the User-Agent header:

#define kUserAgentString @"Mozilla/5.0 (Windows; U; Windows NT 6.1; tr-TR) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27"

NSMutableURLRequest *request = [[[NSMutableURLRequest alloc] initWithURL:theURL] autorelease];
NSString *userAgent = kUserAgentString;
[request setValue:userAgent forHTTPHeaderField:@"User-Agent"];