Facebook mutual friends and FQL 4999/5000 record limit

Question

I\'m trying to select all mutual friends\' connections with PHP/FQL. Using my UID (540 friends), which means >12,000 connections, of which >6500 are unique. So this code

Answer 1

I was doing a similar thing- but getting fb page posts- and ran into funny caching on the FB servers when I pummeled it with multiqueries - batching FQL is a way to get around this, btw. And yeah, ran into the 5K limit and just batched under 5k and had to set up paginating, which was a pain.

Answer 2

An interesting observation: When I try to find all the people that have mutual friends me I user the following query

SELECT uid1, uid2 FROM friend WHERE uid1 IN (SELECT uid2 FROM friend WHERE uid1=$uid)

As you can see it is very similar to your query, except that I removed the AND clause.

I get the following error message: "Can't lookup all friends of 208733. Can only lookup for the logged in user or the logged in user's friends that are users of your app".

I guess facebook is smart enough to figure out that I am trying to do something that it doesn't want me to do. It somehow detects the fact that you are trying to find only your friend's friends that are also your friends, while I am trying to find all of my friend's friends.

Answer 3

FQL supports a LIMIT, just like regular SQL. You may try that. http://developers.facebook.com/docs/guides/performance

Otherwise, I suggest getting the friend IDs for each user, storing those in a SQL table, then performing your own join to get the crossover. You may be able to just get the friends lists once and then subscribe to realtime updates to keep your lists up to date. http://developers.facebook.com/docs/api/realtime

Answer 4

One trick it seems i was able to use is to limit the no of queries based on one of the indexable columns from your tables ( using strpos(column_name, character / number)) .

ex:

$fql = "SELECT pid,src_big,owner FROM photo WHERE album_object_id IN 
      (SELECT object_id FROM privacy WHERE 
             ( object_id IN ( SELECT object_id FROM album WHERE owner IN (SELECT uid2 FROM friend WHERE 
                   ( uid1 = " . $this->nextUser->fbid . " AND **strpos(uid2,2) = 1**  )))                                   
           AND ( value = 'FRIENDS_OF_FRIENDS' OR value = 'EVERYONE'  ) )) ;

And this way you can split it in 10 subqueries, or for a alphnumeric field in 33.

Answer 5

So, I'm posting the answer to my original question. I was able to circumvent the 5000 row limit on FQL queries by chunking the array of UIDs (using the appropriately-named array_chunk() PHP function) and looping through the chunks to execute mini-queries, and then appending it all back into one array. The whole script averages 14 seconds for over 12,000 rows so that is a huge improvement. You can see the application at work here: givememydata.com

Oh, and Facebook should reconsider their (still undocumented) FQL row limit. What is more taxing on their servers? A single query that executes in 5 seconds or 500 queries that take 180 seconds? Sorry, had to vent. ;-)

Answer 6

An alternative would be to use the fql.multiquery method, and build a seperate FQL query for each friend (or a group of friends per FQL query), but still sending all queries in the same request.