I\'m trying to use gson to convert this returned JSON into some kind of data structure such that I can extract useful data.
For Example:
http://search.twitte
Do I need to define a class which maps exactly to the structure of the JSON in order to then populate an instance of that class? If so this seems very inflexible/laborious.
Yes. GSON is a library that can be used to convert Java Objects into their JSON representation. It can also be used to convert a JSON string to an equivalent Java object. This is really powerful because you can automagically instantiate your Java objects from the JSON representation. Assuming your JSON doesn't change its structure, you only have to define the appropriate Java object representation once.
Ideally I'm looking for something which will handle JSON in any form and give me a structure I can use automatically...
However, if you don't want automagical serialisation/deserialisation, then try looking at a simpler library such as java.net/projects/jsonp.
You can extract stuff from it just by querying the keys:
final JSONObject json = new JSONObject(theJsonString);
final String id = json.getString("max_id");
final JSONArray results = json.getJSONArray("results");
final String user = results.getJSONObject(2).getString("from_user");