How to parse a JSON Input stream

Question

I am using java to call a url that returns a JSON object:

url = new URL(\"my URl\");
urlInputStream = url.openConnection().getInputStream();

<

Answer 1

Use a library.

• GSON
• Jackson
• or one of many other JSON libraries that are out there.

Answer 2

This example reads all objects from a stream of objects, it is assumed that you need CustomObjects instead of a Map:

        ObjectMapper mapper = new ObjectMapper();
        JsonParser parser = mapper.getFactory().createParser( source );
        if(parser.nextToken() != JsonToken.START_ARRAY) {
          throw new IllegalStateException("Expected an array");
        }
        while(parser.nextToken() == JsonToken.START_OBJECT) {
          // read everything from this START_OBJECT to the matching END_OBJECT
          // and return it as a tree model ObjectNode
          ObjectNode node = mapper.readTree(parser);
          CustomObject custom = mapper.convertValue( node, CustomObject.class );
           // do whatever you need to do with this object
          System.out.println( "" + custom );
        }
        parser.close();

This answer was composed by using : Use Jackson To Stream Parse an Array of Json Objects and Convert JsonNode into Object

Answer 3

For those that pointed out the fact that you can't use the toString method of InputStream like this see https://stackoverflow.com/a/5445161/1304830 :

My correct answer would be then :

import org.json.JSONObject;

public static String convertStreamToString(java.io.InputStream is) {
    java.util.Scanner s = new java.util.Scanner(is).useDelimiter("\\A");
    return s.hasNext() ? s.next() : "";
}

...

JSONObject json = new JSONObject(convertStreamToString(url.openStream());

Answer 4

All the current answers assume that it is okay to pull the entire JSON into memory where the advantage of an InputStream is that you can read the input little by little. If you would like to avoid reading the entire Json file at once then I would suggest using the Jackson library (which is my personal favorite but I'm sure others like Gson have similar functions).

With Jackson you can use a JsonParser to read one section at a time. Below is an example of code I wrote that wraps the reading of an Array of JsonObjects in an Iterator. If you just want to see an example of Jackson, look at the initJsonParser, initFirstElement, and initNextObject methods.

public class JsonObjectIterator implements Iterator<Map<String, Object>>, Closeable {
    private static final Logger LOG = LoggerFactory.getLogger(JsonObjectIterator.class);

    private final InputStream inputStream;
    private JsonParser jsonParser;
    private boolean isInitialized;

    private Map<String, Object> nextObject;

    public JsonObjectIterator(final InputStream inputStream) {
        this.inputStream = inputStream;
        this.isInitialized = false;
        this.nextObject = null;
    }

    private void init() {
        this.initJsonParser();
        this.initFirstElement();
        this.isInitialized = true;
    }

    private void initJsonParser() {
        final ObjectMapper objectMapper = new ObjectMapper();
        final JsonFactory jsonFactory = objectMapper.getFactory();

        try {
            this.jsonParser = jsonFactory.createParser(inputStream);
        } catch (final IOException e) {
            LOG.error("There was a problem setting up the JsonParser: " + e.getMessage(), e);
            throw new RuntimeException("There was a problem setting up the JsonParser: " + e.getMessage(), e);
        }
    }

    private void initFirstElement() {
        try {
            // Check that the first element is the start of an array
            final JsonToken arrayStartToken = this.jsonParser.nextToken();
            if (arrayStartToken != JsonToken.START_ARRAY) {
                throw new IllegalStateException("The first element of the Json structure was expected to be a start array token, but it was: " + arrayStartToken);
            }

            // Initialize the first object
            this.initNextObject();
        } catch (final Exception e) {
            LOG.error("There was a problem initializing the first element of the Json Structure: " + e.getMessage(), e);
            throw new RuntimeException("There was a problem initializing the first element of the Json Structure: " + e.getMessage(), e);
        }

    }

    private void initNextObject() {
        try {
            final JsonToken nextToken = this.jsonParser.nextToken();

            // Check for the end of the array which will mean we're done
            if (nextToken == JsonToken.END_ARRAY) {
                this.nextObject = null;
                return;
            }

            // Make sure the next token is the start of an object
            if (nextToken != JsonToken.START_OBJECT) {
                throw new IllegalStateException("The next token of Json structure was expected to be a start object token, but it was: " + nextToken);
            }

            // Get the next product and make sure it's not null
            this.nextObject = this.jsonParser.readValueAs(new TypeReference<Map<String, Object>>() { });
            if (this.nextObject == null) {
                throw new IllegalStateException("The next parsed object of the Json structure was null");
            }
        } catch (final Exception e) {
            LOG.error("There was a problem initializing the next Object: " + e.getMessage(), e);
            throw new RuntimeException("There was a problem initializing the next Object: " + e.getMessage(), e);
        }
    }

    @Override
    public boolean hasNext() {
        if (!this.isInitialized) {
            this.init();
        }

        return this.nextObject != null;
    }

    @Override
    public Map<String, Object> next() {
        // This method will return the current object and initialize the next object so hasNext will always have knowledge of the current state

        // Makes sure we're initialized first
        if (!this.isInitialized) {
            this.init();
        }

        // Store the current next object for return
        final Map<String, Object> currentNextObject = this.nextObject;

        // Initialize the next object
        this.initNextObject();

        return currentNextObject;
    }

    @Override
    public void close() throws IOException {
        IOUtils.closeQuietly(this.jsonParser);
        IOUtils.closeQuietly(this.inputStream);
    }

}

If you don't care about memory usage, then it would certainly be easier to read the entire file and parse it as one big Json as mentioned in other answers.

Answer 5

I would suggest you have to use a Reader to convert your InputStream in.

BufferedReader streamReader = new BufferedReader(new InputStreamReader(in, "UTF-8")); 
StringBuilder responseStrBuilder = new StringBuilder();

String inputStr;
while ((inputStr = streamReader.readLine()) != null)
    responseStrBuilder.append(inputStr);
new JSONObject(responseStrBuilder.toString());

I tried in.toString() but it returns:

getClass().getName() + '@' + Integer.toHexString(hashCode())

(like documentation says it derives to toString from Object)

Answer 6

If you like to use Jackson Databind (which Spring uses by default for its HttpMessageConverters), then you may use the ObjectMapper.readTree(InputStream) API. For example,

ObjectMapper mapper = new ObjectMapper();
JsonNode json = mapper.readTree(myInputStream);