How to deserialize multiple objects from a json stream?

问题

Ok, I have asked questions about something like this before, but this is a different subject, so i feel i should make a new topic about it. I'm sorry if this is something i should not have done...

Anyway:

I'm currently reading a twitterfeed and trying to convert it to lose (status) objects. The code i have now is as follows but fails:

webRequest = (HttpWebRequest)WebRequest.Create(stream_url);
webRequest.Credentials = new NetworkCredential(username, password);
webRequest.Timeout = -1;
webResponse = (HttpWebResponse)webRequest.GetResponse();
Encoding encode = Encoding.GetEncoding("utf-8");
responseStream = new StreamReader(webResponse.GetResponseStream(), encode);
int i = 0;

//Read the stream.
while (_running)
{
    jsonText = responseStream.ReadLine();

    byte[] sd = Encoding.Default.GetBytes(jsonText);
    stream.Write(sd, i, i + sd.Length);

    try
    {
        status s = json.ReadObject(stream) as status;
        if (s != null)
        {
            //write s to a file/collection or w/e
            i = 0;
        }
    }
    catch
    {

    }

}

The idea is: Copy the stream into another stream. and keep trying to read it untill an status object is discovered. This was ment to prevent the stream for being to little, so it had chance to grow. Ofcourse the stream does not always start at the start of an object, or can be corrupt.

Now i did find the method IsStartObject, and i think i should use it. Though i have no experience with streams and i can never find a good example of how to use this.

Is there anyone who can explain to me how to read multiple objects from the stream so i can write them into a list or w/e. I really can't find any good examples on the internets..

Thank you very much for trying!!!

回答1:

I used Json.Net library and this extension class that makes use of DynamicObject to parse streaming json objects

HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("https://stream.twitter.com/1/statuses/sample.json");
webRequest.Credentials = new NetworkCredential("...", "......");
webRequest.Timeout = -1;
HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse();
Encoding encode = Encoding.GetEncoding("utf-8");
StreamReader responseStream = new StreamReader(webResponse.GetResponseStream());

string line;
while (true)
{
    line = responseStream.ReadLine();
    dynamic obj = JsonUtils.JsonObject.GetDynamicJsonObject(line);
    if(obj.user!=null)
        Console.WriteLine(obj.user.screen_name + " => " + obj.text);
}

回答2:

This is an implementation of L.B's suggestion for splitting by counting the nesting level of { and }.

public static IEnumerable<string> JsonSplit(this StreamReader input, char openChar = '{', char closeChar = '}',
  char quote='"', char escape='\\')
{
  var accumulator = new StringBuilder();
  int count = 0;
  bool gotRecord = false;
  bool inString = false;
  while (!input.EndOfStream)
  {
    char c = (char)input.Read();
    if (c == escape)
    {
      accumulator.Append(c);
      c = (char)input.Read();
    }
    else if (c == quote)
    {
      inString = !inString;
    }
    else if (inString)
    {
    }
    else if (c == openChar)
    {
      gotRecord = true;
      count++;
    }
    else if (c == closeChar)
    {
      count--;
    }
    accumulator.Append(c);
    if (count != 0 || !gotRecord) continue;
    // now we are not within a block so 
    string result = accumulator.ToString();
    accumulator.Clear();
    gotRecord = false;
    yield return result;
  }
}

Here's a test

[TestClass]
  public class UnitTest1
  {
    [TestMethod]
    public void TestMethod1()
    {
      string text = "{\"a\":1}{\"b\":\"hello\"}{\"c\":\"oh}no!\"}{\"d\":\"and\\\"also!\"}";
      var reader = FromStackOverflow.GenerateStreamFromString(text);
      var e = MyJsonExtensions.JsonSplit(reader).GetEnumerator();
      e.MoveNext();
      Assert.AreEqual("{\"a\":1}", e.Current);
      e.MoveNext();
      Assert.AreEqual("{\"b\":\"hello\"}", e.Current);
      e.MoveNext();
      Assert.AreEqual("{\"c\":\"oh}no!\"}", e.Current);
      e.MoveNext();
      Assert.AreEqual("{\"d\":\"and\\\"also!\"}", e.Current);
    }
  }

The implementation of GenerateStreamFromString is here

来源：https://stackoverflow.com/questions/8886735/how-to-deserialize-multiple-objects-from-a-json-stream

标签

.net

json

twitter

streaming