How do I keep reading an endless stream with fail tolerance

纵饮孤独 提交于 2019-12-23 05:27:38

问题


Currently i'm working on an app that reads the stream from a Twitter api and parses it into objects. At the moment I read the stream and use ReadObject from DataContractJsonSerializer to make my objects.

This works great!!

HOWEVER: I'm kind of worried what would happen in the off chance my program catches up with the stream (internet slows down or w/e) and there is not enough data to parse...The method will probably throw an exception, but i want to wait for new data and then retry the same object and continue.

Also i was wondering how i could make the method more stable, in case corrupt data would enter the stream or something like this.

Thanks in advance for any answers/ideas:)


回答1:


If your program catches up with the twitter feed, the DCJS should just block until you get enough data to complete reading. That isn't normally a concern, because streams are designed to hide latency from their readers.

Much more likely is that you won't catch up, but rather keep falling behind until you run out of memory, throw an OOME, and the program will crash.

I'd suggest rather than trying to parse the stream on the fly, write it to something like a file and read from that (maybe even in parallel, using rolling files or something).



来源:https://stackoverflow.com/questions/8866812/how-do-i-keep-reading-an-endless-stream-with-fail-tolerance

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!