Using Httprequest to get pictures from given URL

Question

I'm trying to get pictures from webcams. There is a php-python web service to get the pictures from webcams and serve them: it serves the picture like http://ip/jpeg/camera=1.

private HttpWebRequest request;
private HttpWebResponse response;
private CookieContainer container;
private Uri uri;
private string _user;
private string _pass;
private string _ip;
//Login code as seen in the previous section should be here
//GetImages is meant to run as a separate thread 
private void GetImages(string camNo)
{
//create the GET request for the JPEG snapshot (found at /jpeg on the IP Camera)
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://" + deviceIP + "/jpeg/camera"+camNo);
request.Method = "GET";
request.ContentType = "application/x-www-form-urlencoded";
request.CookieContainer = container;

//attempt to get a response from the IP110, event if error

try
{
    response = (HttpWebResponse)request.GetResponse();
}
catch (WebException e)
{
    ConnectionError(new ConnectionErrorEventArgs(e.Message));
}

//Get the stream containing the JPEG image.
Stream stream = response.GetResponseStream();

//Read the stream to memory.
byte[] buffer = new byte[100000];
int read, total = 0;

while ((read = stream.Read(buffer, total, 1000)) != 0)
{
    total += read;
}

//create a BMP image from the stream
Bitmap bmp = (Bitmap)Bitmap.FromStream(new MemoryStream(buffer, 0, total));

//send the bmp image to where you would like to display it. 
 }

... then I store that bitmap as a jpeg to a folder. My problem starts here; I want to do that process as fast as possible. I want to take pictures from 50 webcams using that code and store as jpeg, and it has to be fast - I mean every 10 seconds I have to get new pictures from 50 cams.

I used timercontrol, and give it 500 ms every 500ms program use that code above up to camera number and save the jpeg, but it work one after another so performance gets low!

50ms x 500cams = 25000 ms (25sec), if I arrange timer control's interval 100 ms program freeze. When I use the code above I get bitmap bmp in 200 ms but when it tries to write as jpeg on disc it takes a long time.

What can I do to write to disk faster? I looked for memory mapped - would it help? I want to store jpegs on disc becouse I will serve these pictures on a web site and share them with people. Can I use memory mapped and serve it publicly via a web site?

UPDATE: I also used WebClient class for async works, http://www.informit.com/guides/content.aspx?g=dotnet&seqNum=571

The result was: To get a picture like 300 ms and writing it to disk about 700ms So I have to find a solution to write to disk as much as possible.

Which one is better? To write the pictures to disk or to send images to database? I tried to store the pictures to make them ready for serve, because on the website people have to see updated pictures. Which one is better for thousands of client query? To store them on disk or store them on database as binary?

Answer 1

The code to fetch the images is synchronous. I would suggest modifying your code to use HttpWebRequest.BeginGetResponse() to fetch your images as detailed in this answer.

So when your (single) timer fires, begin the async requests to your cams, then (as they complete) the callback will fire for each completed request, and you can save off the returned image.

If you need to optimize further, then you could delegate saving the image to disk to a background ThreadPool work item.

Answer 2

Any time you need to get responses back faster than stacked responses allow, you end up having to complete multiple pieces of work at once, using more than one thread (last word is the clue here).

In .NET, this means either setting up your own threading library, using TPL, etc.

ADDED: This could mean, as pointed out by a comment below, using asynchronous calls and allowing the framework to implicitly create threads for you, instead of controlling them yourself. In fact, you could use thread pooling and still choose asynch or synch without controlling the number of threads created. If you really want to get down to the metal, you can even offload threading to the OS using an interrupt handler at the OS level (a menthod used by Ruby, for example, prior to 1.9, which introduced "true" threading rather than "green" threads - yeah, TMI).

The answer still stands if you have 2 minutes worth of work to get done and 30 seconds to get it done in, you need a multi-threaded solution, whether you are aware you are actually threading (explicit creation) or not (implicit creation handled by framework/OS).