streamreader

I'm trying to implement a limited web crawler in C# (for a few hundred sites only) using HttpWebResponse.GetResponse() and Streamreader.ReadToEnd() , also tried using StreamReader.Read() and a loop to build my HTML string. I'm only downloading pages which are about 5-10K. It's all very slow! For example, the average GetResponse() time is about half a second, while the average StreamREader.ReadToEnd() time is about 5 seconds! All sites should be very fast, as they are very close to my location, and have fast servers. (in Explorer takes practically nothing to D/L) and I am not using any proxy.

I require help to search a text file (log file) using c# and display the line number and the complete line that contains the search keyword. This is a slight modification from: http://msdn.microsoft.com/en-us/library/aa287535%28VS.71%29.aspx int counter = 0; string line; // Read the file and display it line by line. System.IO.StreamReader file = new System.IO.StreamReader("c:\\test.txt"); while((line = file.ReadLine()) != null) { if ( line.Contains("word") ) { Console.WriteLine (counter.ToString() + ": " + line); } counter++; } file.Close(); Bit late to the game on this one, but happened

Is there an elegant to emulate the StreamReader.ReadToEnd method with BinaryReader ? Perhaps to put all the bytes into a byte array? I do this: read1.ReadBytes((int)read1.BaseStream.Length); ...but there must be a better way. Scott Rippey Simply do: byte[] allData = read1.ReadBytes(int.MaxValue); The documentation says that it will read all bytes until the end of the stream is reached. Update Although this seems elegant, and the documentation seems to indicate that this would work, the actual implementation (checked in .NET 2, 3.5, and 4) allocates a full-size byte array for the data, which

can you use streamreader to read a normal textfile and then in the middle of reading close the streamreader after saving the current position and then open streamreader again and start reading from that poision ? if not what else can i use to accomplish the same case without locking the file ? something like this: var fs = File.Open(@"C:\testfile.txt", FileMode.Open, FileAccess.Read); var sr = new StreamReader(fs); Debug.WriteLine(sr.ReadLine());//Prints:firstline var pos = fs.Position; while (!sr.EndOfStream) { Debug.WriteLine(sr.ReadLine()); } fs.Seek(pos, SeekOrigin.Begin); Debug.WriteLine