How to get page via TcpClient?

Question

I'm trying to send a GET request to a page via TCP stream.

Here's what my code looks like:

public class SocketLevelWebClient
{
    public string SendWebRequest(string url, string request)
    {
        using(TcpClient tc = new TcpClient())
        {
            tc.Connect(url, 80);

            using (NetworkStream ns = tc.GetStream())
            {
                using (System.IO.StreamWriter sw = new System.IO.StreamWriter(ns))
                {
                    using (System.IO.StreamReader sr = new System.IO.StreamReader(ns))
                    {
                        sw.Write(request);
                        sw.Flush();
                        return sr.ReadToEnd();
                    }
                }
            }
        }
    }

And the request itself:

            SocketLevelWebClient wc = new SocketLevelWebClient();
            var r=wc.SendWebRequest("www.youtube.com",@"GET http://www.youtube.com/ HTTP/1.1
Accept: application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, */*
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
Host: www.youtube.com"+"\r\n\r\n");

When I call this code, it always freezes waiting for the response from the server.

What am I doing wrong?

Answer 1

The problem is that ReadToEnd only returns when the stream has ended. Unfortunately, the server keeps the TCP connection alive. Therefore ReadToEnd can never detect that the true end has arrived.

Proof:

                        sw.Write(request);
                        sw.Flush();
                        var l = sr.ReadLine();

l is being filled with the first line of the request.

Remove the keep-alive header and add:

Connection: close

Or use the response Content-Length header to correctly read it (binary).

Answer 2

    A simple example is this:

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Threading.Tasks;
    using System.IO;
    using System.Net;
    using System.Net.Sockets;
    using System.Web;
    using System.Data;
    using System.Collections;
    using System.Collections.Specialized;
    using System.Windows.Forms;

    //Some "using" may not be needed
    static public TcpListener listener = new TcpListener(IPAddress.Any, 8080);

     static void Main(string[] args)
            {
             listener.Start();
             TcpClient client = listener.AcceptTcpClient();
             StreamReader sr = new StreamReader(client.GetStream());
             sr.ReadLine();
            }



**For asynchronous Connection:**


 static void Main(string[] args)
     {
       client_listener();
     }
async static public void client_listener()
        {
            while (true)
            {
                listener.Start();
                TcpClient client = await listener.AcceptTcpClientAsync();
                StreamReader sr = new StreamReader(client.GetStream());
                try
                {
                    await sr.ReadLineAsync();
                }
                catch(Exception e)
                {
                }
        }
}