powershell httpwebrequest GET method cookiecontainer problem?

Question

I\'m trying to scrape a website that has user authentication. I am able to do a POST to send my login and stores a cookie. However, after the login I get a 403 error when tr

Answer 1

I found that since cookies can have additional information attached (like the URL or HTTP-only), the $res.Headers["Set-Cookie"] didn't work for me. But using your $CookieContainer variable, you can easily change it to use GetCookieHeader(url), which will strip out the extra information and leave you with a properly formatted cookie string:

$web = new-object net.webclient
$web.Headers.add("Cookie", $CookieContainer.GetCookieHeader($url))
$result = $web.DownloadString($url)