How to tell if a webpage exists?

问题

Just for fun, I'm writing a Perl program to check if a given website exists. For my purposes, a website exists if I can go into my browser, punch in the url and get a meaningful webpage (meaning not an error or "failed to open page" message). What would be the best way to go about doing this? Eventually I would like to be able to give my program a list of hundreds of urls.

I'm thinking about just pinging each of the urls on my list to see if they exist; however, I don't really know too much about networking so is this the best way to do it?

回答1:

Using Library for WWW in Perl (LWP):

#!/usr/bin/perl
use LWP::Simple;
my $url = 'http://www.mytestsite.com/';
if (head($url)) {
  print "Page exists\n";
} else {
  print "Page does not exist\n";;
}

回答2:

There is no such protocol as "pinging web pages" for existence. You actually have to request the resource and if it's served up, it exists. There are several ways to go about it, here are a couple:

1. Retrieving web pages with LWP

2. Checking for an existing web page could as simple as:

   #!/usr/bin/env perl
   use strict;
   use warnings;
   use LWP::Simple qw(head);
   head('http://www.perlmeme.org') or die 'Unable to get page';
   

The same solution as command-line tool is lwp-request/HEAD. HEAD returns the resource headers, such as content size and will be quicker than getting all the page contents.

来源：https://stackoverflow.com/questions/11594932/how-to-tell-if-a-webpage-exists