How can I screen scrape with Perl?

Question

I need to display some values that are stored in a website, for that I need to scrape the website and fetch the content from the table. Any ideas?

Answer 1

I use LWP::UserAgent for most of my screen scraping needs. You can also Couple that with HTTP::Cookies if you need Cookies support.

Here's a simple example on how to get source.

use LWP;
use HTTP::Cookies;
my $cookie_jar = HTTP::Cookies->new;
my $browser = LWP::UserAgent->new;
$browser->cookie_jar($cookie_jar);

$resp = $browser->get("https://www.stackoverflow.com");
if($resp->is_success) {
   # Play with your source here
   $source = $resp->content;
   $source =~ s/^.*//i; # this is just an example 
   print $source;                     # not a solution to your problem.
}

                                                        

              
                
                0
              
                   
                
               讨论(0)
              
                                                  
              
              

                          
             
       
          
              
                                       

     查看其它10个回答


            

                    



               
            

    发布评论:
    

         
                        
    
    
提交评论 
  
  

                    
                    

                    

                        

                        
                         加载中...
                        
                    
                

          

            
            

              
              

            
    


                                 
              
            
                          
    

        
         

                验证码
                

                  
                
                

                   看不清?
                
              
                                  

                    
   
                 
             
              
提交回复
            
          
        

          
 
     
 

        
热议问题