Not able to scrap the images from Flipkart.com website the src attribute is coming emtpy

问题

I am able to scrap all the data from flipkart website except the images using the code below:

jobs = soup.find_all('div',{"class":"IIdQZO _1R0K0g _1SSAGr"})

for job in jobs:
    product_name = job.find('a',{'class':'_2mylT6'})
    product_name = product_name.text if product_name else "N/A"

    product_offer_price = job.find('div',{'class':'_1vC4OE'})
    product_offer_price = product_offer_price.text if product_offer_price else "N/A"

    product_mrp = job.find('div',{'class':'_3auQ3N'})
    product_mrp = product_mrp.text if product_mrp else "N/A"

    product_link = job.find('a',{'class':'_3dqZjq'})
    product_link = product_link.get('href') if product_link else "N/A"
    product_link = url+ product_link

    product_img = job.find('div',{'class':'_3ZJShS _31bMyl'})


    print('product name {}\nproduct offer price {}\nproduct mrp {}\nproduct link {}\nproduct image {}'.\
      format(product_name,product_offer_price,product_mrp,product_link,product_img))
    print('\n')

Results for eg.:

product name UV Protection Wayfarer Sunglasses (54)
product offer price ₹8,000
product mrp ₹8,890
product link https://www.flipkart.com/search?q=rayban/ray-ban-wayfarer- 
product image <img alt="" class="_3togXc" src=""/>

When I am manually inspecting the page the src is there but when scraping it is coming empty as above

回答1:

As I mentioned on comment use selenium.

from selenium import webdriver
from bs4 import BeautifulSoup

driver=webdriver.Chrome()
driver.get('https://www.flipkart.com/search?q=rayban/ray-ban-wayfarer')
time.sleep(3)
soup=BeautifulSoup(driver.page_source,'html.parser')
url='"https://www.flipkart.com'
jobs = soup.find_all('div',{"class":"IIdQZO _1R0K0g _1SSAGr"})

for job in jobs:
    product_name = job.find('a',{'class':'_2mylT6'})
    product_name = product_name.text if product_name else "N/A"

    product_offer_price = job.find('div',{'class':'_1vC4OE'})
    product_offer_price = product_offer_price.text if product_offer_price else "N/A"

    product_mrp = job.find('div',{'class':'_3auQ3N'})
    product_mrp = product_mrp.text if product_mrp else "N/A"

    product_link = job.find('a',{'class':'_3dqZjq'})
    product_link = product_link.get('href') if product_link else "N/A"
    product_link = url+ product_link

    product_img =job.find('div',{'class':'_3ZJShS _31bMyl'}).find('img')['src']

    print('product name {}\nproduct offer price {}\nproduct mrp {}\nproduct link {}\nproduct image {}'.\
      format(product_name,product_offer_price,product_mrp,product_link,product_img))
    print('\n')

回答2:

The images sources are dynamically added by the javascript part.

You should use selenium to get the page source.

Check that code:

from bs4 import BeautifulSoup as soup
from selenium import webdriver

url = 'https://www.flipkart.com/search?q=rayban'

driver = webdriver.Firefox()
driver.get(url)

html = driver.page_source
page = soup(html)

jobs = page.find_all('div',{"class":"IIdQZO _1R0K0g _1SSAGr"})

for job in jobs:
    product_name = job.find('a',{'class':'_2mylT6'})
    product_name = product_name.text if product_name else "N/A"

    product_offer_price = job.find('div',{'class':'_1vC4OE'})
    product_offer_price = product_offer_price.text if product_offer_price else "N/A"

    product_mrp = job.find('div',{'class':'_3auQ3N'})
    product_mrp = product_mrp.text if product_mrp else "N/A"

    product_link = job.find('a',{'class':'_3dqZjq'})
    product_link = product_link.get('href') if product_link else "N/A"
    product_link = url+ product_link

    product_img = job.find('div',{'class':'_3ZJShS _31bMyl'})


    print('product name {}\nproduct offer price {}\nproduct mrp {}\nproduct link {}\nproduct image {}'.\
      format(product_name,product_offer_price,product_mrp,product_link,product_img))
    print('\n')

OUTPUT:

product name Aviator Sunglasses (58)
product offer price ₹4,760
product mrp ₹5,290
product link https://www.flipkart.com/search?q=rayban/ray-ban-aviator-sunglasses/p/itmf3yh25kzjsapz?pid=SGLDPXDY4CMRNEY9&lid=LSTSGLDPXDY4CMRNEY9TNASSX&marketplace=FLIPKART&srno=s_1_1&otracker=search&fm=organic&iid=5ed8f60d-e315-4440-bfdc-76049d80e5da.SGLDPXDY4CMRNEY9.SEARCH&qH=95c0daefc80c4a70
product image <div class="_3ZJShS _31bMyl" style="padding-top:120.00%"><img alt="" class="_3togXc" src="https://rukminim1.flixcart.com/image/454/545/sunglass/e/y/9/0rb3129iw0228-rayban-58-original-imadqb2nzmwzfup6.jpeg?q=50"/></div>
.
.
.

来源：https://stackoverflow.com/questions/57287726/not-able-to-scrap-the-images-from-flipkart-com-website-the-src-attribute-is-comi