beautifulsoup

问题 I am scraping a page using Python & bs4 The html source code that I get from bs4 is as follows (cleaned up a bit for readability purpose): <p style="text-align:justify;font-size:12.0px;font-family:Arial, Helvetica, sans-serif"> <span style="font-size:14.0px"><span style="font-family:Arial, Helvetica, sans-serif"> <strong>COMPANY DESCRIPTION</strong><br> Here goes the first para of company description</span></span></p> <p style="text-align:justify;font-size:12.0px;font-family:Arial, Helvetica,

问题 I'd like to extract text with line break along with "br" tag with BeautifulSoup. html = "<td class="s4 softmerge" dir="ltr"><div class="softmerge-inner" style="width: 5524px; left: -1px;">But when he saw many of the Pharisees and Sadducees come to his baptism, he said unto them, <br/>O generation of vipers, who hath warned you to flee from the wrath to come?<br/>Bring forth therefore fruits meet for repentance:<br/>And think not to say within yourselves, We have Abraham to our father: for I

问题 I'd like to extract text with line break along with "br" tag with BeautifulSoup. html = "<td class="s4 softmerge" dir="ltr"><div class="softmerge-inner" style="width: 5524px; left: -1px;">But when he saw many of the Pharisees and Sadducees come to his baptism, he said unto them, <br/>O generation of vipers, who hath warned you to flee from the wrath to come?<br/>Bring forth therefore fruits meet for repentance:<br/>And think not to say within yourselves, We have Abraham to our father: for I

问题 I'm just starting to learn how to web scrape using BeautifulSoup and want to write a simple program that will get the follower count for a given Instagram page. I currently have the following script (pulled from another Q&A thread): import requests from bs4 import BeautifulSoup user = "espn" url = 'https://www.instagram.com/'+ user r = requests.get(url) soup = BeautifulSoup(r.content) followers = soup.find('meta', {'name': 'description'})['content'] follower_count = followers.split('Followers

问题 I'm just starting to learn how to web scrape using BeautifulSoup and want to write a simple program that will get the follower count for a given Instagram page. I currently have the following script (pulled from another Q&A thread): import requests from bs4 import BeautifulSoup user = "espn" url = 'https://www.instagram.com/'+ user r = requests.get(url) soup = BeautifulSoup(r.content) followers = soup.find('meta', {'name': 'description'})['content'] follower_count = followers.split('Followers

问题 I'm just starting to learn how to web scrape using BeautifulSoup and want to write a simple program that will get the follower count for a given Instagram page. I currently have the following script (pulled from another Q&A thread): import requests from bs4 import BeautifulSoup user = "espn" url = 'https://www.instagram.com/'+ user r = requests.get(url) soup = BeautifulSoup(r.content) followers = soup.find('meta', {'name': 'description'})['content'] follower_count = followers.split('Followers

问题 I've created a script in python using requests module in combination with BeautifulSoup library to fill in some tiny forms traversing different pages in a webpage. There are multiple get and post requests I need to issue to accomplish this as selenium is not an option here. I'm only interested in modifying the fields in step 2 captioned as personal information . How to do it - After logging in using the email and password (available within the script) it is necessary to choose (by default yes

问题 I want to delete the specific div from soup object. I am using python 2.7 and bs4 . According to documentation we can use div.decompose() . But that would delete all the div . How can I delete a div with specific class? 回答1: Sure, you can just select, find, or find_all the div s of interest in the usual way, and then call decompose() on those divs. For instance, if you want to remove all divs with class sidebar , you could do that with # replace with `soup.findAll` if you are using

问题 I want to delete the specific div from soup object. I am using python 2.7 and bs4 . According to documentation we can use div.decompose() . But that would delete all the div . How can I delete a div with specific class? 回答1: Sure, you can just select, find, or find_all the div s of interest in the usual way, and then call decompose() on those divs. For instance, if you want to remove all divs with class sidebar , you could do that with # replace with `soup.findAll` if you are using

问题 Im trying to work with both beautifulsoup and xpath and was trying to using the following code, but now im getting 1 file per URL instead of before where i was getting 1 file for all the URLS I just moved over the reading from CSV to get the list of urls and also just added the parsing of the url and response.. but when i run this now i get alot of individual files and in some cases 1 file may actually contain 2 scraped pages data.. so do i need to move my file saving out (indent) import