beautifulsoup

问题 I am trying to strip certain HTML tags and their content from a file with BeautifulSoup . How can I remove lines that get empty after applying decompose() ? In this example, I want the line between a and 3 to be gone, as this is where the <span>...</span> block was, but not the line in the end. from bs4 import BeautifulSoup Rmd_data = 'a\n<span class="answer">\n2\n</span>\n3\n' print(Rmd_data) #OUTPUT # a # <span class="answer"> # 2 # </span> # 3 # # END OUTPUT soup = BeautifulSoup(Rmd_data,

问题 I'm trying to parse a number of web pages with text, tables and html. Every page has a different number of paragraphs, but while every paragraph begins with an opening <div> , the closing </div> does not occur until the end. I'm just trying to get the content, filtering out certain elements and replacing them by something else Desired result: text1 <b>text2</b> (table_deleted) text3 Actual result text1\n\ntext2some text heretext 3text2some text heretext 3 (table deleted) from bs4 import

问题 I am trying to download a GIF file with urrlib , but it is throwing this error: urllib.error.HTTPError: HTTP Error 403: Forbidden This does not happen when I download from other blog sites. This is my code: import requests import urllib.request url_1 = 'https://goodlogo.com/images/logos/small/nike_classic_logo_2355.gif' source_code = requests.get(url_1,headers = {'User-Agent': 'Mozilla/5.0'}) path = 'C:/Users/roysu/Desktop/src_code/Python_projects/python/web_scrap/myPath/' full_name = path +

问题 I am trying to solve my problems making searches, reading documentations. The problem I want to get all youtube titles from an youtube channel using python beautiful soup. Youtube loads dynamically, i think with JavaScript, without pyqt5 I just can not get any title, So i used the pyqt5 I was able to get titles from youtube channel. The problem is that i need to load all the videos. I can just load the 29 ou 30 first ones. I am thinking on simulating a scroll down or somthing like that. I can

问题 I'm trying to scrape a table on an ajax page with Beautiful Soup and print it out in table form with the TextTable library. import BeautifulSoup import urllib import urllib2 import getpass import cookielib import texttable cj = cookielib.CookieJar() opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj)) urllib2.install_opener(opener) ... def show_queue(): url = 'https://www.animenfo.com/radio/nowplaying.php' values = {'ajax' : 'true', 'mod' : 'queue'} data = urllib.urlencode(values) f

问题 So I'm new to Python (I come from a PHP/JavaScript background), but I just wanted to write a quick script that crawled a website and all children pages to find all a tags with href attributes, count how many there are and then click the link. I can count all of the links, but I can't figure out how to "click" the links and then return the response codes. from bs4 import BeautifulSoup import urllib2 import re def getLinks(url): html_page = urllib2.urlopen(url) soup = BeautifulSoup(html_page,

问题 I have the following element in a web page. <button type="submit" class="zsg-button_primary contact-submit-button track-ga-event" data-ga-category="contact" data-ga-action="email" data-ga-label="rentalbuilding" data-ga-event-content="false" data-ga-event-details="" id="yui_3_18_1_2_1482045459111_1278"> <span class="zsg-loading-spinner hide"></span> <span class="button-text" id="yui_3_18_1_2_1482045459111_1277">Contact Property Manager</span> </button> I can find this element with

问题 I'm coding a Python parser for a website to do some job automatically but I'm not much into "re" module (regex) for Py and can't make it work. req = urllib2.Request(tl2) req.add_unredirected_header('User-Agent', ua) response = urllib2.urlopen(req) try: html = response.read() except urllib2.URLError, e: print "Error while reading data. Are you connected to the interwebz?!", e soup = BeautifulSoup.BeautifulSoup(html) form = soup.find('form', id='form_product_page') pret = form.prettify() print

问题 I recently started working on a program in python which allows the user to conjugate any verb easily. To do this, I am using the urllib module to open the corresponding conjugations web page. For example, the verb "beber" would have the web page: "http://www.spanishdict.com/conjugate/beber" To open the page, I use the following python code: source = urllib.urlopen("http://wwww.spanishdict.com/conjugate/beber").read() This source does contain the information that I want to parse. But, when I

问题 I recently started working on a program in python which allows the user to conjugate any verb easily. To do this, I am using the urllib module to open the corresponding conjugations web page. For example, the verb "beber" would have the web page: "http://www.spanishdict.com/conjugate/beber" To open the page, I use the following python code: source = urllib.urlopen("http://wwww.spanishdict.com/conjugate/beber").read() This source does contain the information that I want to parse. But, when I