How to combine several similar commands using “for match in soup.find_all”?

问题

I have below code in which there are similar commands involved for match in soup.find_all. I would like to ask if it's possible to merge them and thus have cleaner code.

import requests
from bs4 import BeautifulSoup

url = 'https://www.collinsdictionary.com/dictionary/french-english/aimanter'
headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0'}
soup = BeautifulSoup(requests.get(url, headers = headers).content, 'html.parser')

entry_name = soup.h2.text

for script in soup.select('script, .hcdcrt, #ad_contentslot_1, #ad_contentslot_2'):
    script.extract()

for match in soup.find_all('div', {'class' : 'copyright'}):  
    match.extract()
    
for match in soup.find_all('div', {'class' : 'example-info'}):  
    match.extract()

for match in soup.find_all('div', {'class' : 'share-overlay'}):  
    match.extract()
    
for match in soup.find_all('div', {'class' : 'popup-overlay'}):  
    match.extract()    
    

content1 = ''.join(map(str, soup.select_one('.cB.cB-def.dictionary.biling').contents))
content2 = ''.join(map(str, soup.select_one('.cB.cB-e.dcCorpEx').contents))

format = open('aimer.html', 'w+', encoding = 'utf8')
format.write(entry_name + '\n' + str(content1) + str(content2) + '\n</>\n' )
format.close()

回答1:

You can combine the various loops with .find_all() into the first .select().

For example:

import requests
from bs4 import BeautifulSoup


url = 'https://www.collinsdictionary.com/dictionary/french-english/aimanter'
headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0'}
soup = BeautifulSoup(requests.get(url, headers = headers).content, 'html.parser')

entry_name = soup.h2.text

for tag in soup.select('''
        script,
        .hcdcrt,
        #ad_contentslot_1,
        #ad_contentslot_2,
        div.copyright,
        div.example-info,
        div.share-overlay,
        div.popup-overlay'''):
    tag.extract()

content1 = ''.join(map(str, soup.select_one('.cB.cB-def.dictionary.biling').contents))
content2 = ''.join(map(str, soup.select_one('.cB.cB-e.dcCorpEx').contents))

format = open('aimer.html', 'w+', encoding = 'utf8')
format.write(entry_name + '\n' + str(content1) + str(content2) + '\n</>\n' )
format.close()

来源：https://stackoverflow.com/questions/63114903/how-to-combine-several-similar-commands-using-for-match-in-soup-find-all

标签

python-3.x

beautifulsoup