How do you find all list items between two tags with BeautifulSoup?

问题

For example, I'd like to pull out only Child1, Child2, and Child3 out of the below list where it is after the first instance of h3 and before the next tag of h3

<h3>HeaderName1<h3>
<ul class="prodoplist">
 <li>Parent</li>
 <li class="lev1">Child1</li>
 <li class="lev1">Child2</li>
 <li class="lev1">Child3</li>
  </ul>
  <h3>HeaderName2<h3>
   <ul class="prodoplist">
   <li>Parent2</li>
   <li class="lev1">Child4</li>
   <li class="lev1">Child5</li>
   <li class="lev1">Child6</li>
   </ul>

回答1:

using findChildren like:

for ul in soup.find_all('ul'):
    print 'ul start'
    for idx, li in enumerate(ul.findChildren('li')):
        if idx in range(3):
            print li

output:

ul start
<li>Parent</li>
<li class="lev1">Child1</li>
<li class="lev1">Child2</li>
ul start
<li>Parent2</li>
<li class="lev1">Child4</li>
<li class="lev1">Child5</li>

however, as in most cases lxml and xpath is a superior solution:

from lxml import html
doc = html.parse('input.html')
print [ul.xpath('li[1] | li[2] | li[3]') for ul in doc.xpath('//ul')]

回答2:

This should work .

import re
from BeautifulSoup import BeautifulSoup
html_doc = '<h3>HeaderName1</h3><ul class="prodoplist"><li>Parent</li><li class="lev1">Child1</li><li class="lev1">Child2</li><li class="lev1">Child3</li></ul>  <h3>HeaderName2</h3><ul class="prodoplist"><li>Parent2</li><li class="lev1">Child4</li><li class="lev1">Child5</li><li class="lev1">Child6</li></ul>'
m = re.search(r'<h3>.*?<h3>', html_doc, re.DOTALL)
s = m.start()
e = m.end() - len('<h3>')
target_html = html_doc[s:e]
new_bs = BeautifulSoup(target_html)
ul_eles = new_bs.findAll('ul', attrs={'class' : 'prodoplist'})
for ul_ele in ul_eles:
    li_eles = new_bs.findAll('li', attrs={'class' : 'lev1'})
    for li_ele in li_eles:
        print li_ele.text

回答3:

import requests
from BeautifulSoup import BeautifulSoup

children = []

url = "http://someurl.html"
r = requests.get(url)
bs = BeautifulSoup(r.text)
for uls in bs.findAll('ul', 'prodoplist'):
    lis = uls.findAll('li', 'lev1')
    for li in lis:
        children.append(li.text)

print children

来源：https://stackoverflow.com/questions/21422090/how-do-you-find-all-list-items-between-two-tags-with-beautifulsoup

标签

python

beautifulsoup