Nokogiri Xpath to retrieve text after <BR> within <TD> and <SPAN>

Question

I have the following html and like to know how to use xpath to retrieve all the info: - Name(first, last) - Nick Name - email - shipping address...

Primarily, retrieve text after <BR>. Many Thanks in advance.

<table>
<tr>
<td valign="top" width="50%" align="left">
<span>Buyer</span><br/>FirstName LastName<br/>NickName<br/>First.Last@SomeCompany.com</td>

<tr><td valign="top" width="40%" align="left">
<span><span>Shipping address - </span><span>confirmed</span></span><br/>FirstName LastName<br/>Attn: FirstName<br/>1234 Main St.<br/>TheCity, TheState, 12345<br/>United States<br/></td>
</tr></table>

After I posted the above question, I learned that I can do these, but does not look clean:

buyer = html.xpath("//span/text()[contains(., 'Buyer')]").first.parent 
buyer_name = buyer.next.next 
puts "Buyer's Full name: #{buyer_name.text}" 
buyer_nick = buyer_name.next.next 
puts "Buyer's Nick name: #{buyer_nick.text}" 
buyer_email = buyer_nick.next.next 
puts "Buyer's email: #{buyer_email.text}" 

My question now is why the html.xpath("//span/text()[contains(., 'Buyer')]") return the TEXT itself instead of the ELEMENT. Again, thanks!!

Answer 1

Here's a concise way:

name, nick, email, *addr = doc.search('//td/text()[preceding-sibling::br]')

puts name, nick, email, "--", addr

The XPath does exactly what you stated: it takes all text nodes following a br. The address is slurped into one variable, but you can get the components separately if you want.

Output:

FirstName LastName
NickName
First.Last@SomeCompany.com
--
FirstName LastName
Attn: FirstName
1234 Main St.
TheCity, TheState, 12345
United States

Answer 2

<br> are a bit of a unique problem when dealing with HTML. They don't really get used for anything but formatting the content in the page, i.e., breaking lines like a new-line would in a *nix text file. So, my tactic when dealing with them while extracting text, is to transform them into new-lines.

Parse the content into a Nokogiri::HTML document:

doc = Nokogiri::HTML(html_doc_to_parse)

Convert the <br> to new-lines:

doc.search('br').each { |br| br.replace("\n") }

Then, find the cells you want:

doc.search('//td').map{ |td| td.content } 

which will return something like:

doc.search('//td').map(&:content)
=> ["\n  Buyer\nFirstName LastName\nNickName\nFirst.Last@SomeCompany.com",
 "\n  Shipping address - confirmed\nFirstName LastName\nAttn: FirstName\n1234 Main St.\nTheCity, TheState, 12345\nUnited States\n"]

which looks like this when printed:

puts doc.search('//td').map(&:content)

  Buyer
FirstName LastName
NickName
First.Last@SomeCompany.com

  Shipping address - confirmed
FirstName LastName
Attn: FirstName
1234 Main St.
TheCity, TheState, 12345
United States

From there it's a case of determining the correct array elements that you want, and then splitting on the new-lines i.e., String.split("\n").