Want to scrape table using puppeteer.js. How can I get all rows, iterate through rows and then get “td's” for each row

风流意气都作罢 提交于 2019-12-07 01:45:09

问题


I have puppeteer js setup and was able get all rows using

let rows = await page.$$eval('#myTable tr', row => row);

Now I want for each row to get "td's" and then get inner text from those.

Basically I want to do this:

var tds = myRow.querySelectorAll("td");

where myRow is a table row, with puppeteer.js


回答1:


One way to achieve this is to use evaluate that first gets an array of all the TD's then returns the textContent of each TD

const puppeteer = require('puppeteer');

const html = `
<html>
    <body>
      <table>
      <tr><td>One</td><td>Two</td></tr>
      <tr><td>Three</td><td>Four</td></tr>
      </table>
    </body>
</html>`;

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto(`data:text/html,${html}`);

  const data = await page.evaluate(() => {
    const tds = Array.from(document.querySelectorAll('table tr td'))
    return tds.map(td => td.innerHTML)
  });

  //You will now have an array of strings
  //[ 'One', 'Two', 'Three', 'Four' ]
  console.log(data);
  //One
  console.log(data[0]);
  await browser.close();
})();

You could also use something like:-

const data = await page.$$eval('table tr td', tds => tds.map((td) => {
  return td.innerHTML;
}));

//[ 'One', 'Two', 'Three', 'Four' ]
console.log(data);


来源:https://stackoverflow.com/questions/49236981/want-to-scrape-table-using-puppeteer-js-how-can-i-get-all-rows-iterate-through

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!