Not able to find the data from xpath

问题

I tried to extract the data every minute and write the data into csv file but I coun't do it. Since I am new to this broad data science world.

I tried findall with soup library but not showing the data.

import requests
from bs4 import BeautifulSoup
page = requests.get('https://finviz.com/forex_performance.ashx')
soup = BeautifulSoup(page.content, 'html.parser')
forex = soup.find_all("div", {"class": "content "})
print(forex)

I would like to get the data like following format name of the currency and values example GBP 0.27

回答1:

For the first part you can use the charts tab and regex out the required javascript object which houses the chart data. You can parse that with json library.

import re
import json
import requests

r = requests.get('https://finviz.com/forex_charts.ashx')
p = re.compile(r'var tiles = (.*);')
data = json.loads(p2.findall(r.text)[0])
one_day_relative_performance_usd = [(data[item]['label'], data[item]['change']) for item in data]

For the second you can use pandas or requests

import pandas as pd

table = pd.read_html('finviz.com/forex_performance.ashx')[2] 

or

from bs4 import BeautifulSoup as bs
import requests

r = requests.get('https://finviz.com/forex_performance.ashx')
soup = bs(r.content, 'lxml')
table = soup.select_one('#forex_performance table') 

---

There is an API call in the web traffic

https://finviz.com/api/forex_perf.ashx

but that is currently not working at least in my browser.

来源：https://stackoverflow.com/questions/57125985/not-able-to-find-the-data-from-xpath