Weather data scraping and extraction in R [closed]

问题

I'm working on a research project and am assigned to do a bit of data scraping and writing code in R that can help extract current temperature for a particular zip code from a site such as wunderground.com. Now this may be a bit of an abstract question but does anyone know how to do the following: I can extract the current temperature of a particular zip code by doing this:

    temps <- readLines("http://www.wunderground.com/q/zmw:20904.1.99999")
    edit(temps)
    temps //gives me the source code for the website where I can look at the line that contains the temperature
    ldata <- temps[lnumber]
    ldata
    #  then have a few gsub functions that basically extracts 
    # just the numerical data (57.8 for example) from that line of code

I have a cvs file that contains zip code of every city in the country and I have that imported in R. It is arranged in a table according to zip, city and state. My challenge now is to write a method (using java analogy here because I'm new to R) that basically extracts 6-7 consecutive zip codes (after a particular one specified) and runs the above code by modifying the link within the readLines function and putting in the respective zip code after the link segment zmw:XXXXX and running everything after that based on that link. Now I don't quite know how to extract the data from the table. Maybe with a for-loop function? But then I don't know how to use that to modify the link. I think that's where I'm really getting stuck on. I have a bit of Java background so I understand HOW to approach this problem, just not the knowledge of the syntax. I understand this is quite an abstract question as I didn't provide a lot of code but I just want to know they functions/syntax that will help me extract the data from the table and somehow use that to modify the link through a function rather than manually doing it.

回答1:

So this is about the Weather Underground data.

You can download csv files from individual weather stations in wunderground, however you need to know the weather station identifier. Here is an example URL for a weather station in Kirkland, WA (KWAKIRKL8):

http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KWAKIRKL8&day=31&month=1&year=2014&graphspan=day&format=1

Here is some R code:

  url <- 'http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KWAKIRKL8&day=31&month=1&year=2014&graphspan=day&format=1'
  s <- getURL(url)
  s <- gsub("<br>\n","",s)  
  wdf <- read.csv(con<-textConnection(s))

And here is a page with which you can manually find stations and their codes.

http://www.wunderground.com/wundermap/

Since you only need a few you can pick them out manually.

来源：https://stackoverflow.com/questions/30650502/weather-data-scraping-and-extraction-in-r

标签

regex

for-loop

weather

data-extraction