Selenium driver.get() modifying URL

问题

I have a webscraper setup to go through different dates and pull historical stats and save them to a DF. This has worked fine up until today when I tried updating my database. Here is the code:

myDate = startDate
while myDate < endDate:
    dateFormat = "{date.month:02}{date.day:02}{date.year}".format(date=myDate)
    url = "http://www.example.com/?date="+dateFormat

    driver.get(url)
    print(url)

    time.sleep(3)
    html = driver.page_source

So I added the print command just to see what URL it was sending to chrome and it reflects the proper URL. However once the page loads it adds an & at the end of the URL followed by the date and the page will not load as expected. I tested the functionality of copy and pasting the formatted URL and it also adds the & however if you manually type in the address it does not?

So basically my script sends http://www.example.com/?date=11272017

But the URL returned is http://www.example.com/?date=11272017&date=11272017 which redirects me to the current date on the page. Hopefully this makes sense.

I was thinking about switching the driver.get to a sendkeys command but I dont know how to get this to work with a URL. I have also tried manipulating the URL after it has been put into the browser and stripping the & but it just adds it again... Any help on this would be greatly appreciated!

回答1:

Ah! I have figured it out. The webpage used to be regular HTTP, recently they have changed all their pages to HTTPS. Leaving this question up here for anyone that may run into the same problem. Something so small had me stumped for awhile!

来源：https://stackoverflow.com/questions/48099403/selenium-driver-get-modifying-url

标签

python

selenium