How to rotate proxies on a Python requests

后端 未结 2 990
予麋鹿
予麋鹿 2021-02-06 16:20

I\'m trying to do some scraping, but I get blocked every 4 requests. I have tried to change proxies but the error is the same. What should I do to change it properly?

He

2条回答
  •  無奈伤痛
    2021-02-06 16:43

    import requests
    from itertools import cycle
    
    list_proxy = ['socks5://Username:Password@IP1:20000',
                  'socks5://Username:Password@IP2:20000',
                  'socks5://Username:Password@IP3:20000',
                   'socks5://Username:Password@IP4:20000',
                  ]
    
    proxy_cycle = cycle(list_proxy)
    # Prime the pump
    proxy = next(proxy_cycle)
    
    for i in range(1, 10):
        proxy = next(proxy_cycle)
        print(proxy)
        proxies = {
          "http": proxy,
          "https":proxy
        }
        r = requests.get(url='https://ident.me/', proxies=proxies)
        print(r.text)
    

提交回复
热议问题