Code666 (代码贴、代码片段)

[Python] scrapy采集数据时使用代理服务器代码 →→→→→进入此内容的聊天室

来自 , 2020-10-23, 写在 Python, 查看 127 次.
URL http://www.code666.cn/view/01846ae4
  1. # To authenticate the proxy, you must set the Proxy-Authorization header.  You *cannot* use the form http://user:pass@proxy:port in request.meta['proxy']
  2.  
  3. import base64
  4.  
  5. proxy_ip_port = "123.456.789.10:8888"
  6. proxy_user_pass = "awesome:dude"
  7.  
  8. request = Request(url, callback=self.parse)
  9.  
  10. # Set the location of the proxy
  11. request.meta['proxy'] = "http://%s" % proxy_ip_port
  12.  
  13. # setup basic authentication for the proxy
  14. encoded_user_pass=base64.encodestring(proxy_user_pass)
  15. request.headers['Proxy-Authorization'] = 'Basic ' + encoded_user_pass
  16. #//python/8393

回复 "scrapy采集数据时使用代理服务器代码"

这儿你可以回复上面这条便签

captcha

贴代码片段 - Code666