cookies

使用scrapy shell时设置cookies和headers

由 萝らか妹 提交于 2020-02-23 00:54:48
有时为了测试xpath，需要临时下载个页面，这时使用命令行进行测试是最方便的，但是很多网站页面需要认证，不能直接使用scrapy shell命令进行页面的抓取，所以需要重新对请求进行构造，设置cookies和headers。 首先在当前装有scrapy的python环境中安装ipython # python环境下 pip install ipython # conda环境下 conda install ipython 首先进入scrapy shell，会自动使用ipython scrapy shell 把cookies转成字典格式 # 指定请求目标的 URL 链接 url = 'https://novel18.syosetu.com/n7016er/31/' # 自定义 Headers 请求头(一般建议在调试时使用自定义 UA，以绕过最基础的 User-Agent 检测) headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36'} # 构造需要附带的 Cookies 字典 cookies = {"key_1": "value_1", "key_2": "value_2

恋爱日记 隐私政策

由 梦想与她 提交于 2020-02-21 22:40:05
本产品尊重并保护所有使用网络服务用户的个人隐私权。为了给您提供更准确、更有个性化的服务，会按照本隐私权政策的规定使用和披露您的个人信息。但将以高度的勤勉、审慎义务对待这些信息。除本隐私权政策另有规定外，在未征得您事先许可的情况下，本产品不会将这些信息对外披露或向第三方提供。本产品会不时更新本隐私权政策。您在同意本产品网络服务使用协议之时，即视为您已经同意本隐私权政策全部内容。本隐私权政策属于本产品网络服务使用协议不可分割的一部分。 1.适用范围 a)在您注册本产品帐号时， 您根据本产品要求提供的个人注册信息（商家应法律法规要求需公示的企业名称及相关工商注册信息除外）； b)在您使用本产品网络服务，或访问本产品平台网页时，本产品自动接收并记录的您的浏览器和iphone或者ipad上的信息，包括但不限于您的IP地址、浏览器的类型、使用的语言、访问日期和时间、软硬件特征信息及您需求的网页记录等数据； c) 本产品通过合法途径从商业伙伴处取得的用户个人数据。您了解并同意，以下信息不适用本隐私权政策： d)您在使用本产品平台提供的搜索服务时输入的关键字信息；隐私政策 2.信息使用 a) 本产品不会向任何无关第三方提供、出售、出租、分享或交易您的个人信息，除非事先得到您的许可，或该第三方和本产品（含本产品关联公司）单独或共同为您提供服务，且在该服务结束后

会话和Cookies的目的

由 久未见 提交于 2020-02-21 18:23:11
在了解会话和C ookies 之前， 我们还需要了解HTTP 的一个特点，叫作 无状态 。 HTTP 的无状态是指HTTP 协议对事务处理是没有记忆能力的，也就是说服务器不知道客户端是什么状态。当我们向服务器发送请求后，服务器解析此请求，然后返回对应的响应，服务器负责完成这个过程，而且这个过程是完全独立的，服务器不会记录前后状态的变化，也就是缺少状态记录。这意味着如果后续需要处理前面的信息，则必须重传，这导致需要额外传递一些前面的重复请求，才能获取后续响应，然而这种效果显然不是我们想要的。为了保持前后状态，我们肯定不能将前面的请求全部重传一次，这太浪费资源了，对于这种需要用户登录的页面来说，更是棘手。 这时两个用于保持HTTP 连接状态的技术就出现了，它们分别是会话和Cookies 。 会话在服务端，也就是网站的服务器，用来保存用户的会话信息； Cookies 在客户端，也可以理解为浏览器端 ，有了Cookies ，浏览器在下次访问网页时会自动附带上它发送给服务器，服务器通过识别Cookjes 并鉴定出是哪个用户，然后再判断用户是否是登录状态，然后返回对应的响应。 我们可以理解为Cookies 里面保存了登录的 凭证 ，有了它，只需要在下次请求携带Cookies 发送请求而不必重新输入用户名、密码等信息重新登录了。 因此在爬虫中，有时候处理需要登录才能访问的页面时

Safari not sending cookie even after setting SameSite=None; Secure

由 Deadly 提交于 2020-02-21 11:19:20
问题 Our application uses cookies to remember user login. Every auth API call we make, the browser attaches server-set HTTPonly cookie with the API request and gets authenticated. This behaviour seems to be broken in safari after Mojave release. I read about the cross-site cookie security implemented by safari and our server team added SameSite=None;Secure while setting the cookie. Even after that, it still doesn't work. Set-Cookie: my_cookie=XXXXX; path=/; secure; HttpOnly; SameSite=None Please

Safari not sending cookie even after setting SameSite=None; Secure

由 寵の児 提交于 2020-02-21 11:14:03
问题 Our application uses cookies to remember user login. Every auth API call we make, the browser attaches server-set HTTPonly cookie with the API request and gets authenticated. This behaviour seems to be broken in safari after Mojave release. I read about the cross-site cookie security implemented by safari and our server team added SameSite=None;Secure while setting the cookie. Even after that, it still doesn't work. Set-Cookie: my_cookie=XXXXX; path=/; secure; HttpOnly; SameSite=None Please

Can a third-party script set a first-party cookie?

由 生来就可爱ヽ(ⅴ＜●) 提交于 2020-02-20 07:52:04
问题 I've read a lot of content online about cookies, but nothing addressing this question: Let's say I have a server at a.com and a web page served by b.com embeds a script in that web page which lives on my server: <script src='a.com/script.js'></script> What is that script allowed to do in terms of setting cookies? Can it set a cookie with domain=a.com ? I'd assume so since the script is served from that domain. Can it also set a cookie with domain=b.com since the page is served from that

Postman: How do you delete cookies in the pre-request script?

由 时光总嘲笑我的痴心妄想 提交于 2020-02-20 06:32:01
问题 All the postman cookie-management answers I've seen refer to either the browser extension (open chrome, delete cookies viz interceptor etc) or with the app, using the UI to manually manage cookies. I would like to delete certain cookies in my pre-request code as part of scripting my API tests. (delete them programmatically) The Sandobx API docs mention pm.cookies so I tried if (pm.cookies !== null) { console.log("cookies!"); console.log(pm.cookies); } But the pm.cookies array is empty. Yet in

Postman: How do you delete cookies in the pre-request script?

由 只愿长相守 提交于 2020-02-20 06:27:46
问题 All the postman cookie-management answers I've seen refer to either the browser extension (open chrome, delete cookies viz interceptor etc) or with the app, using the UI to manually manage cookies. I would like to delete certain cookies in my pre-request code as part of scripting my API tests. (delete them programmatically) The Sandobx API docs mention pm.cookies so I tried if (pm.cookies !== null) { console.log("cookies!"); console.log(pm.cookies); } But the pm.cookies array is empty. Yet in

Expires string in cookie header

由 只愿长相守 提交于 2020-02-19 09:38:38
问题 Simple question I think, but I just can't seem to find an answer. I am writing a cookie in a Java Servlet with the Cookie class which is sent to the browser in the response headers like the following: Set-Cookie: test=somevalue; Domain=.mydomain.org; Expires=Thu, 06-Jan-2011 18:45:20 GMT; Path=/ I am doing this via the Cookie class in the Servlet 2.5 API. I need to add "HTTPOnly" to the end of this String, which the Servlet 2.5 API does not support. No problem, I'll just create the String

Expires string in cookie header

由 倾然丶 夕夏残阳落幕 提交于 2020-02-19 09:36:27
问题 Simple question I think, but I just can't seem to find an answer. I am writing a cookie in a Java Servlet with the Cookie class which is sent to the browser in the response headers like the following: Set-Cookie: test=somevalue; Domain=.mydomain.org; Expires=Thu, 06-Jan-2011 18:45:20 GMT; Path=/ I am doing this via the Cookie class in the Servlet 2.5 API. I need to add "HTTPOnly" to the end of this String, which the Servlet 2.5 API does not support. No problem, I'll just create the String
订阅 cookies