Log in to a problematic site using requests

问题

I'm trying to create a script in python using requests module to log in to thissite

I'm using my credentials but I don't find any way to do so as I can't see the parameters (in chrome dev tools) required to send along with requests.

username: SIMMTH.iqbal_123
password: SShift_123

The login form looks like this.

This is my initial attempt (I really could not find anything in that page to start with):

import requests
from bs4 import BeautifulSoup

link = "https://jobs.allianz.com/sap/bc/bsp/sap/zhcmx_erc_ui_ex/desktop.html#/SEARCH/RESULTS/"

with requests.Session() as s:
    s.headers['User-Agent'] = 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36'
    r = s.get(link)
    print(r.text)

How can I log in to that site using requests?

回答1:

import requests
from bs4 import BeautifulSoup

with requests.Session() as req:
    r = req.get(
        "https://jobs.allianz.com/sap/hcmx/validate_ea?sap-client=100&sap-language={2}", auth=requests.auth.HTTPBasicAuth("SIMMTH.iqbal_123", "SShift_123"))
    r = req.get(
        "https://jobs.allianz.com/sap/opu/odata/hcmx/erc_ui_auth_srv/UserSet('me')?sap-client=100&sap-language=en")
    soup = BeautifulSoup(r.content, 'lxml')
    print(soup.prettify())

Output:

<?xml version="1.0" encoding="utf-8"?>
<html>
 <body>
  <entry xml:base="https://jobs.allianz.com/sap/opu/odata/hcmx/erc_ui_auth_srv/" xmlns="http://www.w3.org/2005/Atom" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
   <id>
    https://jobs.allianz.com/sap/opu/odata/hcmx/erc_ui_auth_srv/UserSet('SIMMTH.IQBAL_123') 
   </id>
   <title type="text">
    UserSet('SIMMTH.IQBAL_123')
   </title>
   <updated>
    2020-03-23T17:16:38Z
   </updated>
   <category scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" term="erc_ui_auth_srv.User">
   </category>
   <link href="UserSet('SIMMTH.IQBAL_123')" rel="edit" title="User"/>
   <link href="UserSet('SIMMTH.IQBAL_123')/DecimalFormat" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/DecimalFormat" title="DecimalFormat" type="application/atom+xml;type=entry"/>
   <link href="UserSet('SIMMTH.IQBAL_123')/DateFormat" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/DateFormat" title="DateFormat" type="application/atom+xml;type=entry"/>
   <link href="UserSet('SIMMTH.IQBAL_123')/Language" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Language" title="Language" type="application/atom+xml;type=entry"/>
   <link href="UserSet('SIMMTH.IQBAL_123')/$links/DecimalFormat" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/relatedlinks/DecimalFormat" title="DecimalFormat" type="application/xml"/>
   <link href="UserSet('SIMMTH.IQBAL_123')/$links/DateFormat" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/relatedlinks/DateFormat" title="DateFormat" type="application/xml"/>
   <link href="UserSet('SIMMTH.IQBAL_123')/$links/Language" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/relatedlinks/Language" title="Language" type="application/xml"/> 
   <content type="application/xml">
    <m:properties>
     <d:email>
      mthupwork@gmail.com
     </d:email>
     <d:firstname>
      shahin
     </d:firstname>
     <d:lastname>
      iqbal
     </d:lastname>
     <d:newpassword>
     </d:newpassword>
     <d:oldpassword>
     </d:oldpassword>
     <d:logonname>
      SIMMTH.IQBAL_123
     </d:logonname>
     <d:userid>
      SIMMTH.IQBAL_123
     </d:userid>
    </m:properties>
   </content>
  </entry>
 </body>
</html>

来源：https://stackoverflow.com/questions/60815608/log-in-to-a-problematic-site-using-requests

标签

python

python-3.x

web-scraping

beautifulsoup

python-requests