requests-html操作表单,Submit (POST) form with the python requests-html library fails

I'm using python 3.7 and the requests-html library.

I have tried to send a get request in a session to a site with a form. First I use the response to get the CAPTCHA image and download it, and than send a POST request in the same session including the decoded CAPTCHA code.

The first part of sending the get request and getting a "ProcessKey" and the CAPTCHA image works great.

For some reason the second part where I'm send the POST request keeps redirect me to the previous page and it's not working properly.

I tried to change the user agent and the request headers to be similar to what I got with chrome dev panel as you can see in my code.

Before i made it to work with Selenium library but it is not good for uses.

from requests_html import HTMLSession

import time

url = 'https://www.misim.gov.il/svinfonadlan2010/'

url2 = 'https://www.misim.gov.il/svinfonadlan2010/startpageNadlanNewDesign.aspx?ProcessKey='

url3 = 'https://www.misim.gov.il/svinfonadlan2010/InfoNadlanPerutWithMap.aspx?ProcessKey='

session = HTMLSession()

request = session.get(url)

process_key = request.url.split('ProcessKey=')[1]

# Get the captcha image code:

image_url = request.html.find('#ContentUsersPage_RadCaptcha1_CaptchaImageUP', first=True)

image_url = url + image_url.attrs['src']

image_file_name = process_key + '.png'

with open('captcha_temp_files/' + image_file_name, 'wb') as f:

f.write(session.get(image_url).content)

print(request.url)

ans = input('Enter the captcha: ')

all_inputs = request.html.find('input')

data = {}

for i in all_inputs:

if 'value' in i.attrs.keys():

data[i.attrs['name']] = i.attrs['value']

else:

data[i.attrs['name']] = None

data["ctl00$ContentUsersPage$rbYeshuvOrGush"] = "rbMegush"

data['ctl00$ContentUsersPage$txtmegusha'] = 30010

data['ctl00$ContentUsersPage$txthelka'] = 129

data['ctl00$ContentUsersPage$txtadGush'] = 30010

data['ctl00$ContentUsersPage$txtadHelka'] = 129

data['ctl00$ContentUsersPage$DDLTypeNehes'] = 1

data['ctl00$ContentUsersPage$DDLMahutIska'] = 999

data['ctl00$ContentUsersPage$RadCaptcha1$CaptchaTextBox'] = ans

post_request_header = {

"Accept": 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3',

"Accept-Encoding": "gzip, deflate, br",

"Accept-Language": "en-US,en;q=0.9,he-IL;q=0.8,he;q=0.7",

"Cache-Control": "max-age=0",

"Connection": "keep-alive",

#"Content-Length": "10663",

"Content-Type": "application/x-www-form-urlencoded",

"DNT": "1",

"Host": "www.misim.gov.il",

"Origin": "https://www.misim.gov.il",

"Referer": url2 + process_key,

"Sec-Fetch-Mode": "navigate",

"Sec-Fetch-Site": "same-origin",

"Sec-Fetch-User": "?1",

"Upgrade-Insecure-Requests": "1",

"User-Agent": "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Mobile Safari/537.36",

}

session.headers = post_request_header

request2 = session.post(url=url2 + process_key, data=data)

print(request2.url)

time.sleep(2)

request3 = session.get(url=url3 + process_key)

print(request3.url)

Please help me to understand what is wrong here or is there another library that can do this except Selenium

Thank you in advance!