python - Unable to glean data from other pages -


i've written script in python using post requests data webpage. webpage traverses 57 pages next or dropdown button. i've written far can fetch data first page. tried lot find way capture data going through it's next pages failed. how can data of 57 pages? in advance.

here i've tried far:

import requests lxml import html  requests.session() session:     session.headers = {"user-agent":"mozilla/5.0"}     page = session.post("http://registers.centralbank.ie/(x(1)s(cvjcqdbijraticyy2ssdyqav))/fundsearchresultspage.aspx?searchentity=fundserviceprovider&searchtype=name&searchtext=&registers=6%2c29%2c44%2c45&aspxautodetectcookiesupport=1",              data={'ctl00$cphregistersmasterpage$gvwsearchresults$ctl18$ddlpages':'2'},              headers={'content-type': 'application/x-www-form-urlencoded'})       tree = html.fromstring(page.text)     titles = tree.cssselect("table")[1]     list_row =[[tab_d.text_content() tab_d in item.cssselect('td.gvwcolumn,td.entitynamecolumn,td.entitytradingnamecolumn')]                 item in titles.cssselect('tr')]  data in list_row:     print(' '.join(data)) 

if press forward button, can see in dev tools: https://www.dropbox.com/s/0u3zb6qdvczzavq/form%20data.txt?dl=0

this the link page

btw, didn't find paginated links through can go on next page except "data" in requests parameter there page number option changes when button clicked. however, changing number doesn't bring data other pages.


Comments

Popular posts from this blog

ZeroMQ on Windows, with Qt Creator -

unity3d - Unity SceneManager.LoadScene quits application -

python - Error while using APScheduler: 'NoneType' object has no attribute 'now' -