python - Unable to glean data from other pages -

-

i've written script in python using post requests data webpage. webpage traverses 57 pages next or dropdown button. i've written far can fetch data first page. tried lot find way capture data going through it's next pages failed. how can data of 57 pages? in advance.

here i've tried far:

import requests lxml import html  requests.session() session:     session.headers = {"user-agent":"mozilla/5.0"}     page = session.post("http://registers.centralbank.ie/(x(1)s(cvjcqdbijraticyy2ssdyqav))/fundsearchresultspage.aspx?searchentity=fundserviceprovider&searchtype=name&searchtext=&registers=6%2c29%2c44%2c45&aspxautodetectcookiesupport=1",              data={'ctl00$cphregistersmasterpage$gvwsearchresults$ctl18$ddlpages':'2'},              headers={'content-type': 'application/x-www-form-urlencoded'})       tree = html.fromstring(page.text)     titles = tree.cssselect("table")[1]     list_row =[[tab_d.text_content() tab_d in item.cssselect('td.gvwcolumn,td.entitynamecolumn,td.entitytradingnamecolumn')]                 item in titles.cssselect('tr')]  data in list_row:     print(' '.join(data))

if press forward button, can see in dev tools: https://www.dropbox.com/s/0u3zb6qdvczzavq/form%20data.txt?dl=0

this the link page

btw, didn't find paginated links through can go on next page except "data" in requests parameter there page number option changes when button clicked. however, changing number doesn't bring data other pages.


Comments

Post a Comment

Popular posts from this blog

ios - MKAnnotationView layer is not of expected type: MKLayer -

-
Image
so code works fine logger riddled message. there way rid of or suppress it? postannotation.swift class postannotation: mkpointannotation { //mark: properties let post: post //mark: initialization init(post: post) { self.post = post super.init() self.coordinate = cllocationcoordinate2d(latitude: post.latitude, longitude: post.longitude) self.title = post.title self.subtitle = post.timestring() } } adding annotation let annotation = postannotation(post: post) self.map.addannotation(annotation) func mapview func mapview(_ mapview: mkmapview, viewfor annotation: mkannotation) -> mkannotationview? { if annotation mkuserlocation { return nil } var annotationview = mapview.dequeuereusableannotationview(withidentifier: "pin") as? mkpinannotationview if annotationview == nil { annotationview = mkpinannotationview(annotation: annotation, reuseidentifier: "pin")
Read more

ZeroMQ on Windows, with Qt Creator -

-
i have installed zmq link given below on windows 7. http://zeromq.org/distro:microsoft-windows 1) have c++ project in qt-creator , i'm using zmq c++ binding (zhelpers.hpp etc). 2) have added .lib installed zmq folder qtcreator 3) while compiling, compiler looks _imp__zmq_verion, in .dll files present in .lib have __imp_zmq_version. 4) causing undefined reference zmq_version. anyone else faced problem , has solution ?
Read more

unity3d - Unity SceneManager.LoadScene quits application -

-
i have 2 different scenes 1 named 'main' , other 'game' both added scenes in load. when use scenemanager.loadscene("game"); works no problem when tried return main scene code if (input.getkeydown (keycode.escape)) { scenemanager.loadscene("main"); } strangely game quits. thanks. sorry interruption. somewhere on other script found code. if (input.getkeydown (keycode.escape)) { application.quit (); } that's bad, know...
Read more