python - Scraping Multiple Pages On A Website -

-

i'm trying scrape list of coaching instiututes on thsi url: https://www.sulekha.com/entrance-exam-coaching/delhi

the following python code:

import bs4 urllib.request import urlopen ureq bs4 import beautifulsoup soup  my_url = 'https://www.sulekha.com/entrance-exam-coaching/delhi'  uclient = ureq(my_url) page_html = uclient.read() uclient.close() x  page_soup = soup(page_html, "lxml")   insti = page_soup.findall("div", {"class": "list-title"})  filename = "entrance_institutes.csv"  f = open(filename, "w") headers = "institute \n" f.write(headers)  ins in insti:     ins_name = ins.div.a["title"]  f.write(ins_name + "\n")  f.close()

this code runs fine. attached image of csv generates. how should go scraping listings 1 page after other ?

thanks

output csv

i'm not 100% sure mean. if you're asking how fix bug in code need change loop to: 

for ins in insti:     ins_name = ins.div.a["title"]     f.write(ins_name + "\n")

as code loop through , write last 1 due write not being in loop.

however if you're asking how take list , scrap that's more involved , starters need save url rather title i'm going leave rest because kind of sounds homework.


Comments

Post a Comment

Popular posts from this blog

ZeroMQ on Windows, with Qt Creator -

-
i have installed zmq link given below on windows 7. http://zeromq.org/distro:microsoft-windows 1) have c++ project in qt-creator , i'm using zmq c++ binding (zhelpers.hpp etc). 2) have added .lib installed zmq folder qtcreator 3) while compiling, compiler looks _imp__zmq_verion, in .dll files present in .lib have __imp_zmq_version. 4) causing undefined reference zmq_version. anyone else faced problem , has solution ?
Read more

python - Error while using APScheduler: 'NoneType' object has no attribute 'now' -

-
i have python app hosting on heroku. use apscheduler call function every day @ time. strange thing 1 time function runs after while have following error log: exception in thread apscheduler (most raised during interpreter shutdown): 2017-09-06t15:03:28.347848+00:00 app[web.1]: traceback (most recent call last): 2017-09-06t15:03:28.347858+00:00 app[web.1]: file "/app/.heroku/python/lib/python2.7/threading.py", line 810, in __bootstrap_inner 2017-09-06t15:03:28.347859+00:00 app[web.1]: file "/app/.heroku/python/lib/python2.7/threading.py", line 763, in run 2017-09-06t15:03:28.347861+00:00 app[web.1]: file "/app/.heroku/python/lib/python2.7/site-packages/apscheduler/schedulers/blocking.py", line 27, in _main_loop 2017-09-06t15:03:28.348167+00:00 app[web.1]: <type 'exceptions.attributeerror'>: 'nonetype' object has no attribute 'now' any guesses? the full code long here key parts: def startdayauto(): global ...
Read more

ios - MKAnnotationView layer is not of expected type: MKLayer -

-
Image
so code works fine logger riddled message. there way rid of or suppress it? postannotation.swift class postannotation: mkpointannotation { //mark: properties let post: post //mark: initialization init(post: post) { self.post = post super.init() self.coordinate = cllocationcoordinate2d(latitude: post.latitude, longitude: post.longitude) self.title = post.title self.subtitle = post.timestring() } } adding annotation let annotation = postannotation(post: post) self.map.addannotation(annotation) func mapview func mapview(_ mapview: mkmapview, viewfor annotation: mkannotation) -> mkannotationview? { if annotation mkuserlocation { return nil } var annotationview = mapview.dequeuereusableannotationview(withidentifier: "pin") as? mkpinannotationview if annotationview == nil { annotationview = mkpinannotationview(annotation: annotation, reuseidentifier: "pin") ...
Read more