ruby watir - get all divs inner contents from search -
i attempting scrape website (respectfully). tried nokogiri, mechanize, because website scraping loading form dynamically, forced use webdriver. using ruby's watir.
what trying do, fill out dynamic form select, clicking submit, going results part of page (form renders result on same page), , collecting divs information (traversing through sub-divs looking hrefs).
def scrape browser = watir::browser.new browser.goto 'http://www.website-link.com' browser.select_list(:id => 'city').select('cityname') browser.link(:id, 'btnsearch').click # part; results search in div w/ id # however, iterating through list not work way expected browser.div(:id, 'resultsdiv').divs.each |div| p div end browser.close end right returns
#<watir::div: located: true; {:id=>"resultsdiv", :tag_name=>"div"} --> {:tag_name=>"div", :index=>0}> #<watir::div: located: true; {:id=>"resultsdiv", :tag_name=>"div"} --> {:tag_name=>"div", :index=>1}> #<watir::div: located: true; {:id=>"resultsdiv", :tag_name=>"div"} --> {:tag_name=>"div", :index=>2}> which looking @ page source looks there 3 divs inside of resultsdiv indexes are. guess expecting (coming nokogiri/mechanize) object manipulate.
does have experience doing point me right direction?
if known order want, can do:
browser.driver.find_elements(:id => 'resultsdiv')[n].click
or browser.div(:id => 'resultsdiv')[n].click
or browser.div(:id, 'resultsdiv').div(:id,'id_n').click
Comments
Post a Comment