ruby watir - get all divs inner contents from search -
i attempting scrape website (respectfully). tried nokogiri, mechanize, because website scraping loading form dynamically, forced use webdriver. using ruby's watir.
what trying do, fill out dynamic form select
, clicking submit, going results part of page (form renders result on same page), , collecting divs information (traversing through sub-divs looking hrefs).
def scrape browser = watir::browser.new browser.goto 'http://www.website-link.com' browser.select_list(:id => 'city').select('cityname') browser.link(:id, 'btnsearch').click # part; results search in div w/ id # however, iterating through list not work way expected browser.div(:id, 'resultsdiv').divs.each |div| p div end browser.close end
right returns
#<watir::div: located: true; {:id=>"resultsdiv", :tag_name=>"div"} --> {:tag_name=>"div", :index=>0}> #<watir::div: located: true; {:id=>"resultsdiv", :tag_name=>"div"} --> {:tag_name=>"div", :index=>1}> #<watir::div: located: true; {:id=>"resultsdiv", :tag_name=>"div"} --> {:tag_name=>"div", :index=>2}>
which looking @ page source looks there 3 divs inside of resultsdiv
indexes are. guess expecting (coming nokogiri/mechanize) object manipulate.
does have experience doing point me right direction?
if known order want, can do:
browser.driver.find_elements(:id => 'resultsdiv')[n].click
or browser.div(:id => 'resultsdiv')[n].click
or browser.div(:id, 'resultsdiv').div(:id,'id_n').click
Comments
Post a Comment