ruby watir - get all divs inner contents from search -
i attempting scrape website (respectfully). tried nokogiri, mechanize, because website scraping loading form dynamically, forced use webdriver. using ruby's watir.
what trying do, fill out dynamic form select, clicking submit, going results part of page (form renders result on same page), , collecting divs information (traversing through sub-divs looking hrefs).
 def scrape     browser = watir::browser.new      browser.goto 'http://www.website-link.com'      browser.select_list(:id => 'city').select('cityname')      browser.link(:id, 'btnsearch').click      # part; results search in div w/ id     # however, iterating through list not work way expected     browser.div(:id, 'resultsdiv').divs.each |div|       p div     end      browser.close   end right returns
#<watir::div: located: true; {:id=>"resultsdiv", :tag_name=>"div"} --> {:tag_name=>"div", :index=>0}> #<watir::div: located: true; {:id=>"resultsdiv", :tag_name=>"div"} --> {:tag_name=>"div", :index=>1}> #<watir::div: located: true; {:id=>"resultsdiv", :tag_name=>"div"} --> {:tag_name=>"div", :index=>2}> which looking @ page source looks there 3 divs inside of resultsdiv indexes are. guess expecting (coming nokogiri/mechanize) object manipulate. 
does have experience doing point me right direction?
if known order want, can do:
 browser.driver.find_elements(:id => 'resultsdiv')[n].click
 or browser.div(:id => 'resultsdiv')[n].click
 or browser.div(:id, 'resultsdiv').div(:id,'id_n').click
Comments
Post a Comment