Company logo
  • Jobs
  • Bootcamp
  • About Us
  • For professionals
    • Home
    • Jobs
    • Courses
    • Questions
    • Teachers
    • Bootcamp
  • For business
    • Home
    • Our process
    • Plans
    • Assessments
    • Payroll
    • Blog
    • Sales
    • Calculator

0

75
Views
Python Selenium NoSuchElement when trying to webscrape

I know this question has been asked before, but I have been through many of the Stack Overflow Solutions and nothing seems to be working for me. I am attempting to scrape a website for real estate data. I am able to click through the log in page and a second page, but when I arrive on the third page, I am stuck on clicking a link.

HTML Block:

<div id="m_wm_w6_m_pnlContent" class="css_content wgt_content">
     <table width="100%">
          <tbody><tr>
                <td><a id="m_wm_w6_m_lv_ctrl0_m_lnkWatch" href="javascript:__doPostBack(&#39;m_wm$w6$m_lv$ctrl0$m_lnkWatch&#39;,&#39;&#39;)" style="white-space:nowrap; font-size:11px;">New Listing (186)</a> 
                 </td>
                 <td width="50%"><div style="background-color:#008770;width:100%;height:12px;border:solid 1px #C0C0C0;"></div></td>
                </tr>

I want to click on the link using selenium for the tag "New Listing" with href="javascript:__doPostBack...

My Code:

driver = webdriver.Chrome(ChromeDriverManager().install())

driver.get(url + login)

driver.find_element_by_id('clareity').send_keys(username)

driver.find_element_by_id('security').click()
driver.find_element_by_id('security').send_keys(password)

driver.find_element_by_id('loginbtn').click()

driver.implicitly_wait(5)
driver.find_element_by_id('appColumn386').click()



#WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//html/body/form/div[3]/div[7]/table/tbody/tr/td[3]/div/div[2]/div/div[2]/table/tbody/tr[1]/td[1]/a"))).click()
#WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//*[@id=‘m_wm_w6_m_lv_ctrl0_m_lnkWatch’]"))).click()

#WebDriverWait(driver, 10).until(EC.staleness_of(driver.find_element_by_xpath("//html/body/form/div[3]/div[7]/table/tbody/tr/td[3]/div/div[2]/div/div[2]/table/tbody/tr[1]/td[1]/a")))
#driver.find_element_by_xpath("//html/body/form/div[3]/div[7]/table/tbody/tr/td[3]/div/div[2]/div/div[2]/table/tbody/tr[1]/td[1]/a").click()


The commented code is some of the solutions I tried and didn't seem to get to work. I am getting "NoSuchElement" error even though when I check the Chrome dev tools, the element shows up. I have even tried both XPaths from dev tools, but it is still not able to find. I have also tried time.sleep() to let the page load. I might be missing something very easy here but have been stuck for a while, any help is appreciated :)

7 months ago · Juan Pablo Isaza
1 answers
Answer question

0

Since the text is in between a tag, I would recommend you to use LINK_TEXT as a first option.

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, "New Listing (186)"))).click()

or

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "New Listing"))).click()

PS : Make sure that you are not in any new tab or windows. since you've mentioned that you are in 3rd page.

7 months ago · Juan Pablo Isaza Report
Answer question
Find remote jobs