I understand that versions of this questions have been asked before--this may be a repeat--but I've searched through all of those and can't find a working solution.
I'm trying to web scrape some data from air.norfolk.gov. The disclaimer popup at the beginning is really giving me troubles (My whole goal right now is just get rid of the popup at the beginning so I can actually scrape the data!).
Here's my current code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from webdriver_manager.chrome import ChromeDriverManager
import time
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
driver.get('https://air.norfolk.gov')
time.sleep(5)
element_name = driver.find_element(By.TAG_NAME, 'button')
print("Element is visible? " + str(element_name.is_displayed()))
driver.find_element(By.TAG_NAME, 'button').click()
print('button clicked')
time.sleep(5)
# Get search box element from webElement 'q' using Find Element
search_box = driver.find_element(By.ID, "primary_search")
search_box.send_keys("this would be where the address goes")
I've tried a number of things that you'll be able to notice: I've tried checking if the element is visible--it is. I've tried adding waits to make sure it was ready to receive my click. I've tried to use send_keys instead of click() because it looks like this button is wrapped in a div.
Every time I run the script, I look at the website and the disclaimer has not gone away.
The error I'm getting is:
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <button type="button" class="btn primary-button">...</button> is not clickable at point (912, 317). Other element would receive the click: <div class="modal-body">...</div>
I'm just not sure why the element can't receive the click.
Please help!
Thanks
There are 5 elements highlighting the button tag. Hence finding the element with tag-name would not work as expected.
Try with below xpath to click on I Understand.
//button[text()='I Understand']
Try with below code:
#Imports required for Explicit waits:
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver.get("https://air.norfolk.gov")
wait = WebDriverWait(driver,30)
wait.until(EC.element_to_be_clickable((By.XPATH,"//button[text()='I Understand']"))).click()
You find the wrong button.
You can find it with the CSS tag: #btn-primary.
I suggest you go to that website and open the DevTools and do the test with normal javascript, you will figure it out quickly
If the disclaimer popup is an alert then it can be handled as following:
driver.switchTo().alert().dismiss();
Or:
driver.switchTo().alert().accept();
Also try to use driver.getWindowHandles() to switch between the windows and then click on the element.