This is a recent problem, it began I think three or four days ago. It is not isolated to my own system, as I was running the software on a remote server as well (Windows 10, Windows Server). It is not also not isolated to any specific URL, as I can't get past any URL that has this check now.
Title: "Just a moment..." "Checking your browser before accessing URL". "This process is automatic. Your browser will redirect to your requested content shortly." "Please allow up to 5 seconds..." "DDos Protection by Cloudflare" "Ray Id: xxxxxxxxxxxxxxxxxx"
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('wwww.etherdelta.com')
Does anyone know how I can resolve this; or is it time to put poor ol' timmy (the program) down?
It is because the browser uses cloudfare to protect itself from DDOS (Distributed Denial Of Service) Attacks. There are 2 ways to solve this problem:
Use time.sleep -- if it takes 5 seconds for the webpage to load, just use time.sleep(5).
Use WebDriverWait -- for example, a button with id "sample-btn" appears only after this screen. Then what u can do is:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
btn = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, 'sample-btn'))) #Web driver waits for 10 seconds until element is visible
The 2nd one is recommended. But if the 2nd one doesn't work for u, then go with the first one. Hope that this helps!
I had the same issue with firefox. I was able to solve it by switching to Chrome.
Example code:
from selenium import webdriver
url = "<WEBSITE>"
options = webdriver.ChromeOptions()
options.add_argument("--disable-blink-features=AutomationControlled")
driver = webdriver.Chrome(options=options)
driver.get(url)
"--disable-blink-features=AutomationControlled" hides the "navigator.webdriver" flag.
See Selenium webdriver: Modifying navigator.webdriver flag to prevent selenium detection
Edit
You also have to change some default variables of chromedriver.
Example with perl:
perl -pi -e 's/cdc_/dog_/g' /path/to/chromedriver
For more details look at the original post.
See Can a website detect when you are using selenium with chromedriver?
Edit 2
Cloudflare keeps adjusting their algorithm so you could try to use undetected-chromedriver instead of the manual changing of the chromedriver.
undetected-chromedriver is an optimized Selenium Chromedriver patch which should not trigger anti-bot services. It automatically downloads the driver binary and patches it.
Wether this will work or not kinda depends on the website and the current state of the development. Cloudflare seems to track the development of undetected-chromedriver.
import undetected_chromedriver as uc
url = "<WEBSITE>"
driver= uc.Chrome()
driver.get(url)
Try using yout chrome data folder
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from webdriver_manager.utils import ChromeType
# Configure browser
options = webdriver.ChromeOptions()
options.add_argument(f"--user-data-dir=C:\\Users\\daria\\AppData\\Local\\Google\\Chrome\\User Data")
options.add_argument("--disable-blink-features=AutomationControlled")
chromedriver = ChromeDriverManager(chrome_type=ChromeType.GOOGLE,
log_level='0',
print_first_line=False).install()
driver = webdriver.Chrome(chromedriver,
options=options,
service_log_path=None)
input ("End?")