Dont truly know how to convey my problem but each webpage within a database I'm scraping has variations within the Xpaths of the thing I want to collect. How can I effectively scrape webpages when their exact locations vary by a few places each page?
Using Python 3.6 and Selenium. I want to use CSS selectors because someone said they allow for 'scraping by adjacency' but frankly IDK how and no tutorials are really saying how to CSS select from adjacent variable I have THIS javascript which DOES NOTHING when I run using driver.execute_js(///) *but does work when I manually put it in the webconsole *
let email = '';
let contacts = document.querySelectorAll('div.contact-section');
for (let i = 0; i < contacts.length; i++) {
if (contacts[i].getAttribute('automation-id') === 'contact-true-owner') {
ele = contacts[i]
}
}
let links = ele.querySelectorAll('address > a')
let last = links.length - 1;
email = links[last].innerText; here