I'm trying to use selenium to scrape a webpage, and one of my elements dynamically loads a finite amount of content (IE not like twitter where you can just keep scrolling) when I scroll to the bottom of it.
Now I could simply do something like
var scrollbar = document.getElementsByClassName("dataTables_scrollBody")[0]
var last = -1;
var curr = scrollbar.scrollHeight;
while (curr != last) {
last = curr;
scrollbar.scrollTop = curr;
curr = scrollbar.scrollHeight;
}
And in fact, that's what I'm doing. However, the actual load operation on the data table takes so long that this script exits by the time the load finishes, meaning I only ever get halfway down the scrollbar.
What's a good way to make sure that I've scrolled all the way to the bottom of the scroll bar?
This question is different from Scroll to bottom of div? because I don't have access to the raw HTML, and it's different from Scroll Automatically to the Bottom of the Page because my page is dynamically loading content.
Perhaps I could force an ajax load of all the content in the panel, but I don't know how to do that, and I don't want to wade through miles of minified code to figure out how the page owner does it.
Here's an ugly solution:
scroll_elem = document.getElementsByClassName("dataTables_scrollBody")[0];
table_elem = document.getElementsByClassName("dataTable")[0];
function scrollToBottom() {
scroll_elem .scrollTop = scroll_elem .scrollHeight;
};
new ResizeObserver(scrollToBottom).observe(table_elem)
Basically how this works is we have an object watching the element we want to scroll to the bottom of. If and when it resizes, it instantly scrolls to the bottom of the new window. This works well enough, then you could use the sleep function KunduK recommended in comments to wait for the process to complete