I want to crawl this musictheory.net websites. I first tried using requests and BeautifulSoup it did not work. I was able to finally use hellium and BeautifulSoup to crawl the text. When I tried to crawl the images, I did a research and I noticed the text are embedded in a button using role="button" , clicking any line of text usually activate different images which is usually displayed at the top of the webpage. When I inspect the webpage by right clicking on the webpage and clicking "inspect (Q)", I saw that there is no <img src="".............> what I noticed was there is <canvas style="position"........etc....></canvas> and it is inside this canvas that images are loaded in using JavaScript I guess. I was confused looking at the code in all the <script .......xyz.js></script>. I would be glad if anyone can help me solve this problem. Below is code that I used to print the canvas attribute.
from helium import *
from bs4 import BeautifulSoup
url = 'https://www.musictheory.net/lessons/10'
browser = start_chrome(url, headless=True)
soup = BeautifulSoup(browser.page_source, 'html.parser')
canvas_links = []
y = soup.select('script')
for canvas in y:
canvas_links.append(canvas)
for can in canvas_links:
print(can)