import requests
from bs4 import BeautifulSoup
response = requests.get("...")
soup = BeautifulSoup(response.text, "html.parser")
print(soup.prettify())
In my current output for a div that I looking for was only able to output this <div id="root"></div>, on the website it just load all div inside of div id="root". I wanted to load the div that loaded by a JS Script. The output that I wanted to get was
<div id="root">
<div id="dark-id" class="my-class">dark</div>
<div>
<div id="light-id" class="other-class">light</div>
</div>
</div>
Was it possible to load a HTML Tag that loaded by a JS Script? If its possible, does anyone can help me?
If I understand correctly, then there is no way for you to do this using beautiful soup. It's not really a question of how to load divs that were loaded by a script, but rather how to use a web browser to load a page with JavaScript and then grab the elements from that page.
Browser webdrivers hit the page and then execute the javascript, populating the page with the elements. Beautiful soup doesn't do that. You could use a webdriver and then parse the webdriver's html output, but it sounds like you might be looking for something simpler than that.