I'm struggling to parse some var in my HTML
Here is the example of HTML:
<script type="text/javascript">
var ASPath = "\/modules\/pm_advancedsearch4\/";
var ASSearchUrl = "https:\/\/golf-land.fr\/module\/pm_advancedsearch4\/advancedsearch4";
var as4_orderBySalesAsc = "Meilleures ventes en dernier";
var as4_orderBySalesDesc = "Meilleures ventes en premier";
var controller = "my-account";
</script>
I have tried doing this:
soup = BeautifulSoup(s.text, 'html5lib')
soup = BeautifulSoup(str(soup.find_all('script')[2]), "html.parser")
pattern = re.compile(r"var controller = '(.*?)';$", re.MULTILINE | re.DOTALL)
print(pattern)
script = soup.find("script", text=pattern)
print(pattern.search(script.text).group(1))
My aim was to get the "my-account" but all I got was
re.compile("var controller = '(.*?)';$", re.MULTILINE|re.DOTALL)
Traceback (most recent call last):
File "main.py", line 47, in <module>
print(pattern.search(script.text).group(1))
AttributeError: 'NoneType' object has no attribute 'text'
Line 47 refers to the last line of my code.
You are confusing as to what it is to use bs4 to get specific tags, and to parse out a substring from that content.
The pattern you are searching for is looking for an exact match, while what you want is to find that content that contains the substring (to get the specific tag).
So your regex pattern should be pattern = re.compile(r"var controller = (.*?);$", re.MULTILINE | re.DOTALL) to get that <script> tag as a BeautifulSoup object.
Also, your pattern is looking for single ' quotes, when there are none in "my-account".
But really what you are wanting is to pull out that substring after you got the specific tag.
Try this:
from bs4 import BeautifulSoup
import re
soup = BeautifulSoup(s.text, 'html5lib')
script = soup.find_all('script')[2]
pattern = re.compile(r"var controller = (.*?);$", re.MULTILINE | re.DOTALL)
#print(pattern)
scriptParse = re.search(pattern, str(script))
print(scriptParse.groups(1)[0])
Output:
"my-account"