In my project I'm trying to access some of specific route in some external page. The route controlled by java script (i.e you get two drop-down menu, that controls the categories )
so far my code looks like this - and it product list of links of the first page:
def getAllIndexOfStringInPage(string):
read =requests.get("https://example.com/").text
res = [i for i in range(len(read)) if read.startswith(string, i)]
return res, read
def getAllLinks():
html1 = getAllIndexOfStringInPage("a href")
html2 = getAllIndexOfStringInPage("target=\"_blank\">my condition in the tag")
extractedLinks = []
c = -1
for startIndex in html1[0]:
c += 1
f = html1[1][startIndex:html2[0][c]]
extractedLinks.append(f.replace("amp;", ""))
c = 0
for link in extractedLinks:
extractedLinks[c] = "https" + (link.split("&sp=r")[0] + "&sp=r").split("http")[1]
c += 1
return extractedLinks
the path in the page I want to change is the following - option value =0 redirects the page to get all of the results I wish to change the value of the option from 0 to 1 to 7xx
I did some research and I found the library BeautifulSoup and with the following code I could change the value of it to anything I wanted to
html_doc = requests.get("http://examplepage.com/").text
soup = BeautifulSoup(html_doc, 'lxml')
tag = soup.find_all("option")[0]
tag['value']="697"
So now all I couldn't achieve is to load the page with the change i made with the library BeautifulSoup.
I'm kinda lost already so any help would help.
note that I can use only the following languages - python/Java/Kotlin