0
199
ViewsHow To Use Pupeteer To Get The Value Of A Hidden Input Tag
import puppeteer from 'puppeteer'
async function scrapeProduct(url) {
const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.goto(url)
const element = await page.waitForSelector('input[type="hidden"][name="abuseID"]', {
hidden: true,
timeout: 15000
})
const abuseID = await page.evaluate(element => element.value, element)
console.log(abuseID)
}
scrapeProduct('https://steamcommunity.com/id/lupusRe')
I don't know where I am going wrong but I am getting an error of
file:///C:/Users/charl/Desktop/Code/abg/steam-search/node_modules/puppeteer/lib/esm/puppeteer/common/ExecutionContext.js:282
throw new Error('Evaluation failed: ' + getExceptionMessage(exceptionDetails));
^
Error: Evaluation failed: TypeError: Cannot read properties of null (reading 'value')
at pptr://__puppeteer_evaluation_script__:1:21
at ExecutionContext._ExecutionContext_evaluate (file:///C:/Users/charl/Desktop/Code/abg/steam-search/node_modules/puppeteer/lib/esm/puppeteer/common/ExecutionContext.js:282:15)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async ExecutionContext.evaluate (file:///C:/Users/charl/Desktop/Code/abg/steam-search/node_modules/puppeteer/lib/esm/puppeteer/common/ExecutionContext.js:114:16)
at async scrapeProduct (file:///C:/Users/charl/Desktop/Code/abg/steam-search/test.js:13:21)
My goal is to be able to get the steam ID in the below tag <input type="hidden" name="abuseID" value="76561198036553525>"
I am open to using other things beside puppeteer but its just the main thing that I have been using for web scraping.
This is on the official steam community website if that matters.
·
Juan Pablo Isaza
1 answers
Answer question0
Copy Xpath of an element
and use page.$x(Xpath)
Edit: i've tried couple of things and it seems like you can get persons steamID through a <script> tag
import puppeteer from 'puppeteer'
async function scrapeProduct(url) {
const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.goto(url);
const text = await page.evaluate(() => Array.from(document.querySelectorAll('script'), element => element.textContent));
let spl = text[38].split(',');
console.log(spl[1])
browser.close()
}
scrapeProduct('https://steamcommunity.com/id/DaYanack')
try this and tell me if it works
·
Juan Pablo Isaza
Report
Answer question