Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

170
Views
JavaScript in requests package python

I want to get text from a site using Python. But the site uses JavaScript and the requests package to receive only JavaScript code. Is there a way to get text without using Selenium?

import requests as r

a=r.get('https://aparat.com/').text
about 4 years ago · Juan Pablo Isaza
1 answers
Answer question

0

If the site loads content using javascript then the javascript has to be run in order to get the content. I ran into this issue a while back when I did some web scraping, and ended up using Selenium. Yes its slower than BeautifulSoup but it's the easiest solution.

If you know how the server works you could send a request and it should return with content of some kind (whether that be html, json, etc)

Edit: Load the developer tools, go to network tab and refresh the page. Look for an XHR request and the URL it uses. You may be able to use this data for your needs.

For example I found these URLs:

https://www.aparat.com/api/fa/v1/etc/page/config/mode/full https://www.aparat.com/api/fa/v1/video/video/list/tagid/1?next=1

If you navigate to these in your browser you will notice JSON content, you might be able to use this. I think some of the text is encoded in Unicode e.g \u062e\u0644\u0627\u0635\u0647 \u0628\u0627\u0632\u06cc -> خلاصه بازی

I don't know the specific python implementation you might use. Look for libs that support making http requests and recieving data. That way you can avoid selenium. But you must know the URL's beforehand. Like shown above.

For example this is what I would do:

  1. Make a http request to the URL you find in developer tools
  2. With JSON content, use a JSON parser to get a table/array/dictionary natively. You can then traverse this in the native programming language.
  3. Use a unicode decoder to get the text in normal text format, there might be a lib to do this, but for example on this website using the "Decode/Unescape Unicode Entities" I was able to get the text.

I hope this helps.

Sample code:

import requests;

req = requests.get('https://www.aparat.com/api/fa/v1/video/video/show/videohash/IueKs?pr=1&mf=1&referer=direct')
res = req.json()

#do stuff with res
print(res)
about 4 years ago · Juan Pablo Isaza Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!