Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

205
Views
How to load infinite scroll pages faster?

https://immutascan.io/address/0xac98d8d1bb27a94e79fbf49198210240688bb1ed

This URL has 100k+ rows that I'm trying to scrape. They go back about a month (1/10/2022 I believe) but load in badges of 7-8.

Right now I have a macro slowly scrolling down the page, which is working, but takes about 8-10 hours per day's worth of rows.

As of now, when new rows load there are 2-3 items that load immediately and then a few that load over time. I don't need the parts that load slowly and would like them to load faster or not at all.

Is there a way that I can prevent elements from loading to speed up the loading of additional rows?

I'm using an autohotkey script that scrolls down with the mouse-wheel and that's been working best.

I've also tried a Chrome extension but that was slower.

I found a python script at one point but it wasn't any faster than autohotkey.

Answer: Immutable X has an API so I'm using that instead of this site that does the same thing. Here's the working code:

import requests
import time
import pandas as pd
import time

URL = "https://api.x.immutable.com/v1/orders"
bg_output = []
params = {'status': 'filled',
          'sell_token_address': '0xac98d8d1bb27a94e79fbf49198210240688bb1ed'}

with requests.Session() as session:
    while True:
        (r := session.get(URL, params=params)).raise_for_status()
        data = r.json()
        for value in data['result']:
            orderID = value['order_id']
            info = value["sell"]["data"]["properties"]["name"]
            wei = value["buy"]["data"]["quantity"]
            decimals = value["buy"]["data"]["decimals"]
            spacer = "."
            eth = float(wei[decimals:] + spacer + wei[:decimals])
            print(f'Count={len(bg_output)},Order ID={orderID}, Info={info}, Eth={eth}')
            bg_output.append(f'Count={len(bg_output)},Order ID={orderID}, Info={info}, Eth={eth}')
            timestr = time.strftime("%Y%m%d")
            pd.DataFrame(bg_output).to_csv('bg_output' + timestr + '.csv')
            #print(len(bg_output))
        time.sleep(1)    
        if (cursor := data.get('cursor')):
            params['cursor'] = cursor
        else:
            print(bg_output)
            break
            
print(bg_output)
print("END")
about 4 years ago · Juan Pablo Isaza
1 answers
Answer question

0

Have you considered using their API directly? When you scroll the page, have a look at your browser’s dev tools “network” tab. There you can see the actual call to their API. Look at all POST requests to the URL

https://3vkyshzozjep5ciwsh2fvgdxwy.appsync-api.us-west-2.amazonaws.com/graphql

Try adapting these API calls so that you can get the data right via their GraphQL-API and without having to scroll the actual page.

about 4 years ago · Juan Pablo Isaza Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!