Empresas
Empleos
  • Sobre nosotros
  • Soluciones
    • Publicación de vacantes
      Publica tu vacante y recibe candidatos calificados en 48h.
    • Evaluación de candidatos
      500+ pruebas técnicas y psicológicas, más anti-fraude.
    • Headhunting
      Búsqueda ejecutiva a la medida de principio a fin.
    • Nómina + EOR
      Dispersión de nómina y EOR en más de 15 países de LATAM.
  • Precios
  • Empleos

0

208
Vistas
How to load infinite scroll pages faster?

https://immutascan.io/address/0xac98d8d1bb27a94e79fbf49198210240688bb1ed

This URL has 100k+ rows that I'm trying to scrape. They go back about a month (1/10/2022 I believe) but load in badges of 7-8.

Right now I have a macro slowly scrolling down the page, which is working, but takes about 8-10 hours per day's worth of rows.

As of now, when new rows load there are 2-3 items that load immediately and then a few that load over time. I don't need the parts that load slowly and would like them to load faster or not at all.

Is there a way that I can prevent elements from loading to speed up the loading of additional rows?

I'm using an autohotkey script that scrolls down with the mouse-wheel and that's been working best.

I've also tried a Chrome extension but that was slower.

I found a python script at one point but it wasn't any faster than autohotkey.

Answer: Immutable X has an API so I'm using that instead of this site that does the same thing. Here's the working code:

import requests
import time
import pandas as pd
import time

URL = "https://api.x.immutable.com/v1/orders"
bg_output = []
params = {'status': 'filled',
          'sell_token_address': '0xac98d8d1bb27a94e79fbf49198210240688bb1ed'}

with requests.Session() as session:
    while True:
        (r := session.get(URL, params=params)).raise_for_status()
        data = r.json()
        for value in data['result']:
            orderID = value['order_id']
            info = value["sell"]["data"]["properties"]["name"]
            wei = value["buy"]["data"]["quantity"]
            decimals = value["buy"]["data"]["decimals"]
            spacer = "."
            eth = float(wei[decimals:] + spacer + wei[:decimals])
            print(f'Count={len(bg_output)},Order ID={orderID}, Info={info}, Eth={eth}')
            bg_output.append(f'Count={len(bg_output)},Order ID={orderID}, Info={info}, Eth={eth}')
            timestr = time.strftime("%Y%m%d")
            pd.DataFrame(bg_output).to_csv('bg_output' + timestr + '.csv')
            #print(len(bg_output))
        time.sleep(1)    
        if (cursor := data.get('cursor')):
            params['cursor'] = cursor
        else:
            print(bg_output)
            break
            
print(bg_output)
print("END")
about 4 years ago · Juan Pablo Isaza
1 Respuestas
Responde la pregunta

0

Have you considered using their API directly? When you scroll the page, have a look at your browser’s dev tools “network” tab. There you can see the actual call to their API. Look at all POST requests to the URL

https://3vkyshzozjep5ciwsh2fvgdxwy.appsync-api.us-west-2.amazonaws.com/graphql

Try adapting these API calls so that you can get the data right via their GraphQL-API and without having to scroll the actual page.

about 4 years ago · Juan Pablo Isaza Denunciar
Responde la pregunta
Encuentra empleos remotos

¡Descubre la nueva forma de encontrar empleo!

Top de empleos
Top categorías de empleo
Empresas
Publicar vacante Precios Comercial
Legal
Términos y condiciones Política de privacidad
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomiéndame algunas ofertas
Necesito ayuda