Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

88
Views
How can I improve this paginated do while async await GET request in Javascript (NodeJS)?

I am learning JavaScript (Node.js - using the Pipedream platform). I have been writing scripts to help automate some little tasks in my day to day work.

I am creating one that generates a report on recent interactions with clients.

As part of this I am using axios to get "engagements" from the Hubspot API (basically a list of identifiers I will use in later requests).

The API returns paginated responses. I have encountered pagination previously and understand the principle behind it, but have never written a script to handle it. This is my first.

It works. But I feel it could be improved. Below I've commented how I've approached it.

The endpoint returns up to 100 values 'per page' along with a "hasMore":true flag and an "offset":987654321 value which can be passed as a query parameter in subsequent requests (if hasMore === true).

Example API response:

{"results":[1234,1235,1236],"hasMore":true,"offset":987654321}

My code:

import axios from 'axios';
//function to get each page of data
async function getAssoc(req){
      const options = {
        method: 'GET',
        url: `https://api.hubapi.com/${req}`,
        headers: {
          Authorization: `Bearer ${auths}`,
        },
      };
      return await axios(options);
}
//declare array in which to store all 'associations'
const assocs = [];
//store the ID that I get in an earlier step
const id = vid;
//declare variable in which to temporarily store each request response data
var resp;
//declare query parameter value, initially blank, but will be assigned a value upon subsequent iterations of do while
var offset = '';
do {
  //make request and store response in resp variable
  resp = await getAssoc(`crm-associations/v1/associations/${id}/HUBSPOT_DEFINED/9?offset=${offset}`);
  //push the results into my 'assocs' (associations) array
  resp.data.results.forEach(element => assocs.push(element));
  //store offset value for use in next iteration's request
  offset = resp.data.offset;
} while (resp.data.hasMore); //hasMore will be false when there's no more records to request

return assocs;

I feel it could be improved because:

  1. The DO WHILE loop, I believe, is making sequential requests. Is parallel a better/faster/more efficient option? (EDIT Thanks @Evert - of course I cannot make parallel requests because of the offset!)
  2. I'm re-assigning new values to vars instead of using consts which seems simple and intuitive in my beginner's mind, but I don't understand a better way in this instance.
  3. I would welcome any feedback or suggestions on how I can improve this for my own learning.

Thank you in advance for your time and any assistance you can offer.

about 4 years ago · Juan Pablo Isaza
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!