I am trying to paginate some data. The limit of results per query are 300, and I can use an offset to get the next page. I am attempting to do this with a while loop like so:
async function fetchData (
contractAddress,
tokenID )
{
const url = `https://api.website.io/api/v1/events?asset_contract_address=${contractAddress}&token_id=${tokenID}&offset=0&limit=299`;
logger.log(url);
const data = await internalFetch(url, {
headers: {
'Accept': 'application/json',
'X-Api-Key': API_KEY,
},
method: 'get',
timeout: 10000, // 10 secs
})
var array = data.data.asset_events;
var offset = 0;
var tempResponse = array;
while (tempResponse.length != 0) {
var urlPage = `https://api.website.io/api/v1/events?asset_contract_address=${contractAddress}&token_id=${tokenID}&offset=${offset}&limit=299`;
logger.log(urlPage + " url2");
var nextPage = await internalFetch(urlPage, {
headers: {
'Accept': 'application/json',
'X-Api-Key': API_KEY,
},
method: 'get',
timeout: 10000, // 10 secs
})
tempResponse = nextPage.data.asset_events;
array.concat(tempResponse);
offset = array.length;
logger.log("length " + offset);
}
return array;
}
Something in the while loop is not working correctly and running an infinite loop. The url being queried is the same every time [with the offset at 299], so the same response is being returned and and tempResponse.length is never == 0, ending up with a while loop. Can someone spot the issue?
It's this line, where you set the offset:
offset = array.length;
This is always going to return the same number, assuming there are more than two pages of results. The first time you will get the array length, and then when you use that in the fetch you will get the results starting at 300.
But that array will also be 299 long, so you send the same offset number and request the second page again. I think:
offset += array.length;
should work, although this may repeat the last record from the previous call, in which case you would want offset += array.length + 1;.