I'm fetching data from an API that has a limit of 50 records per call. There are 10,000 records in total that I need to retrieve so looking at using pagination via the offset parameter to achieve this.
I'm not too familiar with the syntax I need to use but I would like to keep looping through the fetch until the number of records returned for a single call is < 50 to signal the final page.
I need my JSON responses from each call to be appended together, in addition to a final 'count' that gives me 10,000
let offset = 0
fetch(`https://api.opensea.io/api/v1/assets?collection=womenandweapons&format=json&offset=${offset}&limit=50&order_direction=desc`,
{
method: 'GET',
headers: {
'Accept': 'application/json'
}
})
.then(response => response.json())
.then(data => {
let len = Object.keys(data.assets).length
console.log(len)
console.log(data.assets)
})
offset += 50
I've had some luck doing something like this:
const main = async () => {
const url = new URL(AIRTABLE_URL);
let data = await fetcher(url, AIRTABLE_API_KEY);
storiesTotal = mapData(data.records);
while (data.offset) {
data = await fetcher(url, AIRTABLE_API_KEY, data.offset);
storiesTotal = [...storiesTotal, ...mapData(data.records)];
}
}
main();