Below I tried to write a conditional that would fetch a paginated api and then map it to another api that is being fetched. One issue that is coming up is that it's not continue to loop after it pulls one paginated page or one next page. The second issue is that that the data being fetched from the pages aren't being combined into one array. What am I doing wrong or missing?
const fetchURL = `${baseURL}?owner=${accounts[0]}`;
fetch(fetchURL, {
method: 'GET',
redirect: 'follow',
})
.then(resp => resp.json())
.then(data => {
console.log(data);
const pageKey = data.pageKey
if (pageKey !== 0) {
fetch(`${baseURL}?owner=${accounts[0]}&pageKey=${pageKey}`, {
method: 'GET',
redirect: 'follow',
})
.then(resp => resp.json())
.then(data => {
console.log(data)
})
return data.ownedNfts.concat(data.ownedNfts)
} else {
return data
}
const responses = data.ownedNfts.map((ownedNfts) =>
fetch(`${baseURL1}stats?address=${ownedNfts.contract.address}`)
.then((res) => res.json()),
);
To manage pagination from api, you could try a recursive like this.
You should have a script with a request loop with increment params, and a threshold to break the loop. You have to manage the request delay from your api with a time sleep or something like this.
This example bellow work in a node env with axios, you can try it and adapt it with your environnement.
const { default: axios } = require('axios');
// Init a bigData array to push new data on each iteration
const bigData = [];
async function fetchAllPaginateData(
pageKey = 0 /** init by default page index 0 */,
) {
try {
const fetchURL = `https://api.instantwebtools.net/v1/passenger?page=${pageKey}&size=1`;
const response = await axios.get(fetchURL);
const { data } = response;
const { totalPages } = data; // Your api should give you a total page count, result or something to setup your iteration
bigData.push(data); // push on big data response data
// if current page isn't the last, call the fetch feature again, with page + 1
if (
pageKey < totalPages &&
pageKey < 10 // (this is a test dev condition to limit for 10 result) */
) {
pageKey++;
await new Promise((resolve) => setTimeout(resolve, 200)); // setup a sleep depend your api request/second requirement.
console.debug(pageKey, '/', totalPages);
return await fetchAllPaginateData(pageKey);
}
console.clear();
return console.info('Data complete.');
} catch (err) {
console.error(err);
}
}
fetchAllPaginateData().then(() => console.table(bigData));