I am new to node.js and am trying to get an initial page key from an api response header and loop through the pages until the page key header no longer exists, meaning I've reached the last page and the loop can end.
For some reason the pageKey never gets changes in the while loop the way I have it set up:
async function doThis() {
let gotAllPages = false;
let pageKey;
var response = await got(url, options);
try {
pageKey = response.headers['next-page'];
} catch (e) {
gotAllPages = true;
}
while (!gotAllPages) {
var pageOptions = {
method: 'GET',
headers: {
Authorization: `xxx`,
nextPage: pageKey
}
};
response = await got(url, pageOptions);
try {
pageKey = response.headers['next-page'];
console.log(pageKey);
} catch (e) {
console.log("No header value found, loop is done.");
gotAllPages = true;
break;
}
}
}
This ends up going in an infinite loop because the pageKey never gets changed from the first response so it just keeps trying to grab the same page.
Being new to Node.js, I don't really understand if there's a scoping issue with how I've set the variables up or what, so any advice would be appreciated!
Try-catch blocks do not fail when setting a variable to undefined or null. Instead of using a try-catch block, use an if statement.
pageKey = response.headers['next-page'];
if (!pageKey) {
console.log("No header value found, loop is done.");
gotAllPages = true;
break;
}
The key access response.headers['next-page']; isn't throwing anything, so the catch where gotAllPages is set never runs.
We can fix this, and tidy up a bit...
async function getAllPages() {
let results = [];
let nextPage = 0;
let options = {
method: 'GET',
headers: { Authorization: `xxx` }
};
while (nextPage !== undefined) {
options.headers.nextPage = nextPage;
let response = await got(url, options);
results.push(response);
nextPage = response.headers['next-page'];
}
return result;
}
This assumes a couple things: (1) that the API expects nextPage === 0 on the first request, (2) that the API signals no more pages by leaving the next-page header undefined, (3) that the caller wants the full response, rather than some part like response.data or response.body, (4) that the caller catches errors thrown here and handles them.