I am working on retrieving data from an XML API. In order to render a result to the screen I have to call one endpoint to get a list of data, then I have to map through the list of items and call a second endpoint for each returned item. The first request goes through successfully; however the rest of the requests seem to be called concurrently. This is leading to about 90% of the requests coming back with the following error:
Maximum allowed concurrent requests threshold(10) was breached
My back end function that calls the API endpoint takes in XML as a string and converts the response to JSON. Here is the code:
export function newSoapRequest(options = {
method: "POST",
url: "",
headers: {},
xml: "",
timeout: 10000
}) {
const {
method,
url,
headers,
xml, // soap envelope as string
timeout
} = options
return new Promise((resolve, reject) => {
axios({
method: method || "POST",
url,
headers,
data: xml,
timeout
}).then((res) => {
resolve({
res: {
headers: res.headers,
body: XML.parse(res.data),
statusCode: res.status,
}
})
}).catch((err) => {
if(err.response) {
reject(err.response.data)
} else {
reject(err)
}
})
})
}
Here is the code where I am calling the API from the map function
resortList.map(async (resort) => {
let resortXML = `<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:dae="DAE"><soapenv:Header/><soapenv:Body><dae:DAEGetResortProfile><!--Optional:--><dae:AuthID>GWAYUAT</dae:AuthID><!--Optional:--><dae:EndpointID>${resort.WeekEndpointID}</dae:EndpointID><!--Optional:--><dae:ResortID>${resort.ResortId}</dae:ResortID></dae:DAEGetResortProfile></soapenv:Body></soapenv:Envelope>`
await newSoapRequest({method: "POST", url, headers: sampleHeaders, xml: resortXML, timeout: 10000})
.then(response => console.log(response))
.catch(err => console.log(err))
})
As far as I know the response is sent from the newSoapRequest function and then the connection is closed. When I am using async await inside the map function the map function should wait until the request is completed and the connection is closed before processing the response and then running for the second time. Could this issue be related to the map function it's self? Or is my newSoapRequest function not closing the connection properly?
The Array.prototype.map function does NOT awaits until an async function ends. That is the reason all the items are being called concurrently.
To avoid that, you can use this approach:
const numbers = Array.from({ length: 10 }).map((_, i) => i + 1);
const callback = async n => await new Promise(r => setTimeout(() => r(`${n} * 2 = ${n * 2}`), 10));
async function sequentialAsyncMap(arr, predicate) {
const result = [];
for (const item of arr) {
result.push(await predicate(item));
}
return result;
}
sequentialAsyncMap(numbers, callback)
.then(console.log);Example with callback-chunk package (Disclaimer: I'm the author of this package):
const chunk = require('callback-chunk');
const callbacks = resortList.map(item => () => newSoapRequest({
method: 'POST',
url,
timeout: 10000,
headers: sampleHeaders,
xml: `<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:dae="DAE"><soapenv:Header/><soapenv:Body><dae:DAEGetResortProfile><!--Optional:--><dae:AuthID>GWAYUAT</dae:AuthID><!--Optional:--><dae:EndpointID>${item.WeekEndpointID}</dae:EndpointID><!--Optional:--><dae:ResortID>${item.ResortId}</dae:ResortID></dae:DAEGetResortProfile></soapenv:Body></soapenv:Envelope>`
}));
const items = await chunk(callbacks, 10, console.log);