Following on from my last post, I needed to update another old AWS Lambda to Node 22. This required me to make several https calls from an external API and then combine them together to create my result.
Using https.get to make an asynchronous http request
The first step was to include the https library, and write a relatively simple https get request function as follows.
import https from "https";
export const handler = async (event, context, callback) => {
const httpsRequest = async(host, path, key) =>{
var data = "";
var options = {
hostname: host,
path: path
};
if (key != "" ) options.headers = {"Authorization": key}
https.get(options, async(r) => {
console.log("https request " + r.statusCode);
r.setEncoding('utf8');
r.on('data', (chunk) => { data += chunk; });
}).on('error', (e) => {
console.error(e);
});
return data;
}
const response = await httpsRequest("myapi.com","/endpoint","my-auth-key");
callback(null, {"data":data});
}I ran into an issue with this, where the data returned from the API was larger than a single data chunk. My function terminated with the first data chunk only and the data was truncated. I got around this by wrapping the get call inside a promise, which only resolves on the end event (ie after all the data chunks have been returned).
const httpsRequest = async(host, path, key) =>{
var data = "";
await new Promise((resolve, reject) =>{
var options = {
hostname: host,
path: path
};
if (key != "" ) options.headers = {"Authorization": key}
https.get(options, async(r) => {
if (r.statusCode != 200) {
data = "Error " + r.statusCode;
resolve();
}
else {
r.setEncoding('utf8');
r.on('data', (chunk) => { data += chunk; });
r.on('end', () => { resolve(); });
}
}).on('error', (e) => {
retVal = "HTTPS Error";
reject();
});
});
return data;
}
Using Promise.all to resolve an array of async function calls
The next issue was how to call multiple API calls in parallel, and then combine the results when they had all completed. If I had known the number of calls, I could simply have used the pattern that I mentioned in my previous post:
const response1 = httpsRequest("myapi.com","/endpoint1","my-auth-key");
const response2 = httpsRequest("myapi.com","/endpoint2","my-auth-key");
const result = [await response1, await response2];Unfortunately in my case the number of calls varies depending on the input data. After a bit of research I decided to use an array of promises and resolve these with Promise.all
As most of the examples that I could find either had a known number of promises for the promise array, or were a lot more complex, it took me a while to come down to the following.
const getFromAPI = async(requestList) => {
const promiseArr = [];
for (var i in requestList) {
var promise = httpsRequest("myapi.com", requestList[i].path, "my-auth-key")
.then (result => {return result;});
promiseArr.push(promise);
}
var data = [];
await Promise.all(promiseArr).then(resultsArr => {
for (var j in resultsArr) {
data.push(resultsArr[j]);
}
});
callback(null, {"data": data});
}
Not having used promises much before I didn’t realise I needed the then statement after the function call. I simply pushed all of the promises into the promise array and tried to read the result in the Promise.all block. This resulted in a lot of undefined elements in my data array. Eventually I realised that var promise was only (and would always be) a promise. In order get the resolution of the promise the then call is needed to populate the results array.
Leave a Reply