I have a strange problem while performing request to external API.
Basically I got 403 error response for the HTTP request, but... only when calling it from node environment. I tried with curl through terminal and it's OK. I also tried from Postman and native go with net/http package and everything is just fine, I got 200 response. But no matter what tool I use in node.js (request, axios, unirest, got) I can't make it. I would love to share entire request with you, but unfortunately I can't :(
All I can show you is some response details, when I call it from node.js environment.
These are headers sent by the server when I got 403:
headers: {
server: 'AkamaiNetStorage',
'content-length': '228',
'content-type': 'application/json',
'x-edge-error': 'halt',
'cache-control': 'max-age=122',
date: 'Thu, 21 Oct 2021 16:17:41 GMT',
connection: 'close'
},
And the part of response data looks like this:
data: {
edge_error: 'halt',
ref_id: '18.9d6656b8.1634833061.4b9f14b',
}
I am aware that question is strange and you probably won't be able to help me with not so many details, but I will try anyway. So If anyone have idea what might help, feel free to propose an answer.
Best regards ;)
This is likely due to TLS fingerprinting. Akamai is able to detect that you're using Node.JS based on information gathered during the TLS handshake (cipher suites, etc.) when you send a request. The website must have set a rule through Akamai to block Node.
Try something like this:
const tls = require('tls');
const https = require('https');
const defaultCiphers = tls.DEFAULT_CIPHERS.split(':');
const shuffledCiphers = [
defaultCiphers[0],
// Swap the 2nd & 3rd ciphers:
defaultCiphers[2],
defaultCiphers[1],
...defaultCiphers.slice(3)
].join(':');
request = require('https').get('https://google.com', {
ciphers: shuffledCiphers
}).on('response', (res) => {
console.log(res.statusCode);
});
// source: https://httptoolkit.tech/blog/tls-fingerprinting-node-js/
More information about this can be found here: https://httptoolkit.tech/blog/tls-fingerprinting-node-js/