https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessage.html
https://sqs.us-east-2.amazonaws.com/123456789012/MyQueue/
?Action=SendMessage
&MessageBody=This+is+a+test+message
This works fine, however is it possible to send the payload using the post body?
I have an external service sending json payloads in the post body (at a very high throughput). It would be ideal if I can simply feed them the SQS url directly, and avoid having to create a AWS api gateway --> lambda --> SQS stack.
I'm open to alternative solutions.
This may not have been exactly what the designers of API Gateway had in mind, but given...
the way the AWS Query APIs (as used by the SDKs) work on the wire for services like SQS -- url-encoded key/value pairs in the standard application/x-www-form-urlencoded format, and
the fact that the built-in body mapping template language in API Gateway (VTL) exposes $util.urlEncode(), and
the fact that API Gateway can transparently sign requests when sending them to a backend service
...this means you can simply construct a valid API request -- a form post -- by hand, as were, using a VTL template, that encapsulates the original message body -- urlencoded, and that's all we need. We don't actually care that it's JSON, as long as it is valid character data.
With this solution, a Lambda function is avoided, and the client does not need to know anything about how SQS expects messages to be formatted.
The entire incoming request body becomes the Message in SQS.
In the API Gateway console:
Create the resource (e.g. /) and the method (e.g. POST).
In the Integration Request settings:
Integration type: AWS Service
AWS Region: [select your region]
AWS Service: Simple Queue Service (SQS)
AWS Subdomain [leave blank]
HTTP Method: POST
Action Type: Use path override
Path override: /
Execution Role: [your role ARN, needs to be able to send a message to the queue]
Credentials cache: Do not add caller credentials to cache key
Content Handling: Passthrough
Under HTTP Headers, add one header, Content-Type. This value should be specified as being "mapped from" 'application/x-www-form-urlencoded' -- note that this is a single-quoted string.
Under Body Mapping Templates, choose Never.
Add a Content-Type of application/json and use the following mapping template:
Action=SendMessage##
&QueueUrl=$util.urlEncode('https://sqs.us-east-2.amazonaws.com/000000000000/my-queue-name')##
&MessageBody=$util.urlEncode($input.body)##
And you have an API that transforms the a raw JSON input body into an SQS SendMessage API request.
The ## at the end of each line are for readability -- VTL is a text templating language, so whitespace and newlines are preserved. Placing ## at the end of each line causes the newline to be stripped, which is necessary for building a correct web form. Otherwise, the entire body mapping template would need to be on a single line.
Deploy and then test:
$ curl -X POST https://xxxxxxxxxx.execute-api.us-east-2.amazonaws.com/v1 --data '{"works": true}' -H 'Content-Type: application/json'
Response:
{"SendMessageResponse":{"ResponseMetadata":{"RequestId":"bbbbbbbb-aaaa-5555-8888-334ed25bb6b3"},"SendMessageResult":{"MD5OfMessageAttributes":null,"MD5OfMessageBody":"81f2ecc3cb027268138bdfe7af0f8a3f","MessageId":"cccccccc-dddd-4444-1111-542e08bb39af","SequenceNumber":null}}}
For extra credit, a body mapping template in the Integration Response can be reused to customize the response, as well.
There is a tutorial on this subject at https://blog.rewind.io/2016/11/16/handle-shopify-webhooks-without-a-server