AWS security assessment: what scanners are missing and how threat modeling may help you?

There are many tools available today that are designed to automate security checks. For example, here’s a good list of open-source AWS security tools, not even mentioning commercial ones. But IMHO some people rely too much on tools, as if conducting an AWS security assessment is the same as formatting the scanner’s output into fancy-looking report. But nothing could be further from the truth. In this blog post I want to focus on what scanners are missing and why I think tools cannot fully replace the assessor in performing an effective AWS security assessment.

Scanners don’t know the big picture

First of all, popular security scanners like Prowler, ScoutSuite etc. aren’t context related.

Imagine a company which stores customer Personal Identifiable Information (PII) files in several S3 buckets. The buckets aren’t publicly available and they use SSE-S3 as a default encryption (scanners won’t raise an issue here, since the default encryption is enabled; but since SSE-S3 uses S3-managed keys, every principal in the AWS account with S3 permissions will also be allowed to encrypt/decrypt objects inside the bucket). Additionaly, the company uses dozens of IAM users and roles, which have attached policies granting full access to any S3 action to any resource (because of using AWS-managed AmazonS3FullAccess policy, which grants access to “Resource”: “*”). That brings a big security concern, because a tiny developer’s mistake or compromising an internal principal may end with sensitive information disclosure which may have dramatic consequences for the company (because there are no restrictive guardrails protecting sensitive buckets). But such threat will most likely not be raised by a scanner. On the other hand, an assessor who knows where sensitive data is stored can easily identify potential threats and recommend multiple security controls to mitigate them (such as using SCP or a restrictive bucket policy).

More findings don’t mean better result

Furthermore, scanners can flood you with false positives. It’s easy to run a scanner, but a report with hundreds of high- and medium-severity issues is… useless. Un-customized scanners tend to report multiple issues, which may be not applicable in your case. For example, they can be findings related to a lack of restrictive password policy, even though you don’t have any IAM users, or an absence of enabled access logs, though you catch them using 3ʳᵈ party solutions. A security team will need then to filter-out relevant issues one by one, which can be a very time-consuming task in complex environments. In the worst-case scenario, the report review task will get stuck in a backlog, leaving your team unaware of issues that really matter.

Scanners may not cover everything

Another consideration is if you can rely on scanner’s output. In other words, you need to be aware of what security checks are missing in your scanner. A good example are hardcoded secrets, which may open many doors to unauthorized principals. Scanners usually cannot detect a login and a password passed in a single instruction in EC2 user data or in ECS task definition’s environment variable. Hardcoded secrets can be a real game-changer in an attacker’s hands. Just imagine a compromised low-privileged user/role, which is allowed to read credentials to external CI/CD pipeline with administrative permissions to the entire organization. Unfortunately, scanners are usually blind to those types of issues.

Indeed, some scanners (especially open-source ones which are no longer maintained or occasionally updated) lack rules against some new AWS services or newly introduced features. You need to have a full inventory list of your cloud assets to identify those scanner’s gaps to address them, either manually or by using custom scripts.

Skipped data flows and relations

In multi-account architecture, it can be often observed that some accounts have many relations either on the identity level (such as an IAM role which can be assumed by other accounts in your Organization or by 3ʳᵈ party) or on the data level (e.g., network traffic peered over VPC peering). But scanners are not aware of the big picture of your infrastructure and usually are blind to those potential attack vectors. With that said, you can be unaware that access to your critical assets is shared via IAM roles or resource-based policies to a 3ʳᵈ party which is no longer used. It’s even more scary when you realize that 82% of companies unknowingly give 3ʳᵈ parties access to all their cloud data.

Similarly, access can be given on the data level. For example, an attacker who compromised an EC2 instance in a development account can access production database via VPC peering or non-restrictive internal proxy. Not to mention, of course, the entire class of supply chain attacks (watch my example of releasing malicious NPM package and using it in Lambda function to compromise the function execution role).

Threat modeling to the rescue!

So how can we address these issues? How can an assessor learn the big picture of a cloud infrastructure to provide a valuable AWS security assessment? I find it very useful to start from a threat modeling session. While there are multiple approaches, I like a simple session of whiteboarding with customer’s representatives. In times of remote work, you can find various tools allowing you to do the same virtually. Personally, I use Lucid Spark. The goal of such session(s) is to:

• understand the usage context of cloud assets,
• identify critical assets and how they are used (who need access, what level of access, how assets are processed and so on),
• identify all potential threats and attack vectors.

When you’re leading such a threat modeling session it’s important to cover all security-relevant cloud infrastructure areas. A good list of questions to ask and topics to consider is given in Marco Lancini’s spreadsheet “What to look for when reviewing a company’s infrastructure”.

A (very) simplified output of such session may look like this:

Of course, it’s just a sample to give you an idea how it may look like. Usually, I try to upload the real customer’s architecture diagrams and then add threats and comments using sticky notes, highlight relations, and so on. After discussing (and understanding) every piece of the system and identifying potential threats, you’re ready to begin the assessment, which most likely will result in increasing your initial list of threats.

Another benefit from a cloud infrastructure threat modeling session is that the attendees who look at the big picture of their system can identify security gaps by themselves. We tend to be project-oriented and forget how our initially good-looking solutions can impact the security of the whole organization. That’s also a good reason to do such exercise with your team internally before starting any kind of project.

Summary

Without a doubt security tools are very helpful in automating security checks, but they should be treated as a complement only, and never as a replacement for the assessor. A lack of understanding a big picture of a cloud infrastructure, missing a list of critical assets and potential attack vectors may end a cloud security assessment with a huge report, full of false positives. Consequently, it will waste engineers’ time on handling irrelevant issues, instead of mitigatigating serious threats. That’s why a deep understanding of what you’re going to assess is a fundamental step, which should start every assessment.