I’m about 30 days1 into building my fourth cloud security program. I want to avoid the mistakes or the past and focus on meaningful risk rather than compliance and security theater.
Coming on board, Security Hub was being used, and not wanting to rock the boat too much, I decided to enable it everywhere and use it for my KRI measurements.
Sadly, Security Hub failed to provide any valuable metrics. It generated so many findings that even I, someone who allegedly knows about cloud security, wanted to give up and raise Alpaca in North Georgia.
So, sit back and enjoy my review of AWS Security Hub.
And if you’re on the product team and want to tell me I’m an idiot for using this wrong or discuss how to improve it, you can find me in the AWS Community Builder or AWS Heroes slack.
What’s good
Let’s start with Security Hub’s positives.
Price
Compared to going out and buying a CSPM, Security Hub is cheap. It’s also easy. You just turn it on and go. No procurement cycles or sales folks are calling to see if they can close the deal in their fiscal quarter.
<– LIES!
Turns out this is just the price for the Delegated Admin account, not for the organization. If you want to know the full price of Security Hub, you must go into each account or wait until the 30-day trial is up and look in Cost Explorer. Between Config Recorder & Security Hub, it’s a lot more than this, but still cheaper than most commercial CSPM solutions.
Delegated Admin
Delegated Admin is another awesome feature. I can configure my security account as the Delegated Admin from my Organizations Management/Payer account, and then all the other config can be done in the security account. Every account created or joins the organization automatically has Security Hub enabled and configured how I want it.
Region Aggregation
Even more impressive, Security Hub can aggregate data across regions! This is one of the few AWS services that support a true single-pane-of-glass, rather than, yeah, we aggregate across accounts, but you still need to look in every region.
AWS Foundational Security Best Practices
While I’ve done some work with the CIS Benchmarks, the most complete list of cloud security posture issues I’ve found for AWS is the AWS-developed AWS Foundational Security Best Practices. The complete set of controls can easily be enabled in Security Hub.
Combine Inspector and CSPM into one place
Security Hub can aggregate findings from a number of sources. It can also take findings from Macie, GuardDuty, and AWS Inspector. So, it can be a one-stop-shop for all configurations and vulnerabilities in your AWS footprint.
As someone who previewed this post said, “I advise folks to turn on security hub everywhere and turn off the controls. Then tap into the feed for events since it will aggregate everything, everywhere, into one account and region to forward to a SIEM or whatever.”
What’s bad
Horrible Dashboards & Metrics
We’ll start with the thing I dislike the most. How it displays metrics. When I first go to Security Hub, I see this:
The Security Score is 31%, the foundation is 33%. WTF? AWS Foundational Security Best Practices v1.0.0 is the only standard I’ve enabled because it has all the controls. I’m focused on risk, so I don’t need NIST or PCI DSS. Why are these numbers not the same?
I’ve passed 69 and failed 137 things. 137 is not that much to fix, right? Let me scroll down the page a bit, Oh shit…
1699 Criticals!!!! FML…
Oh, that includes Inspector findings, too. Okay, maybe my Cloud Security Posture isn’t so bad. Let’s click into the standard and see…
Ok, I’m still at 33%, but no. I don’t have 137 things to fix. I have 14,650 things to fix across 137 different finding types.
82% of my resources are secure, and only 18% is bad.
Focused on Compliance vs. Risk
Security Hub should be called Compliance Hub. This isn’t measuring security or risk. It’s measuring compliance. I’m only 100% compliant with one-third of the controls. Also, note that the above chart highlights your failures rather than successes. AWS wants to make you feel like shit, I guess.
I understand many large AWS customers are banks, healthcare, or processing large amounts of PII. I get that those companies may have zero tolerance for a single public S3 bucket or unencrypted EBS volume. But those are not the companies that need Security Hub. They can afford Wiz, Archer, and an army of compliance wonks to track this. We need a tool for the rest of us.
Cost
The actual cost of Security Hub, after it was running for awhile was pretty significant. Between Security Hub itself and the necessary AWS Config Costs this solution was running over 1% of our AWS bill.
Difficult to disable Controls
Not everything in the Foundational Security Best Practices is a risk for my environment. One of the first Critical findings it sent me was that we didn’t have hardware MFA enabled on all our root accounts. I got 289 Critical Findings because this IAM Rule was evaluated in every region. I have no idea where the number 289 comes from because it’s not divisible by the number of accounts I have enabled or the number of regions I’m aggregating.
Either way, as a company with multiple offices and many remote workers, hardware MFA for root isn’t a necessary security protection, and adopting it introduces a level of operational risk that we’re uncomfortable with.
So I went to disable this control…
Wait. I must log in to every account and go to each region? WTH, I thought I had Delegated Admin and Region Aggregation! I assume this is an issue with AWS Config, a service that hasn’t seen a meaningful improvement since 20192.
I disabled the control in one account/region, and the rest of the findings were Suppressed. Now the standard shows “No Data”, but a deeper dive shows they are all still failed in the list of findings. That won’t confuse anyone.
In between the initial draft and publication, AWS did release a Blog Post with scripts and instructions for disabling controls across multiple accounts and regions. Additionally AWS released CloudFormation Support which might make this a bit easier
Infrequent updates
That leads to another issue - it takes almost a day for a fix to be reflected in Security Hub. How is an engineer tasked with fixing things going to know if they were successful when they won’t know till the next day? Thanks, AWS. You’ve generated more resentment against security. We didn’t have enough resentment when we asked our engineers to fix 14,134 High findings.
Inability to figure out where to start
This is where my imposture syndrome really kicked in. How do I find the actual critical criticals I need to address first? How do I filter out the HIGH findings which shouldn’t be? Yeah, I’m looking at you default security group rule.
I will give Security Hub more more bit of credit. Some of it’s insights are useful for this. I identified a single AMI that accounted for a double-digit percentage of my Inspector findings, and that’s a clear place for engineering to target.
Security Hub is dangerous and does AWS’s customers a disservice
AWS needs to provide a tool for those below the security poverty line. One that focuses on risk and not compliance. One that doesn’t require an AWS expert to configure and tune.
Metrics that focus on compliance and not risk will dishearten cloud engineers. This will result in:
- engineers ignoring issues.
- hostility between engineering and security.
- a false narrative that the security team is incapable of securing the environment.
The inability to adjust finding severity means that AWS, a trillion-dollar company with tens of thousands of expert cloud engineers and resources to fix all these problems, is projecting that on your 20-person development team and single security engineer.
Smaller companies move to AWS because they do reduce the undifferentiated heavy lifting of hardware procurement, data-center management, HVAC, etc. They need practical advice on how to secure their AWS footprint that is both achievable and risk-aligned. Sadly, AWS has failed them with this product.
-
The original draft for this blog post was in May of 2023, while I never got around to finishing and publishing it till October. ↩︎
-
Between drafting and publishing this post, AWS Config did provide the ability to exclude certain resource types from recording, which could help lower super expensive AWS Config Recorder costs. ↩︎