Skrillex at re:Invent 2018

AWS re:Invent 2018 Wrapup

It’s been about two now three weeks since AWS re:Invent 2018 wrapped, and I’m finally starting to recover. Six days in Vegas and a red-eye flight back to Atlanta have me not wanting to travel any more in 2018. So what happened of interest?

First off, my two Chalk Talks with Suman and Damindra went well. I’m glad that at least two people out of the 58,000 present thought my session was they best they’d attended. It was interesting doing one session before Security Hub and Control Tower were announced, and another session after Andy Jassey announced them.

On the topic of Antiope and resource inventory, I re-discovered AWS Config while in a workshop on Tuesday afternoon. I think there is value in taking a second look at the output of the Config Recorder and seeing if the Antiope resource object format shouldn’t better align with AWS Config. Polling 250 AWS accounts every 30 minutes isn’t terribly efficient, and instances can come and go within that time-frame. Leveraging Config to save specific resources only when they change might be more efficient.

As always I enjoyed Steve Schmidt’s session. Amazon’s ability to “keep people away from data” and how they automate so much to have so few humans in the security organization is very impressive.

Teri Radichel’s Red Team vs Blue Team session started off with an awesome breakdown of how to do reconnaissance using read-only credentials to find the crown-jewels and ex-filtrate the data via the network plane.

Andrew Krug with Mozilla did an excellent ChalkTalk on “How to Perform Forensics on AWS Using Serverless Infrastructure”. In it he announced ssm_acquire which is a automated serverless disk and memory forensics tool.

Another really valuable workshop on Friday was on using CodeDeploy, CodePipeline and Inspector to assess AMIs (which we learned this year are pronounced Ah-Mee-s) before releasing them to production.

In between all of that was some quiet time on the bus (AWS did a much better job this year with the spreadout nature of this show), and talking to vendors. The most interesting vendor was Cloud Conformity. Their catalog of security checks is the most in-depth I’ve seen. Also intriguing is Turbot. If you’re just starting out in AWS or looking to pivot to multi-cloud, consider them to orchestrate your deployment of accounts and resources. Lastly, is This vendor wasn’t on the show floor, they were doing a ChalkTalk on enterprise governance. They had a very interesting product around governance, at scale, using organizational constructs, all implemented via IAM.


There were a lot of launches. As I said in my last blog post the ones you need to worry about are the ones with Resource policies or the ability to share between accounts. The AWS account is the fundamental security boundary for their cloud. As they blur the lines between accounts, your security risk increases. So where do I see security risk in their new offerings:

ALB & Lambda

Application Load Balancers now support invoking Lambda functions to serve HTTP(S) requests. This enables users to access serverless applications from any HTTP client, including web browsers. With the Application Load Balancers’ support for content-based routing rules, you can also route requests to different Lambda functions based on the request content.

Status: Generally Available

Potential Risks: With the ability for Lambda to be invoked by an Application Load Balancer, the obscurity & surface area reduction provided by API Gateway have been stripped away. OWASP attacks against Lambda are likely to increase.

Recommended Action: This is where AWS WAF will help. I recommend building out a template of rules to protect against this

Lambda Layers

Lambda Layers are a new type of artifact that can contain arbitrary code and data, and may be referenced by zero, one, or more functions at the same time. Lambda functions in a serverless application typically share common dependencies such as SDKs, frameworks, and now runtimes. With layers, you can centrally manage common components across multiple functions enabling better code reuse. To use layers, you simply put your common code in a zip file, and upload it to Lambda as a layer. You then configure your functions to reference it.

Status: Generally Available

Potential Risks: Lambda Layers allow multiple Lambda functions to share the same set of deployed code across multiple functions. Now that the lambda codebase is no longer atomic, new attack vectors against this service arise. Additionally, Lambda Layers can be shared across accounts, potentially introducing the risk of publicly shared Lambda code.

Lambda Runtimes

The Runtime API for AWS Lambda defines a standardized HTTP-based specification which codifies how Lambda and a function’s runtime communicate. It enables you to build custom runtimes that integrate with Lambda to execute functions in response to events. By leveraging the Runtime API, you can use binaries or shell scripts, and your choice of programming languages and language versions.

Status: Generally Available

Potential Risks: Lambda now support additional runtimes, and there is the capability for a developer to “bring their own runtime”. The risk is that non-hardened runtimes might be deployed and expose additional attack surface area. This risk is compounded by the ALB support for Lambda release

Amazon FSx for Windows

Amazon FSx for Windows File Server provides a fully managed native Microsoft Windows file system so you can easily move your Windows-based applications that require file storage to AWS. Built on Windows Server, Amazon FSx provides shared file storage with the compatibility and features that your Windows-based applications rely on, including full support for the SMB protocol and Windows NTFS, Active Directory (AD) integration, and Distributed File System (DFS). Amazon FSx provides multiple levels of security and compliance to help ensure your data is protected. Amazon FSx automatically encrypts your data at-rest and in-transit. Amazon FSx is designed to meet the highest security standards and it has been assessed to comply with ISO and PCI-DSS certifications, and is HIPAA eligible. Integration with AWS CloudTrail monitors and logs your API calls letting you see actions taken by users on your Amazon FSx resources.

Status: Generally Available

Potential Risks: Probably minimal. Like EFS, FSx exists inside the VPC and is wrapped by security groups. The FS has ACL support. The service has KMS & CloudTrail support and has industry certifications.

Amazon Quantum Ledger Database (QLDB)

Amazon QLDB is a fully managed ledger database that provides a transparent, immutable, and cryptographically verifiable transaction log ‎owned by a central trusted authority. Amazon QLDB tracks each and every application data change and maintains a complete and verifiable history of changes over time.

Status: currently in private preview. Few details are available

Open Questions:

  • Does this live in a VPC, or does it have a public endpoint?
  • How does authentication work? Does it leverage IAM like Aurora?
  • Does it support Resource Policies, can the DB be made public?

AWS Managed Blockchain

Amazon Managed Blockchain is a fully managed service that makes it easy to create and manage scalable blockchain networks using the popular open source frameworks Hyperledger Fabric and Ethereum

Status: Currently in Preview

Open Questions:

  • Really? FFS Why?
  • Does this live in a VPC, or does it have a public endpoint?
  • How does authentication work? Does it leverage IAM like Aurora?

Amazon Timestream

Amazon Timestream is a purpose-built time series database service for collecting, storing, and processing time-series data such as server and network logs, sensor data, and industrial telemetry data for IoT and operational applications.

Status: Preview

Potential Risks: as a serverless application, this probably has a public endpoint. The potential for exposed databases via resource polices exists. Additional documentation from the AWS product team will be required.

Transit Gateway

AWS Transit Gateway is a new service that enables customers to connect thousands of Amazon Virtual Private Clouds (VPCs) and their on-premises networks using a single gateway.

Status: Generally Available

Potential Risks: The generally isolated nature of VPCs provides us a security advantage. By simplifying the process of interconnecting networks, the temptation will be to dissolve the network segmentation that currently exists to bypass limits of DirectConnect or performance and latency issues. Transit Gateways can be shared across accounts (via an invitation mechanism). There is no capability in the current architecture to insert traffic inspection (in-line or as a tap) as traffic traverses the Transit Gateway.

AWS Control Tower

AWS Control Tower automates the set-up of a baseline environment, or landing zone, that is a secure, well-architected multi-account AWS environment. The configuration of the landing zone is based on best practices that have been established by working with thousands of enterprise customers to create a secure environment that makes it easier to govern AWS workloads with rules for security, operations, and compliance.

Status: Preview

Potential Risks: None

Analysis: This service is designed for a greenfield deployment of a new enterprise. It was described as “Highly Opinionated AWS”. While there could be potential for this as a governance strategy, the initial offering likely will not be useful to anyone already established in AWS.

Cross Account EFS

You can now connect to an Amazon EFS file system from EC2 instances in a different AWS account or Amazon Virtual Private Cloud (VPC).

Status: Generally Available

Potential Risks: EFS cloud be shared with an external entity that doesn’t meet our Security Standards.

Security Hub

AWS Security Hub gives you a comprehensive view of your high-priority security alerts and compliance status across AWS accounts. With Security Hub, you now have a single place that aggregates, organizes, and prioritizes your security alerts, or findings, from multiple AWS services, such as Amazon GuardDuty, Amazon Inspector, and Amazon Macie, as well as from AWS Partner solutions.

Status: Preview

Potential Risks: None

Overall Impression: Good first start. The tool should give smaller organizations the often desired single-pane-of-glass. From an enterprise perspective, we’re dealing with on-prem and multiple cloud providers. So while Security Hub would provide for a good pane-of-glass for AWS Infrastructure issues (in a single region), it doesn’t encompass all the variables our Security team is watching. I look forward to kicking the tires once they support customized Standards and I can push my own findings. At that point Security Hub will be a great tool for communicating security issues down to the technical teams that own each AWS account.

Security Hub also requires AWS Config to be enabled. At the scale I’m working at AWS Config charges are non-trivial and I need to really think about the value proposition of Config before I turn it on at work. That however, may be the topic of my next post. This post has already taken me three weeks to write, so it’s time to hit publish and go build some Legos.