AWS Misconfigurations Lead to Exposed Data

Early this week it was discovered that AWS environments had been compromised through exposed .env files. Unit 42 researchers discovered a data extortion campaign that exposed files containing AWS access keys, credentials for databases and social media accounts, API keys for SaaS applications and email services, as well as access tokens for a variety of cloud services.

Around 110,000 domains were affected, leading to the potential exposure of over 90,000 unique environment variables with 7,000 corresponding to cloud services used by organizations.

What does this mean?

If you rely on AWS, now's a great time to ensure your configurations are secure, permissions are right-sized, and you’re monitoring for abnormal activity.

Misconfigurations opened the door to attacks

The threat actors utilized automated tools to check for domains, hacking into exposed .env files that contained critical information. Using an AWS access key, the attackers ran a GetCallerIdentity API call to verify the identity or role assigned to the exposed credential.

Once in the compromised AWS environment, the attackers discovered the exposed AWS Identity and Access Management (IAM) role did not have administrative privileges over all resources. However, it had the permission to create new IAM roles and attach IAM policies to existing ones. This allowed the actors to create a new role called lambda-ex and elevate their privileges to full administrative rights.

AWS Lambda is a serverless computing platform designed to execute user-supplied application code on demand. In this case, hackers used it to deploy a bash script that would scan other domains for exposed .env files, extract credentials from them and upload them to a public S3 bucket they previously compromised.

AWS IAM security best practices 

The accidental exposure of .env or .git files is a known issue that other research has warned about in the past, so it’s not unusual for attackers to run web crawlers looking for such exposed files in the root folder of domains. However, the scale of this operation suggests that AWS misconfigurations remain a widespread issue for many organizations.

As a best practice, AWS recommends that you use AWS Identity and Access Management (IAM) roles instead of IAM users with long-term credentials such as access keys. However, this may not be an option for all organizations.

In the case of using access keys, IAM security best practices recommends regularly rotating your IAM credentials to prevent a compromised set of IAM access keys from accessing components in your AWS account.

The pattern supports deployment in a single account or multiple accounts. If you’re using AWS Organizations, this solution identifies all AWS account IDs within your organization and dynamically scales as accounts are removed or new accounts are created. The centralized AWS Lambda function uses an assumed IAM role to locally run the rotation functions across multiple accounts you select.

• New IAM access keys are generated when existing keys are 90 days old.
• The new access keys are stored as a secret in AWS Secrets Manager. A resource-based policy allows only the specified IAM principal to access and retrieve the secret. If you choose to store keys in the management accounts, the keys for all accounts are stored in the management account.
• The email address assigned to the owner of the AWS accounts where the new access keys were created receives a notification.
• The previous access keys are deactivated at 100 days old, and then deleted at 110 days old.

Lambda functions and Amazon CloudWatch automatically performs these actions. You can then retrieve the new access key pair and replace them in your code or applications. The rotation, deletion, and deactivation periods can be customized.

In addition, our security experts recommend the following tips to help make lateral and permission escalation more difficult for potential attacks:

• SCP enforcing tagging makes it much more difficult for the attacker to automatically create new identities.

An AWS Service Control Policy (SCP) that enforces resource tagging on IAM users and roles can ensure that any IAM users or roles created in your AWS organization must have specific tags. This can be useful for governance, cost allocation, and management purposes.

Here’s an example of how you might structure such a SCP:

{
    "Version": "2012-10-17",
    "Statement": [{
        "Sid": "EnforceTaggingOnIAMUsersAndRoles",
        "Effect": "Deny",
        "Action": ["iam:CreateUser", "iam:CreateRole", "iam:TagUser", "iam:TagRole"],
        "Resource": "",
        "Condition": {
            "StringNotEquals": {
                "aws:RequestTag/Environment": ["Production", "Development", "Staging"],
                "aws:RequestTag/Owner": ["Finance", "HR", "Engineering"]
            }
        }
    }, {
        "Sid": "EnforceTaggingOnTagOperations",
        "Effect": "Deny",
        "Action": ["iam:UntagUser", "iam:UntagRole"],
        "Resource": "",
        "Condition": {
            "ForAnyValue:StringEquals": {
                "aws:TagKeys": ["Environment", "Owner"]
            }
        }
    }]
}

Explanation

1. Sid: EnforceTaggingOnIAMUsersAndRoles: 
    - This statement denies the ability to create IAM users or roles unless the specified tags (`Environment` and `Owner`) are included and have one of the allowed values (like `Production`, `Development`, `Finance`, `Engineering`, etc.). 
   
2. Sid: EnforceTaggingOnTagOperations: 
    - This statement denies the ability to remove the required tags (`Environment` and `Owner`) from IAM users and roles.

Key Points 

aws:RequestTag/TagKey: This condition ensures that when creating or tagging IAM resources, the required tags must be present and have acceptable values. 

ForAnyValue:StringEquals: This condition checks if any of the provided tag keys are being removed during an operation, and if so, denies that action.

Applying the SCP

To apply this SCP, attach it to the appropriate Organizational Unit (OU) or root within AWS Organizations. It will then enforce the tagging policy across all accounts under that OU or root.

• Ensure the access key is part of gitignore and never published within the codebase.
• As part of your CICD, never upload backups of the SCM to S3. In addition, avoid pushing the codebase outside of your designated repository.
• While it’s fine to use non vault storages for environment variables, access keys should be stored within the designated hardened area (vault).

How Varonis can help 

Typically, .env files are located on EC2 instances. However, this isn’t always the case, with our security team even finding them in S3 as well. With Varonis, no matter where your .env files live, our platform can scan and detect secrets across your AWS identity (IAM), storage (S3), database (RDS) and computer (EC2).

Our Privileged Policies Views can help you detect suspicious activity – such as unusual access and escalating privileges as seen in this attack – alerting your user admin immediately of the situation.

In addition, Varonis can help you identify any excessive permissions or toxic setting combinations in your AWS accounts, then automatically fix those misconfigurations, helping you reduce exposure and secure your sensitive assets.