When managing your cloud infrastructure on AWS, it’s important to have a comprehensive understanding of all the resources running in your AWS accounts. It’s crucial to be able to have reliable data and clear insight into areas such as:
Identifying resources that are idle, unmonitored, or exposed to security threats.
Understanding the cost breakdown and coverage of tags.
Keeping up-to-date and accurate audit information.
Evaluating whether your resources conform to specific governance controls.
Detecting any infrastructure drift and changes in configurations.
Without answers to those questions, you open the doors to cost wastage, security threats, and compliance issues.
This blog post provides guidance on the different tools available that can help you in locating and identifying all resources within your AWS account.
AWS Native Services
AWS provides several tools to help you identify and track resources in your account. Each tool has its own pros and cons, as we will see. The key distinction for cloud resource tracking is the scope of the tool and which resources are in its zone. Let’s dive in.
AWS Management Console
For anyone working with AWS, the AWS Management Console is a good place to begin. It provides access to a wide range of services and features. However, the console’s UI can be challenging to navigate, and it can be overwhelming. Additionally, you should have prior knowledge of the resources you are searching for and their location in the region. Otherwise, you may end up spending several hours browsing through multiple tabs and levels of hierarchy in the AWS Console to find answers to questions such as “What is the number of EC2 instances operating in our Frankfurt region?”
AWS Resource Groups
AWS Resource Groups is a better alternative to the AWS console. This service enables you to create a custom group of your resources, based on specific criteria such as tags or the resources in an AWS CloudFormation stack. By organizing and consolidating information in this way, you can easily track the resources used by individuals or application teams.
It’s worth noting that the service doesn’t support all AWS services (AWS services that work with AWS Resource Groups), as it was not specifically designed for resource discovery. Rather, the service is intended to group resources together based on predetermined tags or CF stack. Therefore, it may not be the best option if you’re looking for a tool to build your asset inventory.
AWS Config
AWS Config is a full-fledged asset inventory. It discovers all your running AWS resources and their configuration history as well as the resource relationships (e.g: find out if an EBS volume is attached to an EC2 instance associated with a security group).
The service also provides a rules engine that you can use to evaluate the configuration of resources against pre-defined rules or compliance policies. E.g: you can use SQL queries to find resources that are non-compliant AWS resources and export the results to JSON or CSV format for further benchmarks (e.g: CIS AWS Benchmarks, AWS Foundational Security Best Practices, or PCI DSS)
Despite those features, the AWS Config service does come with certain drawbacks, including:
As of the time of writing, AWS Config does not cover all types of resources (A list of supported services can be found here).
The more configuration items generated, the more expensive the service can become (See pricing).
AWS Config is best suited for AWS resources. Therefore, users operating in multi-cloud environments and organizations seeking configuration visibility for SaaS assets may require additional tools.
The service is not enabled by default, so users need to set it up in all regions for all their AWS accounts. For those with a considerable number of AWS accounts, this can result in significant effort.
AWS Cost Explorer and CloudWatch
It is also a good idea to take a look at Cost explorer once in a while and check whether we are charging our account unnecessarily. Billing information cannot provide a complete picture. But you can use the AWS Cost Explorer to slice your AWS cost by both AWS services, regions, and tags (if enabled). This can give you a starting point of where to further explore manually with AWS Config or Resource Explorer.
You can also leverage AWS CloudWatch to identify which resources are generating metrics so no resource goes untracked.
AWS Resource Explorer
AWS Resource Explorer is a service released last year that allows you to explore and discover the resources in your AWS account. It allows you to view, search, and filter the resources across all regions and services in your AWS account. The service is free of charge, making it a great alternative to other resource discovery mechanisms, such as AWS Config.
Resource Explorer was built with cross-region support from the very beginning. However, the list of resource types that can be discovered with Resource Explorer is quite short and does not support searching across multiple accounts inside an organization (It only works on an AWS account scope).
As such, you may want to consider alternative options that are more user-friendly and offer a more intuitive way to manage your resources on AWS.
Komiser
Komiser is an open-source cloud-agnostic asset inventory. It integrates with multiple cloud providers, builds a cloud asset inventory, and helps you break down your cost at the resource level.
Komiser comes with a resource inventory feature where you can have an active resource inventory of all your cloud resources along with relevant information such as source account, region, cost, and the tags that are applied to it. You can analyze cloud resource utilization and costs based on specific criteria, such as teams, applications, or cost centers. This approach enables the creation of custom views for engineering, finance, and product teams and promotes accountability for cloud expenses.
As you’re moving to a multi-cloud model, you would need a single place where you can manage all your cloud resources. By integrating with several cloud service providers (currently supporting AWS, Azure, Oracle, DigitalOcean, Civo, Tencent Linode, Kubernetes, and Scaleway), Komiser can swiftly generate your cloud asset inventory. This allows you to utilize its powerful filter system to uncover idle resources and wasted costs across all your cloud accounts and regions. Consequently, supported resources have nowhere to hide, and there is no way they will slip under the radar. As soon as the resource inventory is fetched, all regions will show exactly what they are holding. The resources come to you in a sense, so there’s no more tab switching or console hoping to make sure you didn’t miss anything.
Having an asset inventory of your AWS resources is crucial to uncover optimization opportunities and answering questions about your infrastructure. AWS has some good services but as the number of resources increases and you shift toward multi-cloud you might want to check out something like Komiser that does it all.
Regardless if you are a Developer, DevOps, or Cloud engineer. Dealing with the cloud can be tough at times, especially on your own. If you are using Komiser and want to share your thoughts doubts and insights with other cloud practitioners feel free to join our Tailwarden discord server. Where you will find tips, community calls, and much more.