How to organize projects in AWS?

Question

In our team, we are using AWS as our main cloud provider and currently, we have 3 projects hosted on their platform.

We are about to have 2 more projects in the next weeks, but first, we want to organize our projects, because our current organization is a little bit disordered.

We want our projects to be organized following these rules:

• Each project must have a staging and production environment.
• Each project is independent of each other so that it is not possible to see the resources of a project from within another project, i.e., VPC and S3 Buckets.
• The client is responsible for paying the bills of the project (staging and production environment).
• Even though the client is responsible for paying the bills, we must have access to the environments to deploy our code and to do other tasks related to development, testing, and operations.
• We can assign a team of developers to each project. It should be possible for a developer to be in one or more projects at the same time. Plus, it should be possible to move our developers between projects and to remove their access from a project.

So, is it possible to organize projects in AWS under the rules previously mentioned? If so, what are good resources to learn how to do this? If not, what cloud providers allow to organize projects the way we want?

Thanks for your attention and time. I'm looking forward to your replies.

Answer 1

AWS Organizations is the standard approach to managing a hierarchy of accounts. However, it does not allow billing to be consolidated at arbitrary points in the organization; it's a tree, not a graph.

Given that, here is what I would recommend:

1. Adopt an "infrastructure as code" tool to manage your users and permissions.

   This is the most important thing that you can do, as it lets you define infrastructure in a declarative form. Terraform is pretty much the standard here.

2. Each of your clients creates an organization, with separate sub-accounts for dev, staging, and production.

   This is necessary to ensure that they are billed for resource usage. You could alternatively manage all of the accounts yourself, and bill them for usage. However, they may feel more comfortable if they're in control.

   You want separate accounts for dev/staging/production because they provide a hard barrier to access. If you have the ability to shut down RDS instances in the dev environment, that does not grant you the same ability in the prod environment (some companies rely on VPCs to isolate environments, but it's a lot more work to construct IAM policies that follow those boundaries).

3. Create an account for your development team.

   This is where you define your developers. You'll also define (at least) one group per client environment, to which you give the ability to assume the client-specific roles.

4. For each client account, create roles that allow access to certain operations.

   For example, you might have one role that grants all access needed to deploy an application, another that grants the access needed to undeploy the application, and a third that provides read-only access for monitoring. This is where using a tool like Terraform really pays off: you can define a single set of roles that you apply to every client and every account within that client.

   Note that you will need to include a trust policy in each of these roles that let users from your "developer" account assume the role.

   You also need to define the roles that each application needs to execute (for example, access to specific SQS queues).

5. Your developers assume a role to access client environments.

   They can do this from the AWS Console or from the command-line using aws sts assume-role. The latter will provide you with temporary access credentials that you use instead of the developer's own credentials.