IAM

The AWS Identity and Access Management service is the place in which we manage users and access controls within AWS. IAM works on the principle of defining who can access what and under which circumstances.

The who refers to individual actors in your organisation. The most obvious type of actor is a user representing yourself or one of your colleagues. Other types of actors are CI pipelines, microservices, and lambda functions.

The what will typically mean some technical resource such as a database, kafka cluster, or load balancer. It can also refer to things that are not directly technical components, such as billing information or logs.

Can access means specific privileges and permissions which have been assigned to an actor, allowing them to observe or modify resources in the organisation. These permissions can be applied on different levels, for example as part of an assumable role or directly on a group of users. In this blog we will show you examples of both of these permission models.

Base Account

We have created a repository which we refer to as base account, which you can find at infrablocks-example/aws-base-account. The base account is called just that because it serves as the base for each of our sub-accounts, leveraging three different InfraBlocks modules to create:

• An admin user
• Account defaults
• Assumable roles

The purpose of the admin user is to bridge the gap between the root user and the individual users which we will provision later. You can technically skip the admin user but depending on how long it will take for you to set up users and roles you may end up relying on your root user for too long, and thus increasing your exposure. We therefore provide an admin user at this early stage which has the same permissions as the root user. Unlike the root user it can be removed or otherwise limited at any time in the event that it becomes compromised.

The account defaults are a small set of reasonable default configurations which we want to apply to each of our organisational units, namely an account alias and a minimum password length policy for our users.

Finally, we have a set of roles which future users will be able to assume, depending on their specific privileges. For illustrative purposes we have taken a naive approach of providing only two roles: a read-only role and an admin role. The admin role has the same privileges as the admin user and the read-only role can inspect all resources but is not allowed to make any changes. In a real-world example you will quickly outgrow this setup and define more granular roles that are scoped to the context of the actions you want to take.

Secrets

The base account repository is one of several in which we will be using Terraform to create new users with passwords immediately assigned to them. In order for Terraform to be able to output the passwords for us in a secure manner, we need to provide a GPG key for it to encrypt the output, and a corresponding private key for you to use to decrypt the password.

In our InfraBlocks repositories we frequently rely on git-crypt as a simple way of storing secrets. As this is an example repository it already has git-crypt with an encrypted GPG key in the secrets folder. This will not be accessible to you since you don't have the required credentials to unlock the secrets folder, so you will need to remove the existing git-crypt configuration and secrets if you fork this repository. If you wish to store your own GPG key in your git repository you will first need to initialise git crypt yourself.

When you're ready to generate the GPG key, simply run the corresponding rake task:

go gpg_keys:admin:generate

This will generate the public/private key pair.

The repository is already configured to look for the GPG key in the correct spot. If you will not be storing the GPG key in the repository you will need to update the path for the key in the defaults.yaml.

Base Account Configuration

Before you can provision the base account repository you'll need to replace the relevant configuration parameters. In the defaults.yaml, replace the development-group with the ORGANISATION_IDENTIFIER you chose in Part 2, replace example-product in the component with the name of your product, and replace the account ids with the ids of the organisational units that we created in Part 2. You can find them by navigating to the organisation panel in AWS.

Since we will be provisioning the base-account for each sub-account, we need to add the root user for each sub-account to our aws-vault config and initialise the profiles before we can use it. This is the same process as in Part 2, but for completeness let's look at the development sub-account as an example. The steps are as follows:

• Add [profile {ORGANISATION_IDENTIFIER}-development-root-user] to your .aws/config file, configured with your own organisation identifier.
• Go to the AWS console, log in as the development root user and create an Access Key ID and Secret Access Key pair.
• Run aws-vault add {ORGANISATION_IDENTIFIER}-development-root-user to add the development root user to AWS vault.

Assuming that everything worked as expected, simply repeat the process for the remaining accounts until you have all of them set up in AWS vault.

Base Account Provisioning

Now it's time for us to provision the base account. Continuing on our development account example, run the following rake task:

aws-vault exec \
    {ORGANISATION_IDENTIFIER}-development-root-user --no-session -- \
    go "common:provision[{ORGANISATION_IDENTIFIER},development,molybdenum]"

You may be wondering what the metal name molybdenum has to do with anything at this point. This is a random identifier for our development configuration, such that if you ever wanted to spin up a second development account in parallel with your original one, you could do so under a different name and configuration.

Once again, repeat the process for all remaining sub-accounts. For every account you provision, the task will print the Terraform outputs, which includes the encrypted admin secrets. Use the private GPG key which we generated to decrypt the secrets and store them in a secure location.

Root Account

As I mentioned in the beginning, we will give you two examples of access control models in this article. The first one is to create a group in which users can be added. Policies can subsequently be applied to that group such that they automatically give permissions to each user assigned to the group. This type of access control is useful for generic permissions that we are okay with certain users leveraging all the time. In this example we will create a group for our developers and apply a policy which allows them to inspect the organisation's billing information.

The second access control model is that we will give users the right to assume the read-only and admin roles which we assigned for each sub-account in the previous section. As an example, if a developer is investigating an end-user issue, they can assume the production read-only role which will allow them to inspect CloudWatch logs in the production account. Similarly, they can assume the management admin-role if they need to update a CI pipeline in the management account, and so on. This type of access control model is more sophisticated. Access can be restricted to only when needed and it's easier to audit when and how specific permissions were leveraged.

The root account repository sets us up for trying out both access controls models, by creating:

• Personalised users
• Groups which the users can be assigned to
• Policies which are assigned to the groups

Root Account Configuration

Our example root account repository can be found at infrablocks-example/aws-root-account. As always, we start by configuring the repository to our organisation. First off, head over to the ibe-root-default.yaml. At the top you will find a list of users which you'll need to replace with the users in your own organisation. Next, we have a list of groups, with their associated users, policies and assumable roles. You can create any number of different groups depending on the needs of your organisation. Each of the users in your organisation needs to create a GPG key and have the public key added to the /config/gpg folder. This GPG key is the one which they'll use to decrypt the default passwords which are created for their users. Just like the base account repository, you will also need to configure the development group and the account ids in the defaults.yaml.

Root Account Provisioning

We will start by creating the policies which are required for the access controls. Start by running the following rake task:

aws-vault exec \
    {ORGANISATION_IDENTIFIER}-root-account-root-user --no-session  -- \
    go “policies:provision[{ORGANISATION_IDENTIFIER},root,default]"

Now that we have the policies provisioned, we can create the users and the groups by running this rake task:

aws-vault exec \
    {ORGANISATION_IDENTIFIER}-root-account-root-user --no-session  -- \
    go "access_control:provision[{ORGANISATION_IDENTIFIER},root,default]"

Just like the base account, provisioning the root account will print the Terraform output which includes the encrypted secrets for the users that have been generated. Your colleagues for whom the users have been created will be able to use their private GPG keys to decrypt the passwords and subsequently use them to log in with their newly created users for the first time. The auto-generated passwords are only used for the first login, during which they will be prompted to set a new password.

Multi-factor Authentication

You may have noticed in the previous step that the configuration for our users defines a variable called enforce_mfa. When enabled, this will force all users to set up MFA before being able to take any further actions such as assuming roles. We recommend always enforcing MFA since it adds a significant layer of security to your organisation.

Access Controls in Practice

Let us now try out the two access control models we've set up in practice. Start by logging in to AWS as one of the users that you generated in the previous step. By clicking the top right navigation bar and selecting the Billing Dashboard, we are navigated to the account billing overview. This is a secured resource, but by virtue of the user being added to the developer group, we are given access to this section simply by being logged in. Had the user not been assigned to the developer group, they would not have been allowed to see this information. This is the first of our two access control models in play.

Let us now instead attempt to inspect some logs in CloudWatch. The way you'd normally do this is by navigating to the CloudWatch service and selecting Log groups in the left-hand menu. If we do this now, we are met by the following error message:

The reason for this is that, unlike the billing dashboard, there's no special permissions given by the developer group for us to access CloudWatch log groups. Instead, we have to assume one of the two roles that we created for a particular sub-account. Since we won't be making any changes but simply inspecting data, the read-only role is preferable in this case.

Click the top right navigation bar again, but this time select Switch Role. Here you'll be met by the following page:

The account refers to the id of the sub-account that you want to inspect. For example, if you're inspecting the logs for a production issue, you would use the production account id. The role is simply the name of the desired role, in this case it would be cross-account-read-only-role. Finally, AWS allows you to assign a display name and colour to your role configurations, so we can for example name this one production-read-only, and give it a colour of red since it allows inspection of production data. If we now head back to the CloudWatch log groups, the previous error is gone and we can freely inspect the production logs.

Conclusion

Access control management is critical as it serves as the basis for all security in your organisation. As your product evolves you will find that you will need to add further layers of security on top of what we have configured thus far in this blog. Make sure to stay on top of the latest best practices in AWS security by regularly reviewing the guidelines AWS guidlines

Organisational units

AWS models sub-parts of your organisation as organisational units. The root account is the most critical one as it typically contains things such as billing information. Other than that we rarely interact with the root account and instead operate in specialised sub-units. As part of the example that we’re creating in this series, we will imagine that our organisation is developing one product, aptly referred to as the Example Product. To develop our example product we need a base account for the product and three sub-accounts. The three sub-units will represent the 3 environments we discussed in Part 1:

• Management (for hosting non-specific infrastructure such as CI, VPNs etc)
• Development (for hosting your development environment)
• Production (for hosting your production environment)

As you follow along you are free to add more accounts as you see fit for your organisation. Perhaps you have multiple products that all need their own environments, or perhaps your product needs an additional “Staging/Pre-production” environment.

Before we get into writing some actual code we will have to create the AWS root account. This is a manual step which you can complete by navigating to aws.amazon.com/free. Creating an AWS account and setting up your organisation is free but bear in mind that if you add other components such as databases or compute resources you may begin to incur costs.

When we start to execute command line tasks against AWS we need to generate an Access Key ID and Secret Access Key for our root user. You can do this by navigating to the Access keys section under Security credentials in the AWS console. We recommend storing these fields in a secure password manager for your organisation.

aws-vault

To run any of the tasks that operate on AWS resources we need to run the task in the context of an AWS session using aws-vault. Start by creating a file for your AWS config under ~/.aws/config. The file initially only needs to contain one profile but we will add further ones as we operate on the other sub-accounts. Put the following line in the config file: [profile root-account-root-user]

Lastly, we need to add this profile to AWS vault by running aws-vault add root-account-root-user. You will be prompted to add the Access Key ID and Secret Access Key for the root user that we created in a previous step. After completing this step we are ready to execute our first task against AWS.

Configuring the organisation

To create the rest of our AWS organisation we will leverage the InfraBlocks module named terraform-aws-organization. We have created an example git repository which you can fork for your own setup: infrablocks-example/aws-organisation

To configure it for your own organisation you will need to update some of the configuration files. Throughout the repository you will see references to ibe, which is an abbreviation for InfraBlocks Example. We recommend that you change this to a descriptive identifier for your own organisation. In this repo we need to update 2 sets of configuration, the default region you want to operate in and the root account id in the defaults.yaml, and the emails in the global-default.yaml. Keep in mind that AWS requires that the email for each sub-account is unique.

You can use any pattern you’d like to apply the Terraform configuration in this repository, but we provide convenient Rake tasks that are part of the InfraBlocks eco-system. These Rake tasks abstract away much of the complexity. To list all the available tasks, navigate into the repository and run ./go -T.

Bootstrapping the state bucket

The first task we need to execute is the bootstrap:provision task. This task will create the S3 bucket in which we will store all future Terraform state related to this repository. Since this task will create a resource in AWS, it needs to be run within the environment provided by AWS vault. Here’s the full command, where you will need to substitute your chosen organisation identifier:

aws-vault exec root-account-root-user -- \
  go “bootstrap:provision[{ORGANISATION_IDENTIFIER},global,default]"

We have now created an empty S3 bucket in AWS. Feel free to navigate to S3 in the AWS console to confirm. The bootstrap step is part of every repository we create and is the only step that creates Terraform state which is saved locally in the repository. This file is intended to be checked in with the rest of your code in version control and is simply a reference to an S3 bucket which will contain the rest of the Terraform state for this repository.

Creating the organisation

The next step is to actually create the organisation. Once again we have a Rake task to do this for us:

aws-vault exec root-account-root-user -- \
  go "organization:provision[{ORGANISATION_IDENTIFIER},global,default]”

Congratulations! We have now created our organisation in AWS using an industry standard infrastructure-as-code setup. What this means in practice is that we now have multiple AWS accounts which are hierarchically linked to the root account which we initially created manually. You can confirm that this has worked correctly by navigating to the organisation panel in AWS.

When creating these sub-accounts, each of them has been given it’s own root user based on the emails that we previously configured. Because no password has been given to these users you will need to follow the AWS Forgotten Password reset flow to set the passwords for them. This is unfortunately a manual step which has to be done for each sub-account. Keep in mind that each sub-account has a unique account id which you can find by going to the organisational unit overview in your root account.

The root users are required for completing some of the basic steps in this series but otherwise have access rights which are too broad for general use. In Part 3 of this series we will show you how to create custom users and IAM roles to manage your access controls and the root users should henceforth only be used in case of emergency.

Salutem

In a nutshell, Salutem allows you to define a set of health checks, a registry in which to store them, and a maintainer which ensures that results of health checks are kept up-to-date. Here’s a basic example which monitors www.google.com using an http call:

(require '[salutem.core :as salutem])
(require '[org.httpkit.client :as http])

(defn http-endpoint-check-fn [url]
  (fn [_ callback-fn]
    (http/get url
      (fn [{:keys [status]}]
        (callback-fn
          (if (<= 200 status 399)
            (salutem/healthy)
            (salutem/unhealthy)))))))

(def registry-store
  (atom
    (->
      (salutem/empty-registry)
      (salutem/with-check
        (salutem/background-check :google
          (http-endpoint-check-fn
            "https://www.google.com/"))))))

(def maintenance-pipeline
  (salutem/maintain registry-store))

(salutem/resolve-checks @registry-store)
; => {:google
;     {:status :healthy,
;      :evaluated-at #time/instant"2021-10-05T12:15:52.910167Z"}}

Realtime vs background

You may have noticed in the previous example that we defined the check as a background check. This means that the check is continuously performed in the background. When resolving checks, Salutem serves up the last cached result. Salutem also supports realtime checks which, unlike the background checks, are executed when the checks are being resolved.

With both of these options you are free to configure Salutem to allow the highest degree of optimisation for your use case. The factors to consider when choosing check type are how often checks will be resolved, if you prioritise freshness over resolve time and how much control you want over the frequency of which a dependency is monitored. Keep in mind that you may use different types of checks for different dependencies. To learn more about Salutem, such as how to set timeouts or how to pass results to callback functions, head on over to our Getting Started documentation.

1. Using an off-the-shelf solution which allows you to compose simple components without requiring deep technical expertise (WordPress, SquareSpace, Hubspot, BigCommerce etc)
   Pros: Fast, cheap, simple
2. Building it yourself from scratch
   Pros: Fully customisable, more ability to scale your offering, sometimes the only plausible path if your product is sufficiently complex

Naturally there's a whole slew of products that fall somewhere in between which, to varying degrees, can take on the responsibility of certain aspects of your product. Examples of this include using Stripe for managing your e-commerce payments, Onfido for doing identity checks, or outsourcing your bank ledger to Mambu. For the purpose of this article series, we consider using such products as still falling under option 2, as they typically mean that you're still creating and hosting proprietary software which composes and integrates these capabilities to create a complete digital offering.

The goal of this article series is to help you on your journey of building a digital product from scratch. At Atomic, we've spent the last decade working with various partners to realise their digital product visions. In the process we've established a set of reusable open-source components which allow you to create your product faster, more easily, and more cheaply. We will do our best to explain the essential concepts, as well as guide you in using the components we've created under the InfraBlocks banner. If you've been working as a software engineer (or in a related role) for a meaningful amount of time these terms may be completely obvious to you already, in which case you can hop directly to Part 2 of the series.

Amazon Web Services

If you write proprietary software, such as Java microservices or a JavaScript web app, you need a central place where you conduct the activities associated with creating, running and hosting your software. This includes things such as building and hosting docker images, running software containers, managing hosted databases, creating DNS entries, setting up load balancers and much more. Amazon Web Services (AWS) is a popular cloud provider which allows you to do all of these. AWS is the provider we will be using in this article series as well as the provider that the majority of the components we've built are targeted towards.

• Wikipedia → cloud computing
• AWS → getting started

Terraform

Running software infrastructure with a cloud provider such as AWS is not a simple activity. AWS provides a graphical interface for conducting all activities, but in most cases this is not the preferable approach. Manually creating and updating software infrastructure is highly error prone and difficult to make reproducible. For this reason, most modern organisations will rely on infrastructure-as-code instead, a concept in which you template your infrastructure in dedicated infrastructure files and check them into a version control system such as Git alongside all your other software.

Terraform is arguably the most popular infrastructure-as-code platform out there today, and is the foundation of all the components we've built. Terraform allows you to build your infrastructure using HashiCorp's proprietary configuration language, HCL. Terraform automatically transforms your infrastructure files into the actual physical components that you require, without needing any manual provisioning work. This has a number of critical benefits. Amongst these are that it's simple to make changes, it's easy to keep track of changes, and it's straight-forward to share common patterns through reusable components such as the ones we've created in InfraBlocks.

• Hashicorp → infrastructure as code tutorial

Deployment environments

If you've previously worked in a collaborative software development project you' ve likely heard colleagues refer to different environments. When we talk about environments in the context of software we mean the same/similar software running in separate, isolated systems. The most important environment is typically referred to as the production environment, which is the one your end-users are actually connecting to. Making untested changes directly in the production environment is dangerous because the results can be unpredictable. For this reason, most organisations will have at least one additional environment, often referred to as the development environment. This environment functions more or less exactly the same as the production environment, but is completely isolated from it, such that you can make changes (such as deploying a new version of a software component, updating a database or testing a new integration) without risking degrading the experience of your end-users. When the change is thoroughly tested and you have a high degree of confidence that it works as expected, you can promote the change to your production environment.

At some point many organisations will outgrow their initial 2 environments, and may add arbitrarily many environments to allow for more isolated testing and validation of specific functionality. Some even go so far as creating and tearing down new environments on an ad-hoc basis. Using Terraform makes it very simple to spin up multiple environments and promote changes between them, since Terraform defines configuration in a way that makes it precisely repeatable between environments. This is one of the many benefits of relying on infrastructure-as-code as the foundation for all your work.

• Wikipedia → deployment environment

Continuous integration / delivery / deployment

The family of Continuous X terms can sometimes be hard to keep track of. Nonetheless, it's essential to understand the basics since we'll assume this is how you deliver software. For the purpose of this series, the critical points are that we assume some type of CI tool is used to build and deploy software changes. A CI tool allows you to create build pipelines, in which you define automated steps to perform the discrete activities required to release software. This typically includes (but is not necessarily limited to):

• Build (building a deployable version of your software)
• Test (running an automated test suite on your software)
• Deploy to development (deploying your new software to the development environment)
• Deploy to production (deploying your new software to the production environment)

Most of the time the first three steps are performed automatically whenever any change is pushed to Git (or whichever version control system you use). The final step of promoting the change to production is often a manual step which is performed after sufficient manual testing has been done in the development environment. Naturally, we apply these principles for the components we build under InfraBlocks as well.

• Wikipedia → continuous integration
• Wikipedia → continuous delivery
• Wikipedia → continuous deployment

Rake

The InfraBlocks Terraform components are perfectly valid to use in any Terraform based environment, but InfraBlocks as a platform leverages Rake as the build utility of choice. Rake allows you to write command line tasks using standard Ruby syntax. We find this to generally be preferable over simple bash scripts because it is more composable and testable. We have created a number of Rake based tools as part of the InfraBlocks platform which simplify many standard infrastructure tasks, including interacting with Terraform. In upcoming parts of this series we will be leveraging these tools as we build an example system.

• Rake
• rake_terraform
• InfraBlocks rake tasklibs

Conclusion

In this article we covered some basic concepts which the reader is required to grasp before we move on to actually building something concrete. In the upcoming parts of this series we will create a digital product together, using the open-source components of InfraBlocks.

The resource in question is a comment for a blog article, which contains the actual comment but more importantly references both the original article and the author of the comment through links. If you want to learn more about why this is so beneficial then I highly suggest reading the full article linked above.

{
  "_links": {
    "self": {
      "href": "https://article-service.io/articles/123/comments/456"
    },
    "article": {
      "href": "https://article-service.io/articles/123"
    },
    "author": {
      "href": "https://user-service.io/users/789"
    }
  },
  "comment": "This was a crazy article"
}

Constructing links

An obvious question you may have when looking at this is "how do you construct the links in a consistent manner?". To answer this question lets have a look at how API routes are constructed in bidi, a popular routing library for Clojure. Bidi supports bi-directional URI routing which is critical for our use case.

If you are serving REST resources, you should be providing links to other resources, and without full support for forming URIs from handlers your code will become coupled with your routing. In short, hard-coded URIs will eventually break.

Here's an example of a set of routes defined according to bidi:

(def routes
  [""
   [["/" :index]
    ["/articles" :articles]
    [["/articles/" :article-id]
     [["" :article]
      [["/comments/" :comment-id] :comment]]]]])

As we can see the routes are defined as a pure data structure, nested vectors consisting of patterns and keywords (which typically correspond to some type of HTTP handler defined in a separate data structure). The benefit of defining our routes like this is that we can easily inspect and perform various other transformations on it. Instead of matching a route to a keyword we can match a keyword to a route, allowing us to construct the links in the comment resource that we looked at above.

Let us look at an example. The self-link of the comment looks like this:

https://article-service.io/articles/123/comments/456

There are four components required to construct this link:

• the bidi routes;
• the keyword matching the route;
• the base URL for the service; and
• the article and comment IDs.

Assuming that you have an incoming request for this resource, all of these components are available to you. The bidi routes are defined within your service, the keyword maps directly to the resource, the base URL is part of the incoming request and the IDs will depend on the particular request. They may be provided in the incoming request already, or they will be part of the persisted resource which is being loaded.

Hype

Hype is an open-source library we created to abstract away all the details for generating links from these four components.

(require '[hype.core :as hype])
(require '[ring.mock.request :as ring-mock])

(def request (ring-mock/request "GET" "https://localhost:8080/"))

(hype/absolute-url-for request routes :comment
  {:path-params {:article-id 123
                 :comment-id 456}})
; https://article-service.io/articles/123/comments/456

Hype also supports more sophisticated use cases that often come up when designing APIs, such as generating templated links which allow the client to fill in details themselves.

(hype/absolute-url-for request routes :article
  {:path-template-params       {:article-id :articleID}
   :path-template-param-key-fn clojure.core/identity})
; https://article-service.io/articles/{articleID}

(hype/absolute-url-for request routes :articles
  {:query-template-params [:page :per-page]})
; https://article-service.io/articles{?page,perPage}

Conclusion

RESTful APIs are a staple of the internet but unfortunately many of them suffer from poor adherence to the fundamental principle of offering Hypermedia Controls. I believe that a common cause for this is that the barrier of entry is perceived as too high and simple questions such as the one we asked ourselves in the beginning of this article seem too daunting for the expected reward. With tools such as Hype, hopefully we can lower the barrier of entry and make Hypermedia Controls more accessible. If you want to try out Hype yourself I suggest heading over to the official documentation.