Autoschematic

Autoschematic solves the problem of state drift by adopting a bidirectional state model. Tools such as Terraform and Pulumi are, by comparison, one-directional - you can update the infrastructure to match your code, but you cannot automatically update your code to match your infrastructure.

Let's look at how this could have prevented the incident we discussed previously. As we go, I'll explain some basic design elements that will help you to understand and use Autoschematic.

In Autoschematic, all resources have an address that serves to uniquely identify the real resource. That address maps directly to a file path inside a prefix within the repo. We'll discuss prefixes later, but for now, just think of a prefix as a folder.

So, the file at:

./support/aws/iam/roles/west-europe-support-staff.ron

is an AWS IAM Role in the support prefix named "west-europe-support-staff".

Likewise, the file at:

./support/aws/iam/policies/west-europe-support-db-access.ron

is an AWS IAM Policy in the support prefix named "west-europe-support-db-access".

Wait, .ron? What the heck is .ron?

RON is the Rusty Object Notation. As part of building Autoschematic, we've also built out language server support and other tooling for RON. We've found it to be a great fit.

Let's take a look at the resources at these addresses.

IamPolicy(
    policy_document: {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "rds-db:connect"
                ],
                "Resource": [
                    "arn:aws:rds:eu-west-1:1234567890:db:customerdb-eu-west-*"
                    "arn:aws:rds:eu-west-2:1234567890:db:customerdb-eu-west-*"
                ]
            }
        ]
    },
    tags: {
    },
)

IamRole(
    attached_policies: [
        "arn:aws:iam::1234567890:policy/west-europe-support-db-access"
    ],
    assume_role_policy_document: {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": "sts:AssumeRole",
                "Principal": {
                    "AWS": [
                      "arn:aws:iam::1234567890:role/aws-reserved/sso.amazonaws.com/WestEuropeSupport"
                    ]
                },
            },
        ],
    },
    tags: {},
)

Ah, but you recall - this isn't the true state! Remember, someone has modified it behind our backs, but the engineer doesn't know that yet. Let's see how we can use Autoschematic to detect and resolve this state drift automatically before we proceed with modifying the infrastructure.

First, just to show off a little, we'll use the Autoschematic extension for Visual Studio Code to see how the files above compare with the real state:

Aha! Here we can see precise way our state has drifted, and we can even hit the "Pull Remote State" button at the top-right to set our local configuration to match the real remote state.

This isn't state-drift remediation as most tools implement it, by simply force-applying your existing code. That would still have caused the incident we described.

This is a pull, not a push. When we pull state, we aren't modifying the real infrastructure at all. All we're doing is updating our code to match the real infrastructure state. This is something Terraform users are very familiar with, but they have to do it manually, by hand-modifying the code and running "terraform plan" until the tool shows no changes. This is a huge waste of time.

Once we've pulled state, we're free to move on to the actual task assigned to us: to create a new IAM policy granting access to the customerdb-auxiliary-eu-west-* databases and attach it to our west-europe-support IAM role.

First, we'll create the new policy. Autoschematic provides a helpful command to draft new resources from templates.

[demo@autoschematic.sh]./autoschematic.sh-core-infra% autoschematic create               
Select prefix: main
Select connector: aws/iam
Select object: aws/iam/policies/[policy_name].ron
policy_name: west-europe-aux-db-access
Writing main/aws/iam/policies/west-europe-aux-db-access.ron

...Now we'll just edit it to reflect the actual policy in the ticket:

IamPolicy(
    policy_document: {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "rds-db:connect"
                ],
                "Resource": [
                    "arn:aws:rds:eu-west-1:1234567890:db:customerdb-auxiliary-eu-west-*"
                    "arn:aws:rds:eu-west-2:1234567890:db:customerdb-auxiliary-eu-west-*"
                ]
            }
        ]
    },
    tags: {
    },
)

Now, even though we haven't applied this yet, we can still reference the policy's ARN in other resources. Let's update our IAM role to attach this policy.

IamRole(
    attached_policies: [
        "arn:aws:iam::1234567890:policy/west-europe-support-db-access-extra",
        "arn:aws:iam::1234567890:policy/west-europe-support-db-access",
        "out://aws/iam/policies/west-europe-aux-db-access.ron[arn]"
    ],
    assume_role_policy_document: {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": "sts:AssumeRole",
                "Principal": {
                    "AWS": [
                      "arn:aws:iam::1234567890:role/aws-reserved/sso.amazonaws.com/WestEuropeSupport"
                    ]
                },
            },
        ],
    },
    tags: {},
)

Cool! As you can see, we have a syntax available in the form of "out://{addr}[{key}]". This is designed to support things like dependent resources and resource bundles. Autoschematic will handle creating resources sequentially to resolve their dependencies.

Let's see what that looks like in practice:

[dev@autoschematic.sh]~/prog/autoschematic.sh-core-infra% autoschematic plan
╔══════════════════════════════════════════════════════════════════════════════╗
║ At main/aws/iam/policies/west-europe-aux-db-access.ron:
║  ⟣ Create new IAM policy west-europe-aux-db-access
║ At main/aws/iam/roles/west-europe-support-staff.ron:
║  ⊬ Missing outputs: 
║    aws/iam/policies/west-europe-aux-db-access.ron[arn]
╚══════════════════════════════════════════════════════════════════════════════╝
 ◇ Plan complete.
[dev@autoschematic.sh]~/prog/autoschematic.sh-core-infra% autoschematic apply
╔══════════════════════════════════════════════════════════════════════════════╗
║ At main/aws/iam/policies/west-europe-aux-db-access.ron:
║  ⟣ Create new IAM policy west-europe-aux-db-access
╚══════════════════════════════════════════════════════════════════════════════╝
 ◇ Plan complete.
Type 7309 to execute all of the above actions and commit.
Hit Ctrl-c to cancel.
>7309
╔══════════════════════════════════════════════════════════════════════════════╗
║ At main/aws/iam/policies/west-europe-aux-db-access.ron:
║  ⟖ Created IAM policy `/west-europe-aux-db-access`
╚══════════════════════════════════════════════════════════════════════════════╝

On the next apply, the new policy will be attached, with the ARN from the output supplied as the template's input value.

As you can see, this is very different to the way that Terraform and Pulumi work under the hood. We hope this intro guide has piqued your curiousity. Autoschematic is an open-source project, and will always remain open-source.

Its goal is not to replace Terraform, but rather to complement it in areas where mistakes can be really critical.