Why Create Autoschematic?
There are many tools for managing cloud infrastructure declaratively. What makes Autoschematic special?
Tools like Cloudformation, Terraform, Pulumi, Bicep, etc are all well-regarded infrastructure-as-code (IaC) frameworks in this domain. However, Autoschematic was developed in response to the way that we saw companies - big and small - fail to properly adopt these frameworks, or misuse them.
First, these companies had a lot of existing infrastructure that was created manually - such as through the command line, or the web console, as described in the previous section. This means that they couldn't easily use current IaC frameworks to manage that infrastructure. They had to rewrite and port it over piece-by-piece!
Second, they had multiple teams all managing different parts of their infrastructure, with some overlap - and other internal users that relied on those teams to provision or reconfigure that infrastructure in order to get their job done. There was no time to pause and get the whole team working on porting everything to Terraform.
Third, because not everyone was proficient with IaC, the infrastructure that was managed with IaC was often manually modified with the web console, meaning that the platform engineers couldn't trust the IaC to represent the real desired state! This problem is called state drift.
Without getting bogged down into technical details, let's try and state the problem more precisely.
We have a business of some kind, and that business uses "The Cloud" to do... something. To fulfill some business need. This business retains engineers that will operate and configure cloud resources in order to fulfill their business needs.
As those business needs change, the engineers are given work to do. Some element of the company's day-to-day operations has changed, and therefore, this boils down to some specific change that needs to be made to their infrastructure.
For example:
- Michael has retired, and his account needs to be deactivated
- We've just opened a new branch office in Tokyo, and we need to set up a storage bucket in that region for them to use
- After our successful ad campaign, our servers are overloaded, and we need to raise our cluster's scaling limit to serve our increased user base
Some change has arisen in the needs of the organization, and so a cloud infrastructure engineer is tasked with translating that change in need to some concrete change in the infrastructure.
They must be able to reason along these lines:
- The infrastructure is currently in state A.
- To fulfill this new requirement, we need to be in state B.
- What are the precise steps needed to go from state A to state B?
But hold on - remember that problem of state drift we mentioned? That begins to pose a real issue for this . If the engineer cannot reason about the current state of the infrastructure, they can make fatal errors. If you execute a series of steps to go from A to B, but in reality, you are in state C to start with, you are not likely to end up in state B. We're getting a little too abstract here, so let's move on to the next section, where we'll look at a real example of infrastructure state drift causing real operational problems for a business.