weekly.tf #16 - 0.13.0-rc1, CDK for TF

The march toward version 0.13 of Terraform continues, with less pain than the 0.12 release.

The Terraform 0.13 train continues to roll.

According to this comment, barring any bugs, this will be the last test release before the real thing rolls out.

So far this process seems much smoother than 0.12 was. To be expected give that the scope is much smaller.

A single back-ported bug-fix in this release related to quadratic memory usage when using large collections of resources.

HashiCorp has collaborated with AWS to add support to AWS's CDK for Terraform.

This enables developers to write Python or TypeScript to define infrastructure managed by Terraform (other languages TBD). Like the CDK proper, which works by generating CloudFormation templates the CDK for Terraform generates Terraform code. You can then apply using standard terraform workflows or use the new cdktf commands.

Here's the thing– Developers have been asking for ways to use the Terraform model with general purpose programming languages. I get why it hasn't happened yet. It is very hard to provide the end-to-end experience of Terraform if you don't control the whole stack.

This model of using a general purpose language to generate code might be a good middle ground? Go crazy generating resources, then rely on the control and predictability of Terraform to apply the changes.

Since cost estimation was added to Terraform Cloud it seems that everyone wants to add it.

Here is another way to estimate the cost effects of code changes, this time via a Github Action.

A great list of pain points from someone that obviously sees the value in Terraform.

Refactoring is difficult

Indeed. Great software is not written, it is rewritten. Terraform is sloooowly getting better at this, but about 10% to where it needs to be. All the issues raised here are things my team experiences as well.

Code reuse is limited

Modules are hard to work with– you can't interpolate into module sources, you can't partially apply them (like currying) and generally lead to a lot of verbose boilerplate.

Type system is too rigid

0.12 was a good step forward here, but I really don't understand the behavior around object types. The requirement that all declared fields be mandatory really limits the usefulness here. Being able to declare all the valid fields would be much more useful.

Upstream development: frustrating priorities

I feel this one too, especially with some of the providers. There are PRs that seem to just die a slow death of neglect. But I understand too, these are very popular projects that cover a very complicated domain. It is not easy and I have sympathy for the provider maintainers. Still, room for improvement exists.

Many readers are at this point probably thinking of Pulumi. The author addresses that with

[M]y team took some time in June 2020 to evaluate Pulumi as a replacement for Terraform. It fixes or sidesteps basically all of the issues described above. However, it’s not as polished as Terraform, and using a real programming language fixes many things but it’s not a panacea (surprise!). Even when starting from scratch, rather than thinking about migrating existing resources, I’d still choose Terraform because it’s simply a better piece of software, HCL and all. I hope to discuss this further in a future article.

No free lunch!

AWS has built a number of building blocks for limiting IAM permissions over-exposure (too many people & things having access to too many things).

AWS Access Advisor will supply data on which principals have used which services and when. This is great data, but what do you do with it? How do you tie it into your infrastructure-as-code practice?

AirIAM fills that gap. It will use data from IAM and the Access Advisor to find unused permissions, suggest IAM groups and generate terraform code to implement them.

It seems like using this might require either some manual massaging of the results or refactoring of existing code to align with what it produces. But either way this level of automation can be a huge boost to a security team's ability to implement least-priviledge.

A great new feature in 0.13 which will eliminate a ton of code that everyone has written to manage third-party providers. My team has already published one of ours.