Author: Marc LeBlanc


What is Terraform Cloud

Terraform Cloud (and its self-hosted Enterprise counter-part), is a SaaS solution which provides a platform to manage your Terraform projects through a single control plane. With functionality including Workspaces, local and remote execution, VCS integration, private module registry, it has become part of my everyday workflow. Organizations will find it gives DevOps teams a great platform to collaborate on - team-based permissions, sentinel policies, cost estimation, SSO integration and more.

It has an nice intuitive GUI front-end (we’ll come back to that) where you will find you can do anything you could do from the Terraform cli and then some.

The API

If you’re like me, while I appreciate a nice GUI, I also loathe logging in and using them. Hashicorp has done right by many a DevOps’r with their implementation of a full API for Terraform Cloud. Now, I do believe that using the GUI will help you learn the tool and become familiar with its capabilities, but once you do, you’ll find you can fly with speed using the API.

The more I used Terraform Cloud, I found there were a few tasks I really wished were quicker to take care of - creating workspaces, listing workspaces when I forgot what I named something, setting variables on the workspace, and a couple of others. Using the GUI was fine, but I wanted something faster. So I crafted a few scripts to help which I’d like to share. These scripts should help you do some common tasks, and perhaps more importantly, figure out how to use the Terraform Cloud API to solve more complex problems. There are samples of scripts written in Bash and Python. So let’s take a look.

Workspaces

I started finding myself wanting to do things quickly, such as list workspaces, create them, create and attach them to a GitHub Repository and define a working directory within the repository. The Terraform Cloud Workspaces API offers methods to handle all this nicely.

Let’s say I want to create a workspace for a development GKE environment based on a standardized deployment as defined in a GitHub repository (read about setting up standardized repositories and workspaces here). The one trick to attaching to a VCS repo is that you need the OAUTH token associated with the connection. In a pipeline scenario there are probably better mechanisms to handle retrieving this, but for this purpose, I’ve fashioned a helper script to grab it.

python fetch_oauth.py
🔧 Fetching VCS and oauth information ...
✅ Pulled VCS information for leblanchq creation completed...
VCS Connection Information
---------------
Name: GitHub - Arctiq
ID: oc-eSxv09876543r5
Token ot-1234567809h2   <----- you need this

Now we can go ahead and create the workspace:

python workspaces.py create -w acme_manufactoring_widgets \
                            --repo marc-leblanc/gcp-demo-org \
                            --working-dir lob/widgets \
                            --oauth ot-1234567809h2
🔧 Creating Terraform Workspace acme_manufactoring_widgets...
✅ Workspace acme_manufactoring_widgets creation completed...

This command just created a workspace, attached a VCS repo and specified the working directory within the repo. Neat, but the next script is really what drove me to go down this path. Variables. Sure, you can set hundreds of variables through the GUI if you want to, clicking buttons, pasting values, typing variable names, it is a perfectly valid workflow. Thankfully, the variables API endpoint was created clearly with my sanity in mind.

Variables

Using a CSV format, we can set variables on the workspace, mark them as sensitive or not, set them as terraform or environment variables, making it much more efficient to set large numbers of variables.

python set_vars.py -w acme_manufactoring_widgets --file widget_vars.csv
🔧 Setting variables on acme_manufactoring_widgets ...
✅ Got ID for workspace acme_manufactoring_widgets
Setting variable widget_network....
✅ OK
Setting variable widget_compute_spec....
✅ OK
Setting variable widget_special_var....
✅ OK
Setting variable widget_token....
✅ OK

Additional Functionality/Ideas

A few other nuggets of functionality I added because I wanted it for some reason or another:

List Workspaces

+-----------------------------+--------------------------------+------------------------------+--------------------------+
| Workspace Name              | VCS Repo                       | Working Directory            | Date Created             |
+-----------------------------+--------------------------------+------------------------------+--------------------------+
| acme_manufactoring_widgets  | marc-leblanc/gcp-demo-org      | lob/widgets                  | 2020-10-13T14:09:25.528Z |
| my_test2                    |                                | None                         | 2020-09-11T00:20:35.278Z |
+-----------------------------+--------------------------------+------------------------------+--------------------------+

List Workspace Runs

python workspaces.py runs -w backstage-nginx-nap-k8s --list
+----------------------+----------------------+------------------------------------------------------+--------------------------+
| Run ID               | Status               | Message                                              | Date Created             |
+----------------------+----------------------+------------------------------------------------------+--------------------------+
| run-79Cb3PoDUrwxjFGT | applied              | Queued manually to destroy infrastructure            | 2020-09-09T18:17:14.732Z |
| run-614EytdHqBsrbXr4 | planned_and_finished | Merge pull request #29 from marc-leblanc/app-protect | 2020-08-21T19:48:50.751Z |
|                      |                      |                                                      |                          |
|                      |                      | App protect                                          |                          |
| run-fCvDYs33vK8e1sJQ | applied              | Merge pull request #28 from marc-leblanc/app-protect | 2020-08-20T17:26:36.157Z |
|                      |                      |                                                      |                          |
|                      |                      | App protect                                          |                          |
| run-g8Pq8yEszFSHEZPX | applied              | Queued manually in Terraform Cloud                   | 2020-08-18T23:12:14.497Z |
| run-ixA8bST62qT4oeGj | errored              | Queued manually in Terraform Cloud                   | 2020-08-18T23:11:31.355Z |
+----------------------+----------------------+------------------------------------------------------+--------------------------+

Conclusion

In a nutshell, that’s kind of where these scripts are for now. Helping with some basic functionality and keeping me focused on writing Terraform and testing as opposed to logging into GUI’s configuring things. I personally find this work pattern much more efficient, and it gives some insight around how the Terraform Cloud product could tie into CIDC pipelines. You can find these scripts here. There are a few examples of working in bash and all of the examples shown here in python. There is a bit of set up in the way of environment variables the scripts look for, but that is documented in the repo.

Hopefully this gives you some insight on how you can leverage the Terraform Cloud API for your workflows and pipelines!

Tagged:



//comments


//blog search


//other topics