OpenTofu lets you declaratively manage your cloud infrastructure.
Go to file
Chris Marchesi 3fa11e456b
helper/schema: Clear existing map/set/list contents before overwriting
There are situations where one may need to write to a set, list, or map
more than once per single TF operation (apply/refresh/etc). In these
cases, further writes using Set (example: d.Set("some_set", newSet))
currently create unstable results in the set writer (the name of the
writer layer that holds the data set by these calls) because old keys
are not being cleared out first.

This bug is most visible when using sets. Example: First write to set
writes elements that have been hashed at 10 and 20, and the second write
writes elements that have been hashed at 30 and 40. While the set length
has been correctly set at 2, since a set is basically a map (as is the
entire map writer) and map results are non-deterministic, reads to this
set will now deliver unstable results in a random but predictable
fashion as the map results are delivered to the caller non-deterministic
- sometimes you may correctly get 30 and 40, but sometimes you may get
10 and 20, or even 10 and 30, etc.

This problem propagates to state which is even more damaging as unstable
results are set to state where they become part of the permanent data
set going forward.

The problem also applies to lists and maps. This is probably more of an
issue with maps as a map can contain any key/value combination and hence
there is no predictable pattern where keys would be overwritten with
default or zero values. This is contrary to complex lists, which has
this problem as well, but since lists are deterministic and the length
of a list properly gets updated during the overwrite, the problem is
masked by the fact that a read will only read to the boundary of the
list, skipping any bad data that may still be available due past the
list boundary.

This update clears the child contents of any set, list, or map before
beginning a new write to address this issue. Tests are included for all
three data types.
2017-11-05 12:04:23 -08:00
.github provider/terraform: reorganize for merge into core 2017-11-02 10:46:31 -07:00
backend Migrate Manta Remote state to be a backend 2017-10-30 18:36:50 +02:00
builtin provider/terraform: import terraform provider back into core 2017-11-02 10:48:20 -07:00
command Merge #16543: Bring the "terraform" provider back into core 2017-11-03 12:08:31 -07:00
communicator communicator/winrm: pass cacert option correctly 2017-10-23 13:28:41 -07:00
config update from review 2017-11-03 10:50:19 -04:00
contrib Autoload only .auto.tfvars files 2017-07-05 17:24:17 -07:00
dag faster DAG transitive reduction 2017-10-03 11:24:59 -04:00
digraph Fix TestWriteDot random order error 2014-07-29 10:26:50 -07:00
docs mons-months: fix typo in maintainer-etiquette 2017-09-25 17:29:19 +02:00
e2e e2e: allow tests to set environment variables for command runs 2017-09-28 14:35:51 -07:00
examples examples: remove openstack example 2017-07-24 16:02:13 -07:00
flatmap flatmap: be resilient to lying "foo.#" key 2017-06-23 14:47:36 -07:00
helper helper/schema: Clear existing map/set/list contents before overwriting 2017-11-05 12:04:23 -08:00
moduledeps plugin/discovery: PluginRequirements can specify SHA256 digests 2017-06-09 14:03:59 -07:00
plugin Use pooled http client for fetching providers 2017-10-31 10:53:42 -04:00
registry remove the registryDetector 2017-10-27 09:08:15 -04:00
repl terraform: improve error messages to assist REPL 2016-11-13 23:17:04 -08:00
scripts provider/terraform: reorganize for merge into core 2017-11-02 10:46:31 -07:00
state Migrate Manta Remote state to be a backend 2017-10-30 18:36:50 +02:00
svchost svchost/disco: allow overriding discovery for a particular hostname 2017-10-26 08:58:52 -07:00
terraform core: test correct behavior of plan+apply with unstable values 2017-11-03 16:11:13 -07:00
test-fixtures main: allow overriding host-based discovery in CLI config 2017-10-26 08:58:52 -07:00
tfdiags tfdiags: show descriptions in diagnosticsAsError 2017-10-16 17:53:06 -07:00
tools/terraform-bundle tools/terraform-bundle: Add e2e tests 2017-08-16 18:20:14 +02:00
vendor provider/terraform: reorganize for merge into core 2017-11-02 10:46:31 -07:00
version release: clean up after v0.11.0-beta1 2017-11-03 23:51:47 +00:00
website Merge #16543: Bring the "terraform" provider back into core 2017-11-03 12:08:31 -07:00
.gitignore provider/terraform: reorganize for merge into core 2017-11-02 10:46:31 -07:00
.travis.yml provider/terraform: reorganize for merge into core 2017-11-02 10:46:31 -07:00
BUILDING.md Makefile/docs: Lock in 1.6 req, doc vendored deps 2016-02-24 16:13:49 -06:00
CHANGELOG.md release: clean up after v0.11.0-beta1 2017-11-03 23:51:47 +00:00
checkpoint.go fixing version numbers RCs should be labeled x.x.x-rcx 2015-02-07 16:56:56 +01:00
commands.go command: Allow TF_DATA_DIR env var to override data directory 2017-11-01 16:55:23 -07:00
config_test.go main: allow overriding host-based discovery in CLI config 2017-10-26 08:58:52 -07:00
config_unix.go prevent log output during init 2017-06-12 15:05:59 -04:00
config_windows.go config looks in a plugin directory if it exists 2014-09-27 12:36:13 -07:00
config.go main: allow overriding host-based discovery in CLI config 2017-10-26 08:58:52 -07:00
Dockerfile build: Stop using deprecated MAINTAINER in Dockerfile 2017-10-27 17:25:44 -07:00
help.go Remind future maintainers to update the docs when changing CLI usage 2016-11-24 09:22:18 -08:00
LICENSE provider/terraform: reorganize for merge into core 2017-11-02 10:46:31 -07:00
main_test.go main: make configuration available when initializing commands 2017-09-29 14:03:09 -07:00
main.go provider/terraform: reorganize for merge into core 2017-11-02 10:46:31 -07:00
Makefile build: run end-to-end tests during Travis run 2017-09-28 14:35:51 -07:00
panic.go panic: Instruct the user to include terraform's version for bug reports. 2015-05-14 18:14:56 -04:00
plugins.go keep .terraform.d/plugins for discovery 2017-08-09 17:46:49 -04:00
README.md provider/terraform: reorganize for merge into core 2017-11-02 10:46:31 -07:00
signal_unix.go Forward SIGTERM and handle that as an interrupt 2016-12-08 12:20:25 -05:00
signal_windows.go Forward SIGTERM and handle that as an interrupt 2016-12-08 12:20:25 -05:00
synchronized_writers.go main: synchronize writes to VT100-faker on Windows 2017-05-04 15:36:51 -07:00
Vagrantfile Start building with Terraform 1.9 2017-08-28 09:59:30 -07:00
version.go use the new version package 2017-10-19 21:48:08 -04:00

Terraform

Terraform

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.

The key features of Terraform are:

  • Infrastructure as Code: Infrastructure is described using a high-level configuration syntax. This allows a blueprint of your datacenter to be versioned and treated as you would any other code. Additionally, infrastructure can be shared and re-used.

  • Execution Plans: Terraform has a "planning" step where it generates an execution plan. The execution plan shows what Terraform will do when you call apply. This lets you avoid any surprises when Terraform manipulates infrastructure.

  • Resource Graph: Terraform builds a graph of all your resources, and parallelizes the creation and modification of any non-dependent resources. Because of this, Terraform builds infrastructure as efficiently as possible, and operators get insight into dependencies in their infrastructure.

  • Change Automation: Complex changesets can be applied to your infrastructure with minimal human interaction. With the previously mentioned execution plan and resource graph, you know exactly what Terraform will change and in what order, avoiding many possible human errors.

For more information, see the introduction section of the Terraform website.

Getting Started & Documentation

If you're new to Terraform and want to get started creating infrastructure, please checkout our Getting Started guide, available on the Terraform website.

All documentation is available on the Terraform website:

Developing Terraform

If you wish to work on Terraform itself or any of its built-in providers, you'll first need Go installed on your machine (version 1.9+ is required). Alternatively, you can use the Vagrantfile in the root of this repo to stand up a virtual machine with the appropriate dev tooling already set up for you.

This repository contains only Terraform core, which includes the command line interface and the main graph engine. Providers are implemented as plugins that each have their own repository in the terraform-providers organization on GitHub. Instructions for developing each provider are in the associated README file. For more information, see the provider development overview.

For local development of Terraform core, first make sure Go is properly installed and that a GOPATH has been set. You will also need to add $GOPATH/bin to your $PATH.

Next, using Git, clone this repository into $GOPATH/src/github.com/hashicorp/terraform. All the necessary dependencies are either vendored or automatically installed, so you just need to type make. This will compile the code and then run the tests. If this exits with exit status 0, then everything is working!

$ cd "$GOPATH/src/github.com/hashicorp/terraform"
$ make

To compile a development version of Terraform and the built-in plugins, run make dev. This will build everything using gox and put Terraform binaries in the bin and $GOPATH/bin folders:

$ make dev
...
$ bin/terraform
...

If you're developing a specific package, you can run tests for just that package by specifying the TEST variable. For example below, onlyterraform package tests will be run.

$ make test TEST=./terraform
...

If you're working on a specific provider which has not been separated into an individual repository and only wish to rebuild that provider, you can use the plugin-dev target. For example, to build only the Test provider:

$ make plugin-dev PLUGIN=provider-test

Dependencies

Terraform stores its dependencies under vendor/, which Go 1.6+ will automatically recognize and load. We use govendor to manage the vendored dependencies.

If you're developing Terraform, there are a few tasks you might need to perform.

Adding a dependency

If you're adding a dependency, you'll need to vendor it in the same Pull Request as the code that depends on it. You should do this in a separate commit from your code, as makes PR review easier and Git history simpler to read in the future.

To add a dependency:

Assuming your work is on a branch called my-feature-branch, the steps look like this:

  1. Add the new package to your GOPATH:

    go get github.com/hashicorp/my-project
    
  2. Add the new package to your vendor/ directory:

    govendor add github.com/hashicorp/my-project/package
    
  3. Review the changes in git and commit them.

Updating a dependency

To update a dependency:

  1. Fetch the dependency:

    govendor fetch github.com/hashicorp/my-project
    
  2. Review the changes in git and commit them.

Acceptance Tests

Terraform has a comprehensive acceptance test suite covering the built-in providers. Our Contributing Guide includes details about how and when to write and run acceptance tests in order to help contributions get accepted quickly.

Cross Compilation and Building for Distribution

If you wish to cross-compile Terraform for another architecture, you can set the XC_OS and XC_ARCH environment variables to values representing the target operating system and architecture before calling make. The output is placed in the pkg subdirectory tree both expanded in a directory representing the OS/architecture combination and as a ZIP archive.

For example, to compile 64-bit Linux binaries on Mac OS X, you can run:

$ XC_OS=linux XC_ARCH=amd64 make bin
...
$ file pkg/linux_amd64/terraform
terraform: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped

XC_OS and XC_ARCH can be space separated lists representing different combinations of operating system and architecture. For example, to compile for both Linux and Mac OS X, targeting both 32- and 64-bit architectures, you can run:

$ XC_OS="linux darwin" XC_ARCH="386 amd64" make bin
...
$ tree ./pkg/ -P "terraform|*.zip"
./pkg/
├── darwin_386
│   └── terraform
├── darwin_386.zip
├── darwin_amd64
│   └── terraform
├── darwin_amd64.zip
├── linux_386
│   └── terraform
├── linux_386.zip
├── linux_amd64
│   └── terraform
└── linux_amd64.zip

4 directories, 8 files

Note: Cross-compilation uses gox, which requires toolchains to be built with versions of Go prior to 1.5. In order to successfully cross-compile with older versions of Go, you will need to run gox -build-toolchain before running the commands detailed above.

Docker

When using docker you don't need to have any of the Go development tools installed and you can clone terraform to any location on disk (doesn't have to be in your $GOPATH). This is useful for users who want to build master or a specific branch for testing without setting up a proper Go environment.

For example, run the following command to build terraform in a linux-based container for macOS.

docker run --rm -v $(pwd):/go/src/github.com/hashicorp/terraform -w /go/src/github.com/hashicorp/terraform -e XC_OS=darwin -e XC_ARCH=amd64 golang:latest bash -c "apt-get update && apt-get install -y zip && make bin"

License

FOSSA Status