IntenseWebs/opentofu
3
0
Fork 0
mirror of https://github.com/opentofu/opentofu.git synced 2025-02-25 18:45:20 -06:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/master' into consul-force-unlock

Browse Source
This commit is contained in:
Rémi Lapeyre 2020-09-29 10:08:32 +02:00
commit 5c7008bd89
4769 changed files with 13435 additions and 1495132 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.circleci/config.yml
View File

@ -6,7 +6,7 @@ orbs:
executors:
go:
docker:
- image: circleci/golang:1.14
- image: circleci/golang:1.15
environment:
CONSUL_VERSION: 1.7.2
GOMAXPROCS: 4
@ -14,7 +14,6 @@ executors:
GOPROXY: https://proxy.golang.org/
TEST_RESULTS_DIR: &TEST_RESULTS_DIR /tmp/test-results
ARTIFACTS_DIR: &ARTIFACTS_DIR /tmp/artifacts
GOFLAGS: "-mod=vendor"
jobs:
go-checks:
@ -38,6 +37,8 @@ jobs:
go-test:
executor:
name: go
environment:
TF_CONSUL_TEST: 1
parallelism: 4
steps:
- checkout

13
.github/CONTRIBUTING.md vendored
View File

@ -201,7 +201,7 @@ make protobuf
## External Dependencies
Terraform uses Go Modules for dependency management, but currently uses "vendoring" to include copies of all of the external library dependencies in the Terraform repository to allow builds to complete even if third-party dependency sources are unavailable.
Terraform uses Go Modules for dependency management.
Our dependency licensing policy for Terraform excludes proprietary licenses and "copyleft"-style licenses. We accept the common Mozilla Public License v2, MIT License, and BSD licenses. We will consider other open source licenses in similar spirit to those three, but if you plan to include such a dependency in a contribution we'd recommend opening a GitHub issue first to discuss what you intend to implement and what dependencies it will require so that the Terraform team can review the relevant licenses to for whether they meet our licensing needs.
@ -213,24 +213,23 @@ go get github.com/hashicorp/hcl/v2@2.0.0
This command will download the requested version (2.0.0 in the above example) and record that version selection in the `go.mod` file. It will also record checksums for the module in the `go.sum`.
To complete the dependency change, clean up any redundancy in the module metadata files and resynchronize the `vendor` directory with the new package selections by running the following commands:
To complete the dependency change, clean up any redundancy in the module metadata files by running:
```
go mod tidy
go mod vendor
```
To ensure that the vendoring has worked correctly, be sure to run the unit test suite at least once in *vendoring* mode, where Go will use the vendored dependencies to build the test programs:
To ensure that the upgrade has worked correctly, be sure to run the unit test suite at least once:
```
go test -mod=vendor ./...
go test ./...
```
Because dependency changes affect a shared, top-level file, they are more likely than some other change types to become conflicted with other proposed changes during the code review process. For that reason, and to make dependency changes more visible in the change history, we prefer to record dependency changes as separate commits that include only the results of the above commands and the minimal set of changes to Terraform's own code for compatibility with the new version:
```
git add go.mod go.sum vendor
git commit -m "vendor: go get github.com/hashicorp/hcl/v2@2.0.0"
git add go.mod go.sum
git commit -m "go get github.com/hashicorp/hcl/v2@2.0.0"
```
You can then make use of the new or updated dependency in new code added in subsequent commits.

14
.github/ISSUE_TEMPLATE/bug_report.md vendored
View File

@ -10,7 +10,17 @@ Hi there,
Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.
If your issue relates to a specific Terraform provider, please open it in the provider's own repository. The index of providers is at https://github.com/terraform-providers .
If your issue relates to a specific Terraform provider, please open it in the provider's own repository. The index of providers is at https://github.com/terraform-providers.
To fix problems, we need clear reproduction cases - we need to be able to see it happen locally. A reproduction case is ideally something a Terraform Core engineer can git-clone or copy-paste and run immediately, without inventing any details or context.
* A short example can be directly copy-pasteable; longer examples should be in separate git repositories, especially if multiple files are needed
* Please include all needed context. For example, if you figured out that an expression can cause a crash, put the expression in a variable definition or a resource
* Set defaults on (or omit) any variables. The person reproducing it should not need to invent variable settings
* If multiple steps are required, such as running terraform twice, consider scripting it in a simple shell script. For example, see [this case](https://github.com/danieldreier/terraform-issue-reproductions/tree/master/25719). Providing a script can be easier than explaining what changes to make to the config between runs.
* Omit any unneeded complexity: remove variables, conditional statements, functions, modules, providers, and resources that are not needed to trigger the bug
* When possible, use the [null resource](https://www.terraform.io/docs/providers/null/resource.html) provider rather than a real provider in order to minimize external dependencies. We know this isn't always feasible. The Terraform Core team doesn't have deep domain knowledge in every provider, or access to every cloud platform for reproduction cases.
-->
### Terraform Version
@ -28,7 +38,7 @@ If you are not running the latest version of Terraform, please try upgrading bec
<!--
Paste the relevant parts of your Terraform configuration between the ``` marks below.
For large Terraform configs, please use a service like Dropbox and share a link to the ZIP file. For security, you can also encrypt the files using our GPG public key.
For Terraform configs larger than a few resources, or that involve multiple files, please make a GitHub repository that we can clone, rather than copy-pasting multiple files in here. For security, you can also encrypt the files using our GPG public key at https://www.hashicorp.com/security.
-->
```terraform

4
.github/SECURITY.md vendored
View File

@ -1,4 +0,0 @@
# Vulnerability Reporting
Please disclose security vulnerabilities responsibly by following the procedure
described at https://www.hashicorp.com/security#vulnerability-reporting

2
.go-version
View File

@ -1 +1 @@
1.14.2
1.15.2

2
.hashibot.hcl
View File

@ -3,6 +3,7 @@ behavior "remove_labels_on_reply" "remove_stale" {
only_non_maintainers = true
}
/*
poll "label_issue_migrater" "provider_migrater" {
schedule = "0 20 * * * *"
new_owner = env.PROVIDERS_OWNER
@ -23,6 +24,7 @@ poll "label_issue_migrater" "provider_migrater" {
EOF
migrated_comment = "This issue has been automatically migrated to ${var.repository}#${var.issue_number} because it looks like an issue with that provider. If you believe this is _not_ an issue with the provider, please reply to ${var.repository}#${var.issue_number}."
}
*/
poll "closed_issue_locker" "locker" {
schedule = "0 50 1 * * *"

82
BUGPROCESS.md Normal file
View File

@ -0,0 +1,82 @@
# Terraform Core GitHub Bug Triage & Labeling
The Terraform Core team has adopted a more structured bug triage process than we previously used. Our goal is to respond to reports of issues quickly.
When a bug report is filed, our goal is to either:
1. Get it to a state where it is ready for engineering to fix it in an upcoming Terraform release, or
2. Close it explain why, if we can't help
## Process
### 1. [Newly created issues](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Anew+label%3Abug+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent++-label%3Abackend%2Fmanta++-label%3Abackend%2Fatlas++-label%3Abackend%2Fetcdv3++-label%3Abackend%2Fetcdv2+-label%3Aconfirmed+-label%3A%22pending+project%22+-label%3A%22waiting+for+reproduction%22+-label%3A%22waiting-response%22+-label%3Aexplained) require initial filtering.
These are raw reports that need categorization and support clarifying them. They need the following done:
* label backends, provisioners, and providers so we can route work on codebases we don't support to the correct teams
* point requests for help to the community forum and close the issue
* close reports against old versions we no longer support
* prompt users who have submitted obviously incomplete reproduction cases for additional information
If an issue requires discussion with the user to get it out of this initial state, leave "new" on there and label it "waiting-response" until this phase of triage is done.
Once this initial filtering has been done, remove the new label. If an issue subjectively looks very high-impact and likely to impact many users, assign it to the [appropriate milestone](https://github.com/hashicorp/terraform/milestones) to mark it as being urgent.
### 2. Clarify [unreproduced issues](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Abug+created%3A%3E2020-05-01+-label%3Aprovisioner%2Fsalt-masterless+-label%3Adocumentation+-label%3Aprovider%2Fazuredevops+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent+-label%3Abackend%2Fmanta+-label%3Abackend%2Fatlas+-label%3Abackend%2Fetcdv3+-label%3Abackend%2Fetcdv2+-label%3Aconfirmed+-label%3A%22pending+project%22+-label%3Anew+-label%3A%22waiting+for+reproduction%22+-label%3Awaiting-response+-label%3Aexplained+sort%3Acreated-asc)
A core team member initially determines whether the issue is immediately reproducible. If they cannot readily reproduce it, they label it "waiting for reproduction" and correspond with the reporter to describe what is needed. When the issue is reproduced by a core team member, they label it "confirmed".
"confirmed" issues should have a clear reproduction case. Anyone who picks it up should be able to reproduce it readily without having to invent any details.
Note that the link above excludes issues reported before May 2020; this is to avoid including issues that were reported prior to this new process being implemented. [Unreproduced issues reported before May 2020](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Abug+created%3A%3C2020-05-01+-label%3Aprovisioner%2Fsalt-masterless+-label%3Adocumentation+-label%3Aprovider%2Fazuredevops+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent+-label%3Abackend%2Fmanta+-label%3Abackend%2Fatlas+-label%3Abackend%2Fetcdv3+-label%3Abackend%2Fetcdv2+-label%3Aconfirmed+-label%3A%22pending+project%22+-label%3Anew+-label%3A%22waiting+for+reproduction%22+-label%3Awaiting-response+-label%3Aexplained+sort%3Areactions-%2B1-desc) will be triaged as capacity permits.
### 3. Explain or fix [confirmed issues](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Abug+-label%3Aexplained+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent++-label%3Abackend%2Fmanta++-label%3Abackend%2Fatlas++-label%3Abackend%2Fetcdv3++-label%3Abackend%2Fetcdv2+label%3Aconfirmed+-label%3A%22pending+project%22+)
The next step for confirmed issues is to either:
* explain why the behavior is expected, label the issue as "working as designed", and close it, or
* locate the cause of the defect in the codebase. When the defect is located, and that description is posted on the issue, the issue is labeled "explained". In many cases, this step will get skipped if the fix is obvious, and engineers will jump forward and make a PR.
[Confirmed crashes](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Acrash+label%3Abug+-label%3Aexplained+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent++-label%3Abackend%2Fmanta++-label%3Abackend%2Fatlas++-label%3Abackend%2Fetcdv3++-label%3Abackend%2Fetcdv2+label%3Aconfirmed+-label%3A%22pending+project%22+) should generally be considered high impact
### 4. The last step for [explained issues](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Abug+label%3Aexplained+no%3Amilestone+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent++-label%3Abackend%2Fmanta++-label%3Abackend%2Fatlas++-label%3Abackend%2Fetcdv3++-label%3Abackend%2Fetcdv2+label%3Aconfirmed+-label%3A%22pending+project%22+) is to make a PR to fix them.
Explained issues that are expected to be fixed in a future release should be assigned to a milestone
## GitHub Issue Labels
label | description
------------------------ | -----------
new | new issue not yet triaged
explained | a Terraform Core team member has described the root cause of this issue in code
waiting for reproduction | unable to reproduce issue without further information
not reproducible | closed because a reproduction case could not be generated
duplicate | issue closed because another issue already tracks this problem
confirmed | a Terraform Core team member has reproduced this issue
working as designed | confirmed as reported and closed because the behavior is intended
pending project | issue is confirmed but will require a significant project to fix
## Lack of response and unreproducible issues
When bugs that have been [labeled waiting response](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Abug+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent+-label%3Abackend%2Fmanta+-label%3Abackend%2Fatlas+-label%3Abackend%2Fetcdv3+-label%3Abackend%2Fetcdv2+-label%3Aconfirmed+-label%3A%22pending+project%22+-label%3A%22waiting+for+reproduction%22+label%3Awaiting-response+-label%3Aexplained+sort%3Aupdated-asc) or [labeled "waiting for reproduction"](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Abug+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent+-label%3Abackend%2Fmanta+-label%3Abackend%2Fatlas+-label%3Abackend%2Fetcdv3+-label%3Abackend%2Fetcdv2+-label%3Aconfirmed+-label%3A%22pending+project%22+label%3A%22waiting+for+reproduction%22+-label%3Aexplained+sort%3Aupdated-asc+) for more than 30 days, we'll use our best judgement to determine whether it's more helpful to close it or prompt the reporter again. If they again go without a response for 30 days, they can be closed with a polite message explaining why and inviting the person to submit the needed information or reproduction case in the future.
The intent of this process is to get fix the maximum number of bugs in Terraform as quickly as possible, and having un-actionable bug reports makes it harder for Terraform Core team members and community contributors to find bugs they can actually work on.
## Helpful GitHub Filters
### Triage Process
1. [Newly created issues](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Anew+label%3Abug+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent++-label%3Abackend%2Fmanta++-label%3Abackend%2Fatlas++-label%3Abackend%2Fetcdv3++-label%3Abackend%2Fetcdv2+-label%3Aconfirmed+-label%3A%22pending+project%22+-label%3A%22waiting+for+reproduction%22+-label%3A%22waiting-response%22+-label%3Aexplained) require initial filtering.
2. Clarify [unreproduced issues](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Abug+created%3A%3E2020-05-01+-label%3Aprovisioner%2Fsalt-masterless+-label%3Adocumentation+-label%3Aprovider%2Fazuredevops+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent+-label%3Abackend%2Fmanta+-label%3Abackend%2Fatlas+-label%3Abackend%2Fetcdv3+-label%3Abackend%2Fetcdv2+-label%3Aconfirmed+-label%3A%22pending+project%22+-label%3Anew+-label%3A%22waiting+for+reproduction%22+-label%3Awaiting-response+-label%3Aexplained+sort%3Acreated-asc)
3. Explain or fix [confirmed issues](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Abug+-label%3Aexplained+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent++-label%3Abackend%2Fmanta++-label%3Abackend%2Fatlas++-label%3Abackend%2Fetcdv3++-label%3Abackend%2Fetcdv2+label%3Aconfirmed+-label%3A%22pending+project%22+). Prioritize [confirmed crashes](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Acrash+label%3Abug+-label%3Aexplained+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent++-label%3Abackend%2Fmanta++-label%3Abackend%2Fatlas++-label%3Abackend%2Fetcdv3++-label%3Abackend%2Fetcdv2+label%3Aconfirmed+-label%3A%22pending+project%22+).
4. Fix [explained issues](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Abug+label%3Aexplained+no%3Amilestone+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent++-label%3Abackend%2Fmanta++-label%3Abackend%2Fatlas++-label%3Abackend%2Fetcdv3++-label%3Abackend%2Fetcdv2+label%3Aconfirmed+-label%3A%22pending+project%22+)
### Other Backlog
[Confirmed needs for documentation fixes](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Abug+label%3Adocumentation++label%3Aconfirmed+-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent++-label%3Abackend%2Fmanta++-label%3Abackend%2Fatlas++-label%3Abackend%2Fetcdv3++-label%3Abackend%2Fetcdv2+)
[Confirmed bugs that will require significant projects to fix](https://github.com/hashicorp/terraform/issues?q=is%3Aopen+label%3Abug+label%3Aconfirmed+label%3A%22pending+project%22++-label%3Abackend%2Foss+-label%3Abackend%2Fazure+-label%3Abackend%2Fs3+-label%3Abackend%2Fgcs+-label%3Abackend%2Fconsul+-label%3Abackend%2Fartifactory+-label%3Aterraform-cloud+-label%3Abackend%2Fremote+-label%3Abackend%2Fswift+-label%3Abackend%2Fpg+-label%3Abackend%2Ftencent++-label%3Abackend%2Fmanta++-label%3Abackend%2Fatlas++-label%3Abackend%2Fetcdv3++-label%3Abackend%2Fetcdv2)
### Milestone Use
Milestones ending in .x indicate issues assigned to that milestone are intended to be fixed during that release lifecycle. Milestones ending in .0 indicate issues that will be fixed in that major release. For example:
[0.13.x Milestone](https://github.com/hashicorp/terraform/milestone/17). Issues in this milestone should be considered high-priority but do not block a patch release. All issues in this milestone should be resolved in a 13.x release before the 0.14.0 RC1 ships.
[0.14.0 Milestone](https://github.com/hashicorp/terraform/milestone/18). All issues in this milestone must be fixed before 0.14.0 RC1 ships, and should ideally be fixed before 0.14.0 beta 1 ships.
[0.14.x Milestone](https://github.com/hashicorp/terraform/milestone/20). Issues in this milestone are expected to be addressed at some point in the 0.14.x lifecycle, before 0.15.0. All issues in this milestone should be resolved in a 14.x release before the 0.15.0 RC1 ships.
[0.15.0 Milestone](https://github.com/hashicorp/terraform/milestone/19). All issues in this milestone must be fixed before 0.15.0 RC1 ships, and should ideally be fixed before 0.15.0 beta 1 ships.

156
CHANGELOG.md
View File

@ -1,144 +1,42 @@
## 0.13.1 (Unreleased)
## 0.14.0 (Unreleased)
BUG FIXES:
* backend: fix inconsistent locking behavior between local and remote backends, which caused lingering locks in terraform console and import [GH-25454]
* lang/funcs: update cidrsubnet and cidrhost to support 64-bit systems [GH-25517]
* states/statefile: consistently sort resources across modules [GH-25498]
* cli: allow targeting of resources with module instances [GH-25760]
* command: fix panic when using `state mv` to move the last resource in a module [GH-25523]
* configs: include `providers` when processing module overrides [GH-25496]
* core: fix inconsistent plan error when dynamic set block has 0 elements [GH-25662]
* core: prevent decoding errors when resource attributes have been removed entirely from the schema [GH-25779]
## 0.13.0 (August 10, 2020)
> This is a list of changes relative to Terraform v0.12.29. To see the
> incremental changelogs for the v0.13.0 prereleases, see
> [the v0.13.0-rc1 changelog](https://github.com/hashicorp/terraform/blob/v0.13.0-rc1/CHANGELOG.md).
This section contains details about various changes in the v0.13 major release. If you are upgrading from Terraform v0.12, we recommend first referring to [the v0.13 upgrade guide](https://www.terraform.io/upgrade-guides/0-13.html) for information on some common concerns during upgrade and guidance on ways to address them. (The final upgrade guide and the documentation for the new features will be published only when v0.13.0 final is released; until then, some links in this section will be non-functional.)
NEW FEATURES:
* [**`count` and `for_each` for modules**](https://www.terraform.io/docs/configuration/modules.html#multiple-instances-of-a-module): Similar to the arguments of the same name in `resource` and `data` blocks, these create multiple instances of a module from a single `module` block. ([#24461](https://github.com/hashicorp/terraform/issues/24461))
* [**`depends_on` for modules**](https://www.terraform.io/docs/configuration/modules.html#other-meta-arguments): Modules can now use the `depends_on` argument to ensure that all module resource changes will be applied after any changes to the `depends_on` targets have been applied. ([#25005](https://github.com/hashicorp/terraform/issues/25005))
* [**Automatic installation of third-party providers**](https://www.terraform.io/docs/configuration/provider-requirements.html): Terraform now supports a decentralized namespace for providers, allowing for automatic installation of community providers from third-party namespaces in the public registry and from private registries. (More details will be added about this prior to release.)
* [**Custom validation rules for input variables**](https://www.terraform.io/docs/configuration/variables.html#custom-validation-rules): A new `validation` block type inside `variable` blocks allows module authors to define validation rules at the public interface into a module, so that errors in the calling configuration can be reported in the caller's context rather than inside the implementation details of the module. ([#25054](https://github.com/hashicorp/terraform/issues/25054))
* [**New Kubernetes remote state storage backend**](https://www.terraform.io/docs/backends/types/kubernetes.html): This backend stores state snapshots as Kubernetes secrets. ([#19525](https://github.com/hashicorp/terraform/issues/19525))
BREAKING CHANGES:
* As part of introducing a new heirarchical namespace for providers, Terraform now requires an explicit `source` specification for any provider that is not in the "hashicorp" namespace in the main public registry. ([#24477](https://github.com/hashicorp/terraform/issues/24477))
For more information, including information on the automatic upgrade process, refer to [the v0.13 upgrade guide](https://www.terraform.io/upgrade-guides/0-13.html).
* `terraform import`: the previously-deprecated `-provider` option is now removed. ([#24090](https://github.com/hashicorp/terraform/issues/24090))
To specify a non-default provider configuration for import, add the `provider` meta-argument to the target `resource` block.
* config: Inside `provisioner` blocks that have `when = destroy` set, and inside any `connection` blocks that are used by such `provisioner` blocks, it is no longer valid to refer to any objects other than `self`, `count`, or `each`. (This was previously deprecated in a v0.12 minor release.) ([#24083](https://github.com/hashicorp/terraform/issues/24083))
If you are using `null_resource` to define provisioners not attached to a real resource, include any values your provisioners need in the `triggers` map and change the provisioner configuration to refer to those values via `self.triggers`.
* configs: At most one `terraform` `required_providers` block is permitted per module ([#24763](https://github.com/hashicorp/terraform/issues/24763))
If you previously had multiple `required_providers` blocks in the same module, consolidate their requirements together into a single block.
* The official MacOS builds of Terraform CLI are no longer compatible with Mac OS 10.10 Yosemite; Terraform now requires at least Mac OS 10.11 El Capitan.
Terraform 0.13 is the last major release that will support 10.11 El Capitan, so if you are upgrading your OS we recommend upgrading to Mac OS 10.12 Sierra or later.
* The official FreeBSD builds of Terraform CLI are no longer compatible with FreeBSD 10.x, which has reached end-of-life. Terraform now requires FreeBSD 11.2 or later.
* backend/oss: The TableStore schema now requires a primary key named `LockID` of type `String`. ([#24149](https://github.com/hashicorp/terraform/issues/24149))
* backend/s3: The previously-deprecated `lock_table`, `skip_get_ec2_platforms`, and `skip_requesting_account_id` arguments are now removed. ([#25134](https://github.com/hashicorp/terraform/issues/25134))
* backend/s3: The credential source preference order now considers EC2 instance profile credentials as lower priority than shared configuration, web identity, and ECS role credentials. ([#25134](https://github.com/hashicorp/terraform/issues/25134))
* backend/s3: The `AWS_METADATA_TIMEOUT` environment variable is no longer used. The timeout is now fixed at one second with two retries. ([#25134](https://github.com/hashicorp/terraform/issues/25134))
NOTES:
* The `terraform plan` and `terraform apply` commands will now detect and report changes to root module outputs as needing to be applied even if there are no resource changes in the plan.
This is an improvement in behavior for most users, since it will now be possible to change `output` blocks and use `terraform apply` to apply those changes.
If you have a configuration where a root module output value is changing for every plan (for example, by referring to an unstable data source), you will need to remove or change that output value in order to allow convergence on an empty plan. Otherwise, each new plan will propose more changes.
* Terraform CLI now supports TLS 1.3 and supports Ed25519 certificates when making outgoing connections to remote TLS servers.
While both of these changes are backwards compatible in principle, certain legacy TLS server implementations can reportedly encounter problems when attempting to negotiate TLS 1.3. (These changes affects only requests made by Terraform CLI itself, such as to module registries or backends. Provider plugins have separate TLS implementations that will gain these features on a separate release schedule.)
* On Unix systems where `use-vc` is set in `resolv.conf`, Terraform will now use TCP for DNS resolution.
We don't expect this to cause any problem for most users, but if you find you are seeing DNS resolution failures after upgrading please verify that you can either reach your configured nameservers using TCP or that your resolver configuration does not include the `use-vc` directive.
* The `terraform 0.12upgrade` command is no longer available. ([#24403](https://github.com/hashicorp/terraform/issues/24403))
To upgrade from Terraform v0.11, first [upgrade to the latest v0.12 release](https://www.terraform.io/upgrade-guides/0-12.html) and then upgrade to v0.13 from there.
UPGRADE NOTES:
* configs: The `version` argument inside provider configuration blocks has been documented as deprecated since Terraform 0.12. As of 0.14 it will now also generate an explicit deprecation warning. To avoid the warning, use [provider requirements](https://www.terraform.io/docs/configuration/provider-requirements.html) declarations instead. ([#26135](https://github.com/hashicorp/terraform/issues/26135))
* The official MacOS builds of Terraform now require MacOS 10.12 Sierra or later. [GH-26357]
* TLS certificate verification for outbound HTTPS requests from Terraform CLI no longer treats the certificate's "common name" as a valid hostname when the certificate lacks any "subject alternative name" entries for the hostname. TLS server certificates must list their hostnames as a "DNS name" in the subject alternative names field. [GH-26357]
* Outbound HTTPS requests from Terraform CLI now enforce [RFC 8446](https://tools.ietf.org/html/rfc8446)'s client-side downgrade protection checks. This should not significantly affect normal operation, but may result in connection errors in environments where outgoing requests are forced through proxy servers and other "middleboxes", if they have behavior that resembles a downgrade attack. [GH-26357]
* Terraform's HTTP client code is now slightly stricter than before in HTTP header parsing, but in ways that should not affect typical server implementations: Terraform now trims only _ASCII_ whitespace characters, and does not allow `Transfer-Encoding: identity`. [GH-26357]
ENHANCEMENTS:
* config: `templatefile` function will now return a helpful error message if a given variable has an invalid name, rather than relying on a syntax error in the template parsing itself. ([#24184](https://github.com/hashicorp/terraform/issues/24184))
* config: The configuration language now uses Unicode 12.0 character tables for certain Unicode-version-sensitive operations on strings, such as the `upper` and `lower` functions. Those working with strings containing new characters introduced since Unicode 9.0 may see small differences in behavior as a result of these table updates.
* config: The new `sum` function takes a list or set of numbers and returns the sum of all elements. ([#24666](https://github.com/hashicorp/terraform/issues/24666))
* config: Modules authored by the same vendor as the main provider they use can now pass metadata to the provider to allow for instrumentation and analytics. ([#22583](https://github.com/hashicorp/terraform/issues/22583))
* cli: The `terraform plan` and `terraform apply` commands now recognize changes to root module outputs as side-effects to be approved and applied. This means you can apply root module output changes using the normal plan and apply workflow. ([#25047](https://github.com/hashicorp/terraform/issues/25047))
* cli: When installing providers from the Terraform Registry, Terraform will verify the trust signature for partner providers, and allow for self-signed community providers. ([#24617](https://github.com/hashicorp/terraform/issues/24617))
* cli: `terraform init` will display detailed trust signature information when installing providers from the Terraform Registry and other provider registries. ([#24932](https://github.com/hashicorp/terraform/issues/24932))
* cli: It is now possible to optionally specify explicitly which installation methods can be used for different providers in the CLI configuration, such as forcing a particular provider to be loaded from a particular directory on local disk instead of consulting its origin provider registry. ([#24728](https://github.com/hashicorp/terraform/issues/24728))
* cli: The new `terraform state replace-provider` subcommand allows changing the selected provider for existing resource instances in the Terraform state. ([#24523](https://github.com/hashicorp/terraform/issues/24523))
* cli: The new `terraform providers mirror` subcommand can automatically construct or update a local filesystem mirror directory containing the providers required for the current configuration. ([#25084](https://github.com/hashicorp/terraform/issues/25084))
* cli: `terraform version -json` now produces machine-readable version information. ([#25252](https://github.com/hashicorp/terraform/issues/25252))
* cli: `terraform import` can now work with provider configurations containing references to other objects, as long as the data in question is already known in the current state. ([#25420](https://github.com/hashicorp/terraform/issues/25420))
* cli: The `terraform state rm` command will now exit with status code 1 if the given resource address does not match any resource instances. ([#22300](https://github.com/hashicorp/terraform/issues/22300))
* cli: The `terraform login` command now requires the full word "yes" to confirm, rather than just "y", for consistency with Terraform's other interactive prompts. ([#25379](https://github.com/hashicorp/terraform/issues/25379))
* core: Several of Terraform's graph operations are now better optimized to support configurations with highly-connected graphs. ([#23811](https://github.com/hashicorp/terraform/issues/23811), [#25544](https://github.com/hashicorp/terraform/issues/25544))
* backend/remote: Now supports `terraform state push -force`. ([#24696](https://github.com/hashicorp/terraform/issues/24696))
* backend/remote: Can now accept `-target` options when creating a plan using _remote operations_, if supported by the target server. (Server-side support for this in Terraform Cloud and Terraform Enterprise will follow in forthcoming releases of each.) ([#24834](https://github.com/hashicorp/terraform/issues/24834))
* backend/azurerm: Now uses the Giovanni Storage SDK to communicate with Azure. ([#24669](https://github.com/hashicorp/terraform/issues/24669))
* backend/s3: The backend will now always consult the shared configuration file, even if the `AWS_SDK_LOAD_CONFIG` environment variable isn't set. That environment variable is now ignored. ([#25134](https://github.com/hashicorp/terraform/issues/25134))
* backend/s3: Region validation now automatically supports the new `af-south-1` (Africa (Cape Town)) region. ([#24744](https://github.com/hashicorp/terraform/issues/24744))
For AWS operations to work in the new region, you must explicitly enable it as described in [AWS General Reference: Enabling a Region](https://docs.aws.amazon.com/general/latest/gr/rande-manage.html#rande-manage-enable). If you haven't enabled the region, the Terraform S3 Backend will return `InvalidClientTokenId` errors during credential validation.
* backend/s3: A `~/` prefix in the `shared_credentials_file` argument is now expanded to the current user's home directory. ([#25134](https://github.com/hashicorp/terraform/issues/25134))
* backend/s3: The backend has a number of new options for customizing the "assume role" behavior, including controlling the lifetime and access policy of temporary credentials. ([#25134](https://github.com/hashicorp/terraform/issues/25134))
* backend/swift: The authentication options match those of the OpenStack provider. ([#23510](https://github.com/hashicorp/terraform/issues/23510))
* `terraform plan` and `terraform apply`: Added an experimental concise diff renderer. By default, Terraform plans now hide most unchanged fields, only displaying the most relevant changes and some identifying context. This experiment can be disabled by setting a `TF_X_CONCISE_DIFF` environment variable to `0`. ([#26187](https://github.com/hashicorp/terraform/issues/26187))
* cli: A new global command line option `-chdir=...`, placed before the selected subcommand, instructs Terraform to switch to a different working directory before executing the subcommand. This is similar to switching to a new directory with `cd` before running Terraform, but it avoids changing the state of the calling shell. ([#26087](https://github.com/hashicorp/terraform/issues/26087))
* config: Added `alltrue` function, which returns `true` if all elements in the given collection are `true`. This is primarily intended to make it easier to write variable validation conditions which operate on collections. ([#25656](https://github.com/hashicorp/terraform/issues/25656))
* core: `terraform plan` no longer uses a separate refresh phase, all resources are updated on-demand during planning ([#26270](https://github.com/hashicorp/terraform/issues/26270))
* `terraform console`: Now has distinct rendering of lists, sets, and tuples, and correctly renders objects with `null` attribute values. ([#26189](https://github.com/hashicorp/terraform/issues/26189))
* `terraform login`: Added support for OAuth2 application scopes. ([#26239](https://github.com/hashicorp/terraform/issues/26239))
* `terraform fmt`: Will now do some slightly more opinionated normalization behaviors, using the documented idiomatic syntax. [GH-26390]
* backend/consul: Split state into chunks when outgrowing the limit of the Consul KV store. This allows storing state larger than the Consul 512KB limit. ([#25856](https://github.com/hashicorp/terraform/issues/25856))
* On Unix-based operating systems other than MacOS, the `SSL_CERT_DIR` environment variable can now be a colon-separated list of multiple certificate search paths. [GH-26357]
* On MacOS, Terraform will now use the `Security.framework` API to access the system trust roots, for improved consistency with other MacOS software. [GH-26357]
BUG FIXES:
* config: The `jsonencode` function can now correctly encode a single null value as the JSON expression `null`. ([#25078](https://github.com/hashicorp/terraform/issues/25078))
* config: The `map` function no longer crashes when incorrectly given a non-string key. ([#24277](https://github.com/hashicorp/terraform/issues/24277))
* config: The `substr` function now correctly returns a zero-length string when given a length of zero, rather than ignoring that argument entirely. ([#24318](https://github.com/hashicorp/terraform/issues/24318))
* config: `ceil(1/0)` and `floor(1/0)` (that is, an infinity as an argument) now return another infinity with the same sign, rather than just a large integer. ([#21463](https://github.com/hashicorp/terraform/issues/21463))
* config: The `rsadecrypt` function now supports the OpenSSH RSA key format. ([#25112](https://github.com/hashicorp/terraform/issues/25112))
* config: The `merge` function now returns more precise type information, making it usable for values passed to `for_each`, and will no longer crash if all of the given maps are empty. ([#24032](https://github.com/hashicorp/terraform/issues/24032), [#25303](https://github.com/hashicorp/terraform/issues/25303))
* vendor: The various set-manipulation functions, like `setunion`, will no longer panic if given an unknown set value ([#25318](https://github.com/hashicorp/terraform/issues/25318))
* config: Fixed a crash with incorrect syntax in `.tf.json` and `.tfvars.json` files. ([#24650](https://github.com/hashicorp/terraform/issues/24650))
* config: The function argument expansion syntax `...` no longer incorrectly fails with "Invalid expanding argument value" in situations where the expanding argument's type will not be known until the apply phase. ([#25216](https://github.com/hashicorp/terraform/issues/25216))
* config: Variable `validation` block error message checks no longer fail when non-ASCII characters are present. ([#25144](https://github.com/hashicorp/terraform/issues/25144))
* cli: The `terraform plan` command (and the implied plan run by `terraform apply` with no arguments) will now print any warnings that were generated even if there are no changes to be made. ([#24095](https://github.com/hashicorp/terraform/issues/24095))
* cli: `terraform state mv` now correctly records the resource's use of either `count` or `for_each` based on the given target address. ([#24254](https://github.com/hashicorp/terraform/issues/24254))
* cli: When using the `TF_CLI_CONFIG_FILE` environment variable to override where Terraform looks for CLI configuration, Terraform will now ignore the default CLI configuration directory as well as the default CLI configuration file. ([#24728](https://github.com/hashicorp/terraform/issues/24728))
* cli: The `terraform login` command in OAuth2 mode now implements the PKCE OAuth 2 extension more correctly. Previously it was not compliant with all of the details of the specification. ([#24858](https://github.com/hashicorp/terraform/issues/24858))
* cli: Fixed a potential crash when the `HOME` environment variable isn't set, causing the native service credentials store to be `nil`. ([#25110](https://github.com/hashicorp/terraform/issues/25110))
* command/fmt: Error messages will now include source code snippets where possible. ([#24471](https://github.com/hashicorp/terraform/issues/24471))
* command/apply: `terraform apply` will no longer silently exit when given an absolute path to a saved plan file on Windows. ([#25233](https://github.com/hashicorp/terraform/issues/25233))
* command/init: `terraform init` will now produce an explicit error message if given a non-directory path for its configuration directory argument, and if a `-backend-config` file has a syntax error. Previously these were silently ignored. ([#25300](https://github.com/hashicorp/terraform/pull/25300), [#25411](https://github.com/hashicorp/terraform/issues/25411))
* command/console: ([#25442](https://github.com/hashicorp/terraform/issues/25442))
* command/import: The `import` command will now properly attach the configured provider for the target resource based on the configuration, making the `-provider` command line option unnecessary. ([#22862](https://github.com/hashicorp/terraform/issues/22862))
* command/import: The `-allow-missing-config` option now works correctly. It was inadvertently disabled as part of v0.12 refactoring. ([#25352](https://github.com/hashicorp/terraform/issues/25352))
* command/show: Resource addresses are now consistently formatted between the plan and prior state in the `-json` output. ([#24256](https://github.com/hashicorp/terraform/issues/24256))
* core: Fixed a crash related to an unsafe concurrent read and write of a map data structure. ([#24599](https://github.com/hashicorp/terraform/issues/24599))
* core: Instances are now destroyed only using their stored state, without re-evaluating configuration. This avoids a number of dependency cycle problems when "delete" actions are included in a plan. ([#24083](https://github.com/hashicorp/terraform/issues/24083))
* provider/terraform: The `terraform_remote_state` data source will no longer attempt to "configure" the selected backend during validation, which means backends will not try to perform remote actions such as verifying credentials during `terraform validate`. Local validation still applies in all cases, and the configuration step will still occur prior to actually reading the remote state in a normal plan/apply operation. ([#24887](https://github.com/hashicorp/terraform/issues/24887))
* backend/remote: Backend will no longer crash if the user cancels backend initialization at an inopportune time, or if there is a connection error. ([#25135](https://github.com/hashicorp/terraform/issues/25135)) ([#25341](https://github.com/hashicorp/terraform/issues/25341))
* backend/azurerm: The backend will now create a Azure storage snapshot of the previous Terraform state snapshot before writing a new one. ([#24069](https://github.com/hashicorp/terraform/issues/24069))
* backend/s3: Various other minor authentication-related fixes previously made in the AWS provider. ([#25134](https://github.com/hashicorp/terraform/issues/25134))
* backend/oss: Now allows locking of multiple different state files. ([#24149](https://github.com/hashicorp/terraform/issues/24149))
* provisioner/remote-exec: The provisioner will now return an explicit error if the `host` connection argument is an empty string. Previously it would repeatedly attempt to resolve an empty hostname until timeout. ([#24080](https://github.com/hashicorp/terraform/issues/24080))
* provisioner/chef: The provisioner will now gracefully handle non-failure (RFC062) exit codes returned from Chef. ([#19155](https://github.com/hashicorp/terraform/issues/19155))
* provisioner/habitat: The provisioner will no longer generate `user.toml` with world-readable permissions. ([#24321](https://github.com/hashicorp/terraform/issues/24321))
* communicator/winrm: Support a connection timeout for WinRM `connection` blocks. Previously this argument worked for SSH only. ([#25350](https://github.com/hashicorp/terraform/issues/25350))
EXPERIMENTS:
* This release concludes the `variable_validation` [experiment](https://www.terraform.io/docs/configuration/terraform.html#experimental-language-features) that was started in Terraform v0.12.20. If you were participating in the experiment, you should remove the experiment opt-in from your configuration as part of upgrading to Terraform 0.13.
* backend/consul: Fix bug which prevented state locking when path has trailing `/` ([#25842](https://github.com/hashicorp/terraform/issues/25842))
* build: Fix crash with terraform binary on OpenBSD. ([#26249](https://github.com/hashicorp/terraform/issues/26249)
* command/clistate: return an error on a state unlock failure [[#25729](https://github.com/hashicorp/terraform/issues/25729)]
* command/taint: If the configuration's `required_version` constraint is not met, the `taint` subcommand will now correctly exit early. [GH-26345]
* command/taint, untaint: Fix issue when using `taint` (and `untaint`) with workspaces where statefile was not found. [GH-22467]
* configs: Report an error when provider configuration attributes are incorrectly added to a `required_providers` object. ([#26184](https://github.com/hashicorp/terraform/issues/26184))
* core: Errors with data sources reading old data during refresh, failing to refresh, and not appearing to wait on resource dependencies are fixed by updates to the data source lifecycle and the merging of refresh and plan ([#26270](https://github.com/hashicorp/terraform/issues/26270))
* lang/funcs: fix panic when element() is called with a negative offset ([#26079](https://github.com/hashicorp/terraform/issues/26079))
* states/remote: fix `state push -force` to work for all backends ([#26190](https://github.com/hashicorp/terraform/issues/26190))
The experiment received only feedback that can be addressed with backward-compatible future enhancements, so we've included it into this release as stable with no changes to its original design so far. We'll consider additional features related to custom validation in future releases after seeing how it's used in real-world modules.
## Previous Releases
For information on prior major releases, see their changelogs:
* [v0.13](https://github.com/hashicorp/terraform/blob/v0.13/CHANGELOG.md)
* [v0.12](https://github.com/hashicorp/terraform/blob/v0.12/CHANGELOG.md)
* [v0.11 and earlier](https://github.com/hashicorp/terraform/blob/v0.11/CHANGELOG.md)

4
CODEOWNERS
View File

@ -4,7 +4,7 @@
# Remote-state backend # Maintainer
/backend/remote-state/artifactory Unmaintained
/backend/remote-state/azure @hashicorp/terraform-azure
/backend/remote-state/consul @hashicorp/consul
/backend/remote-state/consul @hashicorp/consul @remilapeyre
/backend/remote-state/cos @likexian
/backend/remote-state/etcdv2 Unmaintained
/backend/remote-state/etcdv3 @bmcustodio
@ -12,7 +12,7 @@
/backend/remote-state/http @hashicorp/terraform-core
/backend/remote-state/manta Unmaintained
/backend/remote-state/oss @xiaozhu36
/backend/remote-state/pg Unmaintained
/backend/remote-state/pg @remilapeyre
/backend/remote-state/s3 @hashicorp/terraform-aws
/backend/remote-state/swift Unmaintained
/backend/remote-state/kubernetes @jrhouston @alexsomesan

2
Makefile
View File

@ -5,7 +5,7 @@ VERSION?="0.3.44"
# source files, except the protobuf stubs which are built instead with
# "make protobuf".
generate:
GOFLAGS=-mod=vendor go generate ./...
go generate ./...
# go fmt doesn't support -mod=vendor but it still wants to populate the
# module cache with everything in go.mod even though formatting requires
# no dependencies, and so we're disabling modules mode for this right

2
README.md
View File

@ -40,5 +40,7 @@ This repository contains only Terraform core, which includes the command line in
To learn more about compiling Terraform and contributing suggested changes, please refer to [the contributing guide](.github/CONTRIBUTING.md).
To learn more about how we handle bug reports, please read the [bug triage guide](./BUGPROCESS.md).
## License
[Mozilla Public License v2.0](https://github.com/hashicorp/terraform/blob/master/LICENSE)

4
addrs/provider.go
View File

@ -156,8 +156,8 @@ func (pt Provider) LegacyString() string {
if pt.IsZero() {
panic("called LegacyString on zero-value addrs.Provider")
}
if pt.Namespace != LegacyProviderNamespace {
panic(pt.String() + " is not a legacy addrs.Provider")
if pt.Namespace != LegacyProviderNamespace && pt.Namespace != BuiltInProviderNamespace {
panic(pt.String() + " cannot be represented as a legacy string")
}
return pt.Type
}

31
addrs/provider_test.go
View File

@ -54,6 +54,37 @@ func TestProviderString(t *testing.T) {
}
}
func TestProviderLegacyString(t *testing.T) {
tests := []struct {
Input Provider
Want string
}{
{
Provider{
Type: "test",
Hostname: DefaultRegistryHost,
Namespace: LegacyProviderNamespace,
},
"test",
},
{
Provider{
Type: "terraform",
Hostname: BuiltInProviderHost,
Namespace: BuiltInProviderNamespace,
},
"terraform",
},
}
for _, test := range tests {
got := test.Input.LegacyString()
if got != test.Want {
t.Errorf("wrong result for %s\ngot: %s\nwant: %s", test.Input.String(), got, test.Want)
}
}
}
func TestProviderDisplay(t *testing.T) {
tests := []struct {
Input Provider

9
addrs/resource.go
View File

@ -50,6 +50,15 @@ func (r Resource) Absolute(module ModuleInstance) AbsResource {
}
}
// InModule returns a ConfigResource from the receiver and the given module
// address.
func (r Resource) InModule(module Module) ConfigResource {
return ConfigResource{
Module: module,
Resource: r,
}
}
// ImpliedProvider returns the implied provider type name, for e.g. the "aws" in
// "aws_instance"
func (r Resource) ImpliedProvider() string {

12
backend/local/backend_apply.go
View File

@ -80,18 +80,6 @@ func (b *Local) opApply(
// If we weren't given a plan, then we refresh/plan
if op.PlanFile == nil {
// If we're refreshing before apply, perform that
if op.PlanRefresh {
log.Printf("[INFO] backend/local: apply calling Refresh")
_, refreshDiags := tfCtx.Refresh()
diags = diags.Append(refreshDiags)
if diags.HasErrors() {
runningOp.Result = backend.OperationFailure
b.ShowDiagnostics(diags)
return
}
}
// Perform the plan
log.Printf("[INFO] backend/local: apply calling Plan")
plan, planDiags := tfCtx.Plan()

25
backend/local/backend_local.go
View File

@ -78,19 +78,17 @@ func (b *Local) context(op *backend.Operation) (*terraform.Context, *configload.
opts.Targets = op.Targets
opts.UIInput = op.UIIn
opts.SkipRefresh = op.Type == backend.OperationTypePlan && !op.PlanRefresh
if opts.SkipRefresh {
log.Printf("[DEBUG] backend/local: skipping refresh of managed resources")
}
// Load the latest state. If we enter contextFromPlanFile below then the
// state snapshot in the plan file must match this, or else it'll return
// error diagnostics.
log.Printf("[TRACE] backend/local: retrieving local state snapshot for workspace %q", op.Workspace)
opts.State = s.State()
// Prepare a separate opts and context for validation, which doesn't use
// any state ensuring that we only validate the config, since evaluation
// will automatically reference the state when available.
validateOpts := opts
validateOpts.State = nil
var validateCtx *terraform.Context
var tfCtx *terraform.Context
var ctxDiags tfdiags.Diagnostics
var configSnap *configload.Snapshot
@ -108,18 +106,9 @@ func (b *Local) context(op *backend.Operation) (*terraform.Context, *configload.
// Write sources into the cache of the main loader so that they are
// available if we need to generate diagnostic message snippets.
op.ConfigLoader.ImportSourcesFromSnapshot(configSnap)
// create a validation context with no state
validateCtx, _, _ = b.contextFromPlanFile(op.PlanFile, validateOpts, stateMeta)
// diags from here will be caught above
} else {
log.Printf("[TRACE] backend/local: building context for current working directory")
tfCtx, configSnap, ctxDiags = b.contextDirect(op, opts)
// create a validation context with no state
validateCtx, _, _ = b.contextDirect(op, validateOpts)
// diags from here will be caught above
}
diags = diags.Append(ctxDiags)
if diags.HasErrors() {
@ -145,7 +134,7 @@ func (b *Local) context(op *backend.Operation) (*terraform.Context, *configload.
// If validation is enabled, validate
if b.OpValidation {
log.Printf("[TRACE] backend/local: running validation operation")
validateDiags := validateCtx.Validate()
validateDiags := tfCtx.Validate()
diags = diags.Append(validateDiags)
}
}
@ -231,7 +220,7 @@ func (b *Local) contextFromPlanFile(pf *planfile.Reader, opts terraform.ContextO
// If the caller sets this, we require that the stored prior state
// has the same metadata, which is an extra safety check that nothing
// has changed since the plan was created. (All of the "real-world"
// state manager implementstions support this, but simpler test backends
// state manager implementations support this, but simpler test backends
// may not.)
if currentStateMeta.Lineage != "" && priorStateFile.Lineage != "" {
if priorStateFile.Serial != currentStateMeta.Serial || priorStateFile.Lineage != currentStateMeta.Lineage {

26
backend/local/backend_plan.go
View File

@ -97,27 +97,6 @@ func (b *Local) opPlan(
runningOp.State = tfCtx.State()
// If we're refreshing before plan, perform that
baseState := runningOp.State
if op.PlanRefresh {
log.Printf("[INFO] backend/local: plan calling Refresh")
if b.CLI != nil {
b.CLI.Output(b.Colorize().Color(strings.TrimSpace(planRefreshing) + "\n"))
}
refreshedState, refreshDiags := tfCtx.Refresh()
diags = diags.Append(refreshDiags)
if diags.HasErrors() {
b.ReportResult(runningOp, diags)
return
}
baseState = refreshedState // plan will be relative to our refreshed state
if b.CLI != nil {
b.CLI.Output("\n------------------------------------------------------------------------")
}
}
// Perform the plan in a goroutine so we can be interrupted
var plan *plans.Plan
var planDiags tfdiags.Diagnostics
@ -142,6 +121,7 @@ func (b *Local) opPlan(
b.ReportResult(runningOp, diags)
return
}
// Record whether this plan includes any side-effects that could be applied.
runningOp.PlanEmpty = !planHasSideEffects(priorState, plan.Changes)
@ -161,7 +141,7 @@ func (b *Local) opPlan(
// We may have updated the state in the refresh step above, but we
// will freeze that updated state in the plan file for now and
// only write it if this plan is subsequently applied.
plannedStateFile := statemgr.PlannedStateUpdate(opState, baseState)
plannedStateFile := statemgr.PlannedStateUpdate(opState, plan.State)
log.Printf("[INFO] backend/local: writing plan output to: %s", path)
err := planfile.Create(path, configSnap, plannedStateFile, plan)
@ -187,7 +167,7 @@ func (b *Local) opPlan(
return
}
b.renderPlan(plan, baseState, priorState, schemas)
b.renderPlan(plan, plan.State, priorState, schemas)
// If we've accumulated any warnings along the way then we'll show them
// here just before we show the summary and next steps. If we encountered

43
backend/local/backend_plan_test.go
View File

@ -289,12 +289,9 @@ Terraform will perform the following actions:
# test_instance.foo is tainted, so must be replaced
-/+ resource "test_instance" "foo" {
ami = "bar"
# (1 unchanged attribute hidden)
network_interface {
description = "Main network interface"
device_index = 0
}
# (1 unchanged block hidden)
}
Plan: 1 to add, 0 to change, 1 to destroy.`
@ -360,8 +357,8 @@ func TestLocal_planDeposedOnly(t *testing.T) {
if run.Result != backend.OperationSuccess {
t.Fatalf("plan operation failed")
}
if !p.ReadResourceCalled {
t.Fatal("ReadResource should be called")
if p.ReadResourceCalled {
t.Fatal("ReadResource should not be called")
}
if run.PlanEmpty {
t.Fatal("plan should not be empty")
@ -468,12 +465,9 @@ Terraform will perform the following actions:
# test_instance.foo is tainted, so must be replaced
+/- resource "test_instance" "foo" {
ami = "bar"
# (1 unchanged attribute hidden)
network_interface {
description = "Main network interface"
device_index = 0
}
# (1 unchanged block hidden)
}
Plan: 1 to add, 0 to change, 1 to destroy.`
@ -484,6 +478,12 @@ Plan: 1 to add, 0 to change, 1 to destroy.`
}
func TestLocal_planRefreshFalse(t *testing.T) {
// since there is no longer a separate Refresh walk, `-refresh=false
// doesn't do anything.
// FIXME: determine if we need a refresh option for the new plan, or remove
// this test
t.Skip()
b, cleanup := TestLocal(t)
defer cleanup()
@ -549,8 +549,8 @@ func TestLocal_planDestroy(t *testing.T) {
t.Fatalf("plan operation failed")
}
if !p.ReadResourceCalled {
t.Fatal("ReadResource should be called")
if p.ReadResourceCalled {
t.Fatal("ReadResource should not be called")
}
if run.PlanEmpty {
@ -605,12 +605,12 @@ func TestLocal_planDestroy_withDataSources(t *testing.T) {
t.Fatalf("plan operation failed")
}
if !p.ReadResourceCalled {
t.Fatal("ReadResource should be called")
if p.ReadResourceCalled {
t.Fatal("ReadResource should not be called")
}
if !p.ReadDataSourceCalled {
t.Fatal("ReadDataSourceCalled should be called")
if p.ReadDataSourceCalled {
t.Fatal("ReadDataSourceCalled should not be called")
}
if run.PlanEmpty {
@ -627,7 +627,7 @@ func TestLocal_planDestroy_withDataSources(t *testing.T) {
// Data source should not be rendered in the output
expectedOutput := `Terraform will perform the following actions:
# test_instance.foo will be destroyed
# test_instance.foo[0] will be destroyed
- resource "test_instance" "foo" {
- ami = "bar" -> null
@ -641,7 +641,7 @@ Plan: 0 to add, 0 to change, 1 to destroy.`
output := b.CLI.(*cli.MockUi).OutputWriter.String()
if !strings.Contains(output, expectedOutput) {
t.Fatalf("Unexpected output (expected no data source):\n%s", output)
t.Fatalf("Unexpected output:\n%s", output)
}
}
@ -678,6 +678,7 @@ func TestLocal_planOutPathNoChange(t *testing.T) {
Type: "local",
Config: cfgRaw,
}
op.PlanRefresh = true
run, err := b.Operation(context.Background(), op)
if err != nil {
@ -830,7 +831,7 @@ func testPlanState_tainted() *states.State {
Mode: addrs.ManagedResourceMode,
Type: "test_instance",
Name: "foo",
}.Instance(addrs.IntKey(0)),
}.Instance(addrs.NoKey),
&states.ResourceInstanceObjectSrc{
Status: states.ObjectTainted,
AttrsJSON: []byte(`{

3
backend/local/backend_refresh.go
View File

@ -42,6 +42,9 @@ func (b *Local) opRefresh(
}
}
// Refresh now happens via a plan, so we need to ensure this is enabled
op.PlanRefresh = true
// Get our context
tfCtx, _, opState, contextDiags := b.context(op)
diags = diags.Append(contextDiags)

74
backend/local/backend_refresh_test.go
View File

@ -51,76 +51,11 @@ test_instance.foo:
assertBackendStateUnlocked(t, b)
}
func TestLocal_refreshNoConfig(t *testing.T) {
b, cleanup := TestLocal(t)
defer cleanup()
p := TestLocalProvider(t, b, "test", refreshFixtureSchema())
testStateFile(t, b.StatePath, testRefreshState())
p.ReadResourceFn = nil
p.ReadResourceResponse = providers.ReadResourceResponse{NewState: cty.ObjectVal(map[string]cty.Value{
"id": cty.StringVal("yes"),
})}
op, configCleanup := testOperationRefresh(t, "./testdata/empty")
defer configCleanup()
run, err := b.Operation(context.Background(), op)
if err != nil {
t.Fatalf("bad: %s", err)
}
<-run.Done()
if !p.ReadResourceCalled {
t.Fatal("ReadResource should be called")
}
checkState(t, b.StateOutPath, `
test_instance.foo:
ID = yes
provider = provider["registry.terraform.io/hashicorp/test"]
`)
}
// GH-12174
func TestLocal_refreshNilModuleWithInput(t *testing.T) {
b, cleanup := TestLocal(t)
defer cleanup()
p := TestLocalProvider(t, b, "test", refreshFixtureSchema())
testStateFile(t, b.StatePath, testRefreshState())
p.ReadResourceFn = nil
p.ReadResourceResponse = providers.ReadResourceResponse{NewState: cty.ObjectVal(map[string]cty.Value{
"id": cty.StringVal("yes"),
})}
b.OpInput = true
op, configCleanup := testOperationRefresh(t, "./testdata/empty")
defer configCleanup()
run, err := b.Operation(context.Background(), op)
if err != nil {
t.Fatalf("bad: %s", err)
}
<-run.Done()
if !p.ReadResourceCalled {
t.Fatal("ReadResource should be called")
}
checkState(t, b.StateOutPath, `
test_instance.foo:
ID = yes
provider = provider["registry.terraform.io/hashicorp/test"]
`)
}
func TestLocal_refreshInput(t *testing.T) {
b, cleanup := TestLocal(t)
defer cleanup()
p := TestLocalProvider(t, b, "test", refreshFixtureSchema())
testStateFile(t, b.StatePath, testRefreshState())
p.GetSchemaReturn = &terraform.ProviderSchema{
schema := &terraform.ProviderSchema{
Provider: &configschema.Block{
Attributes: map[string]*configschema.Attribute{
"value": {Type: cty.String, Optional: true},
@ -129,12 +64,17 @@ func TestLocal_refreshInput(t *testing.T) {
ResourceTypes: map[string]*configschema.Block{
"test_instance": {
Attributes: map[string]*configschema.Attribute{
"id": {Type: cty.String, Computed: true},
"foo": {Type: cty.String, Optional: true},
"id": {Type: cty.String, Optional: true},
"ami": {Type: cty.String, Optional: true},
},
},
},
}
p := TestLocalProvider(t, b, "test", schema)
testStateFile(t, b.StatePath, testRefreshState())
p.ReadResourceFn = nil
p.ReadResourceResponse = providers.ReadResourceResponse{NewState: cty.ObjectVal(map[string]cty.Value{
"id": cty.StringVal("yes"),

2
backend/local/testdata/refresh-var-unset/main.tf vendored
View File

@ -1,7 +1,7 @@
variable "should_ask" {}
provider "test" {
value = "${var.should_ask}"
value = var.should_ask
}
resource "test_instance" "foo" {

16
backend/remote-state/azure/arm_client.go
View File

@ -59,18 +59,28 @@ func buildArmClient(config BackendConfig) (*ArmClient, error) {
builder := authentication.Builder{
ClientID: config.ClientID,
ClientSecret: config.ClientSecret,
SubscriptionID: config.SubscriptionID,
TenantID: config.TenantID,
CustomResourceManagerEndpoint: config.CustomResourceManagerEndpoint,
MetadataURL: config.MetadataHost,
Environment: config.Environment,
MsiEndpoint: config.MsiEndpoint,
ClientSecretDocsLink: "https://www.terraform.io/docs/providers/azurerm/guides/service_principal_client_secret.html",
// Service Principal (Client Certificate)
ClientCertPassword: config.ClientCertificatePassword,
ClientCertPath: config.ClientCertificatePath,
// Service Principal (Client Secret)
ClientSecret: config.ClientSecret,
// Managed Service Identity
MsiEndpoint: config.MsiEndpoint,
// Feature Toggles
SupportsAzureCliToken: true,
SupportsClientCertAuth: true,
SupportsClientSecretAuth: true,
SupportsManagedServiceIdentity: config.UseMsi,
// TODO: support for Client Certificate auth
}
armConfig, err := builder.Build()
if err != nil {

50
backend/remote-state/azure/backend.go
View File

@ -30,6 +30,13 @@ func New() backend.Backend {
Description: "The blob key.",
},
"metadata_host": {
Type: schema.TypeString,
Required: true,
DefaultFunc: schema.EnvDefaultFunc("ARM_METADATA_HOST", ""),
Description: "The Metadata URL which will be used to obtain the Cloud Environment.",
},
"environment": {
Type: schema.TypeString,
Optional: true,
@ -71,11 +78,11 @@ func New() backend.Backend {
DefaultFunc: schema.EnvDefaultFunc("ARM_CLIENT_ID", ""),
},
"client_secret": {
"endpoint": {
Type: schema.TypeString,
Optional: true,
Description: "The Client Secret.",
DefaultFunc: schema.EnvDefaultFunc("ARM_CLIENT_SECRET", ""),
Description: "A custom Endpoint used to access the Azure Resource Manager API's.",
DefaultFunc: schema.EnvDefaultFunc("ARM_ENDPOINT", ""),
},
"subscription_id": {
@ -92,13 +99,35 @@ func New() backend.Backend {
DefaultFunc: schema.EnvDefaultFunc("ARM_TENANT_ID", ""),
},
// Service Principal (Client Certificate) specific
"client_certificate_password": {
Type: schema.TypeString,
Optional: true,
Description: "The password associated with the Client Certificate specified in `client_certificate_path`",
DefaultFunc: schema.EnvDefaultFunc("ARM_CLIENT_CERTIFICATE_PASSWORD", ""),
},
"client_certificate_path": {
Type: schema.TypeString,
Optional: true,
Description: "The path to the PFX file used as the Client Certificate when authenticating as a Service Principal",
DefaultFunc: schema.EnvDefaultFunc("ARM_CLIENT_CERTIFICATE_PATH", ""),
},
// Service Principal (Client Secret) specific
"client_secret": {
Type: schema.TypeString,
Optional: true,
Description: "The Client Secret.",
DefaultFunc: schema.EnvDefaultFunc("ARM_CLIENT_SECRET", ""),
},
// Managed Service Identity specific
"use_msi": {
Type: schema.TypeBool,
Optional: true,
Description: "Should Managed Service Identity be used?.",
DefaultFunc: schema.EnvDefaultFunc("ARM_USE_MSI", false),
},
"msi_endpoint": {
Type: schema.TypeString,
Optional: true,
@ -106,13 +135,6 @@ func New() backend.Backend {
DefaultFunc: schema.EnvDefaultFunc("ARM_MSI_ENDPOINT", ""),
},
"endpoint": {
Type: schema.TypeString,
Optional: true,
Description: "A custom Endpoint used to access the Azure Resource Manager API's.",
DefaultFunc: schema.EnvDefaultFunc("ARM_ENDPOINT", ""),
},
// Deprecated fields
"arm_client_id": {
Type: schema.TypeString,
@ -167,8 +189,11 @@ type BackendConfig struct {
// Optional
AccessKey string
ClientID string
ClientCertificatePassword string
ClientCertificatePath string
ClientSecret string
CustomResourceManagerEndpoint string
MetadataHost string
Environment string
MsiEndpoint string
ResourceGroupName string
@ -199,8 +224,11 @@ func (b *Backend) configure(ctx context.Context) error {
config := BackendConfig{
AccessKey: data.Get("access_key").(string),
ClientID: clientId,
ClientCertificatePassword: data.Get("client_certificate_password").(string),
ClientCertificatePath: data.Get("client_certificate_path").(string),
ClientSecret: clientSecret,
CustomResourceManagerEndpoint: data.Get("endpoint").(string),
MetadataHost: data.Get("metadata_host").(string),
Environment: data.Get("environment").(string),
MsiEndpoint: data.Get("msi_endpoint").(string),
ResourceGroupName: data.Get("resource_group_name").(string),

41
backend/remote-state/azure/backend_test.go
View File

@ -123,7 +123,44 @@ func TestBackendSASTokenBasic(t *testing.T) {
backend.TestBackendStates(t, b)
}
func TestBackendServicePrincipalBasic(t *testing.T) {
func TestBackendServicePrincipalClientCertificateBasic(t *testing.T) {
testAccAzureBackend(t)
clientCertPassword := os.Getenv("ARM_CLIENT_CERTIFICATE_PASSWORD")
clientCertPath := os.Getenv("ARM_CLIENT_CERTIFICATE_PATH")
if clientCertPath == "" {
t.Skip("Skipping since `ARM_CLIENT_CERTIFICATE_PATH` is not specified!")
}
rs := acctest.RandString(4)
res := testResourceNames(rs, "testState")
armClient := buildTestClient(t, res)
ctx := context.TODO()
err := armClient.buildTestResources(ctx, &res)
defer armClient.destroyTestResources(ctx, res)
if err != nil {
t.Fatalf("Error creating Test Resources: %q", err)
}
b := backend.TestBackendConfig(t, New(), backend.TestWrapConfig(map[string]interface{}{
"storage_account_name": res.storageAccountName,
"container_name": res.storageContainerName,
"key": res.storageKeyName,
"resource_group_name": res.resourceGroup,
"subscription_id": os.Getenv("ARM_SUBSCRIPTION_ID"),
"tenant_id": os.Getenv("ARM_TENANT_ID"),
"client_id": os.Getenv("ARM_CLIENT_ID"),
"client_certificate_password": clientCertPassword,
"client_certificate_path": clientCertPath,
"environment": os.Getenv("ARM_ENVIRONMENT"),
"endpoint": os.Getenv("ARM_ENDPOINT"),
})).(*Backend)
backend.TestBackendStates(t, b)
}
func TestBackendServicePrincipalClientSecretBasic(t *testing.T) {
testAccAzureBackend(t)
rs := acctest.RandString(4)
res := testResourceNames(rs, "testState")
@ -152,7 +189,7 @@ func TestBackendServicePrincipalBasic(t *testing.T) {
backend.TestBackendStates(t, b)
}
func TestBackendServicePrincipalCustomEndpoint(t *testing.T) {
func TestBackendServicePrincipalClientSecretCustomEndpoint(t *testing.T) {
testAccAzureBackend(t)
// this is only applicable for Azure Stack.

208
backend/remote-state/consul/client.go
View File

@ -9,6 +9,7 @@ import (
"errors"
"fmt"
"log"
"strings"
"sync"
"time"
@ -71,7 +72,9 @@ func (c *RemoteClient) Get() (*remote.Payload, error) {
c.mu.Lock()
defer c.mu.Unlock()
pair, _, err := c.Client.KV().Get(c.Path, nil)
kv := c.Client.KV()
chunked, hash, chunks, pair, err := c.chunkedMode()
if err != nil {
return nil, err
}
@ -81,17 +84,36 @@ func (c *RemoteClient) Get() (*remote.Payload, error) {
c.modifyIndex = pair.ModifyIndex
payload := pair.Value
var payload []byte
if chunked {
for _, c := range chunks {
pair, _, err := kv.Get(c, nil)
if err != nil {
return nil, err
}
if pair == nil {
return nil, fmt.Errorf("Key %q could not be found", c)
}
payload = append(payload, pair.Value[:]...)
}
} else {
payload = pair.Value
}
// If the payload starts with 0x1f, it's gzip, not json
if len(pair.Value) >= 1 && pair.Value[0] == '\x1f' {
if data, err := uncompressState(pair.Value); err == nil {
payload = data
} else {
if len(payload) >= 1 && payload[0] == '\x1f' {
payload, err = uncompressState(payload)
if err != nil {
return nil, err
}
}
md5 := md5.Sum(pair.Value)
md5 := md5.Sum(payload)
if hash != "" && fmt.Sprintf("%x", md5) != hash {
return nil, fmt.Errorf("The remote state does not match the expected hash")
}
return &remote.Payload{
Data: payload,
MD5: md5[:],
@ -99,9 +121,65 @@ func (c *RemoteClient) Get() (*remote.Payload, error) {
}
func (c *RemoteClient) Put(data []byte) error {
// The state can be stored in 4 different ways, based on the payload size
// and whether the user enabled gzip:
// - single entry mode with plain JSON: a single JSON is stored at
// "tfstate/my_project"
// - single entry mode gzip: the JSON payload is first gziped and stored at
// "tfstate/my_project"
// - chunked mode with plain JSON: the JSON payload is split in pieces and
// stored like so:
// - "tfstate/my_project" -> a JSON payload that contains the path of
// the chunks and an MD5 sum like so:
// {
// "current-hash": "abcdef1234",
// "chunks": [
// "tfstate/my_project/tfstate.abcdef1234/0",
// "tfstate/my_project/tfstate.abcdef1234/1",
// "tfstate/my_project/tfstate.abcdef1234/2",
// ]
// }
// - "tfstate/my_project/tfstate.abcdef1234/0" -> The first chunk
// - "tfstate/my_project/tfstate.abcdef1234/1" -> The next one
// - ...
// - chunked mode with gzip: the same system but we gziped the JSON payload
// before splitting it in chunks
//
// When overwritting the current state, we need to clean the old chunks if
// we were in chunked mode (no matter whether we need to use chunks for the
// new one). To do so based on the 4 possibilities above we look at the
// value at "tfstate/my_project" and if it is:
// - absent then it's a new state and there will be nothing to cleanup,
// - not a JSON payload we were in single entry mode with gzip so there will
// be nothing to cleanup
// - a JSON payload, then we were either single entry mode with plain JSON
// or in chunked mode. To differentiate between the two we look whether a
// "current-hash" key is present in the payload. If we find one we were
// in chunked mode and we will need to remove the old chunks (whether or
// not we were using gzip does not matter in that case).
c.mu.Lock()
defer c.mu.Unlock()
kv := c.Client.KV()
// First we determine what mode we were using and to prepare the cleanup
chunked, hash, _, _, err := c.chunkedMode()
if err != nil {
return err
}
cleanupOldChunks := func() {}
if chunked {
cleanupOldChunks = func() {
// We ignore all errors that can happen here because we already
// saved the new state and there is no way to return a warning to
// the user. We may end up with dangling chunks but there is no way
// to be sure we won't.
path := strings.TrimRight(c.Path, "/") + fmt.Sprintf("/tfstate.%s/", hash)
kv.DeleteTree(path, nil)
}
}
payload := data
if c.GZip {
if compressedState, err := compressState(data); err == nil {
@ -111,8 +189,6 @@ func (c *RemoteClient) Put(data []byte) error {
}
}
kv := c.Client.KV()
// default to doing a CAS
verb := consulapi.KVCAS
@ -122,9 +198,44 @@ func (c *RemoteClient) Put(data []byte) error {
verb = consulapi.KVSet
}
// If the payload is too large we first write the chunks and replace it
// 524288 is the default value, we just hope the user did not set a smaller
// one but there is really no reason for them to do so, if they changed it
// it is certainly to set a larger value.
limit := 524288
if len(payload) > limit {
md5 := md5.Sum(data)
chunks := split(payload, limit)
chunkPaths := make([]string, 0)
// First we write the new chunks
for i, p := range chunks {
path := strings.TrimRight(c.Path, "/") + fmt.Sprintf("/tfstate.%x/%d", md5, i)
chunkPaths = append(chunkPaths, path)
_, err := kv.Put(&consulapi.KVPair{
Key: path,
Value: p,
}, nil)
if err != nil {
return err
}
}
// We update the link to point to the new chunks
payload, err = json.Marshal(map[string]interface{}{
"current-hash": fmt.Sprintf("%x", md5),
"chunks": chunkPaths,
})
if err != nil {
return err
}
}
var txOps consulapi.KVTxnOps
// KV.Put doesn't return the new index, so we use a single operation
// transaction to get the new index with a single request.
txOps := consulapi.KVTxnOps{
txOps = consulapi.KVTxnOps{
&consulapi.KVTxnOp{
Verb: verb,
Key: c.Path,
@ -137,7 +248,6 @@ func (c *RemoteClient) Put(data []byte) error {
if err != nil {
return err
}
// transaction was rolled back
if !ok {
return fmt.Errorf("consul CAS failed with transaction errors: %v", resp.Errors)
@ -149,6 +259,10 @@ func (c *RemoteClient) Put(data []byte) error {
}
c.modifyIndex = resp.Results[0].ModifyIndex
// We remove all the old chunks
cleanupOldChunks()
return nil
}
@ -157,17 +271,36 @@ func (c *RemoteClient) Delete() error {
defer c.mu.Unlock()
kv := c.Client.KV()
_, err := kv.Delete(c.Path, nil)
chunked, hash, _, _, err := c.chunkedMode()
if err != nil {
return err
}
_, err = kv.Delete(c.Path, nil)
// If there were chunks we need to remove them
if chunked {
path := strings.TrimRight(c.Path, "/") + fmt.Sprintf("/tfstate.%s/", hash)
kv.DeleteTree(path, nil)
}
return err
}
func (c *RemoteClient) lockPath() string {
// we sanitize the path for the lock as Consul does not like having
// two consecutive slashes for the lock path
return strings.TrimRight(c.Path, "/")
}
func (c *RemoteClient) putLockInfo(info *statemgr.LockInfo) error {
info.Path = c.Path
info.Created = time.Now().UTC()
kv := c.Client.KV()
_, err := kv.Put(&consulapi.KVPair{
Key: c.Path + lockInfoSuffix,
Key: c.lockPath() + lockInfoSuffix,
Value: info.Marshal(),
}, nil)
@ -175,7 +308,7 @@ func (c *RemoteClient) putLockInfo(info *statemgr.LockInfo) error {
}
func (c *RemoteClient) getLockInfo() (*statemgr.LockInfo, error) {
path := c.Path + lockInfoSuffix
path := c.lockPath() + lockInfoSuffix
pair, _, err := c.Client.KV().Get(path, nil)
if err != nil {
return nil, err
@ -239,7 +372,7 @@ func (c *RemoteClient) lock() (string, error) {
c.info.ID = lockSession
opts := &consulapi.LockOptions{
Key: c.Path + lockSuffix,
Key: c.lockPath() + lockSuffix,
Session: lockSession,
// only wait briefly, so terraform has the choice to fail fast or
@ -412,8 +545,8 @@ func (c *RemoteClient) unlock(id string) error {
}
// We ignore the errors that may happen during cleanup
kv := c.Client.KV()
kv.Delete(c.Path+lockSuffix, nil)
kv.Delete(c.Path+lockInfoSuffix, nil)
kv.Delete(c.lockPath()+lockSuffix, nil)
kv.Delete(c.lockPath()+lockInfoSuffix, nil)
return nil
}
@ -441,7 +574,7 @@ func (c *RemoteClient) unlock(id string) error {
var errs error
if _, err := kv.Delete(c.Path+lockInfoSuffix, nil); err != nil {
if _, err := kv.Delete(c.lockPath()+lockInfoSuffix, nil); err != nil {
errs = multierror.Append(errs, err)
}
@ -488,3 +621,42 @@ func uncompressState(data []byte) ([]byte, error) {
}
return b.Bytes(), nil
}
func split(payload []byte, limit int) [][]byte {
var chunk []byte
chunks := make([][]byte, 0, len(payload)/limit+1)
for len(payload) >= limit {
chunk, payload = payload[:limit], payload[limit:]
chunks = append(chunks, chunk)
}
if len(payload) > 0 {
chunks = append(chunks, payload[:])
}
return chunks
}
func (c *RemoteClient) chunkedMode() (bool, string, []string, *consulapi.KVPair, error) {
kv := c.Client.KV()
pair, _, err := kv.Get(c.Path, nil)
if err != nil {
return false, "", nil, pair, err
}
if pair != nil {
var d map[string]interface{}
err = json.Unmarshal(pair.Value, &d)
// If there is an error when unmarshaling the payload, the state has
// probably been gziped in single entry mode.
if err == nil {
// If we find the "current-hash" key we were in chunked mode
hash, ok := d["current-hash"]
if ok {
chunks := make([]string, 0)
for _, c := range d["chunks"].([]interface{}) {
chunks = append(chunks, c.(string))
}
return true, hash.(string), chunks, pair, nil
}
}
}
return false, "", nil, pair, nil
}

308
backend/remote-state/consul/client_test.go
View File

@ -1,9 +1,14 @@
package consul
import (
"bytes"
"context"
"encoding/json"
"fmt"
"math/rand"
"net"
"reflect"
"strings"
"sync"
"testing"
"time"
@ -19,20 +24,29 @@ func TestRemoteClient_impl(t *testing.T) {
}
func TestRemoteClient(t *testing.T) {
// Get the backend
b := backend.TestBackendConfig(t, New(), backend.TestWrapConfig(map[string]interface{}{
"address": srv.HTTPAddr,
"path": fmt.Sprintf("tf-unit/%s", time.Now().String()),
}))
// Grab the client
state, err := b.StateMgr(backend.DefaultStateName)
if err != nil {
t.Fatalf("err: %s", err)
testCases := []string{
fmt.Sprintf("tf-unit/%s", time.Now().String()),
fmt.Sprintf("tf-unit/%s/", time.Now().String()),
}
// Test
remote.TestClient(t, state.(*remote.State).Client)
for _, path := range testCases {
t.Run(path, func(*testing.T) {
// Get the backend
b := backend.TestBackendConfig(t, New(), backend.TestWrapConfig(map[string]interface{}{
"address": srv.HTTPAddr,
"path": path,
}))
// Grab the client
state, err := b.StateMgr(backend.DefaultStateName)
if err != nil {
t.Fatalf("err: %s", err)
}
// Test
remote.TestClient(t, state.(*remote.State).Client)
})
}
}
// test the gzip functionality of the client
@ -71,30 +85,176 @@ func TestRemoteClient_gzipUpgrade(t *testing.T) {
remote.TestClient(t, state.(*remote.State).Client)
}
// TestConsul_largeState tries to write a large payload using the Consul state
// manager, as there is a limit to the size of the values in the KV store it
// will need to be split up before being saved and put back together when read.
func TestConsul_largeState(t *testing.T) {
path := "tf-unit/test-large-state"
b := backend.TestBackendConfig(t, New(), backend.TestWrapConfig(map[string]interface{}{
"address": srv.HTTPAddr,
"path": path,
}))
s, err := b.StateMgr(backend.DefaultStateName)
if err != nil {
t.Fatal(err)
}
c := s.(*remote.State).Client.(*RemoteClient)
c.Path = path
// testPaths fails the test if the keys found at the prefix don't match
// what is expected
testPaths := func(t *testing.T, expected []string) {
kv := c.Client.KV()
pairs, _, err := kv.List(c.Path, nil)
if err != nil {
t.Fatal(err)
}
res := make([]string, 0)
for _, p := range pairs {
res = append(res, p.Key)
}
if !reflect.DeepEqual(res, expected) {
t.Fatalf("Wrong keys: %#v", res)
}
}
testPayload := func(t *testing.T, data map[string]string, keys []string) {
payload, err := json.Marshal(data)
if err != nil {
t.Fatal(err)
}
err = c.Put(payload)
if err != nil {
t.Fatal("could not put payload", err)
}
remote, err := c.Get()
if err != nil {
t.Fatal(err)
}
// md5 := md5.Sum(payload)
// if !bytes.Equal(md5[:], remote.MD5) {
// t.Fatal("the md5 sums do not match")
// }
if !bytes.Equal(payload, remote.Data) {
t.Fatal("the data do not match")
}
testPaths(t, keys)
}
// The default limit for the size of the value in Consul is 524288 bytes
testPayload(
t,
map[string]string{
"foo": strings.Repeat("a", 524288+2),
},
[]string{
"tf-unit/test-large-state",
"tf-unit/test-large-state/tfstate.2cb96f52c9fff8e0b56cb786ec4d2bed/0",
"tf-unit/test-large-state/tfstate.2cb96f52c9fff8e0b56cb786ec4d2bed/1",
},
)
// We try to replace the payload with a small one, the old chunks should be removed
testPayload(
t,
map[string]string{"var": "a"},
[]string{"tf-unit/test-large-state"},
)
// Test with gzip and chunks
b = backend.TestBackendConfig(t, New(), backend.TestWrapConfig(map[string]interface{}{
"address": srv.HTTPAddr,
"path": path,
"gzip": true,
}))
s, err = b.StateMgr(backend.DefaultStateName)
if err != nil {
t.Fatal(err)
}
c = s.(*remote.State).Client.(*RemoteClient)
c.Path = path
// We need a long random string so it results in multiple chunks even after
// being gziped
// We use a fixed seed so the test can be reproductible
rand.Seed(1234)
RandStringRunes := func(n int) string {
var letterRunes = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
b := make([]rune, n)
for i := range b {
b[i] = letterRunes[rand.Intn(len(letterRunes))]
}
return string(b)
}
testPayload(
t,
map[string]string{
"bar": RandStringRunes(5 * (524288 + 2)),
},
[]string{
"tf-unit/test-large-state",
"tf-unit/test-large-state/tfstate.58e8160335864b520b1cc7f2222a4019/0",
"tf-unit/test-large-state/tfstate.58e8160335864b520b1cc7f2222a4019/1",
"tf-unit/test-large-state/tfstate.58e8160335864b520b1cc7f2222a4019/2",
"tf-unit/test-large-state/tfstate.58e8160335864b520b1cc7f2222a4019/3",
},
)
// Deleting the state should remove all chunks
err = c.Delete()
if err != nil {
t.Fatal(err)
}
testPaths(t, []string{})
}
func TestConsul_stateLock(t *testing.T) {
path := fmt.Sprintf("tf-unit/%s", time.Now().String())
// create 2 instances to get 2 remote.Clients
sA, err := backend.TestBackendConfig(t, New(), backend.TestWrapConfig(map[string]interface{}{
"address": srv.HTTPAddr,
"path": path,
})).StateMgr(backend.DefaultStateName)
if err != nil {
t.Fatal(err)
testCases := []string{
fmt.Sprintf("tf-unit/%s", time.Now().String()),
fmt.Sprintf("tf-unit/%s/", time.Now().String()),
}
sB, err := backend.TestBackendConfig(t, New(), backend.TestWrapConfig(map[string]interface{}{
"address": srv.HTTPAddr,
"path": path,
})).StateMgr(backend.DefaultStateName)
if err != nil {
t.Fatal(err)
}
for _, path := range testCases {
t.Run(path, func(*testing.T) {
// create 2 instances to get 2 remote.Clients
sA, err := backend.TestBackendConfig(t, New(), backend.TestWrapConfig(map[string]interface{}{
"address": srv.HTTPAddr,
"path": path,
})).StateMgr(backend.DefaultStateName)
if err != nil {
t.Fatal(err)
}
remote.TestRemoteLocks(t, sA.(*remote.State).Client, sB.(*remote.State).Client)
sB, err := backend.TestBackendConfig(t, New(), backend.TestWrapConfig(map[string]interface{}{
"address": srv.HTTPAddr,
"path": path,
})).StateMgr(backend.DefaultStateName)
if err != nil {
t.Fatal(err)
}
remote.TestRemoteLocks(t, sA.(*remote.State).Client, sB.(*remote.State).Client)
})
}
}
func TestConsul_destroyLock(t *testing.T) {
testCases := []string{
fmt.Sprintf("tf-unit/%s", time.Now().String()),
fmt.Sprintf("tf-unit/%s/", time.Now().String()),
}
testLock := func(client *RemoteClient, lockPath string) {
// get the lock val
pair, _, err := client.Client.KV().Get(lockPath, nil)
@ -106,62 +266,66 @@ func TestConsul_destroyLock(t *testing.T) {
}
}
// Get the backend
b := backend.TestBackendConfig(t, New(), backend.TestWrapConfig(map[string]interface{}{
"address": srv.HTTPAddr,
"path": fmt.Sprintf("tf-unit/%s", time.Now().String()),
}))
for _, path := range testCases {
t.Run(path, func(*testing.T) {
// Get the backend
b := backend.TestBackendConfig(t, New(), backend.TestWrapConfig(map[string]interface{}{
"address": srv.HTTPAddr,
"path": path,
}))
// Grab the client
s, err := b.StateMgr(backend.DefaultStateName)
if err != nil {
t.Fatalf("err: %s", err)
}
// Grab the client
s, err := b.StateMgr(backend.DefaultStateName)
if err != nil {
t.Fatalf("err: %s", err)
}
clientA := s.(*remote.State).Client.(*RemoteClient)
clientA := s.(*remote.State).Client.(*RemoteClient)
info := statemgr.NewLockInfo()
id, err := clientA.Lock(info)
if err != nil {
t.Fatal(err)
}
info := statemgr.NewLockInfo()
id, err := clientA.Lock(info)
if err != nil {
t.Fatal(err)
}
lockPath := clientA.Path + lockSuffix
lockPath := clientA.Path + lockSuffix
if err := clientA.Unlock(id); err != nil {
t.Fatal(err)
}
if err := clientA.Unlock(id); err != nil {
t.Fatal(err)
}
testLock(clientA, lockPath)
testLock(clientA, lockPath)
// The release the lock from a second client to test the
// `terraform force-unlock <lock_id>` functionnality
s, err = b.StateMgr(backend.DefaultStateName)
if err != nil {
t.Fatalf("err: %s", err)
}
// The release the lock from a second client to test the
// `terraform force-unlock <lock_id>` functionnality
s, err = b.StateMgr(backend.DefaultStateName)
if err != nil {
t.Fatalf("err: %s", err)
}
clientB := s.(*remote.State).Client.(*RemoteClient)
clientB := s.(*remote.State).Client.(*RemoteClient)
info = statemgr.NewLockInfo()
id, err = clientA.Lock(info)
if err != nil {
t.Fatal(err)
}
info = statemgr.NewLockInfo()
id, err = clientA.Lock(info)
if err != nil {
t.Fatal(err)
}
if err := clientB.Unlock(id); err != nil {
t.Fatal(err)
}
if err := clientB.Unlock(id); err != nil {
t.Fatal(err)
}
testLock(clientA, lockPath)
testLock(clientA, lockPath)
err = clientA.Unlock(id)
err = clientA.Unlock(id)
if err == nil {
t.Fatal("consul lock should have been lost")
}
if err.Error() != "consul lock was lost" {
t.Fatal("got wrong error", err)
if err == nil {
t.Fatal("consul lock should have been lost")
}
if err.Error() != "consul lock was lost" {
t.Fatal("got wrong error", err)
}
})
}
}

17
backend/remote-state/http/backend.go
View File

@ -22,44 +22,49 @@ func New() backend.Backend {
"address": &schema.Schema{
Type: schema.TypeString,
Required: true,
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_ADDRESS", nil),
Description: "The address of the REST endpoint",
},
"update_method": &schema.Schema{
Type: schema.TypeString,
Optional: true,
Default: "POST",
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_UPDATE_METHOD", "POST"),
Description: "HTTP method to use when updating state",
},
"lock_address": &schema.Schema{
Type: schema.TypeString,
Optional: true,
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_LOCK_ADDRESS", nil),
Description: "The address of the lock REST endpoint",
},
"unlock_address": &schema.Schema{
Type: schema.TypeString,
Optional: true,
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_UNLOCK_ADDRESS", nil),
Description: "The address of the unlock REST endpoint",
},
"lock_method": &schema.Schema{
Type: schema.TypeString,
Optional: true,
Default: "LOCK",
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_LOCK_METHOD", "LOCK"),
Description: "The HTTP method to use when locking",
},
"unlock_method": &schema.Schema{
Type: schema.TypeString,
Optional: true,
Default: "UNLOCK",
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_UNLOCK_METHOD", "UNLOCK"),
Description: "The HTTP method to use when unlocking",
},
"username": &schema.Schema{
Type: schema.TypeString,
Optional: true,
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_USERNAME", nil),
Description: "The username for HTTP basic authentication",
},
"password": &schema.Schema{
Type: schema.TypeString,
Optional: true,
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_PASSWORD", nil),
Description: "The password for HTTP basic authentication",
},
"skip_cert_verification": &schema.Schema{
@ -71,19 +76,19 @@ func New() backend.Backend {
"retry_max": &schema.Schema{
Type: schema.TypeInt,
Optional: true,
Default: 2,
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_RETRY_MAX", 2),
Description: "The number of HTTP request retries.",
},
"retry_wait_min": &schema.Schema{
Type: schema.TypeInt,
Optional: true,
Default: 1,
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_RETRY_WAIT_MIN", 1),
Description: "The minimum time in seconds to wait between HTTP request attempts.",
},
"retry_wait_max": &schema.Schema{
Type: schema.TypeInt,
Optional: true,
Default: 30,
DefaultFunc: schema.EnvDefaultFunc("TF_HTTP_RETRY_WAIT_MAX", 30),
Description: "The maximum time in seconds to wait between HTTP request attempts.",
},
},

74
backend/remote-state/http/backend_test.go
View File

@ -1,6 +1,7 @@
package http
import (
"os"
"testing"
"time"
@ -88,3 +89,76 @@ func TestHTTPClientFactory(t *testing.T) {
t.Fatalf("Expected retry_wait_max \"%s\", got \"%s\"", 150*time.Second, client.Client.RetryWaitMax)
}
}
func TestHTTPClientFactoryWithEnv(t *testing.T) {
// env
conf := map[string]string{
"address": "http://127.0.0.1:8888/foo",
"update_method": "BLAH",
"lock_address": "http://127.0.0.1:8888/bar",
"lock_method": "BLIP",
"unlock_address": "http://127.0.0.1:8888/baz",
"unlock_method": "BLOOP",
"username": "user",
"password": "pass",
"retry_max": "999",
"retry_wait_min": "15",
"retry_wait_max": "150",
}
defer testWithEnv(t, "TF_HTTP_ADDRESS", conf["address"])()
defer testWithEnv(t, "TF_HTTP_UPDATE_METHOD", conf["update_method"])()
defer testWithEnv(t, "TF_HTTP_LOCK_ADDRESS", conf["lock_address"])()
defer testWithEnv(t, "TF_HTTP_UNLOCK_ADDRESS", conf["unlock_address"])()
defer testWithEnv(t, "TF_HTTP_LOCK_METHOD", conf["lock_method"])()
defer testWithEnv(t, "TF_HTTP_UNLOCK_METHOD", conf["unlock_method"])()
defer testWithEnv(t, "TF_HTTP_USERNAME", conf["username"])()
defer testWithEnv(t, "TF_HTTP_PASSWORD", conf["password"])()
defer testWithEnv(t, "TF_HTTP_RETRY_MAX", conf["retry_max"])()
defer testWithEnv(t, "TF_HTTP_RETRY_WAIT_MIN", conf["retry_wait_min"])()
defer testWithEnv(t, "TF_HTTP_RETRY_WAIT_MAX", conf["retry_wait_max"])()
b := backend.TestBackendConfig(t, New(), nil).(*Backend)
client := b.client
if client == nil {
t.Fatal("Unexpected failure, EnvDefaultFunc")
}
if client.UpdateMethod != "BLAH" {
t.Fatalf("Expected update_method \"%s\", got \"%s\"", "BLAH", client.UpdateMethod)
}
if client.LockURL.String() != conf["lock_address"] || client.LockMethod != "BLIP" {
t.Fatalf("Unexpected lock_address \"%s\" vs \"%s\" or lock_method \"%s\" vs \"%s\"", client.LockURL.String(),
conf["lock_address"], client.LockMethod, conf["lock_method"])
}
if client.UnlockURL.String() != conf["unlock_address"] || client.UnlockMethod != "BLOOP" {
t.Fatalf("Unexpected unlock_address \"%s\" vs \"%s\" or unlock_method \"%s\" vs \"%s\"", client.UnlockURL.String(),
conf["unlock_address"], client.UnlockMethod, conf["unlock_method"])
}
if client.Username != "user" || client.Password != "pass" {
t.Fatalf("Unexpected username \"%s\" vs \"%s\" or password \"%s\" vs \"%s\"", client.Username, conf["username"],
client.Password, conf["password"])
}
if client.Client.RetryMax != 999 {
t.Fatalf("Expected retry_max \"%d\", got \"%d\"", 999, client.Client.RetryMax)
}
if client.Client.RetryWaitMin != 15*time.Second {
t.Fatalf("Expected retry_wait_min \"%s\", got \"%s\"", 15*time.Second, client.Client.RetryWaitMin)
}
if client.Client.RetryWaitMax != 150*time.Second {
t.Fatalf("Expected retry_wait_max \"%s\", got \"%s\"", 150*time.Second, client.Client.RetryWaitMax)
}
}
// testWithEnv sets an environment variable and returns a deferable func to clean up
func testWithEnv(t *testing.T, key string, value string) func() {
if err := os.Setenv(key, value); err != nil {
t.Fatalf("err: %v", err)
}
return func() {
if err := os.Unsetenv(key); err != nil {
t.Fatalf("err: %v", err)
}
}
}

35
backend/remote-state/oss/backend_state.go
View File

@ -57,28 +57,39 @@ func (b *Backend) Workspaces() ([]string, error) {
}
var options []oss.Option
options = append(options, oss.Prefix(b.statePrefix+"/"))
options = append(options, oss.Prefix(b.statePrefix+"/"), oss.MaxKeys(1000))
resp, err := bucket.ListObjects(options...)
if err != nil {
return nil, err
}
result := []string{backend.DefaultStateName}
prefix := b.statePrefix
for _, obj := range resp.Objects {
// we have 3 parts, the state prefix, the workspace name, and the state file: <prefix>/<worksapce-name>/<key>
if path.Join(b.statePrefix, b.stateKey) == obj.Key {
// filter the default workspace
continue
lastObj := ""
for {
for _, obj := range resp.Objects {
// we have 3 parts, the state prefix, the workspace name, and the state file: <prefix>/<worksapce-name>/<key>
if path.Join(b.statePrefix, b.stateKey) == obj.Key {
// filter the default workspace
continue
}
lastObj = obj.Key
parts := strings.Split(strings.TrimPrefix(obj.Key, prefix+"/"), "/")
if len(parts) > 0 && parts[0] != "" {
result = append(result, parts[0])
}
}
parts := strings.Split(strings.TrimPrefix(obj.Key, prefix+"/"), "/")
if len(parts) > 0 && parts[0] != "" {
result = append(result, parts[0])
if resp.IsTruncated {
if len(options) == 3 {
options[2] = oss.Marker(lastObj)
} else {
options = append(options, oss.Marker(lastObj))
}
resp, err = bucket.ListObjects(options...)
} else {
break
}
}
sort.Strings(result[1:])
return result, nil
}

38
backend/remote-state/oss/backend_test.go
View File

@ -67,6 +67,44 @@ func TestBackendConfig(t *testing.T) {
}
}
func TestBackendConfigWorkSpace(t *testing.T) {
testACC(t)
config := map[string]interface{}{
"region": "cn-beijing",
"bucket": "terraform-backend-oss-test",
"prefix": "mystate",
"key": "first.tfstate",
"tablestore_endpoint": "https://terraformstate.cn-beijing.ots.aliyuncs.com",
"tablestore_table": "TableStore",
}
b := backend.TestBackendConfig(t, New(), backend.TestWrapConfig(config)).(*Backend)
createOSSBucket(t, b.ossClient, "terraform-backend-oss-test")
defer deleteOSSBucket(t, b.ossClient, "terraform-backend-oss-test")
if _, err := b.Workspaces(); err != nil {
t.Fatal(err.Error())
}
if !strings.HasPrefix(b.ossClient.Config.Endpoint, "https://oss-cn-beijing") {
t.Fatalf("Incorrect region was provided")
}
if b.bucketName != "terraform-backend-oss-test" {
t.Fatalf("Incorrect bucketName was provided")
}
if b.statePrefix != "mystate" {
t.Fatalf("Incorrect state file path was provided")
}
if b.stateKey != "first.tfstate" {
t.Fatalf("Incorrect keyName was provided")
}
if b.ossClient.Config.AccessKeyID == "" {
t.Fatalf("No Access Key Id was provided")
}
if b.ossClient.Config.AccessKeySecret == "" {
t.Fatalf("No Secret Access Key was provided")
}
}
func TestBackendConfigProfile(t *testing.T) {
testACC(t)
config := map[string]interface{}{

156
backend/remote-state/s3/backend_test.go
View File

@ -18,50 +18,6 @@ import (
"github.com/hashicorp/terraform/states/remote"
)
const (
mockStsAssumeRoleArn = `arn:aws:iam::555555555555:role/AssumeRole`
mockStsAssumeRolePolicy = `{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "*",
"Resource": "*",
}
}`
mockStsAssumeRolePolicyArn = `arn:aws:iam::555555555555:policy/AssumeRolePolicy1`
mockStsAssumeRoleSessionName = `AssumeRoleSessionName`
mockStsAssumeRoleTagKey = `AssumeRoleTagKey`
mockStsAssumeRoleTagValue = `AssumeRoleTagValue`
mockStsAssumeRoleTransitiveTagKey = `AssumeRoleTagKey`
mockStsAssumeRoleValidResponse = `<AssumeRoleResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<AssumeRoleResult>
<AssumedRoleUser>
<Arn>arn:aws:sts::555555555555:assumed-role/role/AssumeRoleSessionName</Arn>
<AssumedRoleId>ARO123EXAMPLE123:AssumeRoleSessionName</AssumedRoleId>
</AssumedRoleUser>
<Credentials>
<AccessKeyId>AssumeRoleAccessKey</AccessKeyId>
<SecretAccessKey>AssumeRoleSecretKey</SecretAccessKey>
<SessionToken>AssumeRoleSessionToken</SessionToken>
<Expiration>2099-12-31T23:59:59Z</Expiration>
</Credentials>
</AssumeRoleResult>
<ResponseMetadata>
<RequestId>01234567-89ab-cdef-0123-456789abcdef</RequestId>
</ResponseMetadata>
</AssumeRoleResponse>`
mockStsGetCallerIdentityValidResponseBody = `<GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<GetCallerIdentityResult>
<Arn>arn:aws:iam::222222222222:user/Alice</Arn>
<UserId>AKIAI44QH8DHBEXAMPLE</UserId>
<Account>222222222222</Account>
</GetCallerIdentityResult>
<ResponseMetadata>
<RequestId>01234567-89ab-cdef-0123-456789abcdef</RequestId>
</ResponseMetadata>
</GetCallerIdentityResponse>`
)
var (
mockStsGetCallerIdentityRequestBody = url.Values{
"Action": []string{"GetCallerIdentity"},
@ -132,8 +88,8 @@ func TestBackendConfig_AssumeRole(t *testing.T) {
"bucket": "tf-test",
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
"role_arn": awsbase.MockStsAssumeRoleArn,
"session_name": awsbase.MockStsAssumeRoleSessionName,
},
Description: "role_arn",
MockStsEndpoints: []*awsbase.MockEndpoint{
@ -141,15 +97,15 @@ func TestBackendConfig_AssumeRole(t *testing.T) {
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"900"},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"RoleArn": []string{awsbase.MockStsAssumeRoleArn},
"RoleSessionName": []string{awsbase.MockStsAssumeRoleSessionName},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsAssumeRoleValidResponseBody, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},
@ -159,8 +115,8 @@ func TestBackendConfig_AssumeRole(t *testing.T) {
"bucket": "tf-test",
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
"role_arn": awsbase.MockStsAssumeRoleArn,
"session_name": awsbase.MockStsAssumeRoleSessionName,
},
Description: "assume_role_duration_seconds",
MockStsEndpoints: []*awsbase.MockEndpoint{
@ -168,26 +124,26 @@ func TestBackendConfig_AssumeRole(t *testing.T) {
Request: &awsbase.MockRequest{"POST", "/", url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"3600"},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"RoleArn": []string{awsbase.MockStsAssumeRoleArn},
"RoleSessionName": []string{awsbase.MockStsAssumeRoleSessionName},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsAssumeRoleValidResponseBody, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},
{
Config: map[string]interface{}{
"bucket": "tf-test",
"external_id": "AssumeRoleExternalId",
"external_id": awsbase.MockStsAssumeRoleExternalId,
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
"role_arn": awsbase.MockStsAssumeRoleArn,
"session_name": awsbase.MockStsAssumeRoleSessionName,
},
Description: "external_id",
MockStsEndpoints: []*awsbase.MockEndpoint{
@ -195,27 +151,27 @@ func TestBackendConfig_AssumeRole(t *testing.T) {
Request: &awsbase.MockRequest{"POST", "/", url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"900"},
"ExternalId": []string{"AssumeRoleExternalId"},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"ExternalId": []string{awsbase.MockStsAssumeRoleExternalId},
"RoleArn": []string{awsbase.MockStsAssumeRoleArn},
"RoleSessionName": []string{awsbase.MockStsAssumeRoleSessionName},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsAssumeRoleValidResponseBody, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},
{
Config: map[string]interface{}{
"assume_role_policy": mockStsAssumeRolePolicy,
"assume_role_policy": awsbase.MockStsAssumeRolePolicy,
"bucket": "tf-test",
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
"role_arn": awsbase.MockStsAssumeRoleArn,
"session_name": awsbase.MockStsAssumeRoleSessionName,
},
Description: "assume_role_policy",
MockStsEndpoints: []*awsbase.MockEndpoint{
@ -223,27 +179,27 @@ func TestBackendConfig_AssumeRole(t *testing.T) {
Request: &awsbase.MockRequest{"POST", "/", url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"900"},
"Policy": []string{mockStsAssumeRolePolicy},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"Policy": []string{awsbase.MockStsAssumeRolePolicy},
"RoleArn": []string{awsbase.MockStsAssumeRoleArn},
"RoleSessionName": []string{awsbase.MockStsAssumeRoleSessionName},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsAssumeRoleValidResponseBody, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},
{
Config: map[string]interface{}{
"assume_role_policy_arns": []interface{}{mockStsAssumeRolePolicyArn},
"assume_role_policy_arns": []interface{}{awsbase.MockStsAssumeRolePolicyArn},
"bucket": "tf-test",
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
"role_arn": awsbase.MockStsAssumeRoleArn,
"session_name": awsbase.MockStsAssumeRoleSessionName,
},
Description: "assume_role_policy_arns",
MockStsEndpoints: []*awsbase.MockEndpoint{
@ -251,29 +207,29 @@ func TestBackendConfig_AssumeRole(t *testing.T) {
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"900"},
"PolicyArns.member.1.arn": []string{mockStsAssumeRolePolicyArn},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"PolicyArns.member.1.arn": []string{awsbase.MockStsAssumeRolePolicyArn},
"RoleArn": []string{awsbase.MockStsAssumeRoleArn},
"RoleSessionName": []string{awsbase.MockStsAssumeRoleSessionName},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsAssumeRoleValidResponseBody, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},
{
Config: map[string]interface{}{
"assume_role_tags": map[string]interface{}{
mockStsAssumeRoleTagKey: mockStsAssumeRoleTagValue,
awsbase.MockStsAssumeRoleTagKey: awsbase.MockStsAssumeRoleTagValue,
},
"bucket": "tf-test",
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
"role_arn": awsbase.MockStsAssumeRoleArn,
"session_name": awsbase.MockStsAssumeRoleSessionName,
},
Description: "assume_role_tags",
MockStsEndpoints: []*awsbase.MockEndpoint{
@ -281,31 +237,31 @@ func TestBackendConfig_AssumeRole(t *testing.T) {
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"900"},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"Tags.member.1.Key": []string{mockStsAssumeRoleTagKey},
"Tags.member.1.Value": []string{mockStsAssumeRoleTagValue},
"RoleArn": []string{awsbase.MockStsAssumeRoleArn},
"RoleSessionName": []string{awsbase.MockStsAssumeRoleSessionName},
"Tags.member.1.Key": []string{awsbase.MockStsAssumeRoleTagKey},
"Tags.member.1.Value": []string{awsbase.MockStsAssumeRoleTagValue},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsAssumeRoleValidResponseBody, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},
{
Config: map[string]interface{}{
"assume_role_tags": map[string]interface{}{
mockStsAssumeRoleTagKey: mockStsAssumeRoleTagValue,
awsbase.MockStsAssumeRoleTagKey: awsbase.MockStsAssumeRoleTagValue,
},
"assume_role_transitive_tag_keys": []interface{}{mockStsAssumeRoleTagKey},
"assume_role_transitive_tag_keys": []interface{}{awsbase.MockStsAssumeRoleTagKey},
"bucket": "tf-test",
"key": "state",
"region": "us-west-1",
"role_arn": mockStsAssumeRoleArn,
"session_name": mockStsAssumeRoleSessionName,
"role_arn": awsbase.MockStsAssumeRoleArn,
"session_name": awsbase.MockStsAssumeRoleSessionName,
},
Description: "assume_role_transitive_tag_keys",
MockStsEndpoints: []*awsbase.MockEndpoint{
@ -313,18 +269,18 @@ func TestBackendConfig_AssumeRole(t *testing.T) {
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: url.Values{
"Action": []string{"AssumeRole"},
"DurationSeconds": []string{"900"},
"RoleArn": []string{mockStsAssumeRoleArn},
"RoleSessionName": []string{mockStsAssumeRoleSessionName},
"Tags.member.1.Key": []string{mockStsAssumeRoleTagKey},
"Tags.member.1.Value": []string{mockStsAssumeRoleTagValue},
"TransitiveTagKeys.member.1": []string{mockStsAssumeRoleTagKey},
"RoleArn": []string{awsbase.MockStsAssumeRoleArn},
"RoleSessionName": []string{awsbase.MockStsAssumeRoleSessionName},
"Tags.member.1.Key": []string{awsbase.MockStsAssumeRoleTagKey},
"Tags.member.1.Value": []string{awsbase.MockStsAssumeRoleTagValue},
"TransitiveTagKeys.member.1": []string{awsbase.MockStsAssumeRoleTagKey},
"Version": []string{"2011-06-15"},
}.Encode()},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsAssumeRoleValidResponse, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsAssumeRoleValidResponseBody, ContentType: "text/xml"},
},
{
Request: &awsbase.MockRequest{Method: "POST", Uri: "/", Body: mockStsGetCallerIdentityRequestBody},
Response: &awsbase.MockResponse{StatusCode: 200, Body: mockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
Response: &awsbase.MockResponse{StatusCode: 200, Body: awsbase.MockStsGetCallerIdentityValidResponseBody, ContentType: "text/xml"},
},
},
},

4
codecov.yml
View File

@ -12,5 +12,9 @@ coverage:
status:
project:
default:
informational: true
target: auto
threshold: "0.5%"
github_checks:
annotations: false

105
command/013_config_upgrade.go
View File

@ -9,6 +9,7 @@ import (
"sort"
"strings"
"github.com/apparentlymart/go-versions/versions"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/hclsyntax"
"github.com/hashicorp/hcl/v2/hclwrite"
@ -207,6 +208,7 @@ command and dealing with them before running this command again.
// Build up a list of required providers, uniquely by local name
requiredProviders := make(map[string]*configs.RequiredProvider)
rewritePaths := make(map[string]bool)
allProviderConstraints := make(map[string]getproviders.VersionConstraints)
// Step 1: copy all explicit provider requirements across
for path, file := range files {
@ -232,6 +234,24 @@ command and dealing with them before running this command again.
Requirement: rp.Requirement,
DeclRange: rp.DeclRange,
}
// Parse and store version constraints for later use when
// processing the provider redirect
constraints, err := getproviders.ParseVersionConstraints(rp.Requirement.Required.String())
if err != nil {
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid version constraint",
// The errors returned by ParseVersionConstraint
// already include the section of input that was
// incorrect, so we don't need to
// include that here.
Detail: fmt.Sprintf("Incorrect version constraint syntax: %s.", err.Error()),
Subject: rp.Requirement.DeclRange.Ptr(),
})
} else {
allProviderConstraints[rp.Name] = append(allProviderConstraints[rp.Name], constraints...)
}
}
}
}
@ -253,6 +273,23 @@ command and dealing with them before running this command again.
Name: p.Name,
}
}
// Parse and store version constraints for later use when
// processing the provider redirect
constraints, err := getproviders.ParseVersionConstraints(p.Version.Required.String())
if err != nil {
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid version constraint",
// The errors returned by ParseVersionConstraint
// already include the section of input that was
// incorrect, so we don't need to
// include that here.
Detail: fmt.Sprintf("Incorrect version constraint syntax: %s.", err.Error()),
Subject: p.Version.DeclRange.Ptr(),
})
} else {
allProviderConstraints[p.Name] = append(allProviderConstraints[p.Name], constraints...)
}
}
// Step 3: add missing provider requirements from resources
@ -291,7 +328,7 @@ command and dealing with them before running this command again.
// stated in the config. If there are any providers, attempt to detect
// their sources, and rewrite the config.
if len(requiredProviders) > 0 {
detectDiags := c.detectProviderSources(requiredProviders)
detectDiags := c.detectProviderSources(requiredProviders, allProviderConstraints)
diags = diags.Append(detectDiags)
if diags.HasErrors() {
c.Ui.Error("Unable to detect sources for providers")
@ -584,10 +621,11 @@ necessary adjustments, and then commit.
}
// For providers which need a source attribute, detect the source
func (c *ZeroThirteenUpgradeCommand) detectProviderSources(requiredProviders map[string]*configs.RequiredProvider) tfdiags.Diagnostics {
func (c *ZeroThirteenUpgradeCommand) detectProviderSources(requiredProviders map[string]*configs.RequiredProvider, allProviderConstraints map[string]getproviders.VersionConstraints) tfdiags.Diagnostics {
source := c.providerInstallSource()
var diags tfdiags.Diagnostics
providers:
for name, rp := range requiredProviders {
// If there's already an explicit source, skip it
if rp.Source != "" {
@ -599,9 +637,70 @@ func (c *ZeroThirteenUpgradeCommand) detectProviderSources(requiredProviders map
// parse process, because we know that without an explicit source it is
// not explicitly specified.
addr := addrs.NewLegacyProvider(name)
p, err := getproviders.LookupLegacyProvider(addr, source)
p, moved, err := getproviders.LookupLegacyProvider(addr, source)
if err == nil {
rp.Type = p
if !moved.IsZero() {
constraints, ok := allProviderConstraints[name]
// If there's no version constraint, always use the redirect
// target as there should be at least one version we can
// install
if !ok {
rp.Type = moved
continue providers
}
// Check that the redirect target has a version meeting our
// constraints
acceptable := versions.MeetingConstraints(constraints)
available, _, err := source.AvailableVersions(moved)
// If something goes wrong with the registry lookup here, fall
// back to the non-redirect provider
if err != nil {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Warning,
"Failed to query available provider packages",
fmt.Sprintf("Could not retrieve the list of available versions for provider %s: %s",
moved.ForDisplay(), err),
))
continue providers
}
// Walk backwards to consider newer versions first
for i := len(available) - 1; i >= 0; i-- {
if acceptable.Has(available[i]) {
// Success! Provider redirect target has a version
// meeting our constraints, so we can use it
rp.Type = moved
continue providers
}
}
// Find the last version available at the old location
oldAvailable, _, err := source.AvailableVersions(p)
if err != nil {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Warning,
"Failed to query available provider packages",
fmt.Sprintf("Could not retrieve the list of available versions for provider %s: %s",
p.ForDisplay(), err),
))
continue providers
}
lastAvailable := oldAvailable[len(oldAvailable)-1]
// If we fall through here, no versions at the target meet our
// version constraints, so warn the user
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Warning,
"Provider has moved",
fmt.Sprintf(
"Provider %q has moved to %q. No action is required to continue using %q (%s), but if you want to upgrade beyond version %s, you must also update the source.",
moved.Type, moved.ForDisplay(), p.ForDisplay(),
getproviders.VersionConstraintsString(constraints), lastAvailable),
))
}
} else {
// Setting the provider address to a zero value struct
// indicates that there is no known FQN for this provider,

4
command/013_config_upgrade_test.go
View File

@ -57,7 +57,7 @@ func verifyExpectedFiles(t *testing.T, expectedPath string) {
t.Fatalf("failed to read expected %s: %s", filePath, err)
}
if diff := cmp.Diff(expected, output); diff != "" {
if diff := cmp.Diff(string(expected), string(output)); diff != "" {
t.Fatalf("expected and output file for %s do not match\n%s", filePath, diff)
}
}
@ -81,6 +81,8 @@ func TestZeroThirteenUpgrade_success(t *testing.T) {
"multiple files": "013upgrade-multiple-files",
"existing versions.tf": "013upgrade-existing-versions-tf",
"skipped files": "013upgrade-skipped-files",
"provider redirect": "013upgrade-provider-redirect",
"version unavailable": "013upgrade-provider-redirect-version-unavailable",
}
for name, testPath := range testCases {
t.Run(name, func(t *testing.T) {

13
command/apply.go
View File

@ -8,7 +8,6 @@ import (
"github.com/hashicorp/terraform/addrs"
"github.com/hashicorp/terraform/backend"
"github.com/hashicorp/terraform/configs/hcl2shim"
"github.com/hashicorp/terraform/repl"
"github.com/hashicorp/terraform/states"
"github.com/hashicorp/terraform/tfdiags"
@ -345,17 +344,7 @@ func outputsAsString(state *states.State, modPath addrs.ModuleInstance, includeH
continue
}
// Our formatter still wants an old-style raw interface{} value, so
// for now we'll just shim it.
// FIXME: Port the formatter to work with cty.Value directly.
legacyVal := hcl2shim.ConfigValueFromHCL2(v.Value)
result, err := repl.FormatResult(legacyVal)
if err != nil {
// We can't really return errors from here, so we'll just have
// to stub this out. This shouldn't happen in practice anyway.
result = "<error during formatting>"
}
result := repl.FormatValue(v.Value, 0)
outputBuf.WriteString(fmt.Sprintf("%s = %s\n", k, result))
}
}

2
command/apply_test.go
View File

@ -1107,7 +1107,7 @@ func TestApply_sensitiveOutput(t *testing.T) {
}
output := ui.OutputWriter.String()
if !strings.Contains(output, "notsensitive = Hello world") {
if !strings.Contains(output, "notsensitive = \"Hello world\"") {
t.Fatalf("bad: output should contain 'notsensitive' output\n%s", output)
}
if !strings.Contains(output, "sensitive = <sensitive>") {

4
command/clistate/state.go
View File

@ -149,9 +149,7 @@ func (l *locker) Unlock(parentErr error) error {
l.ui.Output(l.color.Color(fmt.Sprintf(
"\n"+strings.TrimSpace(UnlockErrorMessage)+"\n", err)))
if parentErr != nil {
parentErr = multierror.Append(parentErr, err)
}
parentErr = multierror.Append(parentErr, err)
}
return parentErr

25
command/clistate/state_test.go Normal file
View File

@ -0,0 +1,25 @@
package clistate
import (
"context"
"fmt"
"testing"
"github.com/hashicorp/terraform/states/statemgr"
"github.com/mitchellh/cli"
"github.com/mitchellh/colorstring"
)
func TestUnlock(t *testing.T) {
ui := new(cli.MockUi)
l := NewLocker(context.Background(), 0, ui, &colorstring.Colorize{Disable: true})
l.Lock(statemgr.NewUnlockErrorFull(nil, nil), "test-lock")
err := l.Unlock(nil)
if err != nil {
fmt.Printf(err.Error())
} else {
t.Error("expected error")
}
}

66
command/command_test.go
View File

@ -42,6 +42,7 @@ import (
"github.com/zclconf/go-cty/cty"
backendInit "github.com/hashicorp/terraform/backend/init"
backendLocal "github.com/hashicorp/terraform/backend/local"
)
// These are the directories for our test data and fixtures.
@ -388,6 +389,31 @@ func testStateFileDefault(t *testing.T, s *terraform.State) string {
return DefaultStateFilename
}
// testStateFileWorkspaceDefault writes the state out to the default statefile
// for the given workspace in the cwd. Use `testCwd` to change into a temp cwd.
func testStateFileWorkspaceDefault(t *testing.T, workspace string, s *states.State) string {
t.Helper()
workspaceDir := filepath.Join(backendLocal.DefaultWorkspaceDir, workspace)
err := os.MkdirAll(workspaceDir, os.ModePerm)
if err != nil {
t.Fatalf("err: %s", err)
}
path := filepath.Join(workspaceDir, DefaultStateFilename)
f, err := os.Create(path)
if err != nil {
t.Fatalf("err: %s", err)
}
defer f.Close()
if err := writeStateForTesting(s, f); err != nil {
t.Fatalf("err: %s", err)
}
return path
}
// testStateFileRemote writes the state out to the remote statefile
// in the cwd. Use `testCwd` to change into a temp cwd.
func testStateFileRemote(t *testing.T, s *terraform.State) string {
@ -466,9 +492,11 @@ func testStateOutput(t *testing.T, path string, expected string) {
func testProvider() *terraform.MockProvider {
p := new(terraform.MockProvider)
p.PlanResourceChangeResponse = providers.PlanResourceChangeResponse{
PlannedState: cty.EmptyObjectVal,
p.PlanResourceChangeFn = func(req providers.PlanResourceChangeRequest) (resp providers.PlanResourceChangeResponse) {
resp.PlannedState = req.ProposedNewState
return resp
}
p.ReadResourceFn = func(req providers.ReadResourceRequest) providers.ReadResourceResponse {
return providers.ReadResourceResponse{
NewState: req.PriorState,
@ -824,7 +852,7 @@ func testLockState(sourceDir, path string) (func(), error) {
source := filepath.Join(sourceDir, "statelocker.go")
lockBin := filepath.Join(buildDir, "statelocker")
cmd := exec.Command("go", "build", "-mod=vendor", "-o", lockBin, source)
cmd := exec.Command("go", "build", "-o", lockBin, source)
cmd.Dir = filepath.Dir(sourceDir)
out, err := cmd.CombinedOutput()
@ -894,6 +922,12 @@ var legacyProviderNamespaces = map[string]string{
"foo": "hashicorp",
"bar": "hashicorp",
"baz": "terraform-providers",
"qux": "hashicorp",
}
// This map is used to mock the provider redirect feature.
var movedProviderNamespaces = map[string]string{
"qux": "acme",
}
// testServices starts up a local HTTP server running a fake provider registry
@ -945,16 +979,36 @@ func fakeRegistryHandler(resp http.ResponseWriter, req *http.Request) {
return
}
if pathParts[0] != "-" || pathParts[2] != "versions" {
if pathParts[2] != "versions" {
resp.WriteHeader(404)
resp.Write([]byte(`this registry only supports legacy namespace lookup requests`))
return
}
name := pathParts[1]
if namespace, ok := legacyProviderNamespaces[name]; ok {
// Legacy lookup
if pathParts[0] == "-" {
if namespace, ok := legacyProviderNamespaces[name]; ok {
resp.Header().Set("Content-Type", "application/json")
resp.WriteHeader(200)
if movedNamespace, ok := movedProviderNamespaces[name]; ok {
resp.Write([]byte(fmt.Sprintf(`{"id":"%s/%s","moved_to":"%s/%s","versions":[{"version":"1.0.0","protocols":["4"]}]}`, namespace, name, movedNamespace, name)))
} else {
resp.Write([]byte(fmt.Sprintf(`{"id":"%s/%s","versions":[{"version":"1.0.0","protocols":["4"]}]}`, namespace, name)))
}
} else {
resp.WriteHeader(404)
resp.Write([]byte(`provider not found`))
}
return
}
// Also return versions for redirect target
if namespace, ok := movedProviderNamespaces[name]; ok && pathParts[0] == namespace {
resp.Header().Set("Content-Type", "application/json")
resp.WriteHeader(200)
resp.Write([]byte(fmt.Sprintf(`{"id":"%s/%s"}`, namespace, name)))
resp.Write([]byte(fmt.Sprintf(`{"id":"%s/%s","versions":[{"version":"1.0.0","protocols":["4"]}]}`, namespace, name)))
} else {
resp.WriteHeader(404)
resp.Write([]byte(`provider not found`))

31
command/console_test.go
View File

@ -27,7 +27,7 @@ func TestConsole_basic(t *testing.T) {
defer testFixCwd(t, tmp, cwd)
p := testProvider()
ui := new(cli.MockUi)
ui := cli.NewMockUi()
c := &ConsoleCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(p),
@ -73,7 +73,7 @@ func TestConsole_tfvars(t *testing.T) {
},
}
ui := new(cli.MockUi)
ui := cli.NewMockUi()
c := &ConsoleCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(p),
@ -95,7 +95,7 @@ func TestConsole_tfvars(t *testing.T) {
}
actual := output.String()
if actual != "bar\n" {
if actual != "\"bar\"\n" {
t.Fatalf("bad: %q", actual)
}
}
@ -120,7 +120,7 @@ func TestConsole_unsetRequiredVars(t *testing.T) {
},
},
}
ui := new(cli.MockUi)
ui := cli.NewMockUi()
c := &ConsoleCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(p),
@ -140,21 +140,12 @@ func TestConsole_unsetRequiredVars(t *testing.T) {
code := c.Run(args)
outCloser()
// Because we're running "terraform console" in piped input mode, we're
// expecting it to return a nonzero exit status here but the message
// must be the one indicating that it did attempt to evaluate var.foo and
// got an unknown value in return, rather than an error about var.foo
// not being set or a failure to prompt for it.
if code == 0 {
t.Fatalf("unexpected success\n%s", ui.OutputWriter.String())
if code != 0 {
t.Fatalf("bad: %d\n\n%s", code, ui.ErrorWriter.String())
}
// The error message should be the one console produces when it encounters
// an unknown value.
got := ui.ErrorWriter.String()
want := `Error: Result depends on values that cannot be determined`
if !strings.Contains(got, want) {
t.Fatalf("wrong output\ngot:\n%s\n\nwant string containing %q", got, want)
if got, want := output.String(), "(known after apply)\n"; got != want {
t.Fatalf("unexpected output\n got: %q\nwant: %q", got, want)
}
}
@ -165,7 +156,7 @@ func TestConsole_modules(t *testing.T) {
defer testChdir(t, td)()
p := applyFixtureProvider()
ui := new(cli.MockUi)
ui := cli.NewMockUi()
c := &ConsoleCommand{
Meta: Meta{
@ -175,8 +166,8 @@ func TestConsole_modules(t *testing.T) {
}
commands := map[string]string{
"module.child.myoutput\n": "bar\n",
"module.count_child[0].myoutput\n": "bar\n",
"module.child.myoutput\n": "\"bar\"\n",
"module.count_child[0].myoutput\n": "\"bar\"\n",
"local.foo\n": "3\n",
}

18
command/e2etest/automation_test.go
View File

@ -8,6 +8,7 @@ import (
"testing"
"github.com/hashicorp/terraform/e2e"
"github.com/hashicorp/terraform/plans"
)
// The tests in this file run through different scenarios recommended in our
@ -72,11 +73,24 @@ func TestPlanApplyInAutomation(t *testing.T) {
// stateResources := plan.Changes.Resources
diffResources := plan.Changes.Resources
if len(diffResources) != 1 || diffResources[0].Addr.String() != "null_resource.test" {
if len(diffResources) != 1 {
t.Errorf("incorrect number of resources in plan")
}
expected := map[string]plans.Action{
"null_resource.test": plans.Create,
}
for _, r := range diffResources {
expectedAction, ok := expected[r.Addr.String()]
if !ok {
t.Fatalf("unexpected change for %q", r.Addr)
}
if r.Action != expectedAction {
t.Fatalf("unexpected action %q for %q", r.Action, r.Addr)
}
}
//// APPLY
stdout, stderr, err = tf.Run("apply", "-input=false", "tfplan")
if err != nil {

107
command/e2etest/primary_test.go
View File

@ -9,6 +9,8 @@ import (
"github.com/davecgh/go-spew/spew"
"github.com/hashicorp/terraform/e2e"
"github.com/hashicorp/terraform/plans"
"github.com/zclconf/go-cty/cty"
)
// The tests in this file are for the "primary workflow", which includes
@ -70,8 +72,22 @@ func TestPrimarySeparatePlan(t *testing.T) {
}
diffResources := plan.Changes.Resources
if len(diffResources) != 1 || diffResources[0].Addr.String() != "null_resource.test" {
t.Errorf("incorrect diff in plan; want just null_resource.test to have been rendered, but have:\n%s", spew.Sdump(diffResources))
if len(diffResources) != 1 {
t.Errorf("incorrect number of resources in plan")
}
expected := map[string]plans.Action{
"null_resource.test": plans.Create,
}
for _, r := range diffResources {
expectedAction, ok := expected[r.Addr.String()]
if !ok {
t.Fatalf("unexpected change for %q", r.Addr)
}
if r.Action != expectedAction {
t.Fatalf("unexpected action %q for %q", r.Action, r.Addr)
}
}
//// APPLY
@ -126,3 +142,90 @@ func TestPrimarySeparatePlan(t *testing.T) {
}
}
func TestPrimaryChdirOption(t *testing.T) {
t.Parallel()
// This test case does not include any provider dependencies, so it's
// safe to run it even when network access is disallowed.
fixturePath := filepath.Join("testdata", "chdir-option")
tf := e2e.NewBinary(terraformBin, fixturePath)
defer tf.Close()
//// INIT
stdout, stderr, err := tf.Run("-chdir=subdir", "init")
if err != nil {
t.Fatalf("unexpected init error: %s\nstderr:\n%s", err, stderr)
}
//// PLAN
stdout, stderr, err = tf.Run("-chdir=subdir", "plan", "-out=tfplan")
if err != nil {
t.Fatalf("unexpected plan error: %s\nstderr:\n%s", err, stderr)
}
if !strings.Contains(stdout, "0 to add, 0 to change, 0 to destroy") {
t.Errorf("incorrect plan tally; want 0 to add:\n%s", stdout)
}
if !strings.Contains(stdout, "This plan was saved to: tfplan") {
t.Errorf("missing \"This plan was saved to...\" message in plan output\n%s", stdout)
}
if !strings.Contains(stdout, "terraform apply \"tfplan\"") {
t.Errorf("missing next-step instruction in plan output\n%s", stdout)
}
// The saved plan is in the subdirectory because -chdir switched there
plan, err := tf.Plan("subdir/tfplan")
if err != nil {
t.Fatalf("failed to read plan file: %s", err)
}
diffResources := plan.Changes.Resources
if len(diffResources) != 0 {
t.Errorf("incorrect diff in plan; want no resource changes, but have:\n%s", spew.Sdump(diffResources))
}
//// APPLY
stdout, stderr, err = tf.Run("-chdir=subdir", "apply", "tfplan")
if err != nil {
t.Fatalf("unexpected apply error: %s\nstderr:\n%s", err, stderr)
}
if !strings.Contains(stdout, "Resources: 0 added, 0 changed, 0 destroyed") {
t.Errorf("incorrect apply tally; want 0 added:\n%s", stdout)
}
// The state file is in subdir because -chdir changed the current working directory.
state, err := tf.StateFromFile("subdir/terraform.tfstate")
if err != nil {
t.Fatalf("failed to read state file: %s", err)
}
gotOutput := state.RootModule().OutputValues["cwd"]
wantOutputValue := cty.StringVal(tf.Path()) // path.cwd returns the original path, because path.root is how we get the overridden path
if gotOutput == nil || !wantOutputValue.RawEquals(gotOutput.Value) {
t.Errorf("incorrect value for cwd output\ngot: %#v\nwant Value: %#v", gotOutput, wantOutputValue)
}
gotOutput = state.RootModule().OutputValues["root"]
wantOutputValue = cty.StringVal(tf.Path("subdir")) // path.root is a relative path, but the text fixture uses abspath on it.
if gotOutput == nil || !wantOutputValue.RawEquals(gotOutput.Value) {
t.Errorf("incorrect value for root output\ngot: %#v\nwant Value: %#v", gotOutput, wantOutputValue)
}
if len(state.RootModule().Resources) != 0 {
t.Errorf("unexpected resources in state")
}
//// DESTROY
stdout, stderr, err = tf.Run("-chdir=subdir", "destroy", "-auto-approve")
if err != nil {
t.Fatalf("unexpected destroy error: %s\nstderr:\n%s", err, stderr)
}
if !strings.Contains(stdout, "Resources: 0 destroyed") {
t.Errorf("incorrect destroy tally; want 0 destroyed:\n%s", stdout)
}
}

7
command/e2etest/testdata/chdir-option/subdir/main.tf vendored Normal file
View File

@ -0,0 +1,7 @@
output "cwd" {
value = path.cwd
}
output "root" {
value = abspath(path.root)
}

228
command/fmt.go
View File

@ -176,7 +176,7 @@ func (c *FmtCommand) processFile(path string, r io.Reader, w io.Writer, isStdout
return diags
}
result := hclwrite.Format(src)
result := c.formatSourceCode(src, path)
if !bytes.Equal(src, result) {
// Something was changed
@ -266,6 +266,232 @@ func (c *FmtCommand) processDir(path string, stdout io.Writer) tfdiags.Diagnosti
return diags
}
// formatSourceCode is the formatting logic itself, applied to each file that
// is selected (directly or indirectly) on the command line.
func (c *FmtCommand) formatSourceCode(src []byte, filename string) []byte {
f, diags := hclwrite.ParseConfig(src, filename, hcl.InitialPos)
if diags.HasErrors() {
// It would be weird to get here because the caller should already have
// checked for syntax errors and returned them. We'll just do nothing
// in this case, returning the input exactly as given.
return src
}
c.formatBody(f.Body(), nil)
return f.Bytes()
}
func (c *FmtCommand) formatBody(body *hclwrite.Body, inBlocks []string) {
attrs := body.Attributes()
for name, attr := range attrs {
if len(inBlocks) == 1 && inBlocks[0] == "variable" && name == "type" {
cleanedExprTokens := c.formatTypeExpr(attr.Expr().BuildTokens(nil))
body.SetAttributeRaw(name, cleanedExprTokens)
continue
}
cleanedExprTokens := c.formatValueExpr(attr.Expr().BuildTokens(nil))
body.SetAttributeRaw(name, cleanedExprTokens)
}
blocks := body.Blocks()
for _, block := range blocks {
// Normalize the label formatting, removing any weird stuff like
// interleaved inline comments and using the idiomatic quoted
// label syntax.
block.SetLabels(block.Labels())
inBlocks := append(inBlocks, block.Type())
c.formatBody(block.Body(), inBlocks)
}
}
func (c *FmtCommand) formatValueExpr(tokens hclwrite.Tokens) hclwrite.Tokens {
if len(tokens) < 5 {
// Can't possibly be a "${ ... }" sequence without at least enough
// tokens for the delimiters and one token inside them.
return tokens
}
oQuote := tokens[0]
oBrace := tokens[1]
cBrace := tokens[len(tokens)-2]
cQuote := tokens[len(tokens)-1]
if oQuote.Type != hclsyntax.TokenOQuote || oBrace.Type != hclsyntax.TokenTemplateInterp || cBrace.Type != hclsyntax.TokenTemplateSeqEnd || cQuote.Type != hclsyntax.TokenCQuote {
// Not an interpolation sequence at all, then.
return tokens
}
inside := tokens[2 : len(tokens)-2]
// We're only interested in sequences that are provable to be single
// interpolation sequences, which we'll determine by hunting inside
// the interior tokens for any other interpolation sequences. This is
// likely to produce false negatives sometimes, but that's better than
// false positives and we're mainly interested in catching the easy cases
// here.
quotes := 0
for _, token := range inside {
if token.Type == hclsyntax.TokenOQuote {
quotes++
continue
}
if token.Type == hclsyntax.TokenCQuote {
quotes--
continue
}
if quotes > 0 {
// Interpolation sequences inside nested quotes are okay, because
// they are part of a nested expression.
// "${foo("${bar}")}"
continue
}
if token.Type == hclsyntax.TokenTemplateInterp || token.Type == hclsyntax.TokenTemplateSeqEnd {
// We've found another template delimiter within our interior
// tokens, which suggests that we've found something like this:
// "${foo}${bar}"
// That isn't unwrappable, so we'll leave the whole expression alone.
return tokens
}
if token.Type == hclsyntax.TokenQuotedLit {
// If there's any literal characters in the outermost
// quoted sequence then it is not unwrappable.
return tokens
}
}
// If we got down here without an early return then this looks like
// an unwrappable sequence, but we'll trim any leading and trailing
// newlines that might result in an invalid result if we were to
// naively trim something like this:
// "${
// foo
// }"
return c.trimNewlines(inside)
}
func (c *FmtCommand) formatTypeExpr(tokens hclwrite.Tokens) hclwrite.Tokens {
switch len(tokens) {
case 1:
kwTok := tokens[0]
if kwTok.Type != hclsyntax.TokenIdent {
// Not a single type keyword, then.
return tokens
}
// Collection types without an explicit element type mean
// the element type is "any", so we'll normalize that.
switch string(kwTok.Bytes) {
case "list", "map", "set":
return hclwrite.Tokens{
kwTok,
{
Type: hclsyntax.TokenOParen,
Bytes: []byte("("),
},
{
Type: hclsyntax.TokenIdent,
Bytes: []byte("any"),
},
{
Type: hclsyntax.TokenCParen,
Bytes: []byte(")"),
},
}
default:
return tokens
}
case 3:
// A pre-0.12 legacy quoted string type, like "string".
oQuote := tokens[0]
strTok := tokens[1]
cQuote := tokens[2]
if oQuote.Type != hclsyntax.TokenOQuote || strTok.Type != hclsyntax.TokenQuotedLit || cQuote.Type != hclsyntax.TokenCQuote {
// Not a quoted string sequence, then.
return tokens
}
// Because this quoted syntax is from Terraform 0.11 and
// earlier, which didn't have the idea of "any" as an,
// element type, we use string as the default element
// type. That will avoid oddities if somehow the configuration
// was relying on numeric values being auto-converted to
// string, as 0.11 would do. This mimicks what terraform
// 0.12upgrade used to do, because we'd found real-world
// modules that were depending on the auto-stringing.)
switch string(strTok.Bytes) {
case "string":
return hclwrite.Tokens{
{
Type: hclsyntax.TokenIdent,
Bytes: []byte("string"),
},
}
case "list":
return hclwrite.Tokens{
{
Type: hclsyntax.TokenIdent,
Bytes: []byte("list"),
},
{
Type: hclsyntax.TokenOParen,
Bytes: []byte("("),
},
{
Type: hclsyntax.TokenIdent,
Bytes: []byte("string"),
},
{
Type: hclsyntax.TokenCParen,
Bytes: []byte(")"),
},
}
case "map":
return hclwrite.Tokens{
{
Type: hclsyntax.TokenIdent,
Bytes: []byte("map"),
},
{
Type: hclsyntax.TokenOParen,
Bytes: []byte("("),
},
{
Type: hclsyntax.TokenIdent,
Bytes: []byte("string"),
},
{
Type: hclsyntax.TokenCParen,
Bytes: []byte(")"),
},
}
default:
// Something else we're not expecting, then.
return tokens
}
default:
return tokens
}
}
func (c *FmtCommand) trimNewlines(tokens hclwrite.Tokens) hclwrite.Tokens {
if len(tokens) == 0 {
return nil
}
var start, end int
for start = 0; start < len(tokens); start++ {
if tokens[start].Type != hclsyntax.TokenNewline {
break
}
}
for end = len(tokens); end > 0; end-- {
if tokens[end-1].Type != hclsyntax.TokenNewline {
break
}
}
return tokens[start:end]
}
func (c *FmtCommand) Help() string {
helpText := `
Usage: terraform fmt [options] [DIR]

66
command/fmt_test.go
View File

@ -9,9 +9,75 @@ import (
"strings"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/mitchellh/cli"
)
func TestFmt(t *testing.T) {
const inSuffix = "_in.tf"
const outSuffix = "_out.tf"
const gotSuffix = "_got.tf"
entries, err := ioutil.ReadDir("testdata/fmt")
if err != nil {
t.Fatal(err)
}
tmpDir, err := ioutil.TempDir("", "terraform-fmt-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
for _, info := range entries {
if info.IsDir() {
continue
}
filename := info.Name()
if !strings.HasSuffix(filename, inSuffix) {
continue
}
testName := filename[:len(filename)-len(inSuffix)]
t.Run(testName, func(t *testing.T) {
inFile := filepath.Join("testdata", "fmt", testName+inSuffix)
wantFile := filepath.Join("testdata", "fmt", testName+outSuffix)
gotFile := filepath.Join(tmpDir, testName+gotSuffix)
input, err := ioutil.ReadFile(inFile)
if err != nil {
t.Fatal(err)
}
want, err := ioutil.ReadFile(wantFile)
if err != nil {
t.Fatal(err)
}
err = ioutil.WriteFile(gotFile, input, 0700)
if err != nil {
t.Fatal(err)
}
ui := cli.NewMockUi()
c := &FmtCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
},
}
args := []string{gotFile}
if code := c.Run(args); code != 0 {
t.Fatalf("fmt command was unsuccessful:\n%s", ui.ErrorWriter.String())
}
got, err := ioutil.ReadFile(gotFile)
if err != nil {
t.Fatal(err)
}
if diff := cmp.Diff(string(want), string(got)); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
})
}
}
func TestFmt_nonexist(t *testing.T) {
tempDir := fmtFixtureWriteDir(t)

16
command/format/diagnostic.go
View File

@ -175,11 +175,17 @@ func Diagnostic(diag tfdiags.Diagnostic, sources map[string][]byte, color *color
}
if desc.Detail != "" {
detail := desc.Detail
if width != 0 {
detail = wordwrap.WrapString(detail, uint(width))
lines := strings.Split(desc.Detail, "\n")
for _, line := range lines {
if !strings.HasPrefix(line, " ") {
line = wordwrap.WrapString(line, uint(width))
}
fmt.Fprintf(&buf, "%s\n", line)
}
} else {
fmt.Fprintf(&buf, "%s\n", desc.Detail)
}
fmt.Fprintf(&buf, "%s\n", detail)
}
return buf.String()
@ -296,6 +302,10 @@ func compactValueStr(val cty.Value) string {
// helpful but concise messages in diagnostics. It is not comprehensive
// nor intended to be used for other purposes.
if val.ContainsMarked() {
return "(sensitive value)"
}
ty := val.Type()
switch {
case val.IsNull():

32
command/format/diagnostic_test.go
View File

@ -167,3 +167,35 @@ Error: Some error
t.Fatalf("unexpected output: got:\n%s\nwant\n%s\n", output, expected)
}
}
func TestDiagnostic_wrapDetailIncludingCommand(t *testing.T) {
var diags tfdiags.Diagnostics
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Everything went wrong",
Detail: "This is a very long sentence about whatever went wrong which is supposed to wrap onto multiple lines. Thank-you very much for listening.\n\nTo fix this, run this very long command:\n terraform read-my-mind -please -thanks -but-do-not-wrap-this-line-because-it-is-prefixed-with-spaces\n\nHere is a coda which is also long enough to wrap and so it should eventually make it onto multiple lines. THE END",
})
color := &colorstring.Colorize{
Colors: colorstring.DefaultColors,
Reset: true,
Disable: true,
}
expected := `
Error: Everything went wrong
This is a very long sentence about whatever went wrong which is supposed to
wrap onto multiple lines. Thank-you very much for listening.
To fix this, run this very long command:
terraform read-my-mind -please -thanks -but-do-not-wrap-this-line-because-it-is-prefixed-with-spaces
Here is a coda which is also long enough to wrap and so it should eventually
make it onto multiple lines. THE END
`
output := Diagnostic(diags[0], nil, color, 76)
if output != expected {
t.Fatalf("unexpected output: got:\n%s\nwant\n%s\n", output, expected)
}
}

357
command/format/diff.go
View File

@ -14,6 +14,7 @@ import (
"github.com/hashicorp/terraform/addrs"
"github.com/hashicorp/terraform/configs/configschema"
"github.com/hashicorp/terraform/helper/experiment"
"github.com/hashicorp/terraform/plans"
"github.com/hashicorp/terraform/plans/objchange"
"github.com/hashicorp/terraform/states"
@ -98,6 +99,7 @@ func ResourceChange(
color: color,
action: change.Action,
requiredReplace: change.RequiredReplace,
concise: experiment.Enabled(experiment.X_concise_diff),
}
// Most commonly-used resources have nested blocks that result in us
@ -123,8 +125,16 @@ func ResourceChange(
changeV.Change.Before = objchange.NormalizeObjectFromLegacySDK(changeV.Change.Before, schema)
changeV.Change.After = objchange.NormalizeObjectFromLegacySDK(changeV.Change.After, schema)
bodyWritten := p.writeBlockBodyDiff(schema, changeV.Before, changeV.After, 6, path)
if bodyWritten {
// Now that the change is decoded, add back the marks at the defined paths
if len(change.BeforeValMarks) > 0 {
changeV.Change.Before = changeV.Change.Before.MarkWithPaths(change.BeforeValMarks)
}
if len(change.AfterValMarks) > 0 {
changeV.Change.After = changeV.Change.After.MarkWithPaths(change.AfterValMarks)
}
result := p.writeBlockBodyDiff(schema, changeV.Before, changeV.After, 6, path)
if result.bodyWritten {
buf.WriteString("\n")
buf.WriteString(strings.Repeat(" ", 4))
}
@ -144,9 +154,10 @@ func OutputChanges(
) string {
var buf bytes.Buffer
p := blockBodyDiffPrinter{
buf: &buf,
color: color,
action: plans.Update, // not actually used in this case, because we're not printing a containing block
buf: &buf,
color: color,
action: plans.Update, // not actually used in this case, because we're not printing a containing block
concise: experiment.Enabled(experiment.X_concise_diff),
}
// We're going to reuse the codepath we used for printing resource block
@ -189,16 +200,24 @@ type blockBodyDiffPrinter struct {
color *colorstring.Colorize
action plans.Action
requiredReplace cty.PathSet
concise bool
}
type blockBodyDiffResult struct {
bodyWritten bool
skippedAttributes int
skippedBlocks int
}
const forcesNewResourceCaption = " [red]# forces replacement[reset]"
// writeBlockBodyDiff writes attribute or block differences
// and returns true if any differences were found and written
func (p *blockBodyDiffPrinter) writeBlockBodyDiff(schema *configschema.Block, old, new cty.Value, indent int, path cty.Path) bool {
func (p *blockBodyDiffPrinter) writeBlockBodyDiff(schema *configschema.Block, old, new cty.Value, indent int, path cty.Path) blockBodyDiffResult {
path = ctyEnsurePathCapacity(path, 1)
bodyWritten := false
result := blockBodyDiffResult{}
blankBeforeBlocks := false
{
attrNames := make([]string, 0, len(schema.Attributes))
@ -229,8 +248,21 @@ func (p *blockBodyDiffPrinter) writeBlockBodyDiff(schema *configschema.Block, ol
oldVal := ctyGetAttrMaybeNull(old, name)
newVal := ctyGetAttrMaybeNull(new, name)
bodyWritten = true
p.writeAttrDiff(name, attrS, oldVal, newVal, attrNameLen, indent, path)
result.bodyWritten = true
skipped := p.writeAttrDiff(name, attrS, oldVal, newVal, attrNameLen, indent, path)
if skipped {
result.skippedAttributes++
}
}
if result.skippedAttributes > 0 {
noun := "attributes"
if result.skippedAttributes == 1 {
noun = "attribute"
}
p.buf.WriteString("\n")
p.buf.WriteString(strings.Repeat(" ", indent+2))
p.buf.WriteString(p.color.Color(fmt.Sprintf("[dark_gray]# (%d unchanged %s hidden)[reset]", result.skippedAttributes, noun)))
}
}
@ -246,23 +278,41 @@ func (p *blockBodyDiffPrinter) writeBlockBodyDiff(schema *configschema.Block, ol
oldVal := ctyGetAttrMaybeNull(old, name)
newVal := ctyGetAttrMaybeNull(new, name)
bodyWritten = true
p.writeNestedBlockDiffs(name, blockS, oldVal, newVal, blankBeforeBlocks, indent, path)
result.bodyWritten = true
skippedBlocks := p.writeNestedBlockDiffs(name, blockS, oldVal, newVal, blankBeforeBlocks, indent, path)
if skippedBlocks > 0 {
result.skippedBlocks += skippedBlocks
}
// Always include a blank for any subsequent block types.
blankBeforeBlocks = true
}
if result.skippedBlocks > 0 {
noun := "blocks"
if result.skippedBlocks == 1 {
noun = "block"
}
p.buf.WriteString("\n")
p.buf.WriteString(strings.Repeat(" ", indent+2))
p.buf.WriteString(p.color.Color(fmt.Sprintf("[dark_gray]# (%d unchanged %s hidden)[reset]", result.skippedBlocks, noun)))
}
}
return bodyWritten
return result
}
func (p *blockBodyDiffPrinter) writeAttrDiff(name string, attrS *configschema.Attribute, old, new cty.Value, nameLen, indent int, path cty.Path) {
path = append(path, cty.GetAttrStep{Name: name})
p.buf.WriteString("\n")
p.buf.WriteString(strings.Repeat(" ", indent))
showJustNew := false
// getPlanActionAndShow returns the action value
// and a boolean for showJustNew. In this function we
// modify the old and new values to remove any possible marks
func getPlanActionAndShow(old cty.Value, new cty.Value) (plans.Action, bool) {
var action plans.Action
showJustNew := false
if old.ContainsMarked() {
old, _ = old.UnmarkDeep()
}
if new.ContainsMarked() {
new, _ = new.UnmarkDeep()
}
switch {
case old.IsNull():
action = plans.Create
@ -275,7 +325,22 @@ func (p *blockBodyDiffPrinter) writeAttrDiff(name string, attrS *configschema.At
default:
action = plans.Update
}
return action, showJustNew
}
func (p *blockBodyDiffPrinter) writeAttrDiff(name string, attrS *configschema.Attribute, old, new cty.Value, nameLen, indent int, path cty.Path) bool {
path = append(path, cty.GetAttrStep{Name: name})
action, showJustNew := getPlanActionAndShow(old, new)
if action == plans.NoOp && p.concise && !identifyingAttribute(name, attrS) {
return true
}
p.buf.WriteString("\n")
p.writeSensitivityWarning(old, new, indent, action)
p.buf.WriteString(strings.Repeat(" ", indent))
p.writeActionSymbol(action)
p.buf.WriteString(p.color.Color("[bold]"))
@ -300,13 +365,16 @@ func (p *blockBodyDiffPrinter) writeAttrDiff(name string, attrS *configschema.At
p.writeValueDiff(old, new, indent+2, path)
}
}
return false
}
func (p *blockBodyDiffPrinter) writeNestedBlockDiffs(name string, blockS *configschema.NestedBlock, old, new cty.Value, blankBefore bool, indent int, path cty.Path) {
func (p *blockBodyDiffPrinter) writeNestedBlockDiffs(name string, blockS *configschema.NestedBlock, old, new cty.Value, blankBefore bool, indent int, path cty.Path) int {
skippedBlocks := 0
path = append(path, cty.GetAttrStep{Name: name})
if old.IsNull() && new.IsNull() {
// Nothing to do if both old and new is null
return
return skippedBlocks
}
// Where old/new are collections representing a nesting mode other than
@ -335,7 +403,10 @@ func (p *blockBodyDiffPrinter) writeNestedBlockDiffs(name string, blockS *config
if blankBefore {
p.buf.WriteRune('\n')
}
p.writeNestedBlockDiff(name, nil, &blockS.Block, action, old, new, indent, path)
skipped := p.writeNestedBlockDiff(name, nil, &blockS.Block, action, old, new, indent, path)
if skipped {
return 1
}
case configschema.NestingList:
// For the sake of handling nested blocks, we'll treat a null list
// the same as an empty list since the config language doesn't
@ -377,19 +448,28 @@ func (p *blockBodyDiffPrinter) writeNestedBlockDiffs(name string, blockS *config
if oldItem.RawEquals(newItem) {
action = plans.NoOp
}
p.writeNestedBlockDiff(name, nil, &blockS.Block, action, oldItem, newItem, indent, path)
skipped := p.writeNestedBlockDiff(name, nil, &blockS.Block, action, oldItem, newItem, indent, path)
if skipped {
skippedBlocks++
}
}
for i := commonLen; i < len(oldItems); i++ {
path := append(path, cty.IndexStep{Key: cty.NumberIntVal(int64(i))})
oldItem := oldItems[i]
newItem := cty.NullVal(oldItem.Type())
p.writeNestedBlockDiff(name, nil, &blockS.Block, plans.Delete, oldItem, newItem, indent, path)
skipped := p.writeNestedBlockDiff(name, nil, &blockS.Block, plans.Delete, oldItem, newItem, indent, path)
if skipped {
skippedBlocks++
}
}
for i := commonLen; i < len(newItems); i++ {
path := append(path, cty.IndexStep{Key: cty.NumberIntVal(int64(i))})
newItem := newItems[i]
oldItem := cty.NullVal(newItem.Type())
p.writeNestedBlockDiff(name, nil, &blockS.Block, plans.Create, oldItem, newItem, indent, path)
skipped := p.writeNestedBlockDiff(name, nil, &blockS.Block, plans.Create, oldItem, newItem, indent, path)
if skipped {
skippedBlocks++
}
}
case configschema.NestingSet:
// For the sake of handling nested blocks, we'll treat a null set
@ -403,7 +483,7 @@ func (p *blockBodyDiffPrinter) writeNestedBlockDiffs(name string, blockS *config
if (len(oldItems) + len(newItems)) == 0 {
// Nothing to do if both sets are empty
return
return 0
}
allItems := make([]cty.Value, 0, len(oldItems)+len(newItems))
@ -437,7 +517,10 @@ func (p *blockBodyDiffPrinter) writeNestedBlockDiffs(name string, blockS *config
newValue = val
}
path := append(path, cty.IndexStep{Key: val})
p.writeNestedBlockDiff(name, nil, &blockS.Block, action, oldValue, newValue, indent, path)
skipped := p.writeNestedBlockDiff(name, nil, &blockS.Block, action, oldValue, newValue, indent, path)
if skipped {
skippedBlocks++
}
}
case configschema.NestingMap:
@ -451,7 +534,7 @@ func (p *blockBodyDiffPrinter) writeNestedBlockDiffs(name string, blockS *config
newItems := new.AsValueMap()
if (len(oldItems) + len(newItems)) == 0 {
// Nothing to do if both maps are empty
return
return 0
}
allKeys := make(map[string]bool)
@ -489,12 +572,20 @@ func (p *blockBodyDiffPrinter) writeNestedBlockDiffs(name string, blockS *config
}
path := append(path, cty.IndexStep{Key: cty.StringVal(k)})
p.writeNestedBlockDiff(name, &k, &blockS.Block, action, oldValue, newValue, indent, path)
skipped := p.writeNestedBlockDiff(name, &k, &blockS.Block, action, oldValue, newValue, indent, path)
if skipped {
skippedBlocks++
}
}
}
return skippedBlocks
}
func (p *blockBodyDiffPrinter) writeNestedBlockDiff(name string, label *string, blockS *configschema.Block, action plans.Action, old, new cty.Value, indent int, path cty.Path) {
func (p *blockBodyDiffPrinter) writeNestedBlockDiff(name string, label *string, blockS *configschema.Block, action plans.Action, old, new cty.Value, indent int, path cty.Path) bool {
if action == plans.NoOp && p.concise {
return true
}
p.buf.WriteString("\n")
p.buf.WriteString(strings.Repeat(" ", indent))
p.writeActionSymbol(action)
@ -509,15 +600,23 @@ func (p *blockBodyDiffPrinter) writeNestedBlockDiff(name string, label *string,
p.buf.WriteString(p.color.Color(forcesNewResourceCaption))
}
bodyWritten := p.writeBlockBodyDiff(blockS, old, new, indent+4, path)
if bodyWritten {
result := p.writeBlockBodyDiff(blockS, old, new, indent+4, path)
if result.bodyWritten {
p.buf.WriteString("\n")
p.buf.WriteString(strings.Repeat(" ", indent+2))
}
p.buf.WriteString("}")
return false
}
func (p *blockBodyDiffPrinter) writeValue(val cty.Value, action plans.Action, indent int) {
// Could check specifically for the sensitivity marker
if val.IsMarked() {
p.buf.WriteString("(sensitive)")
return
}
if !val.IsKnown() {
p.buf.WriteString("(known after apply)")
return
@ -678,13 +777,26 @@ func (p *blockBodyDiffPrinter) writeValueDiff(old, new cty.Value, indent int, pa
// However, these specialized implementations can apply only if both
// values are known and non-null.
if old.IsKnown() && new.IsKnown() && !old.IsNull() && !new.IsNull() && typesEqual {
// Create unmarked values for comparisons
unmarkedOld, oldMarks := old.UnmarkDeep()
unmarkedNew, newMarks := new.UnmarkDeep()
switch {
case ty == cty.Bool || ty == cty.Number:
if len(oldMarks) > 0 || len(newMarks) > 0 {
p.buf.WriteString("(sensitive)")
return
}
case ty == cty.String:
// We have special behavior for both multi-line strings in general
// and for strings that can parse as JSON. For the JSON handling
// to apply, both old and new must be valid JSON.
// For single-line strings that don't parse as JSON we just fall
// out of this switch block and do the default old -> new rendering.
if len(oldMarks) > 0 || len(newMarks) > 0 {
p.buf.WriteString("(sensitive)")
return
}
oldS := old.AsString()
newS := new.AsString()
@ -709,7 +821,7 @@ func (p *blockBodyDiffPrinter) writeValueDiff(old, new cty.Value, indent int, pa
p.buf.WriteString(strings.Repeat(" ", indent))
p.buf.WriteByte(')')
} else {
// if they differ only in insigificant whitespace
// if they differ only in insignificant whitespace
// then we'll note that but still expand out the
// effective value.
if p.pathForcesNewResource(path) {
@ -819,11 +931,10 @@ func (p *blockBodyDiffPrinter) writeValueDiff(old, new cty.Value, indent int, pa
removed = cty.SetValEmpty(ty.ElementType())
}
suppressedElements := 0
for it := all.ElementIterator(); it.Next(); {
_, val := it.Element()
p.buf.WriteString(strings.Repeat(" ", indent+2))
var action plans.Action
switch {
case !val.IsKnown():
@ -836,11 +947,28 @@ func (p *blockBodyDiffPrinter) writeValueDiff(old, new cty.Value, indent int, pa
action = plans.NoOp
}
if action == plans.NoOp && p.concise {
suppressedElements++
continue
}
p.buf.WriteString(strings.Repeat(" ", indent+2))
p.writeActionSymbol(action)
p.writeValue(val, action, indent+4)
p.buf.WriteString(",\n")
}
if suppressedElements > 0 {
p.writeActionSymbol(plans.NoOp)
p.buf.WriteString(strings.Repeat(" ", indent+2))
noun := "elements"
if suppressedElements == 1 {
noun = "element"
}
p.buf.WriteString(p.color.Color(fmt.Sprintf("[dark_gray]# (%d unchanged %s hidden)[reset]", suppressedElements, noun)))
p.buf.WriteString("\n")
}
p.buf.WriteString(strings.Repeat(" ", indent))
p.buf.WriteString("]")
return
@ -852,7 +980,74 @@ func (p *blockBodyDiffPrinter) writeValueDiff(old, new cty.Value, indent int, pa
p.buf.WriteString("\n")
elemDiffs := ctySequenceDiff(old.AsValueSlice(), new.AsValueSlice())
for _, elemDiff := range elemDiffs {
// Maintain a stack of suppressed lines in the diff for later
// display or elision
var suppressedElements []*plans.Change
var changeShown bool
for i := 0; i < len(elemDiffs); i++ {
// In concise mode, push any no-op diff elements onto the stack
if p.concise {
for i < len(elemDiffs) && elemDiffs[i].Action == plans.NoOp {
suppressedElements = append(suppressedElements, elemDiffs[i])
i++
}
}
// If we have some suppressed elements on the stack…
if len(suppressedElements) > 0 {
// If we've just rendered a change, display the first
// element in the stack as context
if changeShown {
elemDiff := suppressedElements[0]
p.buf.WriteString(strings.Repeat(" ", indent+4))
p.writeValue(elemDiff.After, elemDiff.Action, indent+4)
p.buf.WriteString(",\n")
suppressedElements = suppressedElements[1:]
}
hidden := len(suppressedElements)
// If we're not yet at the end of the list, capture the
// last element on the stack as context for the upcoming
// change to be rendered
var nextContextDiff *plans.Change
if hidden > 0 && i < len(elemDiffs) {
hidden--
nextContextDiff = suppressedElements[hidden]
suppressedElements = suppressedElements[:hidden]
}
// If there are still hidden elements, show an elision
// statement counting them
if hidden > 0 {
p.writeActionSymbol(plans.NoOp)
p.buf.WriteString(strings.Repeat(" ", indent+2))
noun := "elements"
if hidden == 1 {
noun = "element"
}
p.buf.WriteString(p.color.Color(fmt.Sprintf("[dark_gray]# (%d unchanged %s hidden)[reset]", hidden, noun)))
p.buf.WriteString("\n")
}
// Display the next context diff if it was captured above
if nextContextDiff != nil {
p.buf.WriteString(strings.Repeat(" ", indent+4))
p.writeValue(nextContextDiff.After, nextContextDiff.Action, indent+4)
p.buf.WriteString(",\n")
}
// Suppressed elements have now been handled so clear them again
suppressedElements = nil
}
if i >= len(elemDiffs) {
break
}
elemDiff := elemDiffs[i]
p.buf.WriteString(strings.Repeat(" ", indent+2))
p.writeActionSymbol(elemDiff.Action)
switch elemDiff.Action {
@ -869,10 +1064,12 @@ func (p *blockBodyDiffPrinter) writeValueDiff(old, new cty.Value, indent int, pa
}
p.buf.WriteString(",\n")
changeShown = true
}
p.buf.WriteString(strings.Repeat(" ", indent))
p.buf.WriteString("]")
return
case ty.IsMapType():
@ -903,6 +1100,7 @@ func (p *blockBodyDiffPrinter) writeValueDiff(old, new cty.Value, indent int, pa
sort.Strings(allKeys)
suppressedElements := 0
lastK := ""
for i, k := range allKeys {
if i > 0 && lastK == k {
@ -910,21 +1108,30 @@ func (p *blockBodyDiffPrinter) writeValueDiff(old, new cty.Value, indent int, pa
}
lastK = k
p.buf.WriteString(strings.Repeat(" ", indent+2))
kV := cty.StringVal(k)
var action plans.Action
if old.HasIndex(kV).False() {
action = plans.Create
} else if new.HasIndex(kV).False() {
action = plans.Delete
} else if eqV := old.Index(kV).Equals(new.Index(kV)); eqV.IsKnown() && eqV.True() {
} else if eqV := unmarkedOld.Index(kV).Equals(unmarkedNew.Index(kV)); eqV.IsKnown() && eqV.True() {
action = plans.NoOp
} else {
action = plans.Update
}
if action == plans.NoOp && p.concise {
suppressedElements++
continue
}
path := append(path, cty.IndexStep{Key: kV})
oldV := old.Index(kV)
newV := new.Index(kV)
p.writeSensitivityWarning(oldV, newV, indent+2, action)
p.buf.WriteString(strings.Repeat(" ", indent+2))
p.writeActionSymbol(action)
p.writeValue(kV, action, indent+4)
p.buf.WriteString(strings.Repeat(" ", keyLen-len(k)))
@ -932,22 +1139,40 @@ func (p *blockBodyDiffPrinter) writeValueDiff(old, new cty.Value, indent int, pa
switch action {
case plans.Create, plans.NoOp:
v := new.Index(kV)
p.writeValue(v, action, indent+4)
if v.IsMarked() {
p.buf.WriteString("(sensitive)")
} else {
p.writeValue(v, action, indent+4)
}
case plans.Delete:
oldV := old.Index(kV)
newV := cty.NullVal(oldV.Type())
p.writeValueDiff(oldV, newV, indent+4, path)
default:
oldV := old.Index(kV)
newV := new.Index(kV)
p.writeValueDiff(oldV, newV, indent+4, path)
if oldV.IsMarked() || newV.IsMarked() {
p.buf.WriteString("(sensitive)")
} else {
p.writeValueDiff(oldV, newV, indent+4, path)
}
}
p.buf.WriteByte('\n')
}
if suppressedElements > 0 {
p.writeActionSymbol(plans.NoOp)
p.buf.WriteString(strings.Repeat(" ", indent+2))
noun := "elements"
if suppressedElements == 1 {
noun = "element"
}
p.buf.WriteString(p.color.Color(fmt.Sprintf("[dark_gray]# (%d unchanged %s hidden)[reset]", suppressedElements, noun)))
p.buf.WriteString("\n")
}
p.buf.WriteString(strings.Repeat(" ", indent))
p.buf.WriteString("}")
return
case ty.IsObjectType():
p.buf.WriteString("{")
@ -976,6 +1201,7 @@ func (p *blockBodyDiffPrinter) writeValueDiff(old, new cty.Value, indent int, pa
sort.Strings(allKeys)
suppressedElements := 0
lastK := ""
for i, k := range allKeys {
if i > 0 && lastK == k {
@ -983,7 +1209,6 @@ func (p *blockBodyDiffPrinter) writeValueDiff(old, new cty.Value, indent int, pa
}
lastK = k
p.buf.WriteString(strings.Repeat(" ", indent+2))
kV := k
var action plans.Action
if !old.Type().HasAttribute(kV) {
@ -996,8 +1221,14 @@ func (p *blockBodyDiffPrinter) writeValueDiff(old, new cty.Value, indent int, pa
action = plans.Update
}
if action == plans.NoOp && p.concise {
suppressedElements++
continue
}
path := append(path, cty.GetAttrStep{Name: kV})
p.buf.WriteString(strings.Repeat(" ", indent+2))
p.writeActionSymbol(action)
p.buf.WriteString(k)
p.buf.WriteString(strings.Repeat(" ", keyLen-len(k)))
@ -1020,6 +1251,17 @@ func (p *blockBodyDiffPrinter) writeValueDiff(old, new cty.Value, indent int, pa
p.buf.WriteString("\n")
}
if suppressedElements > 0 {
p.writeActionSymbol(plans.NoOp)
p.buf.WriteString(strings.Repeat(" ", indent+2))
noun := "elements"
if suppressedElements == 1 {
noun = "element"
}
p.buf.WriteString(p.color.Color(fmt.Sprintf("[dark_gray]# (%d unchanged %s hidden)[reset]", suppressedElements, noun)))
p.buf.WriteString("\n")
}
p.buf.WriteString(strings.Repeat(" ", indent))
p.buf.WriteString("}")
@ -1065,6 +1307,28 @@ func (p *blockBodyDiffPrinter) writeActionSymbol(action plans.Action) {
}
}
func (p *blockBodyDiffPrinter) writeSensitivityWarning(old, new cty.Value, indent int, action plans.Action) {
// Dont' show this warning for create or delete
if action == plans.Create || action == plans.Delete {
return
}
if new.IsMarked() && !old.IsMarked() {
p.buf.WriteString(strings.Repeat(" ", indent))
p.buf.WriteString(p.color.Color("# [yellow]Warning:[reset] this attribute value will be marked as sensitive and will\n"))
p.buf.WriteString(strings.Repeat(" ", indent))
p.buf.WriteString(p.color.Color("# not display in UI output after applying this change\n"))
}
// Note if changing this attribute will change its sensitivity
if old.IsMarked() && !new.IsMarked() {
p.buf.WriteString(strings.Repeat(" ", indent))
p.buf.WriteString(p.color.Color("# [yellow]Warning:[reset] this attribute value will no longer be marked as sensitive\n"))
p.buf.WriteString(strings.Repeat(" ", indent))
p.buf.WriteString(p.color.Color("# after applying this change\n"))
}
}
func (p *blockBodyDiffPrinter) pathForcesNewResource(path cty.Path) bool {
if !p.action.IsReplace() || p.requiredReplace.Empty() {
// "requiredReplace" only applies when the instance is being replaced,
@ -1096,7 +1360,8 @@ func ctyGetAttrMaybeNull(val cty.Value, name string) cty.Value {
// This allows us to avoid spurious diffs
// until we introduce null to the SDK.
attrValue := val.GetAttr(name)
if ctyEmptyString(attrValue) {
// If the value is marked, the ctyEmptyString function will fail
if !val.ContainsMarked() && ctyEmptyString(attrValue) {
return cty.NullVal(attrType)
}
@ -1266,3 +1531,11 @@ func DiffActionSymbol(action plans.Action) string {
return " ?"
}
}
// Extremely coarse heuristic for determining whether or not a given attribute
// name is important for identifying a resource. In the future, this may be
// replaced by a flag in the schema, but for now this is likely to be good
// enough.
func identifyingAttribute(name string, attrSchema *configschema.Attribute) bool {
return name == "id" || name == "tags" || name == "name"
}

978
command/format/diff_test.go
View File

File diff suppressed because it is too large Load Diff

4
command/format/state.go
View File

@ -198,8 +198,8 @@ func formatStateModule(p blockBodyDiffPrinter, m *states.Module, schemas *terraf
}
path := make(cty.Path, 0, 3)
bodyWritten := p.writeBlockBodyDiff(schema, val.Value, val.Value, 2, path)
if bodyWritten {
result := p.writeBlockBodyDiff(schema, val.Value, val.Value, 2, path)
if result.bodyWritten {
p.buf.WriteString("\n")
}

70
command/import_test.go
View File

@ -802,6 +802,76 @@ func TestImportModuleVarFile(t *testing.T) {
}
}
// This test covers an edge case where a module with a complex input variable
// of nested objects has an invalid default which is overridden by the calling
// context, and is used in locals. If we don't evaluate module call variables
// for the import walk, this results in an error.
//
// The specific example has a variable "foo" which is a nested object:
//
// foo = { bar = { baz = true } }
//
// This is used as foo = var.foo in the call to the child module, which then
// uses the traversal foo.bar.baz in a local. A default value in the child
// module of {} causes this local evaluation to error, breaking import.
func TestImportModuleInputVariableEvaluation(t *testing.T) {
td := tempDir(t)
copy.CopyDir(testFixturePath("import-module-input-variable"), td)
defer os.RemoveAll(td)
defer testChdir(t, td)()
statePath := testTempFile(t)
p := testProvider()
p.GetSchemaReturn = &terraform.ProviderSchema{
ResourceTypes: map[string]*configschema.Block{
"test_instance": {
Attributes: map[string]*configschema.Attribute{
"foo": {Type: cty.String, Optional: true},
},
},
},
}
providerSource, close := newMockProviderSource(t, map[string][]string{
"test": {"1.2.3"},
})
defer close()
// init to install the module
ui := new(cli.MockUi)
m := Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
ProviderSource: providerSource,
}
ic := &InitCommand{
Meta: m,
}
if code := ic.Run([]string{}); code != 0 {
t.Fatalf("init failed\n%s", ui.ErrorWriter)
}
// import
ui = new(cli.MockUi)
c := &ImportCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(p),
Ui: ui,
},
}
args := []string{
"-state", statePath,
"module.child.test_instance.foo",
"bar",
}
code := c.Run(args)
if code != 0 {
t.Fatalf("import failed; expected success")
}
}
func TestImport_dataResource(t *testing.T) {
defer testChdir(t, testFixturePath("import-missing-resource-config"))()

122
command/init.go
View File

@ -264,7 +264,12 @@ func (c *InitCommand) Run(args []string) int {
// on a previous run) we'll use the current state as a potential source
// of provider dependencies.
if back != nil {
sMgr, err := back.StateMgr(c.Workspace())
workspace, err := c.Workspace()
if err != nil {
c.Ui.Error(fmt.Sprintf("Error selecting workspace: %s", err))
return 1
}
sMgr, err := back.StateMgr(workspace)
if err != nil {
c.Ui.Error(fmt.Sprintf("Error loading state: %s", err))
return 1
@ -422,8 +427,9 @@ func (c *InitCommand) getProviders(config *configs.Config, state *states.State,
if moreDiags.HasErrors() {
return false, diags
}
stateReqs := make(getproviders.Requirements, 0)
if state != nil {
stateReqs := state.ProviderRequirements()
stateReqs = state.ProviderRequirements()
reqs = reqs.Merge(stateReqs)
}
@ -451,6 +457,11 @@ func (c *InitCommand) getProviders(config *configs.Config, state *states.State,
// appear to have been re-namespaced.
missingProviderErrors := make(map[addrs.Provider]error)
// Legacy provider addresses required by source probably refer to in-house
// providers. Capture these for later analysis also, to suggest how to use
// the state replace-provider command to fix this problem.
stateLegacyProviderErrors := make(map[addrs.Provider]error)
// Because we're currently just streaming a series of events sequentially
// into the terminal, we're showing only a subset of the events to keep
// things relatively concise. Later it'd be nice to have a progress UI
@ -504,13 +515,19 @@ func (c *InitCommand) getProviders(config *configs.Config, state *states.State,
),
))
case getproviders.ErrRegistryProviderNotKnown:
// Default providers may have no explicit source, and the 404
// error could be caused by re-namespacing. Add the provider
// and error to a map to later check for this case. We don't
// run the check here to keep this event callback simple.
if provider.IsDefault() {
// Default providers may have no explicit source, and the 404
// error could be caused by re-namespacing. Add the provider
// and error to a map to later check for this case. We don't
// run the check here to keep this event callback simple.
missingProviderErrors[provider] = err
} else if _, ok := stateReqs[provider]; ok && provider.IsLegacy() {
// Legacy provider, from state, not found from any source:
// probably an in-house provider. Record this here to
// faciliate a useful suggestion later.
stateLegacyProviderErrors[provider] = err
} else {
// Otherwise maybe this provider really doesn't exist? Shrug!
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Failed to query available provider packages",
@ -690,9 +707,13 @@ func (c *InitCommand) getProviders(config *configs.Config, state *states.State,
source := c.providerInstallSource()
for provider, fetchErr := range missingProviderErrors {
addr := addrs.NewLegacyProvider(provider.Type)
p, err := getproviders.LookupLegacyProvider(addr, source)
p, redirect, err := getproviders.LookupLegacyProvider(addr, source)
if err == nil {
foundProviders[provider] = p
if redirect.IsZero() {
foundProviders[provider] = p
} else {
foundProviders[provider] = redirect
}
} else {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
@ -762,6 +783,53 @@ func (c *InitCommand) getProviders(config *configs.Config, state *states.State,
))
}
// Legacy providers required by state which could not be installed are
// probably in-house providers. If the user has completed the necessary
// steps to make their custom provider available for installation, then
// there should be a provider with the same type selected after the
// installation process completed.
//
// If we detect this specific situation, we can confidently suggest
// that the next step is to run the state replace-provider command to
// update state. We build a map of provider replacements here to ensure
// that we're as concise as possible with the diagnostic.
stateReplaceProviders := make(map[addrs.Provider]addrs.Provider)
for provider, fetchErr := range stateLegacyProviderErrors {
var sameType []addrs.Provider
for p := range selected {
if p.Type == provider.Type {
sameType = append(sameType, p)
}
}
if len(sameType) == 1 {
stateReplaceProviders[provider] = sameType[0]
} else {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Failed to install provider",
fmt.Sprintf("Error while installing %s: %s", provider.ForDisplay(), fetchErr),
))
}
}
if len(stateReplaceProviders) > 0 {
var detail strings.Builder
command := "command"
if len(stateReplaceProviders) > 1 {
command = "commands"
}
fmt.Fprintf(&detail, "Found unresolvable legacy provider references in state. It looks like these refer to in-house providers. You can update the resources in state with the following %s:\n", command)
for legacy, replacement := range stateReplaceProviders {
fmt.Fprintf(&detail, "\n terraform state replace-provider %s %s", legacy, replacement)
}
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Failed to install legacy providers required by state",
detail.String(),
))
}
// The errors captured in "err" should be redundant with what we
// received via the InstallerEvents callbacks above, so we'll
// just return those as long as we have some.
@ -856,16 +924,34 @@ func (c *InitCommand) backendConfigOverrideBody(flags rawFlags, schema *configsc
// The value is interpreted as a filename.
newBody, fileDiags := c.loadHCLFile(item.Value)
diags = diags.Append(fileDiags)
// Verify that the file contains only key-values pairs, and not a
// full backend config block. JustAttributes() will return an error
// if blocks are found
_, attrDiags := newBody.JustAttributes()
if attrDiags.HasErrors() {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Invalid backend configuration file",
fmt.Sprintf("The backend configuration file %q given on the command line must contain key-value pairs only, and not configuration blocks.", item.Value),
))
if fileDiags.HasErrors() {
continue
}
// Generate an HCL body schema for the backend block.
var bodySchema hcl.BodySchema
for name := range schema.Attributes {
// We intentionally ignore the `Required` attribute here
// because backend config override files can be partial. The
// goal is to make sure we're not loading a file with
// extraneous attributes or blocks.
bodySchema.Attributes = append(bodySchema.Attributes, hcl.AttributeSchema{
Name: name,
})
}
for name, block := range schema.BlockTypes {
var labelNames []string
if block.Nesting == configschema.NestingMap {
labelNames = append(labelNames, "key")
}
bodySchema.Blocks = append(bodySchema.Blocks, hcl.BlockHeaderSchema{
Type: name,
LabelNames: labelNames,
})
}
// Verify that the file body matches the expected backend schema.
_, schemaDiags := newBody.Content(&bodySchema)
diags = diags.Append(schemaDiags)
if schemaDiags.HasErrors() {
continue
}
flushVals() // deal with any accumulated individual values first

141
command/init_test.go
View File

@ -18,6 +18,7 @@ import (
"github.com/hashicorp/terraform/addrs"
"github.com/hashicorp/terraform/configs"
"github.com/hashicorp/terraform/configs/configschema"
"github.com/hashicorp/terraform/helper/copy"
"github.com/hashicorp/terraform/internal/getproviders"
"github.com/hashicorp/terraform/internal/providercache"
@ -355,8 +356,8 @@ func TestInit_backendConfigFile(t *testing.T) {
}
})
// the backend config file must be a set of key-value pairs and not a full backend {} block
t.Run("invalid-config-file", func(t *testing.T) {
// the backend config file must not be a full terraform block
t.Run("full-backend-config-file", func(t *testing.T) {
ui := new(cli.MockUi)
c := &InitCommand{
Meta: Meta{
@ -368,10 +369,91 @@ func TestInit_backendConfigFile(t *testing.T) {
if code := c.Run(args); code != 1 {
t.Fatalf("expected error, got success\n")
}
if !strings.Contains(ui.ErrorWriter.String(), "Invalid backend configuration file") {
if !strings.Contains(ui.ErrorWriter.String(), "Unsupported block type") {
t.Fatalf("wrong error: %s", ui.ErrorWriter)
}
})
// the backend config file must match the schema for the backend
t.Run("invalid-config-file", func(t *testing.T) {
ui := new(cli.MockUi)
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
},
}
args := []string{"-backend-config", "invalid.config"}
if code := c.Run(args); code != 1 {
t.Fatalf("expected error, got success\n")
}
if !strings.Contains(ui.ErrorWriter.String(), "Unsupported argument") {
t.Fatalf("wrong error: %s", ui.ErrorWriter)
}
})
// missing file is an error
t.Run("missing-config-file", func(t *testing.T) {
ui := new(cli.MockUi)
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
},
}
args := []string{"-backend-config", "missing.config"}
if code := c.Run(args); code != 1 {
t.Fatalf("expected error, got success\n")
}
if !strings.Contains(ui.ErrorWriter.String(), "Failed to read file") {
t.Fatalf("wrong error: %s", ui.ErrorWriter)
}
})
// blank filename clears the backend config
t.Run("blank-config-file", func(t *testing.T) {
ui := new(cli.MockUi)
c := &InitCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
},
}
args := []string{"-backend-config="}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
// Read our saved backend config and verify the backend config is empty
state := testDataStateRead(t, filepath.Join(DefaultDataDir, DefaultStateFilename))
if got, want := normalizeJSON(t, state.Backend.ConfigRaw), `{"path":null,"workspace_dir":null}`; got != want {
t.Errorf("wrong config\ngot: %s\nwant: %s", got, want)
}
})
// simulate the local backend having a required field which is not
// specified in the override file
t.Run("required-argument", func(t *testing.T) {
c := &InitCommand{}
schema := &configschema.Block{
Attributes: map[string]*configschema.Attribute{
"path": {
Type: cty.String,
Optional: true,
},
"workspace_dir": {
Type: cty.String,
Required: true,
},
},
}
flagConfigExtra := newRawFlags("-backend-config")
flagConfigExtra.Set("input.config")
_, diags := c.backendConfigOverrideBody(flagConfigExtra, schema)
if len(diags) != 0 {
t.Errorf("expected no diags, got: %s", diags.Err())
}
})
}
func TestInit_backendConfigFilePowershellConfusion(t *testing.T) {
@ -944,6 +1026,52 @@ func TestInit_getProviderSource(t *testing.T) {
}
}
func TestInit_getProviderLegacyFromState(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
copy.CopyDir(testFixturePath("init-get-provider-legacy-from-state"), td)
defer os.RemoveAll(td)
defer testChdir(t, td)()
overrides := metaOverridesForProvider(testProvider())
ui := new(cli.MockUi)
providerSource, close := newMockProviderSource(t, map[string][]string{
"acme/alpha": {"1.2.3"},
})
defer close()
m := Meta{
testingOverrides: overrides,
Ui: ui,
ProviderSource: providerSource,
}
c := &InitCommand{
Meta: m,
}
if code := c.Run(nil); code != 1 {
t.Fatalf("got exit status %d; want 1\nstderr:\n%s\n\nstdout:\n%s", code, ui.ErrorWriter.String(), ui.OutputWriter.String())
}
// Expect this diagnostic output
wants := []string{
"Found unresolvable legacy provider references in state",
"terraform state replace-provider registry.terraform.io/-/alpha registry.terraform.io/acme/alpha",
}
got := ui.ErrorWriter.String()
for _, want := range wants {
if !strings.Contains(got, want) {
t.Fatalf("expected output to contain %q, got:\n\n%s", want, got)
}
}
// Should still install the alpha provider
exactPath := fmt.Sprintf(".terraform/plugins/registry.terraform.io/acme/alpha/1.2.3/%s", getproviders.CurrentPlatform)
if _, err := os.Stat(exactPath); os.IsNotExist(err) {
t.Fatal("provider 'alpha' not downloaded")
}
}
func TestInit_getProviderInvalidPackage(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
@ -1343,6 +1471,13 @@ func TestInit_checkRequiredVersion(t *testing.T) {
if code := c.Run(args); code != 1 {
t.Fatalf("got exit status %d; want 1\nstderr:\n%s\n\nstdout:\n%s", code, ui.ErrorWriter.String(), ui.OutputWriter.String())
}
errStr := ui.ErrorWriter.String()
if !strings.Contains(errStr, `required_version = "~> 0.9.0"`) {
t.Fatalf("output should point to unmet version constraint, but is:\n\n%s", errStr)
}
if strings.Contains(errStr, `required_version = ">= 0.13.0"`) {
t.Fatalf("output should not point to met version constraint, but is:\n\n%s", errStr)
}
}
func TestInit_providerLockFile(t *testing.T) {

15
command/jsonconfig/config.go
View File

@ -48,6 +48,7 @@ type moduleCall struct {
ForEachExpression *expression `json:"for_each_expression,omitempty"`
Module module `json:"module,omitempty"`
VersionConstraint string `json:"version_constraint,omitempty"`
DependsOn []string `json:"depends_on,omitempty"`
}
// variables is the JSON representation of the variables provided to the current
@ -270,6 +271,20 @@ func marshalModuleCall(c *configs.Config, mc *configs.ModuleCall, schemas *terra
module, _ := marshalModule(c, schemas, mc.Name)
ret.Module = module
if len(mc.DependsOn) > 0 {
dependencies := make([]string, len(mc.DependsOn))
for i, d := range mc.DependsOn {
ref, diags := addrs.ParseRef(d)
// we should not get an error here, because `terraform validate`
// would have complained well before this point, but if we do we'll
// silenty skip it.
if !diags.HasErrors() {
dependencies[i] = ref.Subject.String()
}
}
ret.DependsOn = dependencies
}
return ret
}

2
command/login.go
View File

@ -383,6 +383,7 @@ func (c *LoginCommand) interactiveGetTokenByCode(hostname svchost.Hostname, cred
ClientID: clientConfig.ID,
Endpoint: clientConfig.Endpoint(),
RedirectURL: callbackURL,
Scopes: clientConfig.Scopes,
}
authCodeURL := oauthConfig.AuthCodeURL(
@ -475,6 +476,7 @@ func (c *LoginCommand) interactiveGetTokenByPassword(hostname svchost.Hostname,
oauthConfig := &oauth2.Config{
ClientID: clientConfig.ID,
Endpoint: clientConfig.Endpoint(),
Scopes: clientConfig.Scopes,
}
token, err := oauthConfig.PasswordCredentialsToken(context.Background(), username, password)
if err != nil {

56
command/login_test.go
View File

@ -76,6 +76,16 @@ func TestLogin(t *testing.T) {
"token": s.URL + "/token",
},
})
svcs.ForceHostServices(svchost.Hostname("with-scopes.example.com"), map[string]interface{}{
"login.v1": map[string]interface{}{
// with scopes
// mock browser launcher below.
"client": "scopes_test",
"authz": s.URL + "/authz",
"token": s.URL + "/token",
"scopes": []interface{}{"app1.full_access", "app2.read_only"},
},
})
svcs.ForceHostServices(svchost.Hostname("tfe.acme.com"), map[string]interface{}{
// This represents a Terraform Enterprise instance which does not
// yet support the login API, but does support the TFE tokens API.
@ -140,6 +150,52 @@ func TestLogin(t *testing.T) {
}
}))
t.Run("example.com results in no scopes", loginTestCase(func(t *testing.T, c *LoginCommand, ui *cli.MockUi) {
host, _ := c.Services.Discover("example.com")
client, _ := host.ServiceOAuthClient("login.v1")
if len(client.Scopes) != 0 {
t.Errorf("unexpected scopes %q; expected none", client.Scopes)
}
}))
t.Run("with-scopes.example.com with authorization code flow and scopes", loginTestCase(func(t *testing.T, c *LoginCommand, ui *cli.MockUi) {
// Enter "yes" at the consent prompt.
defer testInputMap(t, map[string]string{
"approve": "yes",
})()
status := c.Run([]string{"with-scopes.example.com"})
if status != 0 {
t.Fatalf("unexpected error code %d\nstderr:\n%s", status, ui.ErrorWriter.String())
}
credsSrc := c.Services.CredentialsSource()
creds, err := credsSrc.ForHost(svchost.Hostname("with-scopes.example.com"))
if err != nil {
t.Errorf("failed to retrieve credentials: %s", err)
}
if got, want := creds.Token(), "good-token"; got != want {
t.Errorf("wrong token %q; want %q", got, want)
}
}))
t.Run("with-scopes.example.com results in expected scopes", loginTestCase(func(t *testing.T, c *LoginCommand, ui *cli.MockUi) {
host, _ := c.Services.Discover("with-scopes.example.com")
client, _ := host.ServiceOAuthClient("login.v1")
expectedScopes := [2]string{"app1.full_access", "app2.read_only"}
var foundScopes [2]string
copy(foundScopes[:], client.Scopes)
if foundScopes != expectedScopes || len(client.Scopes) != len(expectedScopes) {
t.Errorf("unexpected scopes %q; want %q", client.Scopes, expectedScopes)
}
}))
t.Run("TFE host without login support", loginTestCase(func(t *testing.T, c *LoginCommand, ui *cli.MockUi) {
// Enter "yes" at the consent prompt, then paste a token with some
// accidental whitespace.

44
command/meta.go
View File

@ -39,6 +39,17 @@ type Meta struct {
// command with a Meta field. These are expected to be set externally
// (not from within the command itself).
// OriginalWorkingDir, if set, is the actual working directory where
// Terraform was run from. This might not be the _actual_ current working
// directory, because users can add the -chdir=... option to the beginning
// of their command line to ask Terraform to switch.
//
// Most things should just use the current working directory in order to
// respect the user's override, but we retain this for exceptional
// situations where we need to refer back to the original working directory
// for some reason.
OriginalWorkingDir string
Color bool // True if output should be colored
GlobalPluginDirs []string // Additional paths to search for plugins
PluginOverrides *PluginOverrides // legacy overrides from .terraformrc file
@ -341,7 +352,12 @@ const (
// contextOpts returns the options to use to initialize a Terraform
// context with the settings from this Meta.
func (m *Meta) contextOpts() *terraform.ContextOpts {
func (m *Meta) contextOpts() (*terraform.ContextOpts, error) {
workspace, err := m.Workspace()
if err != nil {
return nil, err
}
var opts terraform.ContextOpts
opts.Hooks = []terraform.Hook{m.uiHook()}
opts.Hooks = append(opts.Hooks, m.ExtraHooks...)
@ -379,10 +395,11 @@ func (m *Meta) contextOpts() *terraform.ContextOpts {
}
opts.Meta = &terraform.ContextMeta{
Env: m.Workspace(),
Env: workspace,
OriginalWorkingDir: m.OriginalWorkingDir,
}
return &opts
return &opts, nil
}
// defaultFlagSet creates a default flag set for commands.
@ -435,14 +452,18 @@ func (m *Meta) process(args []string) []string {
// Set colorization
m.color = m.Color
for i, v := range args {
i := 0 // output index
for _, v := range args {
if v == "-no-color" {
m.color = false
m.Color = false
args = append(args[:i], args[i+1:]...)
break
} else {
// copy and increment index
args[i] = v
i++
}
}
args = args[:i]
// Set the UI
m.oldUi = m.Ui
@ -599,11 +620,16 @@ func (m *Meta) outputShadowError(err error, output bool) bool {
// and `terraform workspace delete`.
const WorkspaceNameEnvVar = "TF_WORKSPACE"
var invalidWorkspaceNameEnvVar = fmt.Errorf("Invalid workspace name set using %s", WorkspaceNameEnvVar)
// Workspace returns the name of the currently configured workspace, corresponding
// to the desired named state.
func (m *Meta) Workspace() string {
current, _ := m.WorkspaceOverridden()
return current
func (m *Meta) Workspace() (string, error) {
current, overridden := m.WorkspaceOverridden()
if overridden && !validWorkspaceName(current) {
return "", invalidWorkspaceNameEnvVar
}
return current, nil
}
// WorkspaceOverridden returns the name of the currently configured workspace,

41
command/meta_backend.go
View File

@ -101,7 +101,11 @@ func (m *Meta) Backend(opts *BackendOpts) (backend.Enhanced, tfdiags.Diagnostics
}
// Setup the CLI opts we pass into backends that support it.
cliOpts := m.backendCLIOpts()
cliOpts, err := m.backendCLIOpts()
if err != nil {
diags = diags.Append(err)
return nil, diags
}
cliOpts.Validation = true
// If the backend supports CLI initialization, do it.
@ -180,7 +184,10 @@ func (m *Meta) selectWorkspace(b backend.Backend) error {
}
// Get the currently selected workspace.
workspace := m.Workspace()
workspace, err := m.Workspace()
if err != nil {
return err
}
// Check if any of the existing workspaces matches the selected
// workspace and create a numbered list of existing workspaces.
@ -249,7 +256,11 @@ func (m *Meta) BackendForPlan(settings plans.Backend) (backend.Enhanced, tfdiags
// If the backend supports CLI initialization, do it.
if cli, ok := b.(backend.CLI); ok {
cliOpts := m.backendCLIOpts()
cliOpts, err := m.backendCLIOpts()
if err != nil {
diags = diags.Append(err)
return nil, diags
}
if err := cli.CLIInit(cliOpts); err != nil {
diags = diags.Append(fmt.Errorf(
"Error initializing backend %T: %s\n\n"+
@ -270,7 +281,11 @@ func (m *Meta) BackendForPlan(settings plans.Backend) (backend.Enhanced, tfdiags
// Otherwise, we'll wrap our state-only remote backend in the local backend
// to cause any operations to be run locally.
log.Printf("[TRACE] Meta.Backend: backend %T does not support operations, so wrapping it in a local backend", b)
cliOpts := m.backendCLIOpts()
cliOpts, err := m.backendCLIOpts()
if err != nil {
diags = diags.Append(err)
return nil, diags
}
cliOpts.Validation = false // don't validate here in case config contains file(...) calls where the file doesn't exist
local := backendLocal.NewWithBackend(b)
if err := local.CLIInit(cliOpts); err != nil {
@ -283,7 +298,11 @@ func (m *Meta) BackendForPlan(settings plans.Backend) (backend.Enhanced, tfdiags
// backendCLIOpts returns a backend.CLIOpts object that should be passed to
// a backend that supports local CLI operations.
func (m *Meta) backendCLIOpts() *backend.CLIOpts {
func (m *Meta) backendCLIOpts() (*backend.CLIOpts, error) {
contextOpts, err := m.contextOpts()
if err != nil {
return nil, err
}
return &backend.CLIOpts{
CLI: m.Ui,
CLIColor: m.Colorize(),
@ -291,10 +310,10 @@ func (m *Meta) backendCLIOpts() *backend.CLIOpts {
StatePath: m.statePath,
StateOutPath: m.stateOutPath,
StateBackupPath: m.backupPath,
ContextOpts: m.contextOpts(),
ContextOpts: contextOpts,
Input: m.Input(),
RunningInAutomation: m.RunningInAutomation,
}
}, nil
}
// IsLocalBackend returns true if the backend is a local backend. We use this
@ -318,7 +337,13 @@ func (m *Meta) IsLocalBackend(b backend.Backend) bool {
// be called.
func (m *Meta) Operation(b backend.Backend) *backend.Operation {
schema := b.ConfigSchema()
workspace := m.Workspace()
workspace, err := m.Workspace()
if err != nil {
// An invalid workspace error would have been raised when creating the
// backend, and the caller should have already exited. Seeing the error
// here first is a bug, so panic.
panic(fmt.Sprintf("invalid workspace: %s", err))
}
planOutBackend, err := m.backendState.ForPlan(schema, workspace)
if err != nil {
// Always indicates an implementation error in practice, because

10
command/meta_backend_migrate.go
View File

@ -180,7 +180,10 @@ func (m *Meta) backendMigrateState_S_S(opts *backendMigrateOpts) error {
func (m *Meta) backendMigrateState_S_s(opts *backendMigrateOpts) error {
log.Printf("[TRACE] backendMigrateState: target backend type %q does not support named workspaces", opts.TwoType)
currentEnv := m.Workspace()
currentEnv, err := m.Workspace()
if err != nil {
return err
}
migrate := opts.force
if !migrate {
@ -261,9 +264,12 @@ func (m *Meta) backendMigrateState_s_s(opts *backendMigrateOpts) error {
return nil, err
}
// Ignore invalid workspace name as it is irrelevant in this context.
workspace, _ := m.Workspace()
// If the currently selected workspace is the default workspace, then set
// the named workspace as the new selected workspace.
if m.Workspace() == backend.DefaultStateName {
if workspace == backend.DefaultStateName {
if err := m.SetWorkspace(opts.twoEnv); err != nil {
return nil, fmt.Errorf("Failed to set new workspace: %s", err)
}

6
command/meta_backend_test.go
View File

@ -1002,7 +1002,11 @@ func TestMetaBackend_configuredChangeCopy_multiToSingle(t *testing.T) {
}
// Verify we are now in the default env, or we may not be able to access the new backend
if env := m.Workspace(); env != backend.DefaultStateName {
env, err := m.Workspace()
if err != nil {
t.Fatal(err)
}
if env != backend.DefaultStateName {
t.Fatal("using non-default env with single-env backend")
}
}

97
command/meta_test.go
View File

@ -11,6 +11,7 @@ import (
"github.com/google/go-cmp/cmp"
"github.com/hashicorp/terraform/backend"
"github.com/hashicorp/terraform/backend/local"
"github.com/hashicorp/terraform/terraform"
)
@ -55,6 +56,21 @@ func TestMetaColorize(t *testing.T) {
if !m.Colorize().Disable {
t.Fatal("should be disabled")
}
// Test disable #2
// Verify multiple -no-color options are removed from args slice.
// E.g. an additional -no-color arg could be added by TF_CLI_ARGS.
m = new(Meta)
m.Color = true
args = []string{"foo", "-no-color", "bar", "-no-color"}
args2 = []string{"foo", "bar"}
args = m.process(args)
if !reflect.DeepEqual(args, args2) {
t.Fatalf("bad: %#v", args)
}
if !m.Colorize().Disable {
t.Fatal("should be disabled")
}
}
func TestMetaInputMode(t *testing.T) {
@ -170,7 +186,10 @@ func TestMeta_Env(t *testing.T) {
m := new(Meta)
env := m.Workspace()
env, err := m.Workspace()
if err != nil {
t.Fatal(err)
}
if env != backend.DefaultStateName {
t.Fatalf("expected env %q, got env %q", backend.DefaultStateName, env)
@ -181,7 +200,7 @@ func TestMeta_Env(t *testing.T) {
t.Fatal("error setting env:", err)
}
env = m.Workspace()
env, _ = m.Workspace()
if env != testEnv {
t.Fatalf("expected env %q, got env %q", testEnv, env)
}
@ -190,12 +209,84 @@ func TestMeta_Env(t *testing.T) {
t.Fatal("error setting env:", err)
}
env = m.Workspace()
env, _ = m.Workspace()
if env != backend.DefaultStateName {
t.Fatalf("expected env %q, got env %q", backend.DefaultStateName, env)
}
}
func TestMeta_Workspace_override(t *testing.T) {
defer func(value string) {
os.Setenv(WorkspaceNameEnvVar, value)
}(os.Getenv(WorkspaceNameEnvVar))
m := new(Meta)
testCases := map[string]struct {
workspace string
err error
}{
"": {
"default",
nil,
},
"development": {
"development",
nil,
},
"invalid name": {
"",
invalidWorkspaceNameEnvVar,
},
}
for name, tc := range testCases {
t.Run(name, func(t *testing.T) {
os.Setenv(WorkspaceNameEnvVar, name)
workspace, err := m.Workspace()
if workspace != tc.workspace {
t.Errorf("Unexpected workspace\n got: %s\nwant: %s\n", workspace, tc.workspace)
}
if err != tc.err {
t.Errorf("Unexpected error\n got: %s\nwant: %s\n", err, tc.err)
}
})
}
}
func TestMeta_Workspace_invalidSelected(t *testing.T) {
td := tempDir(t)
os.MkdirAll(td, 0755)
defer os.RemoveAll(td)
defer testChdir(t, td)()
// this is an invalid workspace name
workspace := "test workspace"
// create the workspace directories
if err := os.MkdirAll(filepath.Join(local.DefaultWorkspaceDir, workspace), 0755); err != nil {
t.Fatal(err)
}
// create the workspace file to select it
if err := os.MkdirAll(DefaultDataDir, 0755); err != nil {
t.Fatal(err)
}
if err := ioutil.WriteFile(filepath.Join(DefaultDataDir, local.DefaultWorkspaceFile), []byte(workspace), 0644); err != nil {
t.Fatal(err)
}
m := new(Meta)
ws, err := m.Workspace()
if ws != workspace {
t.Errorf("Unexpected workspace\n got: %s\nwant: %s\n", ws, workspace)
}
if err != nil {
t.Errorf("Unexpected error: %s", err)
}
}
func TestMeta_process(t *testing.T) {
test = false
defer func() { test = true }()

36
command/output.go
View File

@ -10,7 +10,6 @@ import (
ctyjson "github.com/zclconf/go-cty/cty/json"
"github.com/hashicorp/terraform/addrs"
"github.com/hashicorp/terraform/configs/hcl2shim"
"github.com/hashicorp/terraform/repl"
"github.com/hashicorp/terraform/states"
"github.com/hashicorp/terraform/tfdiags"
@ -64,7 +63,11 @@ func (c *OutputCommand) Run(args []string) int {
return 1
}
env := c.Workspace()
env, err := c.Workspace()
if err != nil {
c.Ui.Error(fmt.Sprintf("Error selecting workspace: %s", err))
return 1
}
// Get the state
stateStore, err := b.StateMgr(env)
@ -109,13 +112,17 @@ func (c *OutputCommand) Run(args []string) int {
}
if !jsonOutput && (state.Empty() || len(mod.OutputValues) == 0) {
c.Ui.Error(
"The state file either has no outputs defined, or all the defined\n" +
"outputs are empty. Please define an output in your configuration\n" +
"with the `output` keyword and run `terraform refresh` for it to\n" +
"become available. If you are using interpolation, please verify\n" +
"the interpolated value is not empty. You can use the \n" +
"`terraform console` command to assist.")
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Warning,
"No outputs found",
"The state file either has no outputs defined, or all the defined "+
"outputs are empty. Please define an output in your configuration "+
"with the `output` keyword and run `terraform refresh` for it to "+
"become available. If you are using interpolation, please verify "+
"the interpolated value is not empty. You can use the "+
"`terraform console` command to assist.",
))
c.showDiagnostics(diags)
return 0
}
@ -187,16 +194,7 @@ func (c *OutputCommand) Run(args []string) int {
c.Ui.Output(string(jsonOutput))
} else {
// Our formatter still wants an old-style raw interface{} value, so
// for now we'll just shim it.
// FIXME: Port the formatter to work with cty.Value directly.
legacyVal := hcl2shim.ConfigValueFromHCL2(v)
result, err := repl.FormatResult(legacyVal)
if err != nil {
diags = diags.Append(err)
c.showDiagnostics(diags)
return 1
}
result := repl.FormatValue(v, 0)
c.Ui.Output(result)
}

25
command/output_test.go
View File

@ -41,7 +41,7 @@ func TestOutput(t *testing.T) {
}
actual := strings.TrimSpace(ui.OutputWriter.String())
if actual != "bar" {
if actual != `"bar"` {
t.Fatalf("bad: %#v", actual)
}
}
@ -80,9 +80,19 @@ func TestOutput_nestedListAndMap(t *testing.T) {
}
actual := strings.TrimSpace(ui.OutputWriter.String())
expected := "foo = [\n {\n \"key\" = \"value\"\n \"key2\" = \"value2\"\n },\n {\n \"key\" = \"value\"\n },\n]"
expected := strings.TrimSpace(`
foo = tolist([
tomap({
"key" = "value"
"key2" = "value2"
}),
tomap({
"key" = "value"
}),
])
`)
if actual != expected {
t.Fatalf("wrong output\ngot: %#v\nwant: %#v", actual, expected)
t.Fatalf("wrong output\ngot: %s\nwant: %s", actual, expected)
}
}
@ -120,7 +130,7 @@ func TestOutput_json(t *testing.T) {
}
}
func TestOutput_emptyOutputsErr(t *testing.T) {
func TestOutput_emptyOutputs(t *testing.T) {
originalState := states.NewState()
statePath := testStateFile(t, originalState)
@ -139,6 +149,9 @@ func TestOutput_emptyOutputsErr(t *testing.T) {
if code := c.Run(args); code != 0 {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
if got, want := ui.ErrorWriter.String(), "Warning: No outputs found"; !strings.Contains(got, want) {
t.Fatalf("bad output: expected to contain %q, got:\n%s", want, got)
}
}
func TestOutput_jsonEmptyOutputs(t *testing.T) {
@ -257,7 +270,7 @@ func TestOutput_blank(t *testing.T) {
t.Fatalf("bad: \n%s", ui.ErrorWriter.String())
}
expectedOutput := "foo = bar\nname = john-doe\n"
expectedOutput := "foo = \"bar\"\nname = \"john-doe\"\n"
output := ui.OutputWriter.String()
if output != expectedOutput {
t.Fatalf("wrong output\ngot: %#v\nwant: %#v", output, expectedOutput)
@ -390,7 +403,7 @@ func TestOutput_stateDefault(t *testing.T) {
}
actual := strings.TrimSpace(ui.OutputWriter.String())
if actual != "bar" {
if actual != `"bar"` {
t.Fatalf("bad: %#v", actual)
}
}

7
command/plan.go
View File

@ -135,7 +135,12 @@ func (c *PlanCommand) Run(args []string) int {
}
var backendForPlan plans.Backend
backendForPlan.Type = backendPseudoState.Type
backendForPlan.Workspace = c.Workspace()
workspace, err := c.Workspace()
if err != nil {
c.Ui.Error(fmt.Sprintf("Error selecting workspace: %s", err))
return 1
}
backendForPlan.Workspace = workspace
// Configuration is a little more awkward to handle here because it's
// stored in state as raw JSON but we need it as a plans.DynamicValue

76
command/plan_test.go
View File

@ -96,10 +96,6 @@ func TestPlan_plan(t *testing.T) {
if code := c.Run(args); code != 1 {
t.Fatalf("wrong exit status %d; want 1\nstderr: %s", code, ui.ErrorWriter.String())
}
if p.ReadResourceCalled {
t.Fatal("ReadResource should not have been called")
}
}
func TestPlan_destroy(t *testing.T) {
@ -142,10 +138,6 @@ func TestPlan_destroy(t *testing.T) {
t.Fatalf("bad: %d\n\n%s", code, ui.ErrorWriter.String())
}
if !p.ReadResourceCalled {
t.Fatal("ReadResource should have been called")
}
plan := testReadPlan(t, outPath)
for _, rc := range plan.Changes.Resources {
if got, want := rc.Action, plans.Delete; got != want {
@ -560,15 +552,15 @@ func TestPlan_varsUnset(t *testing.T) {
tmp, cwd := testCwd(t)
defer testFixCwd(t, tmp, cwd)
// Disable test mode so input would be asked
test = false
defer func() { test = true }()
// The plan command will prompt for interactive input of var.foo.
// We'll answer "bar" to that prompt, which should then allow this
// configuration to apply even though var.foo doesn't have a
// default value and there are no -var arguments on our command line.
defaultInputReader = bytes.NewBufferString("bar\n")
// This will (helpfully) panic if more than one variable is requested during plan:
// https://github.com/hashicorp/terraform/issues/26027
close := testInteractiveInput(t, []string{"bar"})
defer close()
p := planVarsFixtureProvider()
ui := new(cli.MockUi)
@ -587,6 +579,64 @@ func TestPlan_varsUnset(t *testing.T) {
}
}
// This test adds a required argument to the test provider to validate
// processing of user input:
// https://github.com/hashicorp/terraform/issues/26035
func TestPlan_providerArgumentUnset(t *testing.T) {
tmp, cwd := testCwd(t)
defer testFixCwd(t, tmp, cwd)
// Disable test mode so input would be asked
test = false
defer func() { test = true }()
// The plan command will prompt for interactive input of provider.test.region
defaultInputReader = bytes.NewBufferString("us-east-1\n")
p := planFixtureProvider()
// override the planFixtureProvider schema to include a required provider argument
p.GetSchemaReturn = &terraform.ProviderSchema{
Provider: &configschema.Block{
Attributes: map[string]*configschema.Attribute{
"region": {Type: cty.String, Required: true},
},
},
ResourceTypes: map[string]*configschema.Block{
"test_instance": {
Attributes: map[string]*configschema.Attribute{
"id": {Type: cty.String, Optional: true, Computed: true},
"ami": {Type: cty.String, Optional: true, Computed: true},
},
BlockTypes: map[string]*configschema.NestedBlock{
"network_interface": {
Nesting: configschema.NestingList,
Block: configschema.Block{
Attributes: map[string]*configschema.Attribute{
"device_index": {Type: cty.String, Optional: true},
"description": {Type: cty.String, Optional: true},
},
},
},
},
},
},
}
ui := new(cli.MockUi)
c := &PlanCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(p),
Ui: ui,
},
}
args := []string{
testFixturePath("plan"),
}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: %d\n\n%s", code, ui.ErrorWriter.String())
}
}
func TestPlan_varFile(t *testing.T) {
tmp, cwd := testCwd(t)
defer testFixCwd(t, tmp, cwd)

2
command/plugins.go
View File

@ -25,7 +25,7 @@ import (
//
// The provider-related functions live primarily in meta_providers.go, and
// lean on some different underlying mechanisms in order to support automatic
// installation and a heirarchical addressing namespace, neither of which
// installation and a hierarchical addressing namespace, neither of which
// are supported for other plugin types.
// store the user-supplied path for plugin discovery

6
command/providers.go
View File

@ -83,7 +83,11 @@ func (c *ProvidersCommand) Run(args []string) int {
}
// Get the state
env := c.Workspace()
env, err := c.Workspace()
if err != nil {
c.Ui.Error(fmt.Sprintf("Error selecting workspace: %s", err))
return 1
}
s, err := b.StateMgr(env)
if err != nil {
c.Ui.Error(fmt.Sprintf("Failed to load state: %s", err))

4
command/providers_mirror.go
View File

@ -269,8 +269,8 @@ func (c *ProvidersMirrorCommand) Run(args []string) int {
indexArchives[version] = map[string]interface{}{}
}
indexArchives[version][platform.String()] = map[string]interface{}{
"url": archiveFilename, // a relative URL from the index file's URL
"hashes": []string{hash}, // an array to allow for additional hash formats in future
"url": archiveFilename, // a relative URL from the index file's URL
"hashes": []string{hash.String()}, // an array to allow for additional hash formats in future
}
}
mainIndex := map[string]interface{}{

6
command/show.go
View File

@ -130,7 +130,11 @@ func (c *ShowCommand) Run(args []string) int {
}
}
} else {
env := c.Workspace()
env, err := c.Workspace()
if err != nil {
c.Ui.Error(fmt.Sprintf("Error selecting workspace: %s", err))
return 1
}
stateFile, stateErr = getStateFromEnv(b, env)
if stateErr != nil {
c.Ui.Error(stateErr.Error())

6
command/state_list.go
View File

@ -41,7 +41,11 @@ func (c *StateListCommand) Run(args []string) int {
}
// Get the state
env := c.Workspace()
env, err := c.Workspace()
if err != nil {
c.Ui.Error(fmt.Sprintf("Error selecting workspace: %s", err))
return 1
}
stateMgr, err := b.StateMgr(env)
if err != nil {
c.Ui.Error(fmt.Sprintf(errStateLoadingState, err))

5
command/state_meta.go
View File

@ -37,7 +37,10 @@ func (c *StateMeta) State() (statemgr.Full, error) {
return nil, backendDiags.Err()
}
workspace := c.Workspace()
workspace, err := c.Workspace()
if err != nil {
return nil, err
}
// Get the state
s, err := b.StateMgr(workspace)
if err != nil {

6
command/state_pull.go
View File

@ -32,7 +32,11 @@ func (c *StatePullCommand) Run(args []string) int {
}
// Get the state manager for the current workspace
env := c.Workspace()
env, err := c.Workspace()
if err != nil {
c.Ui.Error(fmt.Sprintf("Error selecting workspace: %s", err))
return 1
}
stateMgr, err := b.StateMgr(env)
if err != nil {
c.Ui.Error(fmt.Sprintf(errStateLoadingState, err))

6
command/state_push.go
View File

@ -72,7 +72,11 @@ func (c *StatePushCommand) Run(args []string) int {
}
// Get the state manager for the currently-selected workspace
env := c.Workspace()
env, err := c.Workspace()
if err != nil {
c.Ui.Error(fmt.Sprintf("Error selecting workspace: %s", err))
return 1
}
stateMgr, err := b.StateMgr(env)
if err != nil {
c.Ui.Error(fmt.Sprintf("Failed to load destination state: %s", err))

4
command/state_replace_provider_test.go
View File

@ -75,7 +75,7 @@ func TestStateReplaceProvider(t *testing.T) {
inputBuf := &bytes.Buffer{}
ui.InputReader = inputBuf
inputBuf.WriteString("yes")
inputBuf.WriteString("yes\n")
args := []string{
"-state", statePath,
@ -143,7 +143,7 @@ func TestStateReplaceProvider(t *testing.T) {
inputBuf := &bytes.Buffer{}
ui.InputReader = inputBuf
inputBuf.WriteString("no")
inputBuf.WriteString("no\n")
args := []string{
"-state", statePath,

6
command/state_show.go
View File

@ -89,7 +89,11 @@ func (c *StateShowCommand) Run(args []string) int {
schemas := ctx.Schemas()
// Get the state
env := c.Workspace()
env, err := c.Workspace()
if err != nil {
c.Ui.Error(fmt.Sprintf("Error selecting workspace: %s", err))
return 1
}
stateMgr, err := b.StateMgr(env)
if err != nil {
c.Ui.Error(fmt.Sprintf(errStateLoadingState, err))

38
command/taint.go
View File

@ -3,11 +3,13 @@ package command
import (
"context"
"fmt"
"os"
"strings"
"github.com/hashicorp/terraform/addrs"
"github.com/hashicorp/terraform/command/clistate"
"github.com/hashicorp/terraform/states"
"github.com/hashicorp/terraform/terraform"
"github.com/hashicorp/terraform/tfdiags"
)
@ -27,7 +29,7 @@ func (c *TaintCommand) Run(args []string) int {
cmdFlags.BoolVar(&c.Meta.stateLock, "lock", true, "lock state")
cmdFlags.DurationVar(&c.Meta.stateLockTimeout, "lock-timeout", 0, "lock timeout")
cmdFlags.StringVar(&module, "module", "", "module")
cmdFlags.StringVar(&c.Meta.statePath, "state", DefaultStateFilename, "path")
cmdFlags.StringVar(&c.Meta.statePath, "state", "", "path")
cmdFlags.StringVar(&c.Meta.stateOutPath, "state-out", "", "path")
cmdFlags.Usage = func() { c.Ui.Error(c.Help()) }
if err := cmdFlags.Parse(args); err != nil {
@ -62,6 +64,34 @@ func (c *TaintCommand) Run(args []string) int {
return 1
}
// Load the config and check the core version requirements are satisfied
loader, err := c.initConfigLoader()
if err != nil {
diags = diags.Append(err)
c.showDiagnostics(diags)
return 1
}
pwd, err := os.Getwd()
if err != nil {
c.Ui.Error(fmt.Sprintf("Error getting pwd: %s", err))
return 1
}
config, configDiags := loader.LoadConfig(pwd)
diags = diags.Append(configDiags)
if diags.HasErrors() {
c.showDiagnostics(diags)
return 1
}
versionDiags := terraform.CheckCoreVersionRequirements(config)
diags = diags.Append(versionDiags)
if diags.HasErrors() {
c.showDiagnostics(diags)
return 1
}
// Load the backend
b, backendDiags := c.Backend(nil)
diags = diags.Append(backendDiags)
@ -71,7 +101,11 @@ func (c *TaintCommand) Run(args []string) int {
}
// Get the state
env := c.Workspace()
env, err := c.Workspace()
if err != nil {
c.Ui.Error(fmt.Sprintf("Error selecting workspace: %s", err))
return 1
}
stateMgr, err := b.StateMgr(env)
if err != nil {
c.Ui.Error(fmt.Sprintf("Failed to load state: %s", err))

94
command/taint_test.go
View File

@ -8,6 +8,7 @@ import (
"github.com/mitchellh/cli"
"github.com/hashicorp/terraform/addrs"
"github.com/hashicorp/terraform/helper/copy"
"github.com/hashicorp/terraform/states"
"github.com/hashicorp/terraform/terraform"
)
@ -239,6 +240,48 @@ func TestTaint_defaultState(t *testing.T) {
testStateOutput(t, path, testTaintStr)
}
func TestTaint_defaultWorkspaceState(t *testing.T) {
// Get a temp cwd
tmp, cwd := testCwd(t)
defer testFixCwd(t, tmp, cwd)
state := states.BuildState(func(s *states.SyncState) {
s.SetResourceInstanceCurrent(
addrs.Resource{
Mode: addrs.ManagedResourceMode,
Type: "test_instance",
Name: "foo",
}.Instance(addrs.NoKey).Absolute(addrs.RootModuleInstance),
&states.ResourceInstanceObjectSrc{
AttrsJSON: []byte(`{"id":"bar"}`),
Status: states.ObjectReady,
},
addrs.AbsProviderConfig{
Provider: addrs.NewLegacyProvider("test"),
Module: addrs.RootModule,
},
)
})
testWorkspace := "development"
path := testStateFileWorkspaceDefault(t, testWorkspace, state)
ui := new(cli.MockUi)
meta := Meta{Ui: ui}
meta.SetWorkspace(testWorkspace)
c := &TaintCommand{
Meta: meta,
}
args := []string{
"test_instance.foo",
}
if code := c.Run(args); code != 0 {
t.Fatalf("bad: %d\n\n%s", code, ui.ErrorWriter.String())
}
testStateOutput(t, path, testTaintStr)
}
func TestTaint_missing(t *testing.T) {
state := states.BuildState(func(s *states.SyncState) {
s.SetResourceInstanceCurrent(
@ -407,6 +450,57 @@ func TestTaint_module(t *testing.T) {
testStateOutput(t, statePath, testTaintModuleStr)
}
func TestTaint_checkRequiredVersion(t *testing.T) {
// Create a temporary working directory that is empty
td := tempDir(t)
copy.CopyDir(testFixturePath("taint-check-required-version"), td)
defer os.RemoveAll(td)
defer testChdir(t, td)()
// Write the temp state
state := &terraform.State{
Modules: []*terraform.ModuleState{
{
Path: []string{"root"},
Resources: map[string]*terraform.ResourceState{
"test_instance.foo": {
Type: "test_instance",
Primary: &terraform.InstanceState{
ID: "bar",
},
},
},
},
},
}
path := testStateFileDefault(t, state)
ui := cli.NewMockUi()
c := &TaintCommand{
Meta: Meta{
testingOverrides: metaOverridesForProvider(testProvider()),
Ui: ui,
},
}
args := []string{"test_instance.foo"}
if code := c.Run(args); code != 1 {
t.Fatalf("got exit status %d; want 1\nstderr:\n%s\n\nstdout:\n%s", code, ui.ErrorWriter.String(), ui.OutputWriter.String())
}
// State is unchanged
testStateOutput(t, path, testTaintDefaultStr)
// Required version diags are correct
errStr := ui.ErrorWriter.String()
if !strings.Contains(errStr, `required_version = "~> 0.9.0"`) {
t.Fatalf("output should point to unmet version constraint, but is:\n\n%s", errStr)
}
if strings.Contains(errStr, `required_version = ">= 0.13.0"`) {
t.Fatalf("output should not point to met version constraint, but is:\n\n%s", errStr)
}
}
const testTaintStr = `
test_instance.foo: (tainted)
ID = bar

3
command/testdata/013upgrade-provider-redirect-version-unavailable/expected/main.tf vendored Normal file
View File

@ -0,0 +1,3 @@
provider "qux" {
version = "~> 0.9.0"
}

8
command/testdata/013upgrade-provider-redirect-version-unavailable/expected/versions.tf vendored Normal file
View File

@ -0,0 +1,8 @@
terraform {
required_providers {
qux = {
source = "hashicorp/qux"
}
}
required_version = ">= 0.13"
}

3
command/testdata/013upgrade-provider-redirect-version-unavailable/input/main.tf vendored Normal file
View File

@ -0,0 +1,3 @@
provider "qux" {
version = "~> 0.9.0"
}

3
command/testdata/013upgrade-provider-redirect/expected/main.tf vendored Normal file
View File

@ -0,0 +1,3 @@
provider "qux" {
version = ">= 1.0.0"
}

8
command/testdata/013upgrade-provider-redirect/expected/versions.tf vendored Normal file
View File

@ -0,0 +1,8 @@
terraform {
required_providers {
qux = {
source = "acme/qux"
}
}
required_version = ">= 0.13"
}

3
command/testdata/013upgrade-provider-redirect/input/main.tf vendored Normal file
View File

@ -0,0 +1,3 @@
provider "qux" {
version = ">= 1.0.0"
}

41
command/testdata/fmt/general_in.tf vendored Normal file
View File

@ -0,0 +1,41 @@
# This test case is intended to cover many of the main formatting
# rules of "terraform fmt" at once. It's fine to add new stuff in
# here, but you can also add other _in.tf/_out.tf pairs in the
# same directory if you want to test something complicated that,
# for example, requires specific nested context.
#
# The input file of this test intentionally has strange whitespace
# alignment, because the goal is to see the fmt command fix it.
# If you're applying batch formatting to all .tf files in the
# repository (or similar), be sure to skip this one to avoid
# invalidating the test.
terraform {
required_providers {
foo = { version = "1.0.0" }
barbaz = {
version = "2.0.0"
}
}
}
variable instance_type {
}
resource foo_instance foo {
instance_type = "${var.instance_type}"
}
resource foo_instance "bar" {
instance_type = "${var.instance_type}-2"
}
resource "foo_instance" /* ... */ "baz" {
instance_type = "${var.instance_type}${var.instance_type}"
beep boop {}
beep blep {
thingy = "${var.instance_type}"
}
}

41
command/testdata/fmt/general_out.tf vendored Normal file
View File

@ -0,0 +1,41 @@
# This test case is intended to cover many of the main formatting
# rules of "terraform fmt" at once. It's fine to add new stuff in
# here, but you can also add other _in.tf/_out.tf pairs in the
# same directory if you want to test something complicated that,
# for example, requires specific nested context.
#
# The input file of this test intentionally has strange whitespace
# alignment, because the goal is to see the fmt command fix it.
# If you're applying batch formatting to all .tf files in the
# repository (or similar), be sure to skip this one to avoid
# invalidating the test.
terraform {
required_providers {
foo = { version = "1.0.0" }
barbaz = {
version = "2.0.0"
}
}
}
variable "instance_type" {
}
resource "foo_instance" "foo" {
instance_type = var.instance_type
}
resource "foo_instance" "bar" {
instance_type = "${var.instance_type}-2"
}
resource "foo_instance" "baz" {
instance_type = "${var.instance_type}${var.instance_type}"
beep "boop" {}
beep "blep" {
thingy = var.instance_type
}
}

57
command/testdata/fmt/variable_type_in.tf vendored Normal file
View File

@ -0,0 +1,57 @@
variable "a" {
type = string
}
variable "b" {
type = list
}
variable "c" {
type = map
}
variable "d" {
type = set
}
variable "e" {
type = "string"
}
variable "f" {
type = "list"
}
variable "g" {
type = "map"
}
variable "h" {
type = object({})
}
variable "i" {
type = object({
foo = string
})
}
variable "j" {
type = tuple([])
}
variable "k" {
type = tuple([number])
}
variable "l" {
type = list(string)
}
variable "m" {
type = list(
object({
foo = bool
})
)
}

57
command/testdata/fmt/variable_type_out.tf vendored Normal file
View File

@ -0,0 +1,57 @@
variable "a" {
type = string
}
variable "b" {
type = list(any)
}
variable "c" {
type = map(any)
}
variable "d" {
type = set(any)
}
variable "e" {
type = string
}
variable "f" {
type = list(string)
}
variable "g" {
type = map(string)
}
variable "h" {
type = object({})
}
variable "i" {
type = object({
foo = string
})
}
variable "j" {
type = tuple([])
}
variable "k" {
type = tuple([number])
}
variable "l" {
type = list(string)
}
variable "m" {
type = list(
object({
foo = bool
})
)
}

11
command/testdata/import-module-input-variable/child/main.tf vendored Normal file
View File

@ -0,0 +1,11 @@
variable "foo" {
default = {}
}
locals {
baz = var.foo.bar.baz
}
resource "test_instance" "foo" {
foo = local.baz
}

8
command/testdata/import-module-input-variable/main.tf vendored Normal file
View File

@ -0,0 +1,8 @@
variable "foo" {
default = {}
}
module "child" {
source = "./child"
foo = var.foo
}

1
command/testdata/import-module-input-variable/terraform.tfvars vendored Normal file
View File

@ -0,0 +1 @@
foo = { bar = { baz = true } }

2
command/testdata/init-backend-config-file/backend.config vendored
View File

@ -1,5 +1,5 @@
// the -backend-config flag on init cannot be used to point to a "full" backend
// block, only key-value pairs (like terraform.tfvars)
// block
terraform {
backend "local" {
path = "hello"

2
command/testdata/init-backend-config-file/invalid.config vendored Normal file
View File

@ -0,0 +1,2 @@
path = "hello"
foo = "bar"

4
command/testdata/init-check-required-version/main.tf vendored
View File

@ -1,3 +1,7 @@
terraform {
required_version = "~> 0.9.0"
}
terraform {
required_version = ">= 0.13.0"
}

12
command/testdata/init-get-provider-legacy-from-state/main.tf vendored Normal file
View File

@ -0,0 +1,12 @@
terraform {
required_providers {
alpha = {
source = "acme/alpha"
version = "1.2.3"
}
}
}
resource "alpha_resource" "a" {
index = 1
}

25
command/testdata/init-get-provider-legacy-from-state/terraform.tfstate vendored Normal file
View File

@ -0,0 +1,25 @@
{
"version": 4,
"terraform_version": "0.12.28",
"serial": 1,
"lineage": "481bf512-f245-4c60-42dc-7005f4fa9181",
"outputs": {},
"resources": [
{
"mode": "managed",
"type": "alpha_resource",
"name": "a",
"provider": "provider.alpha",
"instances": [
{
"schema_version": 0,
"attributes": {
"id": "a",
"index": 1
},
"private": "bnVsbA=="
}
]
}
]
}

41
command/testdata/providers-schema/required/output.json vendored Normal file
View File

@ -0,0 +1,41 @@
{
"format_version": "0.1",
"provider_schemas": {
"registry.terraform.io/hashicorp/test": {
"provider": {
"version": 0,
"block": {
"attributes": {
"region": {
"description_kind": "plain",
"optional": true,
"type": "string"
}
},
"description_kind": "plain"
}
},
"resource_schemas": {
"test_instance": {
"version": 0,
"block": {
"attributes": {
"ami": {
"type": "string",
"optional": true,
"description_kind": "plain"
},
"id": {
"type": "string",
"optional": true,
"computed": true,
"description_kind": "plain"
}
},
"description_kind": "plain"
}
}
}
}
}
}

7
command/testdata/providers-schema/required/provider.tf vendored Normal file
View File

@ -0,0 +1,7 @@
terraform {
required_providers {
test = {
source = "hashicorp/test"
}
}
}

3
command/testdata/show-json/module-depends-on/foo/main.tf vendored Normal file
View File

@ -0,0 +1,3 @@
variable "test_var" {
default = "foo-var"
}

11
command/testdata/show-json/module-depends-on/main.tf vendored Normal file
View File

@ -0,0 +1,11 @@
module "foo" {
source = "./foo"
depends_on = [
test_instance.test
]
}
resource "test_instance" "test" {
ami = "foo-bar"
}

76
command/testdata/show-json/module-depends-on/output.json vendored Normal file
View File

@ -0,0 +1,76 @@
{
"format_version": "0.1",
"terraform_version": "0.13.1-dev",
"planned_values": {
"root_module": {
"resources": [
{
"address": "test_instance.test",
"mode": "managed",
"type": "test_instance",
"name": "test",
"provider_name": "registry.terraform.io/hashicorp/test",
"schema_version": 0,
"values": {
"ami": "foo-bar"
}
}
]
}
},
"resource_changes": [
{
"address": "test_instance.test",
"mode": "managed",
"type": "test_instance",
"name": "test",
"provider_name": "registry.terraform.io/hashicorp/test",
"change": {
"actions": [
"create"
],
"before": null,
"after": {
"ami": "foo-bar"
},
"after_unknown": {
"id": true
}
}
}
],
"configuration": {
"root_module": {
"resources": [
{
"address": "test_instance.test",
"mode": "managed",
"type": "test_instance",
"name": "test",
"provider_config_key": "test",
"expressions": {
"ami": {
"constant_value": "foo-bar"
}
},
"schema_version": 0
}
],
"module_calls": {
"foo": {
"depends_on": [
"test_instance.test"
],
"source": "./foo",
"module": {
"variables": {
"test_var": {
"default": "foo-var"
}
}
}
}
}
}
}
}

Some files were not shown because too many files have changed in this diff Show More