* Fail global required_version check if it contains any prerelease fields
* go mod tidy
* Improve required_version prerelease not supported error string
* Add prerelease version constraint unit tests
* Fix side-effects by populating global diags too soon
This allows us to remove the manual replace directives
github.com/dgrijalva/jwt-go and google.golang.org/grpc, so that we can
remove the CVE warnings and update the grpc packages.
While the etcdv3 backend is also marked as deprecated, the changes here
are done in a manner to keep that backend working for the time being.
This means we can now use type parameter syntax where appropriate.
This commit also includes an upgrade to the golang.org/x/tools module,
in order to get a newer version of "stringer" that supports the type
parameters syntax.
The version that was in use previously broke my runs with a nice `ssh:
handshake failed: agent unsupported algorithm "ssh-ed25519"` error. I
took a further look into this, and saw that the change that broke ssh
for me was fixed in https://github.com/golang/crypto/commit/46612604a.
Yay!
This tagged release is a superset of the specific commit we had previously
selected, adding in just an improved error message for inconsistent result
types in a conditional expression.
This is intended to eventually replace the CircleCI-based checks we use
as part of the PR process in this repository. We're already using GitHub
Actions for various other processes in this repository, so this change is
motivated by consistency of having all of our automation running in the
same system and written in the same language.
This is not a complete replacement for our CircleCI workflow yet, and
probably won't ever be because the CircleCI workflow contains some steps
that are arguably redundant with other processes we follow elsewhere.
However, the CircleCI workflow remains for now and won't be removed until
we're satisfied that enough of it is replicated by this GitHub Actions
workflow.
`go-slug` has been updated to not upload `terraform.tfstate` to the slug
so that a user will no longer receive the error message about the
leftover state file after migrating from the local backend to TFC.
This change will await the completion of pre-apply run tasks if they
exist on a run and then report the results.
It also adds an abstraction when interacting with cloud integrations such
as policy checking and cost estimation that simplify and unify output,
although I did not go so far as to refactor those callers to use it yet.
There was an unintended regression in go-getter v1.5.9's GitGetter which
caused us to temporarily fork that particular getter into Terraform to
expedite a fix. However, upstream v1.5.10 now includes a
functionally-equivalent fix and so we can heal that fork by upgrading.
We'd also neglected to update the Module Sources docs when upgrading to
go-getter v1.5.9 originally and so we were missing documentation about the
new "depth" argument to enable shadow cloning, which I've added
retroactively here along with documenting its restriction of only
supporting named refs.
This new go-getter release also introduces a new credentials-passing
method for the Google Cloud Storage getter, and so we must incorporate
that into the Terraform-level documentation about module sources.
This is a pragmatic temporary solution to allow us to more quickly resolve
an upstream regression in go-getter locally within Terraform, so that the
work to upstream it for other callers can happen asynchronously and with
less time pressure.
This commit doesn't yet include any changes to address the bug, and
instead aims to be functionally equivalent to getter.GitGetter. A
subsequent commit will then address the regression, so that the diff of
that commit will be easier to apply later to the upstream to get the same
effect there.
cty: The documented definition and comparison logic of cty.Number is now
refined to acknowledge that its true range is limited only to values
that have both a binary floating point and decimal representation,
because cty values are primarily designed to traverse JSON serialization
where numbers are always defined as decimal strings.
In particular, that means that two cty.Number values now always compare
as equal if their representation in JSON (under cty's own JSON encoder)
would be equal, even though the decimal approximation we use for that
conversion is slightly lossy. This pragmatic compromise avoids confusing
situations where a round-trip through JSON serialization (or other
serializations that use the same number format) may produce a value that
doesn't compare equal to the original.
This new definition of equals should not cause any significant behavior
change for any integer in our in-memory storage range, but may cause
some fractional values to compare equal where they didn't before if they
differ only by a small fraction.
* determining source or destination to cloud
* handling single to single state migrations to cloud,
using a name strategy or a tags strategy
* Add end-to-end tests for state migration.
These changes include additions to fulfill the interface for the client
mock, plus moving all that logic (which needn't be duplicated across
both the remote and cloud packages) over to the cloud package under a
dedicated mock client file.
We don't use this library anywhere else in Terraform, and this backend was
using it only for trivial helpers that are easy to express inline anyway.
The new direct code is also type-checkable, whereas these helper functions
seem to be written using reflection.
This gives us one fewer dependency to worry about and makes the test code
for this backend follow a similar assertions style as the rest of this
codebase.
This also transitively upgrades golang.org/x/sys and golang.org/x/net,
but there do not seem to be any significant changes compared to the
commits we were previously using.
CVE-2020-26160 is a high-severity advisory reported against this module.
The dgrijalva package is no longer maintained but our legacy etcv2 backend
depends on it indirectly, via go.etcd.io/etcd/client. The golang-jwt
package is the blessed successor of the original, and has a v3 line which
is compatible with the v3 line of dgrijalva, and so through this replace
we can get a fix for the advisory without other significant behavior
change.
We've preserved the etcdv2 backend as-is on a best-effort basis in order
to support anyone who is already using it, but recommend that users switch
to etcdv3 or to some other backend for ongoing use.
We also have future plans to make state storage be a matter for provider
plugins rather than built in to Terraform CLI, at which point this backend
will either become obsolete or be factored out into its own plugin, at
which point we can remove this "replace" directive and the associated
dependency altogether.
For now, only check the JSON views package, since this was the instance
that most recently tripped us up. There are a few dozen failures
elsewhere in Terraform which would need to be addressed before expanding
this to other packages.
We have a few different .proto files in this repository that all need to
get recompiled into .pb.go files each time we change them, but we were
previously handling that with some scripts that just assumed that protoc
and the relevant plugins were already installed on the system somewhere,
at the right versions.
In practice we've been constantly flopping between different versions of
these tools due to folks having different versions installed in their
development environments. In particular, the state of the .pb.go files
in the prior commit wasn't reproducible by any single version of the tools
because they've all slightly diverged from one another.
In the interests of being more consistent here and avoiding accidental
inconsistencies, we'll now centralize the protocol buffer compile steps
all into a single tool that knows how to fetch and install the expected
versions of the various tools we need and then run those tools with the
right options to get a stable result.
If we want to upgrade to either a newer protoc or a newer protoc-gen-go
in future then we'll do that in a central location and update all of the
.pb.go files at the same time, so that we're always consistently tracking
the same version of protocol buffers everywhere.
While doing this I attempted to keep as close as possible to the toolchain
we'd most recently used, but since they were not consistent with each
other they've now all changed which version numbers they record at minimum,
and the planproto stub in particular now also has a slightly different
descriptor serialization but is otherwise offering the same API.
TencentCloud deleted all their v3 tags, so any system that does not use
the global Go proxy will fail to find the
github.com/tencentcloud/tencentcloud-sdk-go module source.
Update go.mod to point to the specific commit rather than the
incompatible and missing version tag.
Go 1.17 has a new treatment of go.mod where it tracks more indirect
dependencies in return for needing to fetch and load fewer transitive
go.mod files.
This commit opts in to that new treatment and adds all of the additional
indirect dependencies which contain packages that Terraform directly or
indirectly makes use of.
There are more details on what's going on here in the "Lazy Module Loading"
design document:
http://golang.org/design/36460-lazy-module-loading
(the design document was written assuming this would land in Go 1.15, but
it actually landed in Go 1.17 and so it's "go 1.17" that selects the new
behavior in spite of the examples in that document.)
etcd rewrote its import path from coreos/etcd to go.etcd.io/etcd.
Changed the imports path in this commit, which also updates the code
version.
This lets us remove the github.com/ugorji/go/codec dependency, which
was pinned to a fairly old version. The net change is a loss of 30,000
lines of code in the vendor directory. (I first noticed this problem
because the outdated go/codec dependency was causing a dependency
failure when I tried to put Terraform and another project in the same
vendor directory.)
Note the version shows up funkily in go.mod, but I verified
visually it's the same commit as the "release-3.4" tag in
github.com/coreos/etcd. The etcd team plans to fix the release version
tagging in v3.5, which should be released soon.
Some users would want to use Consul namespaces when using the Consul
backend but the version of the Consul API client we use is too old and
don't support them. In preparation for this change this patch just update
it the client and replace testutil.NewTestServerConfig() by
testutil.NewTestServerConfigT() in the tests.
This includes the improvements to various collection-related functions to
make them handle marks more precisely. For Terraform in particular that
translates into handling sensitivity more precisely, so that non-sensitive
collections that happen to contain sensitive elements won't get simplified
into wholly-sensitive collections when using these functions.
Although we don't typically do configuration-level string wrangling
directly in Terraform, we delegate to several other upstream libraries
that do. These upgrades all switch to newer versions that support the
latest definitions from Unicode 13, primarily affecting operations such
as converting strings to upper/lowercase or splitting strings into
component characters (substr, reverse, etc).
The tests for the upstream libraries didn't show any regressions from
these updates, so the Unicode 13 changes seem to be backward-compatible
additions rather than significant breaking changes.
(Our go.mod file had also become non-canonical in some ways, and the Go
toolchain fixed that as part of this work, causing a few extra style-only
diffs here that shouldn't cause any change in behavior.)
This is just a prototype to gather some feedback in our ongoing research
on integration testing of Terraform modules. The hope is that by having a
command integrated into Terraform itself it'll be easier for interested
module authors to give it a try, and also easier for us to iterate quickly
based on feedback without having to coordinate across multiple codebases.
Everything about this is subject to change even in future patch releases.
Since it's a CLI command rather than a configuration language feature it's
not using the language experiments mechanism, but generates a warning
similar to the one language experiments generate in order to be clear that
backward compatibility is not guaranteed.
As part of ongoing research into Terraform testing we'd like to use an
experimental feature to validate our current understanding that expressing
tests as part of the Terraform language, as opposed to in some other
language run alongside, is a good and viable way to write practical
module integration tests.
This initial experimental incarnation of that idea is implemented as a
provider, just because that's an easier extension point for research
purposes than a first-class language feature would be. Whether this would
ultimately emerge as a provider similar to this or as custom language
constructs will be a matter for future research, if this first
experiment confirms that tests written in the Terraform language are the
best direction to take.
The previous incarnation of this experiment was an externally-developed
provider apparentlymart/testing, listed on the Terraform Registry. That
helped with showing that there are some useful tests that we can write
in the Terraform language, but integrating such a provider into Terraform
will allow us to make use of it in the also-experimental "terraform test"
command, which will follow in subsequent commits, to see how this might
fit into a development workflow.
Changes:
```
* backend/s3: Support for AWS Single-Sign On (SSO) cached credentials
```
Updated via:
```
go get github.com/aws/aws-sdk-go@v1.37.0
go mod tidy
```
Please note that Terraform CLI will not initiate or perform the AWS SSO login flow. It is expected that you have already performed the SSO login flow using AWS CLI using the `aws sso login` command, or by some other mechanism before executing Terraform. More precisely, this credential handling must find a valid non-expired access token for the AWS SSO user portal URL in `~/.aws/sso/cache`. If a cached token is not found, is expired, or the file is malformed an error will be returned.
You can use configure AWS SSO credentials from the AWS shared configuration file by specifying the required keys in the profile:
```
sso_account_id
sso_region
sso_role_name
sso_start_url
```
For example, the following defines a profile "devsso" and specifies the AWS SSO parameters that defines the target account, role, sign-on portal, and the region where the user portal is located. Note: all SSO arguments must be provided, or an error will be returned.
```
[profile devsso]
sso_start_url = https:my-sso-portal.awsapps.com/start
sso_role_name = SSOReadOnlyRole
sso_region = us-east-1
sso_account_id = 123456789012
```
Additional Resources
* [Configuring the AWS CLI to use AWS Single Sign-On](https:docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html)
* [AWS Single Sign-On User Guide](https:docs.aws.amazon.com/singlesignon/latest/userguide/what-is.html)
Here we propagate in the initialized terminal.Streams from package main,
and then onwards to backends running in CLI mode.
This also replaces our use of helper/wrappedstreams to determine whether
stdin is a terminal or a pipe. helper/wrappedstreams returns incorrect
file descriptors on Windows, causing StdinPiped to always return false on
that platform and thus causing one of the odd behaviors discussed in
Finally, this includes some wrappers around the ability to look up the
number of columns in the terminal in preparation for use elsewhere. These
wrappers deal with the fact that our unit tests typically won't populate
meta.Streams.
This is a helper package that creates a very thin abstraction over
terminal setup, with the main goal being to deal with all of the extra
setup we need to do in order to get a UTF-8-supporting virtual terminal
on a Windows system.
The upstream bug with opening a browser on Windows Subsystem for Linux
has been fixed, so this reverts our local patch for this. The approach
upstream adds fallback support for x-www-browser and www-browser if
xdg-open fails, and this fixes the problem on WSL.
This reverts commit 12e090ce48.
This is needed to make it possible to use the scram-sha-256
authentication method for the pg backend. It's not easy to write
unit-tests for this since it requires a specific configuration of the
PostgreSQL server, I did test it manually thought and everything seems
to work like it should.
Closes https://github.com/hashicorp/terraform/issues/24016
* github.com/Azure/azure-sdk-for-go to v47.1.0
* github.com/Azure/go-autorest to v0.11.10
* github.com/hashicorp/go-azure-helpers to v0.13.0
* github.com/tombuildsstuff/giovanni to v0.14.0
Added an Off level to hclog, so we can individually disable logging at
various levels.
Added IndependentLevels so that sublogger levels are not linked to their
parents.
The main process is now handling what output to print, so it doesn't do
any good to try and run it through prefixedio, which is only adding
extra coordination to echo the same data.
helper/copy CopyDir was used heavily in tests. It differes from
internal/copydir in a few ways, the main one being that it creates the
dst directory while the internal version expected the dst to exist
(there are other differences, which is why I did not just switch tests
to using internal's CopyDir).
I moved the CopyDir func from helper/copy into command_test.go; I could
also have moved it into internal/copy and named it something like
CreateDirAndCopy so if that seems like a better option please let me
know.
helper/copy/CopyFile was used in a couple of spots so I moved it into
internal, at which point I thought it made more sense to rename the
package copy (instead of copydir).
There's also a `go mod tidy` included.
Update tests to match the fix in mitchellh/cli#71, which aligns MockUi
with BasicUi and allows newlines in user input.
We are not using the new ErrorWriter, added in mitchellh/cli#81, as it
does not appear to interact correctly with panicwrap. All error output
from CLI parsing will continue to appear on stdout, not stderr.
We've not been using HIL in the main codepaths since Terraform 0.12, but
some references to it (and some supporting functionality in Terraform)
stuck around due to interactions with types we'd kept around to support
legacy shims.
However, removing the configs.RawConfig field from
terraform.ResourceConfig disconnects that subtree of dependencies from
everything else, allowing us to remove it. This is safe because the only
remaining uses of terraform.ResourceConfig are shims from values that
were already evaluated using the HCL 2 API, and thus they never need
the "just in time" HIL evaluation that ResourceConfig.interpolateForce
used to do.
We also had some HIL references in configs/hcl2shim that were previously
in support of the "terraform 0.12upgrade" command, but the implementation
of that command is now removed.
There was one remaining reference to HIL in a now-unused function in the
helper/schema package, which I removed entirely here.
This then allows us to remove the HIL dependency entirely, and also to
clean up some remaining old remants of the legacy "config" package that
we'd recently moved into the "configs" package pending further pruning.
* updating `github.com/Azure/azure-sdk-for-go` to `v45.0.0`
* updating `github.com/Azure/go-autorest` to `v0.11.3`
* updating `github.com/hashicorp/go-azure-helpers` to `v0.12.0`
* updating `github.com/tombuildsstuff/giovanni` to `v0.12.0`
