* Add golden JSON test for Terraform plan
* Add data source to golden JSON plan
* Move output comparison code into shared helper function
* Add note for maintainer to contact TFC when UI changes
UI changes may potentially impact the behavior of structured run output
on TFC.
* Add test_data_source to other mock providers
* refactor: Use tfaddr for provider address parsing
* refactor: Use tfaddr for module address parsing
* deps: introduce hashicorp/terraform-registry-address
There are no good options for inserting diagnostics into the backend
lookup, or creating a backend which reports it's removal because none of
the init or GetSchema functions return any errors.
Keep a registry of the removed backend so that we can at least notify
users that a backend was removed vs an invalid name.
By observing the sorts of questions people ask in the community, and the
ways they ask them, we've inferred that various different people have been
confused by Terraform reporting that a value won't be known until apply
or that a value is sensitive as part of an error message when that message
doesn't actually relate to the known-ness and sensitivity of any value.
Quite reasonably, someone who sees Terraform discussing an unfamiliar
concept like unknown values can assume that it must be somehow relevant to
the problem being discussed, and so in that sense Terraform's current
error messages are giving "too much information": information that isn't
actually helpful in understanding the problem being described, and in the
worst case is a distraction from understanding the problem being described.
With that in mind then, here we introduce an explicit annotation on
diagnostic objects that are directly talking about unknown values or
sensitive values, and then the diagnostic renderer will react to that to
avoid using the terminology "known only after apply" or "sensitive" in the
generated diagnostic annotations unless we're rendering a message that is
explicitly related to one of those topics.
This ends up being a bit of a cross-cutting concern because the code that
generates these diagnostics and the code that renders them are in separate
packages and are not directly aware of each other. With that in mind, the
logic for actually deciding for a particular diagnostic whether it's
flagged in one of these special ways lives inside the tfdiags package as
an intermediation point, which both the diagnostic generator (in the core
package) and the diagnostic renderer can both depend on.
When an error occurs in a function call, the error message text often
includes references to particular parameters in the function signature.
This commit improves that reporting by also including a summary of the
full function signature as part of the diagnostic context in that case,
so a reader can see which parameter is which given that function
arguments are always assigned positionally and so the parameter names
do not appear in the caller's source code.
We originally introduced the idea of language experiments as a way to get
early feedback on not-yet-proven feature ideas, ideally as part of the
initial exploration of the solution space rather than only after a
solution has become relatively clear.
Unfortunately, our tradeoff of making them available in normal releases
behind an explicit opt-in in order to make it easier to participate in the
feedback process had the unintended side-effect of making it feel okay
to use experiments in production and endure the warnings they generate.
This in turn has made us reluctant to make use of the experiments feature
lest experiments become de-facto production features which we then feel
compelled to preserve even though we aren't yet ready to graduate them
to stable features.
In an attempt to tweak that compromise, here we make the availability of
experiments _at all_ a build-time flag which will not be set by default,
and therefore experiments will not be available in most release builds.
The intent (not yet implemented in this PR) is for our release process to
set this flag only when it knows it's building an alpha release or a
development snapshot not destined for release at all, which will therefore
allow us to still use the alpha releases as a vehicle for giving feedback
participants access to a feature (without needing to install a Go
toolchain) but will not encourage pretending that these features are
production-ready before they graduate from experimental.
Only language experiments have an explicit framework for dealing with them
which outlives any particular experiment, so most of the changes here are
to that generalized mechanism. However, the intent is that non-language
experiments, such as experimental CLI commands, would also in future
check Meta.AllowExperimentalFeatures and gate the use of those experiments
too, so that we can be consistent that experimental features will never
be available unless you explicitly choose to use an alpha release or
a custom build from source code.
Since there are already some experiments active at the time of this commit
which were not previously subject to this restriction, we'll pragmatically
leave those as exceptions that will remain generally available for now,
and so this new approach will apply only to new experiments started in the
future. Once those experiments have all concluded, we will be left with
no more exceptions unless we explicitly choose to make an exception for
some reason we've not imagined yet.
It's important that we be able to write tests that rely on experiments
either being available or not being available, so here we're using our
typical approach of making "package main" deal with the global setting
that applies to Terraform CLI executables while making the layers below
all support fine-grain selection of this behavior so that tests with
different needs can run concurrently without trampling on one another.
As a compromise, the integration tests in the terraform package will
run with experiments enabled _by default_ since we commonly need to
exercise experiments in those tests, but they can selectively opt-out
if they need to by overriding the loader setting back to false again.
The JSON output for sequences previously omitted unknown values for
tuples and sets, which made it impossible to interpret the corresponding
unknown marks. For example, consider this resource:
resource "example_resource" "example" {
tags = toset(["alpha", timestamp(), "charlie"])
}
This would previously be encoded in JSON as:
"after": {
"tags": ["alpha", "charlie"]
},
"after_unknown": {
"id": true,
"tags": [false, true, false]
},
That is, the timestamp value would be omitted from the output
altogether, while the corresponding unknown marks would include a value
for each of the set members.
This commit changes the behaviour to:
"after": {
"tags": ["alpha", null, "charlie"]
},
"after_unknown": {
"id": true,
"tags": [false, true, false]
},
This aligns tuples and sets with the prior behaviour for lists, and
makes it clear which elements are known and which are unknown.
Planned output changes are represented in the JSON output format using
the same change object as planned resource changes. This structure
includes an `after` value and a parallel `after_unknown` value, which
can be combined to determine which specific parts of a value are known
only at apply time.
Previously, structured output values would be marked in the JSON plan as
coarsely known or unknown, even if only some subset of the structure
will be known only at apply time. This simplification was unnecessary,
and this commit reuses the same logic for resource changes to give more
information to consumers of this format.
For example, consider this output:
output "bar" {
value = tolist([
"hello",
timestamp(),
"world",
])
}
The plan output for this output would be:
+ bar = [
+ "hello",
+ (known after apply),
+ "world",
]
For the same plan, the JSON output was previously:
"bar": {
"actions": [
"create"
],
"before": null,
"after_unknown": true,
"before_sensitive": false,
"after_sensitive": false
}
After this commit, the output is instead:
"bar": {
"actions": [
"create"
],
"before": null,
"after": [
"hello",
null,
"world"
],
"after_unknown": [
false,
true,
false
],
"before_sensitive": false,
"after_sensitive": false
}
This also sets an additional variable if it detects that this is an alpha
or development build, which currently does nothing but might eventually
turn on the ability to use experimental features, if we make that
something available only in prereleases.
We have two different reasons why a data resource might be read only
during apply, rather than during planning as usual: the configuration
contains unknown values, or the data resource as a whole depends on a
managed resource which itself has a change pending.
However, we didn't previously distinguish these two in a way that allowed
the UI to describe the difference, and so we confusingly reported both
as "config refers to values not yet known", which in turn led to a number
of reasonable questions about why Terraform was claiming that but then
immediately below showing the configuration entirely known.
Now we'll use our existing "ActionReason" mechanism to tell the UI layer
which of the two reasons applies to a particular data resource instance.
The "dependency pending" situation tends to happen in conjunction with
"config unknown", so we'll prefer to refer that the configuration is
unknown if both are true.
When specifying variable values on the command line, name-value pairs
must be joined with an equals sign, without surrounding spaces.
Previously Terraform would interpret "foo = bar" as assigning the value
" bar" to the variable named "foo ". This is never valid, as variable
names may not include whitespace.
This commit looks for this specific error and returns a diagnostic with
a suggestion for correcting it. We cannot simply trim whitespace,
because it is valid to write "foo= bar" to assign the value " bar" to
the variable "foo", as unlikely as it seems.
Previously the supported JSON plan and state formats included only
serialized output values, which was a lossy serialization of the
Terraform type system. This commit adds a type field in the usual cty
JSON format, which allows reconstitution of the original value.
For example, previously a list(string) and a set(string) containing the
same values were indistinguishable. This change serializes these as
follows:
{
"value": ["a","b","c"],
"type": ["list","string"]
}
and:
{
"value": ["a","b","c"],
"type": ["set","string"]
}
Hyphen characters are allowed in environment variable names, but are not valid POSIX variable names. Usually, it's still possible to set variable names with hyphens using utilities like env or docker. But, as a fallback, host names may encode their hyphens as double underscores in the variable name. For the example "café.fr", the variable name "TF_TOKEN_xn____caf__dma_fr" or "TF_TOKEN_xn--caf-dma_fr"
may be used.
Introduces a new method of configuring token service credentials using a host-specific environment variable. This configuration was previously possible using the [terraform-credentials-env](https://github.com/apparentlymart/terraform-credentials-env) credentials helper.
This new method is now consulted first, as it is seen to be the most proximate source of credentials before CLI configuration while still falling back to the credentials helper.
A previous change added missing quoting around object keys which do not
parse as barewords. At the same time we introduced a bug where map keys
could be double-quoted, due to calling the `displayAttributeName` helper
function (to quote non-bareword keys) then using the `writeValue` method
(which quotes all strings).
This commit fixes this and adds test coverage for map keys which require
quoting.
When rendering diffs for resources which use nested attribute types, we
must cope with collections backing those attributes which are entirely
sensitive. The most common way this will be seen is through sensitive
values being present in sets, which will result in the entire set being
marked sensitive.
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but `t.TempDir` handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
In order to include condition block results in the JSON plan output, we
must store them in the plan and its serialization.
Terraform can evaluate condition blocks multiple times, so we must be
able to update the result. Accordingly, the plan.Conditions object is a
map with keys representing the condition block's address. Condition
blocks are not referenceable in any other context, so this address form
cannot be used anywhere in the configuration.
The commit includes a new test case for the JSON output of a
refresh-only plan, which is currently the only way for a failing
condition result to be rendered through this path.
When rendering a diff, we should quote object attribute names if the
string representation is not a valid identifier. While this is not
strictly necessary, it makes the diff output more closely resemble the
configuration language, which is less confusing.
This commit applies to both top-level schema attributes and any object
value attributes. We use a simplistic "%q" Go format string to quote the
strings, which is not strictly identical to HCL's quoting requirements,
but is the pattern used elsewhere in HCL and Terraform.
Co-Authored-By: Katy Moe <katy@katy.moe>
Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
The initial rough implementation contained a bug where it would
incorrectly return a NilVal in some cases.
Improve the heuristics here to insert null values more precisely when
parent objects change to or from null. We also check for dynamic types
changing, in which case the entire object must be taken when we can't
match the individual attribute values.
Add the resource instances and individual attributes which may have
contributed to the planned changes to the json format of the plan. We
use the existing path encoding for individual attributes, which is
already used in the replace_paths change field.
