* command: keep our promises
* remove some nil config checks
Remove some of the safety checks that ensure plan nodes have config attached at the appropriate time.
* add GeneratedConfig to plan changes objects
Add a new GeneratedConfig field alongside Importing in plan changes.
* add config generation package
The genconfig package implements HCL config generation from provider state values.
Thanks to @mildwonkey whose implementation of terraform add is the basis for this package.
* generate config during plan
If a resource is being imported and does not already have config, attempt to generate that config during planning. The config is generated from the state as an HCL string, and then parsed back into an hcl.Body to attach to the plan graph node.
The generated config string is attached to the change emitted by the plan.
* complete config generation prototype, and add tests
* Plannable import: Add generated config to json and human-readable plan output
---------
Co-authored-by: Katy Moe <katy@katy.moe>
* Add support for scoped resources
* refactor existing checks addrs and add check block addr
* Add configuration for check blocks
* introduce check blocks into the terraform node and transform graph
* address comments
* address comments
* don't execute checks during destroy operations
* don't even include check nodes for destroy operations
* Use the new structured renderer in place of the old diffs package
* remove old plan tests
* refresh only plans should show moved resources in the refresh section
* remove attributes that do not match the relevant attributes filter
* fix formatting
* fix renderer function, don't drop irrelevant attributes just mark them as no-ops
* fix imports
* raw unmodified broken tests
* tests execute, no panics
* fix whitespace differences
* fix all the tests
* fix tests
* actually fix tests
* add missing plan metadata into the renderer
* address comments
* complete merge
* remove TODO raising questions about outputs, they are fixed
* missing bold on plan
Add a new ChangeReason, ReasonDeleteBecauseNoMoveTarget, to provide better
information in cases where a planned deletion is due to moving a resource to
a target not in configuration.
Consider a case in which a resource instance exists in state at address A, and
the user adds a moved block to move A to address B. Whether by the user's
intention or not, address B does not exist in configuration.
Terraform combines the move from A to B, and the lack of configuration for B,
into a single delete action for the (previously nonexistent) entity B.
Prior to this commit, the Terraform plan will report that resource B will be
destroyed because it does not exist in configuration, without explicitly
connecting this to the move.
This commit provides the user an additional clue as to what has happened, in a
case in which Terraform has elided a user's action and inaction into one
potentially destructive change.
Previously we were attempting to infer the checkable object address kind
of a given address by whether it included "output" in the position where
a resource type name would otherwise go.
That was already potentially risky because we've historically not
prevented a resource type named "output", and it's also a
forward-compatibility hazard in case we introduce additional object kinds
with entirely-new addressing schemes in future.
Given that, we'll instead always be explicit about what kind of address
we're storing in a wire or file format, so that we can make sure to always
use the intended parser when reading an address back into memory, or
return an error if we encounter a kind we're not familiar with.
This is a new-shaped representation of check results which follows the
two-tiered structure of static objects and dynamic instances of objects,
thereby allowing consumers to see which checkable objects exist in the
configuration even if a dynamic evaluation error prevented actually
expanding them all to determine their declared instances.
Eventually we'll include this in the state too, but this initially adds it
only to the plan in order to replace the now-deprecated experimental
conditions result that was present but undocumented in Terraform v1.2.
A significant goal of the design changes around checks in earlier commits
(with the introduction of package "checks") was to allow us to
differentiate between a configuration object that we didn't expand at all
due to an upstream error, which has _unknown_ check status, and a
configuration object that expanded to zero dynamic objects, which
therefore has a _passing_ check status.
However, our initial lowering of checks.State into states.CheckResults
stayed with the older model of just recording each leaf check in isolation,
without any tracking of the containers.
This commit therefore lightly reworks our representation of check results
in the state and plan with two main goals:
- The results are grouped by the static configuration object they came
from, and we capture an aggregate status for each of those so that
we can differentiate an unknown aggregate result from a passing
aggregate result which has zero dynamic associated objects.
- The granularity of results is whole checkable objects rather than
individual checks, because checkable objects have durable addresses
between runs, but individual checks for an object are more of a
syntactic convenience to make it easier for module authors to declare
many independent conditions that each have their own error messages.
Since v1.2 exposed some details of our checks model into the JSON plan
output there are some unanswered questions here about how we can shift to
reporting in the two-level heirarchy described above. For now I've
preserved structural compatibility but not semantic compatibility: any
parser that was written against that format should still function but will
now see fewer results. We'll revisit this in a later commit and consider
other structures and what to do about our compatibility constraint on the
v1.2 structure.
Otherwise though, this is an internal-only change which preserves all of
the existing main behaviors of conditions as before, and just gets us
ready to build user-facing features in terms of this new structure.
The "checks" package is an expansion what we previously called
plans.Conditions to accommodate a new requirement that we be able to track
which checks we're expecting to run even if we don't actually get around
to running them, which will be helpful when we start using checks as part
of our module testing story because test reporting tools appreciate there
being a relatively consistent set of test cases from one run to the next.
So far this should be essentially a no-op change from an external
functionality standpoint, aside from some minor adjustments to how we
report some of the error and warning cases from condition evaluation in
light of the fact that the "checks" package can now track errors as a
different outcome than a failure of a valid check.
As is often the case with anything which changes what we track
in the EvalContext and persist between plan and apply, Terraform Core is
pretty brittle and so this had knock-on effects elsewhere too. Again, the
goal is for these changes to not create any material externally-visible
difference, and just to accommodate the new assumption that there will
always be a "checks" object available for tracking during a graph walk.
Go 1.19's "fmt" has some awareness of the new doc comment formatting
conventions and adjusts the presentation of the source comments to make
it clearer how godoc would interpret them. Therefore this commit includes
various updates made by "go fmt" to acheve that.
In line with our usual convention that we make stylistic/grammar/spelling
tweaks typically only when we're "in the area" changing something else
anyway, I also took this opportunity to review most of the comments that
this updated to see if there were any other opportunities to improve them.
The JSON output for sequences previously omitted unknown values for
tuples and sets, which made it impossible to interpret the corresponding
unknown marks. For example, consider this resource:
resource "example_resource" "example" {
tags = toset(["alpha", timestamp(), "charlie"])
}
This would previously be encoded in JSON as:
"after": {
"tags": ["alpha", "charlie"]
},
"after_unknown": {
"id": true,
"tags": [false, true, false]
},
That is, the timestamp value would be omitted from the output
altogether, while the corresponding unknown marks would include a value
for each of the set members.
This commit changes the behaviour to:
"after": {
"tags": ["alpha", null, "charlie"]
},
"after_unknown": {
"id": true,
"tags": [false, true, false]
},
This aligns tuples and sets with the prior behaviour for lists, and
makes it clear which elements are known and which are unknown.
Planned output changes are represented in the JSON output format using
the same change object as planned resource changes. This structure
includes an `after` value and a parallel `after_unknown` value, which
can be combined to determine which specific parts of a value are known
only at apply time.
Previously, structured output values would be marked in the JSON plan as
coarsely known or unknown, even if only some subset of the structure
will be known only at apply time. This simplification was unnecessary,
and this commit reuses the same logic for resource changes to give more
information to consumers of this format.
For example, consider this output:
output "bar" {
value = tolist([
"hello",
timestamp(),
"world",
])
}
The plan output for this output would be:
+ bar = [
+ "hello",
+ (known after apply),
+ "world",
]
For the same plan, the JSON output was previously:
"bar": {
"actions": [
"create"
],
"before": null,
"after_unknown": true,
"before_sensitive": false,
"after_sensitive": false
}
After this commit, the output is instead:
"bar": {
"actions": [
"create"
],
"before": null,
"after": [
"hello",
null,
"world"
],
"after_unknown": [
false,
true,
false
],
"before_sensitive": false,
"after_sensitive": false
}
We have two different reasons why a data resource might be read only
during apply, rather than during planning as usual: the configuration
contains unknown values, or the data resource as a whole depends on a
managed resource which itself has a change pending.
However, we didn't previously distinguish these two in a way that allowed
the UI to describe the difference, and so we confusingly reported both
as "config refers to values not yet known", which in turn led to a number
of reasonable questions about why Terraform was claiming that but then
immediately below showing the configuration entirely known.
Now we'll use our existing "ActionReason" mechanism to tell the UI layer
which of the two reasons applies to a particular data resource instance.
The "dependency pending" situation tends to happen in conjunction with
"config unknown", so we'll prefer to refer that the configuration is
unknown if both are true.
Previously the supported JSON plan and state formats included only
serialized output values, which was a lossy serialization of the
Terraform type system. This commit adds a type field in the usual cty
JSON format, which allows reconstitution of the original value.
For example, previously a list(string) and a set(string) containing the
same values were indistinguishable. This change serializes these as
follows:
{
"value": ["a","b","c"],
"type": ["list","string"]
}
and:
{
"value": ["a","b","c"],
"type": ["set","string"]
}
In order to include condition block results in the JSON plan output, we
must store them in the plan and its serialization.
Terraform can evaluate condition blocks multiple times, so we must be
able to update the result. Accordingly, the plan.Conditions object is a
map with keys representing the condition block's address. Condition
blocks are not referenceable in any other context, so this address form
cannot be used anywhere in the configuration.
The commit includes a new test case for the JSON output of a
refresh-only plan, which is currently the only way for a failing
condition result to be rendered through this path.
Add the resource instances and individual attributes which may have
contributed to the planned changes to the json format of the plan. We
use the existing path encoding for individual attributes, which is
already used in the replace_paths change field.
When calculating the unknown values for JSON plan output, we would
previously recursively call the `unknownAsBool` function on the current
sub-tree twice, if any values were unknown. This was wasteful, but not
noticeable for normal Terraform resource shapes.
However for deeper nested object values, such as Kubernetes manifests,
this was a severe performance problem, causing `terraform show -json` to
take several hours to render a plan.
This commit reuses the already calculated unknown value for the subtree,
and adds benchmark coverage to demonstrate the improvement.
The JSON plan configuration data now includes a `full_name` field for
providers. This addition warrants a backwards compatible increment to
the version number.
Previously we had three different layers all thinking they were
responsible for substituting a default value for an unset root module
variable:
- the local backend, via logic in backend.ParseVariableValues
- the context.Plan function (and other similar functions) trying to
preprocess the input variables using
terraform.mergeDefaultInputVariableValues .
- the newer prepareFinalInputVariableValue, which aims to centralize all
of the variable preparation logic so it can be common to both root and
child module variables.
The second of these was also trying to handle type constraint checking,
which is also the responsibility of the central function and not something
we need to handle so early.
Only the last of these consistently handles both root and child module
variables, and so is the one we ought to keep. The others are now
redundant and are causing prepareFinalInputVariableValue to get a slightly
corrupted view of the caller's chosen variable values.
To rectify that, here we remove the two redundant layers altogether and
have unset root variables pass through as cty.NilVal all the way to the
central prepareFinalInputVariableValue function, which will then handle
them in a suitable way which properly respects the "nullable" setting.
This commit includes some test changes in the terraform package to make
those tests no longer rely on the mergeDefaultInputVariableValues logic
we've removed, and to instead explicitly set cty.NilVal for all unset
variables to comply with our intended contract for PlanOpts.SetVariables,
and similar. (This is so that we can more easily catch bugs in callers
where they _don't_ correctly handle input variables; it allows us to
distinguish between the caller explicitly marking a variable as unset vs.
not describing it at all, where the latter is a bug in the caller.)
There are a few different reasons why a resource instance tracked in the
prior state might be considered an "orphan", but previously we reported
them all identically in the planned changes.
In order to help users understand the reason for a surprising planned
delete, we'll now try to specify an additional reason for the planned
deletion, covering all of the main reasons why that could happen.
This commit only introduces the new detail to the plans.Changes result,
though it also incidentally exposes it as part of the JSON plan result
in order to keep that working without returning errors in these new
cases. We'll expose this information in the human-oriented UI output in
a subsequent commit.
Configuration-driven moves are represented in the plan file by setting
the resource's `PrevRunAddr` to a different value than its `Addr`. For
JSON plan output, we here add a new field to resource changes,
`previous_address`, which is present and non-empty only if the resource
is planned to be moved.
Like the CLI UI, refresh-only plans will include move-only changes in
the resource drift JSON output. In normal plan mode, these are elided to
avoid redundancy with planned changes.
Previously, if any resources were found to have drifted, the JSON plan
output would include a drift entry for every resource in state. This
commit aligns the JSON plan output with the CLI UI, and only includes
those resources where the old value does not equal the new value---i.e.
drift has been detected.
Also fixes a bug where the "address" field was missing from the drift
output, and adds some test coverage.
* jsonplan and jsonstate: include sensitive_values in state representations
A sensitive_values field has been added to the resource in state and planned values which is a map of all sensitive attributes with the values set to true.
It wasn't entirely clear to me if the values in state would suffice, or if we also need to consult the schema - I believe that this is sufficient for state files written since v0.15, and if that's incorrect or insufficient, I'll add in the provider schema check as well.
I also updated the documentation, and, since we've considered this before, bumped the FormatVersions for both jsonstate and jsonplan.
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
