When working with a ConfigResource, the generalization of a
ModuleInstance to a Module was inadvertently dropped, and there was to
test coverage for that type of target.
Ensure we can target a specific module instance alone.
Before expansion happens, we only have expansion resource nodes that
know their ConfigResource address. In order to properly compare these to
targets within a module instance, we need to generalize the target to
also be a ConfigResource.
We can also remove the IgnoreIndices field from the transformer, since
we have addresses that are properly scoped and can compare them in the
correct context.
While removal of attributes can be handled by providers through the
UpgradeResourceState call, data sources may need to be evaluated before
reading, and they have no upgrade path in the provider protocol.
Strip out extra attributes during state decoding when they are no longer
present in the schema, and there is no schema upgrade pending.
When looking up a resource during plan, we need to return an empty
container type when we're certain there are going to be no instances.
It's now more common to reference resources in a context that needs to
be known during plan (e.g. for_each), and always returning a DynamicVal
her would block plan from succeeding.
This is the known case broken by the changes to allow resources pending
destruction to be evaluated from state. When a resource references
another that is create_before_destroy, and that resource is being scaled
in, the first resource will not be updated correctly.
Since we have to allow destroy nodes to be evaluated for providers
during a full destroy, this is adding a transformer to connect temporary
values to any destroy versions of their references when possible. The
ensures that the destroy happens before evaluation, even when there
isn't a full create-then-destroy set of instances.
The cases where the connection can't be made are when the temporary
value has a provider descendant, which means it must evaluate early in
the case of a full destroy. This means the value may contain incorrect
data when referencing resource that are create_before_destroy, or being
scaled-in via count or for_each. That will need to be addressed later by
reevaluating how we handle the full destroy case in terraform.
During a full destroy, providers may reference resources that are going
to be destroyed as well. We currently cannot change this behavior, so we
need to allow the evaluation and try to prevent it from leaking into as
many other places as possible. Another transformer to try and protect
the values in locals, variables and outputs will be added to enforce
destroy ordering when possible.
Outputs and locals cannot refer to destroy nodes. Since those nodes
types do not have different ordering for create and destroy operations,
connecting them directly to destroy nodes can cause cycles.
The destroy graph builder test requires state in order to be correct,
which it didn't have. The other tests hits the edge case where a planned
destroy cannot remove outputs, because the apply phase does not know it
was created from a destroy.
Since data source destruction is only state removal, and other resources
cannot depend on them creating any physical resources, the destroy
dependencies were not tracked in the state. It turns out that there is a
special case which requires this; running terraform destroy where the
provider depends on a data source. In that case the resources using that
provider need to record their indirect dependence on the data source, so
that they can be deleted before the data source is removed from the
state.
Our reference transformer analyses and our destroy transformer analyses
are built around static (not-yet-expanded) addresses so that they can
correctly handle mixtures of expanded and not-yet-expanded objects in the
same graph.
However, this characteristic also makes them unnecessarily conservative
in their handling of references between resources within different
instances of the same module: we know they can never interact with each
other in practice because the dependencies for all instances of a module
are the same and so one instance cannot possibly depend on another.
As a compromise then, here we introduce a new helper function that can
recognize when a proposed edge is between two resource instances that
belong to different instances of the same module, and thus allow us to
skip actually creating those edges even though our imprecise analyses
believe them to be needed.
As well as significantly reducing the number of edges in situations where
multi-instance resources appear inside multi-instance modules, this also
fixes some potential cycles in situations where a single plan includes
both destroying an instance of a module and creating a new instance of the
same module: the dependencies between the objects in the instance being
destroyed and the objects in the instance being created can, if allowed
to connect, cause Terraform to believe that the create and the destroy
both depend on one another even though there is no need for that to be
true in practice.
This involves a very specialized helper function to encode the situation
where this exception applies. This function has an ugly name to reflect
how specialized it is; it's not intended to be of any use outside of these
three situations in particular.
The AbstractResourceInstance type was storing the entire Resource from
the state, when it only needs the actual instance state. This would
cause resources to consume memory on the order of n^2, where n in the
number of instances of the resource.
Rather than attaching the entire resource state, which includes copying
each individual instance, only attach the ResourceInstance state, and
extract out the provider address from the Resource.
The pruneUnusedNodes transformer was skipping root level locals and
variables, causing them to be left in the graph during a full destroy.
Use the return value from temporaryValue to indicate if the node is
truly temporary or not, rather then keeping the entire root module.
If we're adding a node to remove a root output from the state, the
output itself does not need to be re-evaluated. The exception for root
outputs caused them to be missed when we refactored resource destruction
to only use the existing state.
Have the output reference the expansion of a resource (via the whole
resource object), so that we can be sure we don't attempt to evaluate
that expansion during destroy.
When configuring providers, it is normally valid to refer to any value
which is known at apply time. This can include resource instance
attributes, variables, locals, and so on.
The import command has a simpler graph evaluation, which means that
many of these values are unknown. We previously prevented this from
happening by restricting provider configuration references to input
variables (#22862), but this was more restrictive than is necessary.
This commit changes how we verify provider configuration for import.
We no longer inspect the configuration references during graph building,
because this is too early to determine if these values will become known
or not.
Instead, when the provider is configured during evaluation, we
check if the configuration value is wholly known. If not, we fail with a
diagnostic error.
Includes a test case which verifies that providers can now be configured
using locals as well as vars, and an updated test case which verifies
that providers cannot be configured with references to resources.
element types
The error message when evaluateForEachExpression encounted an unknown
value of cty.DynamicPseudoType was not clear:
The given "for_each" argument value is unsuitable: "for_each" supports maps
and sets of strings, but you have provided a set containing type dynamic.
By moving the check for unknowns before the check for set element types,
the following error is returned instead:
"The "for_each" value depends on resource attributes that cannot be
determined until apply (...)"
Orphaned instances that are create_before_destroy will still be in the
state when their references are evaluated. We need to skip instances
that are planned to be destroyed altogether, as they can't be part of an
evaluation.
When the DestroyEdgeTransformer was updated to handle stored
dependencies the addrs.ConfigResource type did not yet exist. The lookup
map keys in the transformer needed to be updated to remove module
indexes.
This is for consistency with other commands which use prompts, all of
which require "yes" rather than "y" to confirm.
We also migrate the login command to use UIInput, which now supports
securely asking for passwords or secrets via the speakeasy library.
This simplifies the initial targeting logic, and removes the complex
algorithm for finding descendants that result in output changes, which
hid bugs that failed with modules.
The targeting is handled in 2 phases. First we find all individual
resource nodes that are targeted, then add all their dependencies to the
set of targets. This in essence is all we need for targeting, and is
straightforward to understand.
The next phase is to add any root module outputs that can be solely
derived from the set of targeted resources. There is currently no way to
target outputs themselves, so this is how we can allow these to be
updated as part of a target.
Rather than attempting to backtrack through the graph to find candidate
outputs, requiring each node on the chain to properly advertise if it
could be traversed, then backtracking again to determine if the
candidate is valid (which often got "off course"), we can start directly
from the outputs themselves. The algorithm here is simpler: if all the
root output's resource dependencies are targeted, add that output and
its dependencies to the targeted set.
We previously intentionally removed support for the allow-missing-config
option to terraform import, requiring that all imported resources have
matching config. See #24412.
However, the option was not removed from the import command, and it is
widely used. This commit reintroduces support for importing with a
missing configuration by falling back to implying the provider FQN based
on the resource type.
If a data source is storing a value that doesn't comply precisely with
the schema, it will now show up as a perpetual diff during plan.
Since we can easily detect if there is no resulting change from the
stored value, rather than presenting a planned read each time, we can
change the plan to a NoOp and log the incongruity as a warning.
If depends_on is allowed for outputs, we should validate that the
expressions are valid. Since outputs are always evaluated, and
validation is just done by this evaluation, we can check the
depends_on validation during evaluation too.
There aren't going to be any nodes specifically for module call
instances during plan, so we have to switch the reference subject to the
general module call.
The output destroy node only needs to connect to each of the output's
up-edges in order to be connected transitively to all of the outputs
dependencies. In large, highly-connected graphs, this may save
considerable time for each output.
