Legacy providers may return null values for nested blocks during
refresh. Because the ReadResource call needs to accept any value to
allow the provider to report external changes, we allowed all changes to
the value as long as the underlying cty.Type was correct, allowing
null block values to be inserted into the state.
While technically invalid, we needed to accept these null values for
compatibility, and they were mostly seen as a nuisance, causing noise in
external changes and plan output. These null block values however can be
inserted into the effective configuration with the use of
`ignore_changes`, which can cause problems where the configuration is
assumed to be completely valid.
Rather than accept the null values, we can insert empty container values
for these blocks when refreshing the instance, which will prevent any
invalid values from entering state at all. Because these must still be
accepted for compatibility, we can only log the difference as a warning.
Currently the NormalizeObjectFromLegacySDK does not report which
specific blocks it fixed, so we just log a generic message.
* Use the apparentlymart/go-versions library to parse module constraints
* goimports
* Update comments, and parse versions carefully
* add acceptance tests to verify behaviour of partial matches
* goimports
This includes the fix for a bug in what Terraform calls the "yamldecode"
function, where it was not correctly handling any situation where the
decode result is a null value. It was previously returning an unknown
value in that case, whereas now it returns a null value as expected.
Outputs were being evaluated from changes, even during apply. Make sure
we update the state correctly, and remove the existing change. This
requires adding more Planning fields to the output nodes to
differentiate whether the output is being planned or applied because the
same type handles both cases. We can evaluate separately whether new
types should be introduced to deal with both cases.
The module node cleanup was also prematurely removing module outputs
from the state before evaluation. This was not noticed before because
the evaluation was always falling back to changes. Have the root module
node do the final cleanup for all its children.
It turns out sensitive was also being handled incorrectly, and only
sensitive from configuration was being considered. Make sure to mark the
output as sensitive when storing sensitive values into state, and OR
sensitive marks with the state config when evaluating the output values.
When output values are updated in the refreshed state, we don't need to
re-set the changes which were already set in conjunction with the
current state.
Once the ProviderTransformer has resolved and set the exact provider,
the ProvidedBy method should return that exact provider again.
We can hoist the stored provider addr into the AbstractInstance and
avoid the method duplication and slight differences between the
implementations.
If when refreshing an orphaned instance the provider indicates it has
already been deleted, there is no reason to create a change for that
instance. A NoOp change should only represent an object that exists and
is not changing.
This was likely left in before in order to try and provide a record of
the change for external consumers of the plan, but newer plans also
contain all changes made outside of Terraform which better accounts for
the difference. The NoOp change now can cause problems, because it may
represent an instance with conditions to check even though that instance
does not exist.
In a heavily-connected graph with lots of inter-dependent providers, the
cycle checks for destroy edges across providers can seriously impact
performance. Since the specific cases we need to avoid will involve
create/update nodes, skip the extra checks during a full destroy
operation. Once we find a way to better track these dependencies, the
transformer will not need to do the cycle checks in the first place.
Data resource dependencies are not stored in the state, so we need to
take the latest dependency set to use for any direct connections to
destroy nodes.
Some prior refactors left the detroyPlan method a bit confusing, and ran
into a case where the previous run state could be returned as nil.
Get rid of the no longer used pendingPlan value, and track the prior and
prev states directly, making sure we always have a value for both.
In order to complete the terraform destroy command, a refresh must first
be done to update state and remove any instances which have already been
deleted externally. This was being done with a refresh plan, which will
avoid any condition evaluations and avoid planning new instances. That
however can fail due to invalid references from resources that are
already missing from the state.
A new plan type to handle the concept of the pre-destroy-refresh is
needed here, which should probably be incorporated directly into the
destroy plan, just like the original refresh walk was incorporated into
the normal planning process. That however is major refactoring that is
not appropriate for a patch release.
Instead we make two discrete changes here to prevent blocking a destroy
plan. The first is to use a normal plan to refresh, which will enable
evaluation because missing and inconsistent instances will be planned
for creation and updates, allowing them to be evaluated. That is not
optimal of course, but does revert to the method used by previous
Terraform releases until a better method can be implemented.
The second change is adding a preDestroyRefresh flag to the planning
process. This is checked in any location which evalCheckRules is called,
and lets us change the diagnosticSeverity of the output to only be
warnings, matching the behavior of a normal refresh plan.
When we originally introduced the trust-on-first-use checksum locking
mechanism in v0.14, we had to make some tricky decisions about how it
should interact with the pre-existing optional read-through global cache
of provider packages:
The global cache essentially conflicts with the checksum locking because
if the needed provider is already in the cache then Terraform skips
installing the provider from upstream and therefore misses the opportunity
to capture the signed checksums published by the provider developer. We
can't use the signed checksums to verify a cache entry because the origin
registry protocol is still using the legacy ziphash scheme and that is
only usable for the original zipped provider packages and not for the
unpacked-layout cache directory. Therefore we decided to prioritize the
existing cache directory behavior at the expense of the lock file behavior,
making Terraform produce an incomplete lock file in that case.
Now that we've had some real-world experience with the lock file mechanism,
we can see that the chosen compromise was not ideal because it causes
"terraform init" to behave significantly differently in its lock file
update behavior depending on whether or not a particular provider is
already cached. By robbing Terraform of its opportunity to fetch the
official checksums, Terraform must generate a lock file that is inherently
non-portable, which is problematic for any team which works with the same
Terraform configuration on multiple different platforms.
This change addresses that problem by essentially flipping the decision so
that we'll prioritize the lock file behavior over the provider cache
behavior. Now a global cache entry is eligible for use if and only if the
lock file already contains a checksum that matches the cache entry. This
means that the first time a particular configuration sees a new provider
it will always be fetched from the configured installation source
(typically the origin registry) and record the checksums from that source.
On subsequent installs of the same provider version already locked,
Terraform will then consider the cache entry to be eligible and skip
re-downloading the same package.
This intentionally makes the global cache mechanism subordinate to the
lock file mechanism: the lock file must be populated in order for the
global cache to be effective. For those who have many separate
configurations which all refer to the same provider version, they will
need to re-download the provider once for each configuration in order to
gather the information needed to populate the lock file, whereas before
they would have only downloaded it for the _first_ configuration using
that provider.
This should therefore remove the most significant cause of folks ending
up with incomplete lock files that don't work for colleagues using other
platforms, and the expense of bypassing the cache for the first use of
each new package with each new configuration. This tradeoff seems
reasonable because otherwise such users would inevitably need to run
"terraform providers lock" separately anyway, and that command _always_
bypasses the cache. Although this change does decrease the hit rate of the
cache, if we subtract the never-cached downloads caused by
"terraform providers lock" then this is a net benefit overall, and does
the right thing by default without the need to run a separate command.
* Convert variable types before applying defaults
* revert change to unrelated test
* Add another test case to verify behaviour
* update go-cty
* Update internal/terraform/eval_variable.go
Co-authored-by: alisdair <alisdair@users.noreply.github.com>
Co-authored-by: alisdair <alisdair@users.noreply.github.com>
