Commit Graph

31149 Commits

Author SHA1 Message Date
Martin Atkins
1962b1167c go.mod: go get github.com/hashicorp/hcl/v2@v2.17.0
This introduces HCL's own updates to generate refinements for unknown
results from some operations.
2023-05-31 10:14:13 -07:00
Martin Atkins
a801ba45cb Build with Go 1.20.4
This minor release includes a number of security fixes and other small bug
fixes in the standard library and Go compiler.

The security fixes do not appear to involve functionality critical to
Terraform and so this is not an urgent upgrade, but still worth doing to
reset to a more recent baseline.
2023-05-31 08:37:34 -07:00
Brandon Croft
ea0ebcdc05
Merge pull request #33267 from hashicorp/sebasslash/snapshot-interval-header-check
[cloud] Add interval header check to enable snapshots
2023-05-30 12:41:32 -06:00
Brandon Croft
86eed095b3
Rename disableIntermediateSnapshots > enableIntermediateSnapshots 2023-05-30 12:35:23 -06:00
modrake
0d7c718263
build: go-version action no longer uses deprecated "set-output" mechanism
GitHub has deprecated this in favor of appending to a temporary file instead:
     https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
2023-05-30 11:32:12 -07:00
Martin Atkins
9f6a3ba701 build: Generate copyright headers automatically
HashiCorp legal now requires a copyright claim in a comment at the top of
every substantial file in this repository. If we don't add this ourselves
then a bot will open a PR to add missing entries, but that process adds
git history, pull request, and GitHub notification noise so instead we'll
deal with it proactively as part of our usual code generation steps.

This means that pull requests will fail their checks if there are any
files that lack copyright headers, so we can deal with those before we
merge rather than in a subsequent PR.
2023-05-30 08:21:40 -07:00
Sebastian Rivera
d03fd37ee6 Add interval header check to enable snapshots 2023-05-26 15:01:05 -04:00
Martin Atkins
bc216caa65
Update CHANGELOG.md 2023-05-24 14:14:19 -07:00
Martin Atkins
b3f99cecb2
config: Refined Unknown Values
This introduces the "refinements" concept from upstream cty, which allows Terraform to track some additional information about unknown values that constrains their possible range even though we don't yet know the final value.
2023-05-24 14:11:04 -07:00
Martin Atkins
e0ef2748e6 docs: Describe the plugin protocol encoding of refined unknown values
This is actually a description of the "cty" library's encoding of refined
values, but from the perspective of the plugin protocol it's an
implementation detail that Terraform Core outsources that to a third-party
library, and current server-side implementations of the protocol use an
independent implementation of this format which will need to be compatible
with what cty does.
2023-05-24 13:48:16 -07:00
Martin Atkins
4c439b099f plans/objchange: Don't consider refinements when validating plans
Providers that existed prior to refinements (all of them, at the time of
writing) cannot preserve refinements sent in unknown values in the
configuration, and even if one day providers _are_ aware of refinements
there we might add new ones that existing providers don't know how to
handle.

For that reason we'll absolve providers of the responsibility of
preserving refinements from config into plan by fixing some cases where
we were incorrectly using RawEquals to compare values; that function isn't
appropriate for comparing values that might be unknown.

However, to avoid a disruptive change right now this initial fix just
strips off the refinements before comparing. Ideally this should be using
Value.Equals and handling unknown values more explicitly, but we'll save
that for a possible later improvement.

This does not include a similar exception for validating whether a final
value conforms to a plan because the plan value and the final value are
both produced by the same provider and so providers ought to be able to
be consistent with their _own_ treatment of refinements, if any.
Configuration is special because Terraform itself generates that, and so
it can potentially contain refinements that a particular provider has no
awareness of.
2023-05-24 13:48:13 -07:00
Martin Atkins
dfe5e1ddc4 plans/objchange: Providers must honor their unknown value refinements
If the original value was unknown but its range was refined then the
provider must return a value that is within the refined range, because
otherwise downstream planning decisions could be invalidated.

This relies on cty's definition of whether a value is in a refined range,
which has pretty good coverage for the "false" case and so should give a
pretty good signal, but it'll probably improve over time and so providers
must not rely on any loopholes in the current implementation and must
keep their promises even if Terraform can't currently check them.
2023-05-24 13:44:08 -07:00
Martin Atkins
81c15f987e lang/funcs: startswith considers string prefix refinement
If the string to be tested is an unknown value that's been refined with
a prefix and the prefix we're being asked to test is in turn a prefix of
that known prefix then we can return a known answer despite the inputs
not being fully known.

There are also some other similar deductions we can make about other
combinations of inputs.

This extra analysis could be useful in a custom condition check that
requires a string with a particular prefix, since it can allow the
condition to fail even on partially-unknown input, thereby giving earlier
feedback about a problem.
2023-05-24 13:44:08 -07:00
Martin Atkins
691018dd00 builtin/providers/terraform: terraform_data "id" is guaranteed non-null
The "id" attribute of this resource type is generated by the provider
itself and can never be null, so we'll refine the range of its unknown
result in case that helps downstream expressions to produce known results
even when the exact value hasn't yet been planned.
2023-05-24 13:44:08 -07:00
Martin Atkins
696cd68913 command/views: Describe unknown collection bounds in diagnostics 2023-05-24 13:44:08 -07:00
Martin Atkins
c912970153 lang/funcs: Non-null refinements for various functions
cty's new "refinements" concept allows us to reduce the range of unknown
values from our functions. This initial changeset focuses only on
declaring which functions are guaranteed to return a non-null result,
which is a helpful baseline refinement because it allows "== null" and
"!= null" tests to produce known results even when the given value is
otherwise unknown.

This commit also includes some updates to test results that are now
refined based on cty's own built-in refinement behaviors, just as a
result of us having updated cty in the previous commit.
2023-05-24 13:44:08 -07:00
Martin Atkins
1ef550e59a go.mod: go get github.com/zclconf/go-cty@v1.3.2 2023-05-24 13:44:03 -07:00
Liam Cervante
07aa7ee1d5
Propagate generated config filename into the Terraform graph (#33255) 2023-05-24 13:58:26 +02:00
James Bardin
2106b4c05f
Merge pull request #33235 from hashicorp/jbardin/destroy-root-referenced-locals
Remove destroy-plan locals only referenced by root outputs
2023-05-24 06:58:37 -04:00
kmoe
be2ad69eda
plannable import: safer config generation and schema filters (#33232)
* genconfig: fix nil nested block panic

* genconfig: null NestingSingle blocks should be absent

A NestingSingle block that is null in state should be completely absent from config.

* configschema: make FilterOr variadic

* configschema: apply filters to nested types

* configschema: filter helper/schema id attribute

The legacy SDK adds an Optional+Computed "id" attribute to the
resource schema even if not defined in provider code.
During validation, however, the presence of an extraneous "id"
attribute in config will cause an error.
Remove this attribute so we do not generate an "id" attribute
where there is a risk that it is not in the real resource schema.

* configschema: filter test

* terraform: do not pre-validate generated config

Config generated from a resource's import state may fail validation in
the case of schema behaviours such as ExactlyOneOf and ConflictsWith.
We don't want to fail the plan now, because that would give the user no
way to proceed and fix the config to make it valid. We allow the plan to
complete and output the generated config.

* generate config alongside import process

Rather than waiting until we call `plan()`, generate the configuration
at the point of the import call, so we have the necessary data to return
in case planning fails later.

The `plan` and `state` predeclared variables in the plan() method were
obfuscating the actual return of nil throughout, so those identifiers
were removed for clarity.

* move generateHCLStringAttributes closer to caller

* store generated config in plan on error

* test for config gen with error

* add simple warning when generating config

---------

Co-authored-by: James Bardin <j.bardin@gmail.com>
2023-05-24 11:16:05 +01:00
Martin Atkins
f6737d47e7 cloud: Allow Cloud API to lower the intermediate state snapshot interval
Previously we just made a hard rule that the state storage for Terraform
Cloud would never save any intermediate snapshots at all, as a coarse way
to mitigate concerns over heightened Terraform Enterprise storage caused
by saving intermediate snapshots.

As a better compromise, we'll now create intermediate snapshots at the
default interval unless the Terraform Cloud API responds with a special
extra header field X-Terraform-Snapshot-Interval, which specifies a
different number of seconds (up to 1 hour) to wait before saving the next
snapshot.

This will then allow Terraform Cloud and Enterprise to provide some dynamic
backpressure when needed, either to reduce the disk usage in Terraform
Enterprise or in situations where Terraform Cloud is under unusual load
and needs to calm the periodic intermediate snapshot writes from clients.

This respects the "force persist" mode so that if Terraform CLI is
interrupted with SIGINT then it'll still be able to urgently persist
a snapshot of whatever state it currently has, in anticipation of probably
being terminated with a more aggressive signal very soon.
2023-05-23 15:25:48 -07:00
Martin Atkins
c046b846a7 go.mod: upgrade github.com/hashicorp/go-tfe for response header hooks 2023-05-23 15:25:48 -07:00
Martin Atkins
efdc6e52bc cloud: Skip intermediate state snapshots in Terraform Cloud/Enterprise
We've seen some concern about the additional storage usage implied by
creating intermediate state snapshots for particularly long apply phases
that can arise when managing a large number of resource instances together
in a single workspace.

This is an initial coarse approach to solving that concern, just restoring
the original behavior when running inside Terraform Cloud or Enterprise
for now and not creating snapshots at all.

This is here as a solution of last resort in case we cannot find a better
compromise before the v1.5.0 final release. Hopefully a future commit
will implement a more subtle take on this which still gets some of the
benefits when running in a Terraform Enterprise environment but in a way
that will hopefully be less concerning for Terraform Enterprise
administrators.

This does not affect any other state storage implementation except the
Terraform Cloud integration and the "remote" backend's state storage when
running inside a TFC/TFE-driven remote execution environment.
2023-05-23 15:25:48 -07:00
Martin Atkins
8884bef59d backend/local: Allow storage impls to customize intermediate persistence
Previously we just always used the same intermediate state persistence
behavior for all state storages. However, some storages might have access
to additional information that allows them to tailor when they persist,
such as reacting to API rate limit status headers in responses, or just
knowing that a particular storage isn't suited to intermediate snapshots
at all for some reason.

This commit doesn't actually change any observable behavior yet, but it
introduces an optional means for a state storage to customize the behavior
which we may make use of in certain storage implementations in future
commits.
2023-05-23 15:25:48 -07:00
James Bardin
3e95ead4c2
Merge pull request #33224 from hashicorp/jbardin/update-cty
udpate go-cty
2023-05-23 13:56:47 -04:00
CJ Horton
258bdbe89f
Merge pull request #33238 from hashicorp/radditude/import-plan-plumbing
plannable import: correct plumbing when using a plan output file
2023-05-23 10:36:19 -07:00
James Bardin
2f308cf948
Merge pull request #32962 from hashicorp/jbardin/validate-unknown-coll-attrs
validate unknown nested attribute collections
2023-05-23 11:38:13 -04:00
CJ Horton
40ff381887 plumb import changes to and from binary plan 2023-05-22 22:19:42 -07:00
James Bardin
0a921976cd destroy locals referenced by root outputs
When planning a destroy operations, locals only referenced by root
outputs do not need to be kept in the graph, because the root output
does not get evaluated. Rather than try and prune the local based on
this condition, we can prevent the connection from being created by
ensuring that a root output destroy node has no references.

The separate plan+apply destroy fields used for outputs can be
simplified by combining, since they are only ever referenced together.
2023-05-22 13:03:49 -04:00
Masayuki Morita
53755180fd
Fix an error message for import block with moved block (#33221)
Fixes #33220
2023-05-19 13:47:46 -07:00
kmoe
4015f1aa30
genconfig: do not generate null NestingSingle blocks (#33213)
* genconfig: fix nil nested block panic

* always InternalValidate test schemas

* genconfig: null NestingSingle blocks should be absent

A NestingSingle block that is null in state should be completely absent from config.
2023-05-19 11:32:28 -07:00
James Bardin
6c08c3e45e udpate go-cty 2023-05-19 13:37:10 -04:00
Alisdair McDiarmid
048cc65787
Update PR template to target 1.5.x by default 2023-05-19 09:42:53 -04:00
Alisdair McDiarmid
13cb0d03c7 Update changelog for future 1.6.0 release 2023-05-15 10:39:38 -04:00
Alisdair McDiarmid
091c5d1d5e Update main branch version to 1.6.0 2023-05-15 10:37:27 -04:00
kmoe
b4d1146f58
plannable import: improve gen config human plan output (#33194)
* renderer: remove hard-coded config gen path

* mention config gen file in plan next steps
2023-05-15 15:21:41 +01:00
kmoe
789e30dfc5
error if importing to invalid keyed address (#33191)
Import addresses targeting expanded resource instances must target instances that already exist in configuration.
2023-05-13 00:57:51 +01:00
CJ Horton
adcecddb4f
Merge pull request #33193 from hashicorp/radditude/no-import-blocks-in-child-modules
plannable import: disallow import blocks in child modules
2023-05-12 16:44:12 -07:00
kmoe
dc3d37c2b7
Changelog: plannable import (#33189)
* Update CHANGELOG.md



Co-authored-by: CJ Horton <17039873+radditude@users.noreply.github.com>

---------

Co-authored-by: CJ Horton <17039873+radditude@users.noreply.github.com>
2023-05-13 00:31:21 +01:00
kmoe
1172d40d7b
error if import target is move source (#33192)
It is invalid for any import block to have a "to" argument matching any moved block's "from" argument.
2023-05-13 00:30:15 +01:00
CJ Horton
2dd89d9776 import blocks are only allowed in the root module 2023-05-12 16:04:47 -07:00
CJ Horton
bd6ba6cf99
check for duplicate import blocks (#33190)
Importing to the same target address twice or importing the same ID
to multiple different resources of the same type is not allowed.
2023-05-12 23:14:44 +01:00
Liam Cervante
d5fed58fc5
plannable import: write generated config to out flag (#33186)
* plannable import: write generated config to out flag

* Add example command to diagnostic
2023-05-12 23:05:00 +01:00
kmoe
2b71e9edf3
terraform: config-driven import is idempotent (#33188)
If a resource is already in state, do not attempt to import it again. Resources already in state are filtered out of the plan's import targets.

A change is only considered "importing" if it is adding a new resource instance to the state.
2023-05-12 21:31:29 +01:00
CJ Horton
5d7864316e
Merge pull request #33160 from hashicorp/radditude/apply-counts
Populate import counts during applies and clean up output
2023-05-12 09:33:33 -07:00
CJ Horton
e5a6806206 clarify apply hook usage 2023-05-11 19:02:59 -07:00
Rose M Koron
2a49d4b784
Merge pull request #32539 from cmaritan/patch-1
DOC: Update supported options of command graph -type=...
2023-05-11 18:01:25 -07:00
Rose M Koron
2b3da31c83
Merge pull request #32518 from irth/patch-1
docs: provisioners: fix formatting on a note
2023-05-11 17:18:50 -07:00
Rose M Koron
6079061f96
Merge pull request #32648 from mikeantonelli/patch-1
Update providers.mdx
2023-05-11 17:16:53 -07:00
Rose M Koron
2eb7fe3b5e
Update website/docs/cli/commands/graph.mdx 2023-05-11 17:13:16 -07:00