Commit Graph

1354 Commits

Author SHA1 Message Date
Joshua Feierman
2576544db8 fix: remote state behavior
This makes the behavior of remote state consistent with local state in regards to the initial serial number of the generated / pushed state. Previously remote state's initial push would have a serial number of 0, whereas local state had a serial of > 0. This causes issues with the logic around, for example, ensuring that a plan file cannot be applied if state is stale (see https://github.com/hashicorp/terraform/issues/30670 for example).
2023-02-01 13:54:40 -05:00
Joshua Feierman
68e227d93d wip: refactored tests 2023-02-01 13:34:42 -05:00
Joshua Feierman
09e0dffe6c wip: failing unit test for condition 2023-02-01 13:34:42 -05:00
Brandon Croft
0cafe0dfdb
Merge pull request #32571 from hashicorp/TF-3527-detect-alias-localterraform-com-during-terraform-init
Add support for Terraform Enterprise generic hostname localterraform.com
2023-01-31 10:59:12 -07:00
Liam Cervante
24b88b7a72
Reverse the order of conversion/defaults, and update HCL with more flexible defaults package (#32454)
* Add failing test case for the given issue

* pause

* don't use local when sending PR for review

* go get github.com/hashicorp/hcl/v2@v2.16.0

* Update go.mod

---------

Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>
2023-01-31 06:37:24 -05:00
Brandon Croft
2fe3a23094
Add generic hostname (localterraform.com) support to cloud backend
Aliases the backend hostname config as "localterraform.com" and duplicates any existing auth credentials during cloud backend configuration.
2023-01-30 17:21:24 -07:00
Brandon Croft
9fd76e56cd
Factor out terraform-config-inspect/tfconfig during init
As explained by the deleted comments, this package was used to identify situations where the `terraform 0.12upgrade` command can help migrate 0.11 syntax. Current versions of terraform don't include this command, and it's not likely that users are attempting upgrades from 0.11 to 1.4+

The replacement init swaps the order of the module and backend initialization in order to prepare for the next commit.

Config initialization now takes the following approach:
1. Load the root module, but withhold diagnostic errors until after version check
2. Initialize the backend, but withhold diagnostic errors until after version check
3. Get modules
4. Load all config (root and modules)
5. Check terraform version requirements (this can be defined by nested modules) and display any errors. It's important to show these first because prior errors could be the result of a newer terraform version syntax
6. Finally, show any errors related to backed init or config loading
2023-01-30 17:21:23 -07:00
James Bardin
fc8fed0047
Merge pull request #32583 from hashicorp/jbardin/store-null-module-outputs
save null module outputs in state
2023-01-26 15:35:48 -05:00
James Bardin
47fed6d31e save null module outputs in state
Although they are not serialized to the final stored state, all module
outputs must be saved in the state for evaluation. There is no defined
schema which is used to identify the overall type of module outputs, so
all outputs must exist in the state to build the correct type for proper
evaluation.
2023-01-26 11:03:23 -05:00
Sheridan C Rawlins
75e5ae27a2
[fixes 31700] Add mTLS support for http backend by way of client cert & key, as well as enterprise cacert. (#31699)
* Add mTLS support for http backend by way of client cert & key, as well as enterprise cacert.

* Fix style.

* Skip cert validation to be sure error is related to missing client cert; not untrusted server cert.

* Remove misplaced err check.

* Fix the size of test using http backend.

* Just for correctness, include all certs in the pem encoded cert - sometimes certs come with a chain of their signers.

* Adjusted names as recommended in PR comments.

* Adjusted names to be full-length and more descriptive.

* Added full-fledged testing with mTLS http server

* Fix goimports.

* Fix the names of the backend config.

* Exclusive lock for write and delete.

* Revert "Fix goimports."

This reverts commit 7d40f6099fbbb675fb2e25e35ee40aeafe3d0a22.

* goimports just for server test.

* Added the go:generation for the mock.

* Move the TLS configuration out to make it more readable - don't replace the HTTPClient as the retryablehttp already creates one - just configure its TLS.

* Just switch the client/data params - felt more natural this way.

* Update internal/backend/remote-state/http/backend.go

Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>

* Update internal/backend/remote-state/http/testdata/gencerts.sh

Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>

* Update internal/backend/remote-state/http/backend.go

Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>

* Update internal/backend/remote-state/http/backend.go

Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>

* Update internal/backend/remote-state/http/backend.go

Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>

* Update internal/backend/remote-state/http/backend.go

Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>

* the location of the file name is not sensitive.

* Added error if only one of client_certificate_pem and client_private_key_pem are set.

* Remove testify from test cases; use t.Error* for assert and t.Fatal* for require.

* Fixed import consistency

* Just use default openssl.

* Since file(...) is so trivial to use, changed the client cert, key, and ca cert to be the data.

See also https://github.com/hashicorp/terraform-provider-http/pull/211

Co-authored-by: Sheridan C Rawlins <scr@ouryahoo.com>
Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>
2023-01-26 14:08:07 +00:00
James Bardin
b6906f3572
Merge pull request #32563 from hashicorp/jbardin/optional-computed-comparison-next
a new method of `ProposedNew` set comparison
2023-01-25 15:05:50 -05:00
James Bardin
799f4a76ab
Merge pull request #32554 from hashicorp/jbardin/optional-computed-comparison
refactor ProposedNew
2023-01-25 15:05:39 -05:00
James Bardin
33c1ffa7c2
Merge pull request #32553 from hashicorp/jbardin/nesting-map-objchange
Fix crashes with NestingMap values
2023-01-25 15:05:27 -05:00
James Bardin
f6af5c1ef7
Merge pull request #32551 from hashicorp/jbardin/optional-computed-null
better determine when to plan optional+computed
2023-01-25 15:05:06 -05:00
James Bardin
aacbc167eb
Merge pull request #32536 from hashicorp/jbardin/optional-computed-nested-objects
Plan correct optional and computed attributes in nested objects and sets
2023-01-25 15:04:37 -05:00
James Bardin
d1d55653e4 update comment for fixed test 2023-01-25 12:11:31 -05:00
Pooya
6bbb76eca3
Added Quiet value (#32116)
* Added Quiet value

* Added Quiet value

* Added Quiet value

* Removed comments on 170 171 173

Co-authored-by: itspooya <fallenangel201190@gmail.com>
2023-01-25 16:51:14 +00:00
Martin Atkins
e2380b1038 cliconfig: Allow forcing use of the plugin cache despite the lock file
Currently Terraform will use an entry from the global plugin cache only if
it matches a checksum already recorded in the dependency lock file. This
allows Terraform to produce a complete lock file entry on the first
encounter with a new provider, whereas using the cache in that case would
cause the lock file to only cover the single package in the cache and
thereefore be unusable on any other operating system or CPU architecture.

This temporary CLI config option is a pragmatic exception to support those
who cannot currently correctly use the dependency lock file but who still
want to benefit from the plugin cache. With this setting enabled,
Terraform has permission to produce a dependency lock file that is only
suitable for the current system if that would allow use of an existing
entry in the plugin cache.

We are introducing this option to resolve a conflict between the needs of
folks who are using the dependency lock file as expected and the needs of
folks who cannot use the dependency lock file for some reason. The hope
then is to give respite to those who need this exception in the meantime
while we understand better why they cannot use the dependency lock file
and improve its design so that everyone will be able to use it
successfully in a future version of Terraform. This option will become a
silent no-op in a future version of Terraform, once the dependency lock
file behavior is sufficient for all supported Terraform development
workflows.
2023-01-25 08:23:01 -08:00
Anna Winkler
3cc7e55465
Expand error when discovery document request fails (#32394)
* Expand error when discovery document request fails.
* Use error type from svchost library
* Use svchost v0.0.1
2023-01-24 15:19:12 -07:00
James Bardin
60d6e52021 a new method of ProposedNew set comparison
The existing set comparison method uses the prior elements with the computed
portions nulled out to find candidates to match the configuration. This
has the shortcoming of always removing optional+computed attributes,
because we have not yet found the configuration to know if attribute was
set or not.

Rather than having to take the most pessimistic value before comparison
to precompute the nulled values, we can compare each candidate directly,
walking the values in tandem. Each prior value is compared against the
config and checked to see if it could have been derived from that
configuration value, which allows us to treat optional+computed as
optional if there is config and computed if there is not.

This removes the ambiguity from having optional+computed attributes
within sets, giving us consistent plans when all values are known.
Unknown values of course are still undecidable, as are edge cases were
providers refresh with altered values or retained changed prior values
plan that were deemed not functionally significant.
2023-01-24 14:05:57 -05:00
James Bardin
9459f0b214 refactor ProposedNew
Unify the ProposedNew paths for Blocks and Objects. Break out the
individual case blocks into functions, then use a common interface to
dispatch the object creation to the correct function based on schema
type. This cuts the code in half, and prevents the block and object
behavior from diverging.
2023-01-20 15:59:32 -05:00
Nick Fagerlund
f26786d6d9
Merge pull request #32545 from hashicorp/nf/jan23-cloud-backend-test-typo
Fix typo in cloud backend's `TestCloud_setConfigurationFields`
2023-01-20 11:40:52 -08:00
James Bardin
375c2da3e3 update NestingMap logic
Simplify the logic in the NestingMap cases. Prevent uninitialized
cty.NilVal from appearing in block case.
2023-01-20 13:51:10 -05:00
James Bardin
fcbfc365e6 fix panics when handling null values in maps
NestingMap structures are not well tested, and we panic in many
situations when null crops up. Fix the first test cases and start
refactoring best we can. This probably won't go so far as making all the
objchange functions generic over Block and Object, but we can simplify a
lot and verify parity in implementations for now.
2023-01-20 13:38:55 -05:00
Nick Fagerlund
3b26f68737
Merge pull request #32543 from hashicorp/nf/jan23-attr-path-value-marks-corruption
Fix accidental mutation of shared `cty.Path`s in ValueMarks funcs
2023-01-20 10:17:58 -08:00
James Bardin
8e917e5513 check for non-computed within optional+computed
We can check if an object in state must have at least partially come
from configuration, by seeing if the prior value has any non-null
attributes which are not computed in the schema.

This is used when the configuration contains a null optional+computed
value, and we want to know if we should plan to send the null value or
the prior state.
2023-01-20 10:50:57 -05:00
Liam Cervante
7d1ea52077
Structured Plan Renderer: another round of fixes after more equivalence testing (#32537)
* Structured Plan Renderer: another round of fixes after more equivalence testing

* fix sort function

* use new constants
2023-01-20 09:24:22 +01:00
Nick Fagerlund
acf7314587 Fix typo in cloud backend's TestCloud_setConfigurationFields
This was clearly wrong, but it was also harmless -- in the event of a failing
test due to missing tags, they would get double-reported as both missing and
unexpected. This commit separates out the reporting as intended.
2023-01-19 18:33:14 -08:00
Nick Fagerlund
83428c91f1 Fix accidental mutation of shared cty.Paths in ValueMarks funcs
Go's `append()` reserves the right to mutate its primary argument in-place, and
expects the caller to assign its return value to the same variable that was
passed as the primary argument. Due to what was almost definitely a typo
(followed by copy-paste mishap), the configschema `Block.ValueMarks` and
`Object.ValueMarks` functions were treating it like an immutable function that
returns a new slice.

In rare and hard-to-reproduce cases, this was causing bizarre malfunctions when
marking sensitive schema attributes in deeply-nested block structures --
omitting the marks for some sensitive values (🚨), and marking other entire
blocks as sensitive (which is supposed to be impossible). The chaotic and
unreliable nature of the bugs is likely related to `append()`'s automatic slice
reallocation behavior (if the append operation overflows the original array
allocation, the resulting behavior can _look_ immutable), but there might be
other contributing factors too.

This commit fixes existing instances of the problem, and wraps the desired
copy-and-append behavior in a helper function to simplify handling shared parent
paths in an immutable way.
2023-01-19 12:39:19 -08:00
James Bardin
e16b848d00 expand test case 2023-01-19 15:03:27 -05:00
James Bardin
7ca9abe463 optional+computed
Simplify the proposedNewAttributes cases, and add another test for
coverage.
2023-01-19 10:51:38 -05:00
James Bardin
470ed22730 failing tests for nested sets in objchange
Add a number of test cases which fail without the prior changes.
2023-01-19 09:53:55 -05:00
James Bardin
048ee3432c better handling of nested sets in objchange
Combine and simplify the set comparison functions for NestingSet blocks
and attribute types.

The set handling for structural attributes was not recursing into nested
values. Once a simplified method for comparing set elements was devised
for nested types, it turns out the same method could be applied to
nested set blocks as well.
2023-01-19 09:53:55 -05:00
Liam Cervante
6dc49150b7
Structured Plan Renderer: Read the data source schemas from the right place (#32532)
* read the data source schemas from the right place

* address comments and add test
2023-01-19 15:37:02 +01:00
James Bardin
ac4f5fe2f9 reformat 2023-01-18 17:48:47 -05:00
James Bardin
986127e619 remove unused isConfig arg 2023-01-18 17:48:47 -05:00
James Bardin
e661e91bff failing test for nested set types 2023-01-18 17:48:28 -05:00
Liam Cervante
8d61c5bfc4
Use the new structured renderer in place of the old diffs package (#32520)
* Use the new structured renderer in place of the old diffs package

* remove old plan tests

* refresh only plans should show moved resources in the refresh section
2023-01-18 10:14:54 +01:00
James Bardin
93f739e927 optional+computed can contain nested objects
When structural attributes were added, optional+computed were not
correctly handled when containing nested values which could themselves
be computed. This would cause terraform to ignore previously computed
values from state when generating the proposed plan.

The special case for optional+computed was incorrect, but isn't needed
in the context of planning new values anyway. Attributes are either
computed, or not computed. When optional+computed is set and there is
no configuration, the attribute is treated as computed. It is up to the
provider to determine how and when to deal with any changes to that
computed value.
2023-01-17 17:22:01 -05:00
Brandon Croft
5ac03755e7
Merge pull request #32303 from mrinalirao/mr/policy-evaluation
Add policy evaluation task stage to the CLI
2023-01-17 14:47:23 -07:00
Liam Cervante
99823e4a15
Structured Plan Renderer: Fix minor bugs causing diffs in the equivalence tests. (#32519)
* remove attributes that do not match the relevant attributes filter

* fix formatting

* fix renderer function, don't drop irrelevant attributes just mark them as no-ops

* fix imports

* fix bugs in the renderer exposed by the equivalence tests

* imports

* gofmt
2023-01-17 09:31:29 +01:00
Liam Cervante
e015b15f12
Structured Plan Renderer: Remove attributes that do not match the relevant attributes filter (#32509)
* remove attributes that do not match the relevant attributes filter

* fix formatting

* fix renderer function, don't drop irrelevant attributes just mark them as no-ops

* fix imports
2023-01-16 15:18:38 +01:00
Daniel Banck
4fd8322802
Add function descriptions (#32453)
* Add consolidated function description list

* Add function parameter descriptions

* Add descriptions to all functions

* Add sanity test for function descriptions

* Apply suggestions from code review

Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>

Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>
2023-01-16 10:48:31 +00:00
mrinalirao
dcd2826277 refactor runTaskStage func & use multierrors lib 2023-01-16 14:36:17 +11:00
James Bardin
e09b67bebd simplify output assignment 2023-01-13 13:20:25 -05:00
James Bardin
e2a6397a06 typed null input should be reflected in output
The configuration may be supplying a typed null value to the
terraform_data.input attribute, which must be reflected in the output to
have a valid plan.
2023-01-13 11:48:51 -05:00
mrinalirao
4d2e75bbda Merge branch 'mr/refactor-task-stage' into mr/policy-evaluation
# Conflicts:
#	internal/cloud/backend_taskStages.go
2023-01-13 10:33:01 +11:00
Liam Cervante
95782f2491
Structured plan renderer: Implement the main functionality for the renderer (#32496)
* raw unmodified broken tests

* tests execute, no panics

* fix whitespace differences

* fix all the tests

* fix tests

* actually fix tests

* add missing plan metadata into the renderer

* address comments

* complete merge

* remove TODO raising questions about outputs, they are fixed

* missing bold on plan
2023-01-12 17:59:07 +01:00
Liam Cervante
af0ff90d6e
Remove outputs from the jsonplan that are not from the root module (#32503) 2023-01-12 17:02:29 +01:00
Liam Cervante
c125397da1
push the colorize actions as locally as possible (#32502) 2023-01-12 16:47:06 +01:00