Commit Graph

3147 Commits

Author SHA1 Message Date
Alisdair McDiarmid
fb98fc98fa terraform: Fix sensitive values in ignore changes
Because ignore_changes configuration can refer to resource arguments
which are assigned sensitive values, we need to unmark the resource
object before processing.
2020-10-20 12:27:17 -04:00
James Bardin
d9ac57ffae
Merge pull request #26632 from hashicorp/jbardin/logging
Structured logging
2020-10-20 09:23:57 -04:00
Pam Selle
cdebf2820d
Merge pull request #26590 from hashicorp/pselle/sensitivity-providers
Mark attributes providers mark as sensitive
2020-10-19 16:13:39 -04:00
Pam Selle
394e60608c Allocate new copies of paths to avoid append drama
Create new copies of the Path to avoid possible append
related dramas. Also add a test to cover nested block
within blocks
2020-10-19 15:24:14 -04:00
James Bardin
0b31ffa587 use a single log writer
Use a single log writer instance for all std library logging.

Setup the std log writer in the logging package, and remove boilerplate
from test packages.
2020-10-19 14:29:54 -04:00
James Bardin
211edf5d75 use hclog as the default logger
Inject hclog as the default logger in the main binary.
2020-10-19 14:29:54 -04:00
James Bardin
6ca477f042 move helper/logging to internal
remove a dead code file too
2020-10-19 14:27:53 -04:00
Alisdair McDiarmid
5e047b0a0b
Merge pull request #26611 from hashicorp/alisdair/sensitive-values-provisioners
Fixes for sensitive values used as input to provisioners
2020-10-19 13:39:18 -04:00
Pam Selle
02c48f8071 Comment fixing 2020-10-18 13:00:09 -04:00
Alisdair McDiarmid
4f53234d8c terraform: Hide maybe-sensitive provisioner output
If the provisioner configuration includes sensitive values, it's a
reasonable assumption that we should suppress its log output. Obvious
examples where this makes sense include echoing a secret to a file using
local-exec or remote-exec.

This commit adds tests for both logging output from provisioners with
non-sensitive configuration, and suppressing logs for provisioners with
sensitive values in configuration.

Note that we do not suppress logs if connection info contains sensitive
information, as provisioners should not be logging connection
information under any circumstances.
2020-10-16 15:29:44 -04:00
Alisdair McDiarmid
9c580335e3 terraform: Unmark provisioner arguments
If provisioner configuration or connection info includes sensitive
values, we need to unmark them before calling the provisioner. Failing
to do so causes serialization to error.

Unlike resources, we do not need to capture marked paths here, so we
just discard the marks.
2020-10-16 15:12:09 -04:00
Pam Selle
a9823515ec Update context apply test 2020-10-15 17:38:09 -04:00
Pam Selle
a1a46425bd Set and single test coverage 2020-10-15 17:25:53 -04:00
Pam Selle
a5c5d2c28c Cover NestingMap case 2020-10-15 17:19:27 -04:00
Pam Selle
f790332bff NestingList support tested 2020-10-15 16:55:11 -04:00
Pam Selle
10cffc477d Basic test for GetResource, plus sensitivity 2020-10-15 16:50:57 -04:00
James Bardin
1ecd86d08e
Merge pull request #26155 from hashicorp/unused-env-var
Remove unused env var TF_SKIP_PROVIDER_VERIFY
2020-10-14 18:03:22 -04:00
Pam Selle
ee9ec8a193 Reordering so attributes are first (understandability) 2020-10-14 17:45:57 -04:00
Pam Selle
e44e03b283 If our block doesn't contain any sensitive attrs, skip recursing into it 2020-10-14 17:44:50 -04:00
Pam Selle
f60ae7ac08 Mark sensitive attributes in blocks
This implements marking sensitive attributes within
blocks when referenced by adding recursive calls
to get more paths from blocks' attributes
2020-10-14 17:37:06 -04:00
Pam Selle
5e2905d222 Mark attributes providers mark as sensitive
This updates GetResource so that the value
returned has marks where the provider's schema
has marked an attribute as sensitive
2020-10-14 15:30:16 -04:00
James Bardin
073beb90a6 re-enable and fix module variable tests
A few tests were inadvertently renamed, causing them to be be skipped.
For some reason this is not caught by the `vet` pass that happens during
normal testing.
2020-10-14 09:10:37 -04:00
Kristin Laemmert
57fd4c34d1 terraform: fix ProviderConfigTransformer
The ProviderConfigTransformer was using only the provider FQN to attach
a provider configuration to the provider, but what it needs to do is
find the local name for the given provider FQN (which may not match the
type name) and use that when searching for matching provider
configuration.

Fixes #26556

This will also be backported to the v0.13 branch.
2020-10-13 10:07:25 -04:00
James Bardin
03640057be
Merge pull request #26533 from hashicorp/jbardin/plan-output-changes
Use recorded changes for outputs and plan root output removals
2020-10-12 17:35:36 -04:00
James Bardin
28e4281674 handle sensitivity in the OutputChange
The state is not loaded here with any marks, so we cannot rely on marks
alone for equality comparison. Compare both the state and the
configuration sensitivity before creating the OutputChange.
2020-10-12 17:29:45 -04:00
James Bardin
d82778f4fc insert before values into the output changes
Lookup before values for output changes.
Use Update action when output has a non-null before value.
2020-10-12 17:29:45 -04:00
James Bardin
0f5bf21983 remove last use of the apply graph Destroy flag!
The apply graph builder no longer uses the destroy flag, which is not
always known since the destroy flag is not stored in the plan file.
2020-10-12 17:29:45 -04:00
James Bardin
ff21cc3c8d remove the need for destroyRootOutputTransformer
Since root outputs can now use the planned changes, we can directly
insert the correct applyable or destroyable node into the graph during
plan and apply, and it will remove the outputs if they are being
destroyed.
2020-10-12 17:29:45 -04:00
Martin Atkins
0bbbb9c64b configs: Experimental support for optional object type attributes
This builds on an experimental feature in the underlying cty library which
allows marking specific attribtues of an object type constraint as
optional, which in turn modifies how the cty conversion package handles
missing attributes in a source value: it will silently substitute a null
value of the appropriate type rather than returning an error.

In order to implement the experiment this commit temporarily forks the
HCL typeexpr extension package into a local internal/typeexpr package,
where I've extended the type constraint syntax to allow annotating object
type attributes as being optional using the HCL function call syntax.
If the experiment is successful -- both at the Terraform layer and in
the underlying cty library -- we'll likely send these modifications to
upstream HCL so that other HCL-based languages can potentially benefit
from this new capability.

Because it's experimental, the optional attribute modifier is allowed only
with an explicit opt-in to the module_variable_optional_attrs experiment.
2020-10-12 10:12:28 -07:00
James Bardin
d8e6d66362 use recorded changes for outputs
We record output changes in the plan, but don't currently use them for
anything other than display. If we have a wholly known output value
stored in the plan, we should prefer that for apply in order to ensure
consistency with the planned values. This also avoids cases where
evaluation during apply cannot happen correctly, like when all resources
are being removed or we are executing a destroy.

We also need to record output Delete changes when the plan is for
destroy operation. Otherwise without a change, the apply step will
attempt to evaluate the outputs, causing errors, or leaving them in the
state with stale values.
2020-10-09 13:13:27 -04:00
Pam Selle
eb2a027684
Merge pull request #26518 from hashicorp/pselle/sensitive-experiment
Remove sensitive_variables experiment
2020-10-08 16:14:37 -04:00
James Bardin
f5361286cc fix tests that happened during the cleanup 2020-10-08 14:58:43 -04:00
James Bardin
57a47d0b82
Merge pull request #26519 from hashicorp/jbardin/FIXME
Cleanup and remove legacy types from test mocks
2020-10-08 14:54:40 -04:00
James Bardin
a0caed541d remove legacy ApplyFn from mock provisioner 2020-10-08 13:13:13 -04:00
James Bardin
177797100d remove old mock validateFn 2020-10-08 12:43:46 -04:00
James Bardin
ea5ee39f38 remove old mock provider ConfigureFn 2020-10-08 12:26:12 -04:00
James Bardin
0a6853a3f8 replace testDiffFn and testApplyFn
Replace the old mock provider test functions with modern equivalents.
There were a lot of inconsistencies in how they were used, so we needed
to update a lot of tests to match the correct behavior.
2020-10-08 11:47:34 -04:00
Pam Selle
bc57c20d10 Remove sensitive_variables experiment
Ahead of the beta, remove the sensitive_variable experiment
and update tests accordingly
2020-10-08 11:22:20 -04:00
Pam Selle
ece9f8c1f4
Merge pull request #26507 from hashicorp/pselle/sensitive-vars-change
Update state when sensitivity changes
2020-10-07 15:39:07 -04:00
Pam Selle
a927e88c06 Compare before and after paths completely 2020-10-07 14:58:43 -04:00
Pam Selle
2c352ef182 Apply should not communicate w provider if only sensitivity changes
If sensitivity changes, we have an update plan,
but should avoid communicating with the provider
on the apply, as the values are equal (and otherwise
a NoOp plan)
2020-10-07 13:30:35 -04:00
Kristin Laemmert
d2e999ba1f
remove unused code (#26503)
* remove unused code

I've removed the provider-specific code under registry, and unused nil
backend, and replaced a call to helper from backend/oss (the other
callers of that func are provisioners scheduled to be deprecated).

I also removed the Dockerfile, as our build process uses a different
file.

Finally I removed the examples directory, which had outdated examples
and links. There are better, actively maintained examples available.

* command: remove various unused bits

* test wasn't running

* backend: remove unused err
2020-10-07 11:00:06 -04:00
Pam Selle
7c2ec1640a Tidy eval_diff assignments and update plan action
When the sensitivity has changed, we want to write to
state and also display to the user that the sensitivity
has changed, so make this an update action. Also
clean up some assignments, since Contains traverses the val
anyway, this is a little tighter.
2020-10-07 10:59:28 -04:00
Pam Selle
33cbf2698d Typo fix in eval_variable 2020-10-07 10:58:05 -04:00
James Bardin
ca7b5bc28b dead code 2020-10-07 10:50:21 -04:00
James Bardin
8d5e844158 no need to loop this test any longer 2020-10-07 10:47:09 -04:00
Pam Selle
7a7ad23113
Merge pull request #26491 from hashicorp/pselle/sensitive-vals-out-modules
Consider sensitivity when evaluating module outputs
2020-10-07 09:55:03 -04:00
James Bardin
35714e61e6 audit graph builder to make them more similar
Auditing the graph builder to remove unused transformers (planning does
not need to close provisioners for example), and re-order them. While
many of the transformations are commutative, using the same order
ensures the same behavior between operations when the commutative
property is lost or changed.
2020-10-06 17:39:53 -04:00
James Bardin
a32028aeed evaluate vars and outputs during import
Outputs were not being evaluated during import, because it was not added
to the walk filter.

Remove any unnecessary walk filters from all the Execute nodes.
2020-10-06 17:22:50 -04:00
Pam Selle
b9eeba0da2 Consider sensitivity when evaluating module outputs
This change "marks" values related to outputs that
have Sensitive set to true.
2020-10-06 13:09:18 -04:00