Commit Graph

2253 Commits

Author SHA1 Message Date
Martin Atkins
f65f51c023 core: fix syntax of input-bad-var-default test fixture
This was incorrectly using block syntax to assign a value to the "default"
attribute.
2018-10-16 18:49:20 -07:00
Martin Atkins
6cf4835714 core: Fix output from context input tests
The previous test output made it hard to understand what was going on when
a test failed. We'll now produce more verbose output.
2018-10-16 18:49:20 -07:00
Martin Atkins
74873838e0 core: Fix TestContext2Input
This test was previously not setting InputModeVarUnset, causing us to
overwrite the "amis" map that _is_ already set. This worked before because
we used to treat the empty result as an empty map and then merge it with
the given value, but since we no longer do that merging behavior we were
ending up with an empty map after input.

Since the intent of this test is to see that the "foo" variable gets
populated by input, here we add InputModeVarUnset which then matches how
the input walk is triggered by the "real" codepath in the local backend.

This also includes some updates to make the test fixture v0.12-idiomatic
(applied after it was seen to work with the old fixture) and to properly
handle the "diags" return value from the various context methods.
2018-10-16 18:49:20 -07:00
Martin Atkins
1b55f09891 core: remove redundant rule lines in BasicGraphBuilder logs
The "------" is useful when we produce the multi-line graph description,
but just adds noise when we're showing "(no changes)".
2018-10-16 18:49:20 -07:00
Martin Atkins
7673a0d850 core: Include "id" attribute in schema for cbd depends datasource test
This attribute is referenced in order to include a computed value into
another resource, and so it must be present in the schema so that it can
be properly resolved.
2018-10-16 18:49:20 -07:00
Martin Atkins
588f4930f2 core: Rename typoed fixture for TestContext2Plan "CBD" test
The intent here was for this to be "cbd" for "create_before_destroy", but
it was typoed.
2018-10-16 18:49:20 -07:00
Martin Atkins
dd6b171f62 core: Make provisioner schemas available to plan resource instance nodes
This requires making the "components" object available to the resource
node so it can be used during DynamicExpand. It also involved splitting
the provisioner schema attachment into a separate interface from
GraphNodeProvisionerConsumer so that it can now be handled within
AttachSchemaTransformer, along with all of the other schema attachment
steps.
2018-10-16 18:49:20 -07:00
Martin Atkins
559f65bf3d core: Fix ignore_changes = all
The initial rework of this function to support traversals didn't correctly
handle the "all" case, due to a logic error where the ignoreAll branch
could be visited only if ignoreChanges were non-empty, but yet the two
are mutually exclusive in practice.

Now we process ignoreAll separately from ignoreChanges, and invert the
two loops so that we will visit all attributes regardless of what is
in the ignoreChanges slice.
2018-10-16 18:49:20 -07:00
Martin Atkins
3f4b7f847d core: Fix TestContext2Plan_ignoreChangesWithFlatmaps
Prior to our v0.12 changes this test was confusingly using an attribute
named "set", but assigning a map to it. The expected test result suggested
that it was actually expecting legacy HCL2's weird interpretation of a
single map as a list of maps, and so to retain the intent of the test here
(in spite of the contrary name) we type "set" as list of map of string,
update the fixture to _actually_ be a list of maps, and then we get the
expected test result.
2018-10-16 18:49:20 -07:00
James Bardin
c6787d0266 update test fixtures
Update test fixtures to work in our new world.
This is mostly changing out attribute names for those in the schema,
adding Providers to states, and updating the test-fixture
configurations.
2018-10-16 18:49:20 -07:00
James Bardin
d0693d0db5 fix InstanceInfo.HumanId
The module no longer has a "root" component, and use the
normalizeModulePath function for consistency
2018-10-16 18:49:20 -07:00
Martin Atkins
d236a9e4cd core: update TestContext2Plan_computedList for changed assumptions
Previously this test's fixture was depending on the fact that attribute
access of an unknown value would always succeed and return another unknown
value, but under the new language interpreter an unknown value still
retains type information and so accessing this "bar" attribute would fail
the semantic check.

We also have to fuss a bit here to work around the limitations of the
testDiffFn implementation, which doesn't have enough context to understand
that "list" in the ResourceConfig is the same as "list.#" in its result.
Since this part of the provider API will change soon to use cty values
directly, this change just accepts a slightly-odd-looking diff in the mean
time, with both "list" and "list.#" populated.
2018-10-16 18:49:20 -07:00
Martin Atkins
ebef145980 core: Update expected error message for plan countComputedModule test
Previously we only handled the "count cannot be computed" check during
validate, leaving other walks to just report "a number is required"
(because "unknown" was represented as a special string) but now we have
unknown as first-class we handle it during all walks, and so this error
message is now the more appropriate one saying that the value is not
yet known.
2018-10-16 18:49:20 -07:00
Martin Atkins
083ea05295 core: Context plan tests update for "count" behavior change
The behavior of the "count" meta-argument has changed so that its presence
(rather than its value) chooses whether the associated resource has
indexes. As a consequence, these tests which set count = 1 now produce
a single indexed instance with index 0.
2018-10-16 18:49:20 -07:00
Martin Atkins
0ac663bb2b core: context plan tests update wanted output for new behavior
Previously Terraform Core was unaware of the structure of a resource type
schema and so the strange behavior of our testDiffFn caused some
attributes to not appear at all in the result. With core now more aware,
it "fills in" these missing items before calling, and as a result they
now appear in the diff even with the testDiffFn.

In real code, where helper/schema is constructing diffs, this situation
doesn't arise because the framework always produces schema-compatible
diffs.
2018-10-16 18:49:20 -07:00
Martin Atkins
e850becf64 core: update context plan tests for new module address rendering
The diff stringer now uses the standard serialization of a module address,
so we need to update the golden representations to restore their
associated tests to passing.
2018-10-16 18:49:20 -07:00
Martin Atkins
1a547b5351 core: Rename "count" variables in context plan fixtures
This variable name is now reserved so we can support the count
meta-argument inside module blocks in a later release.
2018-10-16 18:49:20 -07:00
James Bardin
cdc8abdae0 add missing Operation to the Evaluator 2018-10-16 18:48:28 -07:00
James Bardin
d55d623ef0 handle unset computed values in tests
The old testDiffFn used th raw config to dynamically set computed values
in the diff. Since the schema now defines what values should be there,
all test diffs end up with unkown computed values. Filter these out by
looking for a value set to "compute"
2018-10-16 18:48:28 -07:00
Martin Atkins
77bc36abce core: Make context_plan_test failure output more helpful
Lots of the tests were previously only producing the actual result, and
not what was expected. Now we'll show both, to make debugging easier.
2018-10-16 18:48:28 -07:00
Martin Atkins
cc5b1d452f core: NewInstanceInfo now produces correct "id" for data resources
We need to mimic the old API here, and we weren't doing that quite right
since the "data" prefix was missing from data resources.
2018-10-16 18:48:28 -07:00
Martin Atkins
eb54715902 core: fix string rendering of modules in diffs
This was assuming our old practice of a slice starting with the string
"root". We'll normalize here and then stringify the result to ensure that
we get a string consistent with what's used elsewhere.

This is primarily aimed at fixing some of the context plan tests.
2018-10-16 18:48:28 -07:00
Martin Atkins
39dfc9f4e0 core: Fix schema-related issues in context plan tests
Most changes here just introduce some custom schema into the test mocks.
In some cases, a fixture is lightly updated to more modern assumptions.

The test for accessing count.index in a resource block without count set
is removed, because that is no longer valid under the new language
implementation.
2018-10-16 18:48:28 -07:00
Martin Atkins
2b3b6c2407 core: TestContext2Plan_countComputedModule expectedErr
The phrasing of this message changed as part of updating it to use HCL
diagnostics rather than just a string.
2018-10-16 18:48:28 -07:00
James Bardin
e6478599c6 strip unknown values from the resource config
Computed values are now all added to the config value as unknowns. Strip
these out to match the legacy expatiations.
2018-10-16 18:48:28 -07:00
Martin Atkins
d6d655f21d core: Context plan tests expect a new error message for prevent_destroy
This error message has been revised as part of changing it to use
diagnostics instead of an error string directly.
2018-10-16 18:48:28 -07:00
Martin Atkins
5a77045a61 core: Move invalid output context tests to "validate"
This problem should now be caught at validate time rather than plan time,
because we can use the schema to detect the problem before the resource
has been resolved.
2018-10-16 18:48:28 -07:00
Martin Atkins
824986b698 core: use correct provider config address when eval pending resources
The evaluate data source was using a guessed provider configuration
address from configuration in this case, but that isn't necessarily
correct since the resource might actually be associated with a config
inherited from a parent module.

We still need to retain that fallback to config because we are sometimes
asked to evaluate when state is incomplete (like in "terraform console"),
but if possible we'll take the stored provider address from the state
and use that, even if the resource is otherwise "pending".
2018-10-16 18:48:28 -07:00
James Bardin
cae5c2feaa update apply tests 2018-10-16 18:48:28 -07:00
James Bardin
492a3b09fa deposed nodes need the resolved provider too
The provider is looked up by the ResolvedProvider
2018-10-16 18:48:28 -07:00
James Bardin
f8b77030db destroy nodes can't be referenced directly
Destroy nodes were being referenced by their regular paths, which was
causing cycles in the graphs. Destroy nodes can't be referenced directly
in any way, so override the inherited method for a referenceable address.
2018-10-16 18:48:28 -07:00
James Bardin
7137d85522 ignore id when creating diffs
The id field is always computed, and never directly related to a diff.
Since that field is being converted to a regular schema attribute, we
need to handle the behavior in the mocks too.
2018-10-16 18:48:28 -07:00
Martin Atkins
e528fe50e9 core: Misc. updates to the "plan" context tests
Mostly this is about updating ctx.Plan callers to expect diags instead of
err, but also includes a few light updates to test fixtures, and a fix to
testModuleInline.
2018-10-16 18:48:28 -07:00
James Bardin
6bf70427dd lock the provisioner mutex in ProvisionerSchema 2018-10-16 18:48:28 -07:00
James Bardin
cdaeed4f26 a provisioner may not always have a connection 2018-10-16 18:48:28 -07:00
James Bardin
07042a95fa fixing context apply tests 2018-10-16 18:48:28 -07:00
James Bardin
3bd2293152 compare module by normalized path
The onld logic for locating comparing module paths no longer worked, and
we can simplify the comparison by using the addrs.ModuleInstance string.
2018-10-16 18:48:28 -07:00
Martin Atkins
ea727d9918 core: Add mock schemas to the refresh context tests
There are still some other issues with some of these tests right now, but
all the ones that need to have schema should now have it.

It seems that there is a bug with the evaluation of child module input
variables where they can't find their schema even when a mock is provided.
Will attack this in a subsequent commit.
2018-10-16 18:48:28 -07:00
Martin Atkins
8b6ef7c8d3 core: EvalWriteOutput handle dynamic pseudo-type
This should actually have been caught by !val.IsWhollyKnown, since
DynamicVal is always unknown, but something isn't working quite right here
and so for now we'll redundantly check also if it's of the dynamic
pseudo-type, and then revisit cty later to see if there's a real bug
hiding down there.
2018-10-16 18:48:28 -07:00
Martin Atkins
5281afcc07 core: Actually read provider schema for data resources
We were passing the pointer into EvalReadDataDiff here, but because it was
not also passed to EvalGetProvider it was just always nil.
2018-10-16 18:48:28 -07:00
Martin Atkins
01a19e18ea core: Add ProviderAddr to all import context tests
The recent changes for v0.12 have moved the responsibilities around here
so that it's the caller's responsibility to specify the provider address
in all cases, with the real UI (in the "command" package) providing a
suitable default if nothing is specified.

Therefore the tests at _this_ level must all include an explicit provider
address, since these tests are acting as if they _are_ the UI code.
2018-10-16 18:48:28 -07:00
Martin Atkins
2c1ef35965 core: skip resource validation when count is unknown
The approach here is a little hacky, since this edge case applies only to
validate and all of the other evaluateResourceCountExpression callers
don't care about it: we overload the "count" return value as a flag to
allow NodeValidatableResource to allow it to detect this situation and
silently ignore errors in this case.
2018-10-16 18:48:28 -07:00
Martin Atkins
b54342460c core: Check that resource exists in config before evaluating it
We were previously doing this for all of the reference types except this
one. Now we do it for resources and resource instances too, which both
allows us to produce a proper error message when one is missing (rather
than returning an unknown value) and allows us to properly handle the
case where there are no instances yet present in the state (e.g. because
we're in the validate walk) but "count" isn't set, and so a single
unknown value is expected rather than an empty tuple.
2018-10-16 18:48:28 -07:00
Martin Atkins
0c4c7c5a5f core: Context.InitProvider must cache providers using absolute addresses
Previously InitProvider was incorrectly using only the relative address,
which (due to the ambiguity in the string representation of absolute vs.
relative addresses) caused it to always initialize providers in the root
module.

Now we use the absolute address as the key, which then agrees with the
Provider method and ensures that each module gets its own separate
instance of each provider if explicit configuration is present.
2018-10-16 18:48:28 -07:00
Martin Atkins
b8c3b8d45d core: Tolerate missing provider config schema in EvalValidateProvider
This should never happen in real code, but it comes up a lot in test code
where incomplete mock schemas are being used to test with very simple
configurations.
2018-10-16 18:48:28 -07:00
Martin Atkins
e4e3876332 core: EvalValidateProvider must always call provider.Validate
Previously we were skipping all of the validation steps if a provider was
being configured implicitly, and thus had no block in configuration.

This is incorrect, since a provider must still get an opportunity to
configure itself with an empty configuration and possibly reject that
empty configuration with errors.
2018-10-16 18:48:28 -07:00
Martin Atkins
b031e18332 core: Pass ProviderSchema to EvalValidateSelfRef
EvalValidateSelfRef needs schema in order to extract references. It was
previously expecting a *configschema.Block directly, but we weren't
actually passing that in from anywhere except the tests because it's not
available directly in that form during the evaltree for
node_resource_validate.

Instead, we now pass in the whole *ProviderSchema for the associated
provider and have this EvalNode find the schema itself based on the
address. This breaks some of the generality of this node (now only really
works for resource addresses) but that's okay since we have no other
use-case right now anyway.
2018-10-16 18:48:28 -07:00
Martin Atkins
54464e3f93 core: EvalWriteOutput must convert values to string
Our old state format requires all primitive values to be strings. We were
trying to enforce that before, but this didn't work properly because
gocty does not perform automatic type conversions.

Instead, we now convert to string first and then convert the result into
a native Go string afterwards.
2018-10-16 18:48:28 -07:00
Martin Atkins
6a859f1fe3 core: don't use functions in the input-interpolate-var fixture
Using functions here causes this to test more than it needs to, and
requires additional setup in the corresponding tests.
2018-10-16 18:48:28 -07:00
Martin Atkins
0b4ac6d9e3 core: Allow unknown values as variable outputs in EvalWriteOutput
At the moment this must be handled as a special case because we're still
using the old representation of output state, but we do still need to
handle this so that unknown values can properly pass between modules
during validate and plan.
2018-10-16 18:48:28 -07:00
Martin Atkins
d5fda47751 core: Add schema to all of the "Context2Validate" tests
Some of them are still failing for other reasons, but they at least all
now have schema representing their configurations.
2018-10-16 18:48:28 -07:00
Martin Atkins
509c6557dd core: Don't crash EvalValidateProvisioner if no connection block present 2018-10-16 18:48:28 -07:00
Martin Atkins
5b1b564663 core: fix formatting of unexpected diagnostics in context tests 2018-10-16 18:48:28 -07:00
Martin Atkins
7efa010f24 core: Add schema to TestTestContext2Validate_interpolateMap 2018-10-16 18:48:28 -07:00
Martin Atkins
fc7453c073 core: Update tests that use the apply-vars fixture
These now need to include a suitable schema in order to properly process
the configuration.
2018-10-16 18:48:28 -07:00
Martin Atkins
a9a9cdaa15 core: Fix ProcessIgnoreChanges tests
Since our ignoring is now implemented in terms of cty objects and HCL
traversals, rather than flatmap keys, it no longer makes sense to test
for the flatmap detail of keeping the map count in a separate key.

It _does_ make sense to ignore an entire block or map, but that's already
covered by another existing tests for just []string{"resource"} above.
2018-10-16 18:48:28 -07:00
Martin Atkins
39a83b2fd1 core: fix test for buildProviderConfig
This was incorrectly comparing a cty.Value to an hcl.Body. Now we decode
the body first so we can compare two of cty.Value.

Also includes a fix to a stale comment in buildProviderConfig that was no
longer accurate.
2018-10-16 18:48:28 -07:00
Martin Atkins
0a59da418a core: Fix tests for EvalConfigProvider
The interface of this eval node has changed for v0.12, now requiring both
a provider address and the actual provider object.

We also need to give it a working ctx.EvalBlock implementation on the
mock EvalContext, so we just use installSimpleEval here to get our simple
implementation that just knows how to evaluate constant expressions.
2018-10-16 18:48:28 -07:00
Martin Atkins
9833d9991d core: Fix EvalWriteState tests
EvalWriteState now requires a provider address so that it can record the
last-used provider configuration in the state.
2018-10-16 18:48:28 -07:00
Martin Atkins
dc7455887e core: fix tests for debug.go
The plan debug test now needs to provide a schema for the resource types
it uses.
2018-10-16 18:48:28 -07:00
Martin Atkins
0da8fe2087 core: Fix EvalValidateSelfRef tests
We must provide a schema for the synthetic body we use for testing so that
the references can be found within it.
2018-10-16 18:48:28 -07:00
Martin Atkins
4a002bf10b core: EvalValidateSelfRef must catch instances referencing their resources
An instance like aws_instance.foo[0] is not permitted to refer to
aws_instance.foo, since that result contains the individual instance along
with all other instances.
2018-10-16 18:48:28 -07:00
Martin Atkins
fc5a3c5cec core: Fix tests for types in eval_validate.go
A schema is now required for any validation, so these tests now use the
simpleMockProvider function to produce a provider with a simple schema
already configured, and test against that schema.
2018-10-16 18:48:28 -07:00
Martin Atkins
e6049d79b7 core: EvalContextMock code-based mocking of evaluation
EvaluateBlock and EvaluateExpr are often called multiple times in a single
operation, and our usual approach of mocking with static values is a poor
fit for those cases.

To accommodate more complex tests, we allow the test to optionally provide
a callback function to use instead of the static return values.

Since a pretty common need is to just evaluate the given block or
expression in a simple way, we also now have a helper method
installSimpleEval that installs reasonable implementations of these
two methods that can (assuming no other customization) just evaluate
constant expressions, which is sufficient for many tests.
2018-10-16 18:48:28 -07:00
Martin Atkins
8d0bec5728 core: Don't include "type" in connection block schema
The config loader already extracts this during its initial pass and saves
it as a separate field in a configs.Connection value, so requiring it
again here causes confusing errors about this attribute being provided but
not set.
2018-10-16 18:48:28 -07:00
Martin Atkins
f148b26287 core: Fix RefreshGraphBuilder tests
A correct outcome from this test now requires resource type schema to be
available to the graph builder.
2018-10-16 18:48:28 -07:00
Martin Atkins
5e8938de98 core: Remove GraphWalkerPanicwrap, etc
These are some remnants of the shadow graph functionality that was added
to support the graph builder changes in v0.8, but that has since been
removed and so there are no remaining callers for these types and
functions.
2018-10-16 18:48:28 -07:00
Martin Atkins
e1e4762b03 core: Update TestNodeApplyableModuleVariablePath for new address type 2018-10-16 18:48:28 -07:00
Martin Atkins
feff252700 core: fix tests for checkInputVariables
The contract for this function has changed as part of the 0.12
reorganization so that, while before it would accept a partial variables
map if all missing variables were optional, it now expects that the caller
has already collected values for all variables and is just checking them
for type-correctness here.
2018-10-16 18:48:28 -07:00
Martin Atkins
ce41553d56 core: Fix test for NodeDestroyResource.DynamicExpand
Provider addresses are now required for ResourceState, and the string
formatting is now different in the naming of destroy-deposed graph nodes.
2018-10-16 18:48:28 -07:00
Martin Atkins
26e820d829 addrs: Don't return zero-length slice in Reference.Remaining.
Although this rarely matters, making it always be nil when empty makes
deep assertions simpler in tests.

This also includes a minor update to the test in the core package that
first encountered this problem, to improve the quality of its output
on failure.
2018-10-16 18:48:28 -07:00
Martin Atkins
e1bc27ef6c core: ApplyGraphBuilder tests to use the "test" mock provider
These tests now need schema available to run properly, so rather than
crafting a separate schema for each of these we'll just use the one
created by simpleMockProvider, which contains a resource type called
test_object that we'll now use as the basis for all of the tests here.
2018-10-16 18:48:28 -07:00
Martin Atkins
f475b3b931 core: Fix DiffTransformer node addresses
In this codepath we're still using the old "internal" ResourceAddress
parser to parse resource addresses from the diff, but the string being
parsed does not include a module path and so the result is interpreted
as a resource in the root module.

We need to rewrite this to be in the correct module path as part of
converting to the new address representation, in order to get a correct
absolute address.
2018-10-16 18:48:28 -07:00
Martin Atkins
4472cffb35 core: NodeApplyableResourceInstance.References returns its result
A trivial mistake in the rework of this function meant that it was just
discarding its result rather than returning it. It will now return its
result as expected, allowing reference analysis to work for this node
type.
2018-10-16 18:48:28 -07:00
Martin Atkins
5863c581ee core: Fix misleading log message in ProviderTransformer
The final "<resource> provided by <provider>" message was showing the
original selection made by the resource itself rather than the provider
address finally resolved after inheritance, which caused the log to
misrepresent what was actually being created in the graph.
2018-10-16 18:48:28 -07:00
Martin Atkins
f7b8e3b8be core: Fix tests for TestPlanGraphBuilder
Previously this was just stubbing out provider types, but we now need to
include schema for each of the providers and resource types the tests
use in order for the references to be properly detected.

The test fixtures are adjusted slightly here so we can use the
simpleTestSchema as the schema for all of the different blocks in these
tests. The relationships between the resources are still preserved, but
the attributes are renamed to comply with this schema.
2018-10-16 18:48:28 -07:00
Martin Atkins
168354c2e8 core: Tests for scale in and out need to mock EvalExpression
Since expression evaluation is now done via an EvalContext method, and
since these tests use the MockEvalContext, we need to provide a value for
the mock call to return. Previously expression evaluation happened at a
different layer that was not mocked here.
2018-10-16 18:48:28 -07:00
Martin Atkins
bec0f56808 core: Pass components through to the destroy transformers
These transformers both construct temporary graphs using many of the same
transformers used in the apply graph, and properly doing this now requires
access to the providers and provisioners in order to obtain their schemas.

Along with this, we also update the tests here to use the
simpleMockComponentFactory helper to get a mock provider with a schema
already configured, which means we also need to update the test fixtures
and assertions to use the resource type and attributes defined in that
mock factory.
2018-10-16 18:48:28 -07:00
Martin Atkins
1a9ee72ecb core: Additional trace logging in attach resource config transformer 2018-10-16 18:48:28 -07:00
Martin Atkins
71d7a9e480 core: AttachSchemaTransformer fail gracefully with no schema
Having a missing schema is a programming error, but at the time of this
commit we're in the midst of introducing schema all over Terraform and so
there are inevitably some places -- particularly in older unit tests --
where schema isn't yet being provided.

This error allows us to catch those cases and fail gracefully, rather
than panicking further down here when we access t.Components methods.

Also includes some additional logging to aid debugging of this
transformer.
2018-10-16 18:48:28 -07:00
Martin Atkins
b73e4ede5d core: use correct fallback provider address for abstract resource instance
Previously we were attempting to construct a default manually here, and
actually getting it wrong by using the resource type name as a whole
rather than the expected inferrence by prefix.

Now we use the method provided in the addrs package for this purpose,
which implements the standard behavior of shaving off the first
underscore-separated word from the resource type name.
2018-10-16 18:48:28 -07:00
Martin Atkins
f82c192c49 core: mock provider factory functions for testing
Now that any configuration processing requires schema, we need either a
standalone schema or a provider/provisioner configured with one a lot more
often in tests.

To avoid adding loads of extra boilerplate to the tests, these new helper
functions produce objects pre-configured with a schema that should be
useful for a number of different cases, and can be customized further for
more interesting situations.

A lot of our tests can then just use these pre-defined object and
attribute names in their fixtures in situations where the canned schemas
here are good enough.
2018-10-16 18:48:28 -07:00
Martin Atkins
8fabcc0c08 core: Correct test TestOrphanResourceCountTransformer_zeroAndNone
The logic under test has intentionally changed here so that setting count
to any value -- even 1 -- causes Terraform to prefer to keep indexed
instances if both non-indexed and indexed are present.

Previously Terraform treated count = 1 as equivalent to count not set at
all, but we now recognize these as two different situations and treat
count = 1 as the same as any other non-zero count, to ensure that if
count is set conditionally it'll always produce indexed instances, even
if the dynamic expression ends up evaluating to 1.
2018-10-16 18:48:28 -07:00
Martin Atkins
ad6bb4a1d5 core: NodeProvisioner.Name update for new address types
This function was previously checking for a path length greater than one
because the older path format included an always present "root" element
at the start.

We now need to check for a totally-empty list, because otherwise we fail
to add the expected prefix to the front of a path with only one element.

This also includes some adjustments to the related tests and transforms
that do not change behavior but do make the test results easier to
understand and debug.
2018-10-16 18:48:28 -07:00
Martin Atkins
e40e3b9ad8 core: fix inheritence checks in ConfigTreeDependencies
Due to some disagreement about what representation of provider addresses
we were using, the inherited providers map wasn't matching. Now we'll
consistenly use the "compact" form (just the provider name and optional
alias).

Also includes some other tweaks to make this test better-behaved.
2018-10-16 18:48:28 -07:00
Martin Atkins
880d971328 core: TestStateAddModule old-style output
Although the AddModule method now takes a new-style
module address as an argument, internally we're still
shimming it to a []string.

Therefore this test needs to still expect [][]string as
a result, rather tan []addrs.ModuleInstance.
2018-10-16 18:48:28 -07:00
Martin Atkins
3ed396ad5c core: Update ProviderTransformer tests for new ImportTarget interface
Prior to the refactoring to move provider address parsing/selection up
into the frontend, there was some logic here to just-in-time default a
provider config based on the given resource type.

This is now expected to happen at a higher layer, with ImportTarget
expecting an already-valid provider configuration address.

The normal import codepath was already updated with this in mind, but some
of the provider transform tests are using ImportStateTransformer as a
shortcut for getting some resource nodes added to the graph, and so those
tests now need to include a valid provider address in their ImportTarget
values.

Also includes some adjustments to test output to make the tests easier
to debug.
2018-10-16 18:48:28 -07:00
Martin Atkins
b0b1486c46 core: Clarify the MissingProviderTransformer logic
The prior implementation of this function (before the rewrite to use
addrs.AbsProviderConfig values) was relying on some implicit behavior of
our provider address normalization to generate an address in the root
module. Since that wasn't explicit, I introduced a bug here when
introducing the new address type, where I generated an address in the
node's own module, rather than in the root as expected.

This new implementation is functionally equivalent to the prior, but is
written to make the intent more obvious: take _just_ the type from the
node's provider address and create an implicit configuration for it
_in the root module_.

Along with the change in approach, this new implementation also has an
updated documentation comment that better describes its current intent
(previously it was outdated) and hopefully clearer trace logging to
better communicate what it's doing.
2018-10-16 18:48:28 -07:00
Martin Atkins
e4e972db67 core: Fix MissingProviderTransformer matching existing providers
In the change to using addrs types rather than string keys directly here
I incorrectly made this use the _relative_ provider config instead of
the absolute one, causing MissingProviderTransformer to only match
providers defined in the root module (due to ambiguity in the string
representations of these address types).

The rest of this change is improved logging and test output that helped
with debugging this issue.
2018-10-16 18:48:28 -07:00
Martin Atkins
604c347128 core: Update test fixtures for ProviderTransformer tests
Some of these tests were using an outdated form to reference a local
directory as a module. While here, I also updated the provider references
to the new unquoted form, which is preferred from v0.12 onwards.
2018-10-16 18:48:28 -07:00
Martin Atkins
37dfda93ce core: Fix ReferenceTransformer tests
The changes to our GraphNodeReferencable and GraphNodeReferencer
interfaces were not also reflected in our testing structs here, and so
these tests were no longer working.

This updates these implementations to the new required signatures,
adapting the simplified model used in the structs to generate local
variable references, since the reference model no longer uses the
arbitrary strings that this test originally depended on.

We also remove some of the tests here since the functionality they were
testing no longer applies: inter-module dependencies are now handled by
the graph nodes producing extra ReferencableAddrs, and the "backup" form
is no longer used because our new address model is able to distinguish
resources from resource instances without the need for the magical
backup reference forms we previously used.
2018-10-16 18:48:28 -07:00
Martin Atkins
a6ca01f49d core: Fix TargetsTransformer tests
Resolution of dependencies automatically from expressions now requires
a schema to be available. To avoid the need to provide mock schemas for
all of these different resource types, we instead use the depends_on
argument here to mark the dependencies explicitly.
2018-10-16 18:48:28 -07:00
Martin Atkins
17d8ec6b6f core: NodeAbstractResource correct default result from ProvidedBy
Previously we had a bug where we'd use the resource type name instead of
the provider name. Now we use the DefaultProviderConfig helper function
to extract the provider name from the resource type name.
2018-10-16 18:48:28 -07:00
Martin Atkins
c8f186b303 core: TestTransitiveReductionTransformer needs a schema
The ReferenceTransformer can't do its work here unless our resources have
schema attached, since otherwise it doesn't know which attributes to
search to find references.
2018-10-16 18:48:28 -07:00
Martin Atkins
7188cabdaf core: Testing helpers for building provider schemas
Now that a schema is required to decode resource configuration, a lot more
tests than before need access to mock providers in order to get a schema.
These new helpers allow a mock provider with a schema to be constructed
concisely.
2018-10-16 18:48:28 -07:00
Martin Atkins
c58bdc6390 core: fix the tests in variables_test.go
This test required some adaptation for the new variable handling model in
the HCL2-oriented codepaths, where more of the handling is now done in the
"command" package.

The initial reorganization of this test was not correct, so this is a
further revision to adapt the original assertions to the new model as much
as possible, removing a few tests that no longer make sense in the new
world.
2018-10-16 18:48:28 -07:00
Martin Atkins
b33a970b62 core: fix test fixtures for HCL2 syntax idiom
A number of our test fixtures were previously using the non-idiomatic form
of including a single child attribute all on one line with the block
header and bounding braces.

This non-idiomatic form is an error in HCL2, and hclfmt has always "fixed"
it to the expected form of each attribute being on a line of its own, and
so here we just update all the affected test fixtures to canonical form
(using hclfmt), allowing them to be parsed as intended.

Since the these entire files were processed with hclfmt, there are some
other unrelated style changes included in situations where the file
layouts were non-idiomatic in other ways.
2018-10-16 18:46:46 -07:00
Martin Atkins
c82e3ec92f core: even more nil checks to catch missing objects
These are all things that ought to be present in normal use but can end up
being nil in incorrect tests. Test debugging is simpler if these things
return errors gracefully, rather than crashing.
2018-10-16 18:46:46 -07:00
Martin Atkins
26c1584317 core: EvalConfigProvider shouldn't crash if its provider isn't set
While there's no good reason for this to happen in practice, it can arise
in tests if mocks aren't set up quite right, and so we'll catch it and
report it nicely to make test debugging a little easier.
2018-10-16 18:46:46 -07:00
Martin Atkins
7dacb6b6a8 core: Don't crash if EvaluateBlock returns nil during validation
There's actually no good reason for this to happen in the normal case, but
it can happen reasonably easy if a test doesn't properly configure the
MockEvalContext, and so having this check here makes test debugging a
little easier.
2018-10-16 18:46:46 -07:00
Martin Atkins
69d2fa0c15 terraform: fix crasher in TestProcessIgnoreChanges
Since this test case is using t.Run, it must be sure to pass the nested
*testing.T over to the testDiffs function or else calls to t.Fatal will
crash the whole test process, since they would otherwise be called on the
wrong instance of *testing.T.
2018-10-16 18:46:46 -07:00
Martin Atkins
0dd7677d1f core: fail gracefully if provider schemas aren't available
The only reason these cases are arising right now is because we have tests
that haven't yet been updated to properly support schema, but it can't
hurt to add some robustness here to reduce the risk of real crashes.
2018-10-16 18:46:46 -07:00
Martin Atkins
4a21b763aa core: Get tests compiling again
After the refactoring to integrate HCL2 many of the tests were no longer
using correct types, attribute names, etc.

This is a bulk update of all of the tests to make them compile again, with
minimal changes otherwise. Although the tests now compile, many of them
do not yet pass. The tests will be gradually repaired in subsequent
commits, as we continue to complete the refactoring and retrofit work.
2018-10-16 18:46:46 -07:00
Martin Atkins
20f6de735d core: Update MockResourceProvisioner to include GetConfigSchema
This method is now required for the ResourceProvisioner interface.
2018-10-16 18:46:46 -07:00
Martin Atkins
73053eb5ef core: Context.Eval method
Some of the objects that are referencable from expressions are transient
values computed only during a graph walk, and not persisted in state. In
order to support arbitrary evaluation of expressions, such as in the
"terraform console" CLI command, it's necessary to be able to evaluate
these values before we start evaluating.

This new Eval method achieves this by performing a special graph walk that
ignores resources (except for dependency resolution) and just focuses on
evaluating all of these transient values, before returning an evaluation
scope that can then resolve expressions in terms of that result.

This replaces the Context.Interpolator method, which was fraught with
various issues due to it not properly priming the state before evaluating.
2018-10-16 18:46:46 -07:00
Martin Atkins
630e3b4be2 core: evaluation for HCL2-flavored reference expressions
Here we replace the stub implementations in evaluationStateData with real
implementations that are based on their equivalents in the old
Interpolator type.

The behavior here is a little different due to the different semantics
expected under HCL2, but the principle remains the same: the main
references are resolved from the state, using config for validation
in order to produce some helpful error messages.
2018-10-16 18:46:46 -07:00
Martin Atkins
25e3ac56d2 core: Wire back in module input variables to the evaluator
I took some missteps here while doing the initial refactor for HCL2 types.
This restores the map of maps that retains all of the variable values, and
then makes it available to the evaluator.
2018-10-16 18:46:46 -07:00
Martin Atkins
4b5868f653 core: BuiltinEvalContext constructs evaluationStateData itself
Previously our evaluationStateData object was constructed inside
Evaluator.Scope, but this was awkward because all of the fields inside it
need to be populated from BuiltinEvalContext fields, and so the signature
of Evaluator.Scope kept growing new arguments over time.

Instead, we reassign the responsibilities here so that Evaluator.Scope
takes an already-constructed lang.Data, and then teach BuiltinEvalContext
to build this object itself from its own internal values.
2018-10-16 18:46:46 -07:00
Martin Atkins
b0435cd533 core: ProviderConfigTransformer properly locates parent module
Due to a logic error here we were trying to find our our module's parent
as a descendent of itself, rather than as a descendent of the root. It
turns out that we can do this even more simply by just accessing the
Parent field on the given config, avoiding the need to traverse the tree
down from the root at all.

While here, this also switches to using the path.Call helper method rather
than manually slicing the path array, since this better communicates our
intent.
2018-10-16 18:46:46 -07:00
Martin Atkins
d4cfe85361 core: pass InstanceKey to EvaluateBlock
This gives us the value we need to evaluate a "count.index" reference, and
later also the equivalent for the "for_each" argument once implemented.
2018-10-16 18:46:46 -07:00
Martin Atkins
d4285dd27f core: Attach resource and provider config schemas during graph build
This is a little awkward since we need to instantiate the providers much
earlier than before. To avoid a lot of reshuffling here we just spin each
one up and then immediately shut it down again, letting our existing init
functionality during the graph walk still do the main initialization.
2018-10-16 18:46:46 -07:00
Martin Atkins
ca67899827 core: NewLegacyResourceInstanceAddress correct handling of addrs.NoKey
Previously we were setting zero here, but the correct way to represent
"no index" in a *ResourceAddress is to set the index to -1.
2018-10-16 18:46:46 -07:00
Martin Atkins
a28c93c3ce core: render variables, locals and outputs nicely in "terraform graph" 2018-10-16 18:46:46 -07:00
Martin Atkins
226d66bac8 core: include "root" element when constructing legacy ModuleState path
We've not yet adjusted any of the state structs to reflect our new address
types because they are used with encoding/json to produce our state file
format, but the shimming here previously was incorrect because it failed
to include the special "root" string that's always required at element
zero of a module path in the state.
2018-10-16 18:46:46 -07:00
Martin Atkins
c937c06a03 terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.

The three main goals here are:
- Use the configuration models from the "configs" package instead of the
  older models in the "config" package, which is now deprecated and
  preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
  new "lang" package, instead of the Interpolator type and related
  functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
  rather than hand-constructed strings. This is not critical to support
  the above, but was a big help during the implementation of these other
  points since it made it much more explicit what kind of address is
  expected in each context.

Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.

I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-10-16 18:46:46 -07:00
Martin Atkins
a09498a8a3 core: load a provider's schema at initialization
This is currently not very ergonomic due to the API exposed by providers.
We'll smooth this out in a later change to improve the provider API, since
we know we always want the entire schema.
2018-10-16 18:44:26 -07:00
Martin Atkins
c07b0a7806 configs: Re-unify the ManagedResource and DataResource types
Initially the intent here was to tease these apart a little more since
they don't really share much behavior in common in core, but in practice
it'll take a lot of refactoring to tease apart these assumptions in core
right now and so we'll keep these things unified at the configuration
layer in the interests of minimizing disruption at the core layer.

The two types are still kept in separate maps to help reinforce the fact
that they are separate concepts with some behaviors in common, rather than
the same concept.
2018-10-16 18:44:26 -07:00
Martin Atkins
4ed06a9227 terraform: HCL2-flavored module dependency resolver
For the moment this is just a lightly-adapted copy of
ModuleTreeDependencies named ConfigTreeDependencies, with the goal that
the two can live concurrently for the moment while not all callers are yet
updated and then we can drop ModuleTreeDependencies and its helper
functions altogether in a later commit.

This can then be used to make "terraform init" and "terraform providers"
work properly with the HCL2-powered configuration loader.
2018-10-16 18:44:26 -07:00
Martin Atkins
ebafa51723 command: Various updates for the new backend package API
This is a rather-messy, complex change to get the "command" package
building again against the new backend API that was updated for
the new configuration loader.

A lot of this is mechanical rewriting to the new API, but
meta_config.go and meta_backend.go in particular saw some major
changes to interface with the new loader APIs and to deal with
the change in order of steps in the backend API.
2018-10-16 18:44:26 -07:00
Alex Somesan
00822c82c0 Remove a leftover test log file 2018-09-11 17:02:32 +02:00
James Bardin
cfefeec926 walkDestroy is a form of "apply"
When computing the count value, make sure to include walkDestroy with
walkApply, as the former is only a special case of the latter.

When applying a saved plan, the computed count values are lost and we
can no longer query the state for those values. The apply walk was
already considered in the `resourceCountMax` function, but the destroy
walk was not.  This worked when destroying in a single operation
("terraform destroy"), since the state would still be updated with the
latest counts from the plan.
2018-04-10 11:46:29 -04:00
James Bardin
2f8532407a
Merge pull request #17071 from MarkFreebairn/Issue-16360-empty-target-points-to-all-resources-in-state
command: Fix #16360 - Fail fast if -target is specified but empty
2018-04-05 14:19:32 -04:00
James Bardin
060a3bc6d2
Merge pull request #17748 from reancloud/winrm-ntlm
Support NTLM for WinRM communicators.
2018-04-05 09:44:11 -04:00
James Bardin
f25718b065
Merge pull request #17768 from hashicorp/jbardin/unused-destroy-outputs
unused outputs in a destroy should be pruned
2018-04-04 15:20:12 -04:00
James Bardin
620f1985a1 unused outputs in a destroy should be pruned
During a full destroy when outputs are removed, the
NodeDestroyableOutput was preventing it's sibling output from being
destroyed. Prune the output node if it only has its destroy node as a
dependent.

The destroy output test is simply run a second time with no state, which
would cause the output interpolation to fail if it remained in the
graph.
2018-04-03 13:19:04 -04:00
James Bardin
79b948c9cc detect scaled in resources when evaluating *s
If an existing resources is scaled back to 0, locals and outputs will
still have a multi-variable reference to evaluate, which should return
an empty list. Due to how the resource is removed, the resource will
still exist in the state but with no primary instance, which needs to be
ignored in the instance count.
2018-04-03 10:00:45 -04:00
Joe Khoobyar
852a74c49d first attempt at supporting NTLM authentication in Terraform 2018-03-30 21:11:53 -04:00
James Bardin
38e6309f03
Merge pull request #17642 from hashicorp/jbardin/GH-17462
remove more unneeded partial outputs
2018-03-20 11:11:54 -04:00
James Bardin
a5c4f7e08e remove unneeded partial outputs
filterPartialOutputs was not taking into account that some dependent
resources might yet be removed from the graph. Check that they are not
in the targeted set before declaring that the output remain.
2018-03-19 21:20:06 -04:00
James Bardin
e5f8adfc1a add failing test for invalid output with targets
Outputs that are missing references aren't always removed from the
graph, due to being filtered before their dependents are removed.
2018-03-19 20:32:37 -04:00
James Bardin
8fb8b2cffc make sure ReadState returns an error
ReadState would hide any errors, assuming that it was an empty state.
This can mask errors on Windows, where the OS enforces read locks on the
state file.
2018-03-19 18:17:25 -04:00
James Bardin
88e911af45 fix a provisionerFail test which was incorrect
The provisionerFail_createBeforeDestroy test was verifying the incorrect
output. The create_before_destroy instance in the state has an ID of
"bar" with require_new="abc", and a new instance would get an ID of
"foo" with require_new="xyz". The existing test was expecting the
following state:

aws_instance.bar: (1 deposed)
  ID = bar
  provider = provider.aws
  require_new = abc
  Deposed ID 1 = foo (tainted)

Which showed "bar" still the primary instance in the state, with the new
instance "foo" as being the deposed instance, though properly tainted.

The new output is:

aws_instance.bar: (tainted) (1 deposed)
  ID = foo
  provider = provider.aws
  require_new = xyz
  type = aws_instance
  Deposed ID 1 = bar

Showing the new "foo instance as being the primary instance in the
state, with "bar" as the deposed instance.
2018-03-15 16:04:05 -04:00
James Bardin
b2d111c2bd return provisioner Apply errors
EvaApplyProvisioners was not returning errors if there was already a
multierror stored in the Error field. Always return the error to the
caller.
2018-03-15 16:04:05 -04:00
James Bardin
a2718e4f79 ignore errors interpolating RawCount during apply
If a count field references another count field which is interpolated
but is attached to a resource already in the state, the result of that
first interpolation will be lost when a plan is serialized. This is
because the result of the first interpolation is stored directly in the
module config, in an unexported config field.

This is not a general fix for the above situation, which would require
refactoring how counts are handles throughout the config. Ignoring the
error works, because in most cases the count will be properly
handled during the resource's interpolation.
2018-03-09 19:16:04 -05:00
James Bardin
f3d1fb3aff failing test for interpolated count from plan
An interpolated count value that is determined during plan, is lost
during plan serialization, causing apply to fail when the interpolation
string can't be evaluated.
2018-03-09 19:04:39 -05:00
James Bardin
7fd6f97899 Check for nil config in node_resource_refresh
While not normally possible, manual manipulation of the state and config
can cause us to end up with a nil config in
evalTreeManagedResourceNoState.

Regardless of how it got here, we can't ever assume the Config field is
not nil, and EvalInterpolate happily accepts a nil RawConfig
2018-03-06 17:53:09 -05:00
Paul Tyng
c868092d2d
Standardize http.Client creation with User-Agent 2018-02-28 12:09:50 -05:00
James Bardin
a37acb1837 gofmt with go1.10 2018-02-21 10:22:08 -05:00
James Bardin
1a68fdb4f6 add support for ssh host key checking
Add `host_key` and `bastion_host_key` fields to the ssh communicator
config for strict host key checking.

Both fields expect the contents of an openssh formated public key. This
key can either be the remote host's public key, or the public key of the
CA which signed the remote host certificate.

Support for signed certificates is limited, because the provisioner
usually connects to a remote host by ip address rather than hostname, so
the certificate would need to be signed appropriately. Connecting via
a hostname needs to currently be done through a secondary provisioner,
like one attached to a null_resource.
2018-02-14 15:35:41 -05:00
Ryan Uber
71ac5034e9 terraform: use hashicorp/go-uuid for lineage generation 2018-02-09 15:02:32 -08:00
James Bardin
7fbc35a36c Make sure outputs are removed when targeting
Similar to NodeApplyableOuptut, NodeDestroyableOutputs also need to stay
in the graph if any ancestor nodes

Use the same GraphNodeTargetDownstream method to keep them from being
pruned, since they are dependent on the output node and all its
descendants.
2018-01-31 13:51:40 -05:00
James Bardin
ca4178b9ec gofmt bug will be fixed in 1.10 2018-01-30 15:46:24 -05:00
James Bardin
a2f8482333 catch missing id attribute during interpolation
The id attribute can be missing during the destroy operation.
While the new destroy-time ordering of outputs and locals should prevent
resources from having their id attributes set to an empty string,
there's no reason to error out if we have the canonical ID field
available.

This still interrogates the attributes map first to retain any previous
behavior, but in the future we should settle on a single ID location.
2018-01-30 15:46:12 -05:00
James Bardin
99867f0082 add PruneUnusedValuesTransformer
Since outputs and local nodes are always evaluated, if the reference a
resource form the configuration that isn't in the state, the
interpolation could fail.

Prune any local or output values that have no references in the graph.
2018-01-30 10:47:17 -05:00
James Bardin
2d138d9917 add a more complex locals test
Using destroy provisioners again for edge cases during destroy.
2018-01-30 10:47:17 -05:00
James Bardin
d31fe5ab9d delete outputs during destroy
Now that outputs are always evaluated, we still need a way to remove
them from state when they are destroyed.

Previously, outputs were removed during destroy from the same
"Applyable" node type that evaluates them. Now that we need to possibly
both evaluate and remove output during an apply, we add a new node -
NodeDestroyableOutput.

This new node is added to the graph by the DestroyOutputTransformer,
which make the new destroy node depend on all descendants of the output
node.  This ensures that the output remains in the state as long as
everything which may interpolate the output still exists.
2018-01-29 19:30:04 -05:00
James Bardin
08139557f8 always evaluate outputs too
Always evaluate outputs during destroy, just like we did for locals.
This breaks existing tests, which we will handle separately.

Don't reverse output/local node evaluation order during destroy, as they
are both being evaluated.
2018-01-29 18:10:34 -05:00
James Bardin
7ac0a46981 add destroy provisioner test with locals, outputs
Add a complex destroy provisioner testcase using locals, outputs and
variables.

Add that pesky "id" attribute to the instance states for interpolation.
2018-01-29 18:01:58 -05:00
James Bardin
7da1a39480 always evaluate locals, even during destroy
Destroy-time provisioners require us to re-evaluate during destroy.

Rather than destroying local values, which doesn't do much since they
aren't persisted to state, we always evaluate them regardless of the
type of apply. Since the destroy-time local node is no longer a
"destroy" operation, the order of evaluation need to be reversed. Take
the existing DestroyValueReferenceTransformer and change it to reverse
the outgoing edges, rather than in incoming edges. This makes it so that
any dependencies of a local or output node are destroyed after
evaluation.

Having locals evaluated during destroy failed one other test, but that
was the odd case where we need `id` to exist as an attribute as well as
a field.
2018-01-29 16:16:41 -05:00
James Bardin
d29994e247
Merge pull request #17127 from hashicorp/jbardin/ignore_changes
don't ignore partial containers in diffs
2018-01-19 16:08:18 -05:00