Commit Graph

2279 Commits

Author SHA1 Message Date
James Bardin
02a6657a71 don't check for targeted downstream from providers
Since outputs now rely on providers in order to ensure that a schema is
available for evaluation, we need to exclude providers from checking
TargetDownstream.
2018-10-16 18:49:20 -07:00
James Bardin
fd4b427162 index provider schemas by type
A provider's schema is the same regardless of its address in the
config. Key them by type so that an evaluation referencing a provider
from an address not included in the graph can still find the schema.
2018-10-16 18:49:20 -07:00
Martin Atkins
c8dfc944fe core: ProviderTransformer debug log to use abs addr always
This one address in the log output was relative, making the result
misleading.
2018-10-16 18:49:20 -07:00
Martin Atkins
05bf08b2ec core: Remove context apply test for map variable merging
We no longer have this merge behavior, because it is inconsistent with how
variables behave in all other contexts and similar behavior can now be
achieved by merging the user's input with a predefined map in a local
value expression.
2018-10-16 18:49:20 -07:00
Martin Atkins
7dc3c51a86 core: Only create stub provider if real provider is not present
Previously we'd create the stub provider in any case where we didn't need
a configured provider, but we also need to skip creating it if there's
already a provider node present, or else we can end up with multiple
stub nodes in the graph.
2018-10-16 18:49:20 -07:00
Martin Atkins
b144497041 core: Attach schemas before dealing with provider edges
Since ProviderTransformer now needs the schema in order to infer indirect
references to providers, we must run AttachSchemaTransformer before the
provider transformers in order to calculate the correct ordering of
operations.
2018-10-16 18:49:20 -07:00
Martin Atkins
a2728759cd core: Apply inheritance logic to both direct and referenced providers
The provider schema cache is keyed by provider configuration address
rather than provider type, so we need to do the same inheritance logic
to resolve providers needed because of reference as we do for providers
needed for direct use.

This allows resources that override "provider" or resources in child
modules that have their own provider configurations to be associated
with the provider config they will eventually get schema from, rather
than (as before) always the default configuration for the provider in
the root module.

Eventually it'd probably be better to switch to using a provider cache
that is keyed by provider _type_ rather than provider config, but since
it's currently fetched by visiting the individual provider graph nodes
we currently visit each provider configuration separately and fetch a
schema for each.
2018-10-16 18:49:20 -07:00
James Bardin
4d2da4d733 connect non-resources to providers they reference
Any non-resource (outputs, variables, locals) that references a resource
type must also be connected to that resources provider. This is required
during apply, because the graph built from the diff may not include the
referenced resources because they are being evaluated from the state.

If the provider isn't present already, add a NodeEvalableProvider to
fetch the provider schema.

The provider transformers now need to happen after the outputs, locals,
and variables are transformed.
2018-10-16 18:49:20 -07:00
Martin Atkins
687830a5d5 core: Mock schema for TestContext2Apply_multiProviderDestroy
This test seems to have been buggy before our current work, with the test
fixture containing a reference to a resource that doesn't exist.

This both fixes the fixture and adds a mock schema for it, though this
just revealed another error which isn't fixed here, where the a_ids value
seems to come through as unknown after apply. That will be fixed in a
subsequent commit.
2018-10-16 18:49:20 -07:00
Martin Atkins
cdb32390b9 core: Add schema mocks to more of the context apply tests 2018-10-16 18:49:20 -07:00
Martin Atkins
5cf06e86c6 core: Remove TestContext2Apply_provisionerMultiSelfRefSingle
We no longer support using "self.count" in a provisioner to access the
resolved count meta-argument value of the associated resource.

This was only possible before because of a special exception in how
Terraform resolved variables, and in new HCL that exception isn't possible
because resource instances are real values in the scope and we don't want
to add this implied "count" attribute to all of them.

"count" is a property of the resource config rather than of the resource
instances, and since "self" is a resource _instance_ it doesn't make sense
to expose it there.

There is no replacement for this feature. In the rare case where it is
needed, the user must factor the count out into a named local value and
refer to that both in the count meta-argument and in the provisioner.
2018-10-16 18:49:20 -07:00
Martin Atkins
1caffba0e6 core: update various context apply tests for schema/fixtures
Most of these changes are just adding schema to describe the expectations
of the existing test fixtures. However, some of them require the fixtures
themselves to be changed due to changing assumptions in the language.
2018-10-16 18:49:20 -07:00
Martin Atkins
8774c857f6 core: "computed ref type mismatch" test is now a plan test
This was previously an apply-time failure due to our inability to
type-check unknowns in 0.11, but we now retain type information for
unknown values and so this check now fails during plan instead.
2018-10-16 18:49:20 -07:00
Martin Atkins
02932b5a50 core: Schema for TestContext2Apply_outputDiffVars
The fixtures for this test assume some atypical arguments to the resources
and also need a provisioner schema.

This doesn't actually fix the test, but by fixing the schema/fixture this
exposes a problem that seems to exist in the main code, which will be
fixed in a subsequent commit.
2018-10-16 18:49:20 -07:00
Martin Atkins
4314c9e985 core: Update context refresh test output to be more useful 2018-10-16 18:49:20 -07:00
Martin Atkins
1ed56f9903 core: NewInstanceInfo should take ResourceInstance, not Resource
On the initial pass here I reached a faulty conclusion about what from
the new world should shim into a NewInstanceInfo, based on a poor read
of existing code.

It actually _should've_ been based on an absolute instance after all,
as evidenced by the expected result of TestContext2Refresh_targetedCount.
Therefore the signature is changed here, and all of the callers (which,
in retrospect, were all holding a full instance address anyway!) are
updated to that new signature.
2018-10-16 18:49:20 -07:00
James Bardin
8d062fc577 update apply test fixtures 2018-10-16 18:49:20 -07:00
James Bardin
6ed1a81831 connect references to the resource type
References can't be connected directly to the instances, because the
resources are expanded when ReferenceTransformer is run. Lookup
references by the resource type.
2018-10-16 18:49:20 -07:00
Martin Atkins
153021dd08 core: NewResourceConfigShimmed not to crash without schema
Although there isn't really a good reason why there should be no schema
in practice, it's better for us not to crash right now while we're still
updating all of the callers (mostly tests) to make schema available.
2018-10-16 18:49:20 -07:00
James Bardin
c6b2799ee9 add "unknown" computed attribute
Having a compute value of "unknown" will trigger the testDifFn to always
insert a computed value to cause an unknown value suring apply
2018-10-16 18:49:20 -07:00
James Bardin
edacd9ff4d add Provider schema to the contextFixture 2018-10-16 18:49:20 -07:00
James Bardin
672221e38e it is valid to have no provider config 2018-10-16 18:49:20 -07:00
Martin Atkins
fe105bfc3f core: Fix TestContext2Input_submoduleTriggersInvalidCount
This test now needs to provide a mock schema so its fixture can assign
to the "foo" argument in null_data_source.
2018-10-16 18:49:20 -07:00
Martin Atkins
93ff9f4233 core: Fix TestContext2Input_submoduleTriggersInvalidCount
Its fixture was using a module variable called "count", and that name is
now reserved.
2018-10-16 18:49:20 -07:00
Martin Atkins
e7cfbb9737 core: Fix TestContext2Input_hcl
This test now needs to provide a mock schema for its resource type so that
the two test arguments can be assigned.
2018-10-16 18:49:20 -07:00
Martin Atkins
b6fe705985 core: fix buggy test TestContext2Input_providerMulti
This was trying to test gathering input from a default and an alias
provider configuration, but it was incorrectly setting the provider ref
on the instance that was supposed to belong to the alias. This was working
before because Terraform would gather input from the aliased provider
anyway, but now the invalid "alias" argument in the resource is producing
a validation error.

This doesn't actually make the test work again yet, because we still have
provider input disabled at this time, pending a forthcoming change to
how provider input is handled.
2018-10-16 18:49:20 -07:00
Martin Atkins
f65f51c023 core: fix syntax of input-bad-var-default test fixture
This was incorrectly using block syntax to assign a value to the "default"
attribute.
2018-10-16 18:49:20 -07:00
Martin Atkins
6cf4835714 core: Fix output from context input tests
The previous test output made it hard to understand what was going on when
a test failed. We'll now produce more verbose output.
2018-10-16 18:49:20 -07:00
Martin Atkins
74873838e0 core: Fix TestContext2Input
This test was previously not setting InputModeVarUnset, causing us to
overwrite the "amis" map that _is_ already set. This worked before because
we used to treat the empty result as an empty map and then merge it with
the given value, but since we no longer do that merging behavior we were
ending up with an empty map after input.

Since the intent of this test is to see that the "foo" variable gets
populated by input, here we add InputModeVarUnset which then matches how
the input walk is triggered by the "real" codepath in the local backend.

This also includes some updates to make the test fixture v0.12-idiomatic
(applied after it was seen to work with the old fixture) and to properly
handle the "diags" return value from the various context methods.
2018-10-16 18:49:20 -07:00
Martin Atkins
1b55f09891 core: remove redundant rule lines in BasicGraphBuilder logs
The "------" is useful when we produce the multi-line graph description,
but just adds noise when we're showing "(no changes)".
2018-10-16 18:49:20 -07:00
Martin Atkins
7673a0d850 core: Include "id" attribute in schema for cbd depends datasource test
This attribute is referenced in order to include a computed value into
another resource, and so it must be present in the schema so that it can
be properly resolved.
2018-10-16 18:49:20 -07:00
Martin Atkins
588f4930f2 core: Rename typoed fixture for TestContext2Plan "CBD" test
The intent here was for this to be "cbd" for "create_before_destroy", but
it was typoed.
2018-10-16 18:49:20 -07:00
Martin Atkins
dd6b171f62 core: Make provisioner schemas available to plan resource instance nodes
This requires making the "components" object available to the resource
node so it can be used during DynamicExpand. It also involved splitting
the provisioner schema attachment into a separate interface from
GraphNodeProvisionerConsumer so that it can now be handled within
AttachSchemaTransformer, along with all of the other schema attachment
steps.
2018-10-16 18:49:20 -07:00
Martin Atkins
559f65bf3d core: Fix ignore_changes = all
The initial rework of this function to support traversals didn't correctly
handle the "all" case, due to a logic error where the ignoreAll branch
could be visited only if ignoreChanges were non-empty, but yet the two
are mutually exclusive in practice.

Now we process ignoreAll separately from ignoreChanges, and invert the
two loops so that we will visit all attributes regardless of what is
in the ignoreChanges slice.
2018-10-16 18:49:20 -07:00
Martin Atkins
3f4b7f847d core: Fix TestContext2Plan_ignoreChangesWithFlatmaps
Prior to our v0.12 changes this test was confusingly using an attribute
named "set", but assigning a map to it. The expected test result suggested
that it was actually expecting legacy HCL2's weird interpretation of a
single map as a list of maps, and so to retain the intent of the test here
(in spite of the contrary name) we type "set" as list of map of string,
update the fixture to _actually_ be a list of maps, and then we get the
expected test result.
2018-10-16 18:49:20 -07:00
James Bardin
c6787d0266 update test fixtures
Update test fixtures to work in our new world.
This is mostly changing out attribute names for those in the schema,
adding Providers to states, and updating the test-fixture
configurations.
2018-10-16 18:49:20 -07:00
James Bardin
d0693d0db5 fix InstanceInfo.HumanId
The module no longer has a "root" component, and use the
normalizeModulePath function for consistency
2018-10-16 18:49:20 -07:00
Martin Atkins
d236a9e4cd core: update TestContext2Plan_computedList for changed assumptions
Previously this test's fixture was depending on the fact that attribute
access of an unknown value would always succeed and return another unknown
value, but under the new language interpreter an unknown value still
retains type information and so accessing this "bar" attribute would fail
the semantic check.

We also have to fuss a bit here to work around the limitations of the
testDiffFn implementation, which doesn't have enough context to understand
that "list" in the ResourceConfig is the same as "list.#" in its result.
Since this part of the provider API will change soon to use cty values
directly, this change just accepts a slightly-odd-looking diff in the mean
time, with both "list" and "list.#" populated.
2018-10-16 18:49:20 -07:00
Martin Atkins
ebef145980 core: Update expected error message for plan countComputedModule test
Previously we only handled the "count cannot be computed" check during
validate, leaving other walks to just report "a number is required"
(because "unknown" was represented as a special string) but now we have
unknown as first-class we handle it during all walks, and so this error
message is now the more appropriate one saying that the value is not
yet known.
2018-10-16 18:49:20 -07:00
Martin Atkins
083ea05295 core: Context plan tests update for "count" behavior change
The behavior of the "count" meta-argument has changed so that its presence
(rather than its value) chooses whether the associated resource has
indexes. As a consequence, these tests which set count = 1 now produce
a single indexed instance with index 0.
2018-10-16 18:49:20 -07:00
Martin Atkins
0ac663bb2b core: context plan tests update wanted output for new behavior
Previously Terraform Core was unaware of the structure of a resource type
schema and so the strange behavior of our testDiffFn caused some
attributes to not appear at all in the result. With core now more aware,
it "fills in" these missing items before calling, and as a result they
now appear in the diff even with the testDiffFn.

In real code, where helper/schema is constructing diffs, this situation
doesn't arise because the framework always produces schema-compatible
diffs.
2018-10-16 18:49:20 -07:00
Martin Atkins
e850becf64 core: update context plan tests for new module address rendering
The diff stringer now uses the standard serialization of a module address,
so we need to update the golden representations to restore their
associated tests to passing.
2018-10-16 18:49:20 -07:00
Martin Atkins
1a547b5351 core: Rename "count" variables in context plan fixtures
This variable name is now reserved so we can support the count
meta-argument inside module blocks in a later release.
2018-10-16 18:49:20 -07:00
James Bardin
cdc8abdae0 add missing Operation to the Evaluator 2018-10-16 18:48:28 -07:00
James Bardin
d55d623ef0 handle unset computed values in tests
The old testDiffFn used th raw config to dynamically set computed values
in the diff. Since the schema now defines what values should be there,
all test diffs end up with unkown computed values. Filter these out by
looking for a value set to "compute"
2018-10-16 18:48:28 -07:00
Martin Atkins
77bc36abce core: Make context_plan_test failure output more helpful
Lots of the tests were previously only producing the actual result, and
not what was expected. Now we'll show both, to make debugging easier.
2018-10-16 18:48:28 -07:00
Martin Atkins
cc5b1d452f core: NewInstanceInfo now produces correct "id" for data resources
We need to mimic the old API here, and we weren't doing that quite right
since the "data" prefix was missing from data resources.
2018-10-16 18:48:28 -07:00
Martin Atkins
eb54715902 core: fix string rendering of modules in diffs
This was assuming our old practice of a slice starting with the string
"root". We'll normalize here and then stringify the result to ensure that
we get a string consistent with what's used elsewhere.

This is primarily aimed at fixing some of the context plan tests.
2018-10-16 18:48:28 -07:00
Martin Atkins
39dfc9f4e0 core: Fix schema-related issues in context plan tests
Most changes here just introduce some custom schema into the test mocks.
In some cases, a fixture is lightly updated to more modern assumptions.

The test for accessing count.index in a resource block without count set
is removed, because that is no longer valid under the new language
implementation.
2018-10-16 18:48:28 -07:00
Martin Atkins
2b3b6c2407 core: TestContext2Plan_countComputedModule expectedErr
The phrasing of this message changed as part of updating it to use HCL
diagnostics rather than just a string.
2018-10-16 18:48:28 -07:00
James Bardin
e6478599c6 strip unknown values from the resource config
Computed values are now all added to the config value as unknowns. Strip
these out to match the legacy expatiations.
2018-10-16 18:48:28 -07:00
Martin Atkins
d6d655f21d core: Context plan tests expect a new error message for prevent_destroy
This error message has been revised as part of changing it to use
diagnostics instead of an error string directly.
2018-10-16 18:48:28 -07:00
Martin Atkins
5a77045a61 core: Move invalid output context tests to "validate"
This problem should now be caught at validate time rather than plan time,
because we can use the schema to detect the problem before the resource
has been resolved.
2018-10-16 18:48:28 -07:00
Martin Atkins
824986b698 core: use correct provider config address when eval pending resources
The evaluate data source was using a guessed provider configuration
address from configuration in this case, but that isn't necessarily
correct since the resource might actually be associated with a config
inherited from a parent module.

We still need to retain that fallback to config because we are sometimes
asked to evaluate when state is incomplete (like in "terraform console"),
but if possible we'll take the stored provider address from the state
and use that, even if the resource is otherwise "pending".
2018-10-16 18:48:28 -07:00
James Bardin
cae5c2feaa update apply tests 2018-10-16 18:48:28 -07:00
James Bardin
492a3b09fa deposed nodes need the resolved provider too
The provider is looked up by the ResolvedProvider
2018-10-16 18:48:28 -07:00
James Bardin
f8b77030db destroy nodes can't be referenced directly
Destroy nodes were being referenced by their regular paths, which was
causing cycles in the graphs. Destroy nodes can't be referenced directly
in any way, so override the inherited method for a referenceable address.
2018-10-16 18:48:28 -07:00
James Bardin
7137d85522 ignore id when creating diffs
The id field is always computed, and never directly related to a diff.
Since that field is being converted to a regular schema attribute, we
need to handle the behavior in the mocks too.
2018-10-16 18:48:28 -07:00
Martin Atkins
e528fe50e9 core: Misc. updates to the "plan" context tests
Mostly this is about updating ctx.Plan callers to expect diags instead of
err, but also includes a few light updates to test fixtures, and a fix to
testModuleInline.
2018-10-16 18:48:28 -07:00
James Bardin
6bf70427dd lock the provisioner mutex in ProvisionerSchema 2018-10-16 18:48:28 -07:00
James Bardin
cdaeed4f26 a provisioner may not always have a connection 2018-10-16 18:48:28 -07:00
James Bardin
07042a95fa fixing context apply tests 2018-10-16 18:48:28 -07:00
James Bardin
3bd2293152 compare module by normalized path
The onld logic for locating comparing module paths no longer worked, and
we can simplify the comparison by using the addrs.ModuleInstance string.
2018-10-16 18:48:28 -07:00
Martin Atkins
ea727d9918 core: Add mock schemas to the refresh context tests
There are still some other issues with some of these tests right now, but
all the ones that need to have schema should now have it.

It seems that there is a bug with the evaluation of child module input
variables where they can't find their schema even when a mock is provided.
Will attack this in a subsequent commit.
2018-10-16 18:48:28 -07:00
Martin Atkins
8b6ef7c8d3 core: EvalWriteOutput handle dynamic pseudo-type
This should actually have been caught by !val.IsWhollyKnown, since
DynamicVal is always unknown, but something isn't working quite right here
and so for now we'll redundantly check also if it's of the dynamic
pseudo-type, and then revisit cty later to see if there's a real bug
hiding down there.
2018-10-16 18:48:28 -07:00
Martin Atkins
5281afcc07 core: Actually read provider schema for data resources
We were passing the pointer into EvalReadDataDiff here, but because it was
not also passed to EvalGetProvider it was just always nil.
2018-10-16 18:48:28 -07:00
Martin Atkins
01a19e18ea core: Add ProviderAddr to all import context tests
The recent changes for v0.12 have moved the responsibilities around here
so that it's the caller's responsibility to specify the provider address
in all cases, with the real UI (in the "command" package) providing a
suitable default if nothing is specified.

Therefore the tests at _this_ level must all include an explicit provider
address, since these tests are acting as if they _are_ the UI code.
2018-10-16 18:48:28 -07:00
Martin Atkins
2c1ef35965 core: skip resource validation when count is unknown
The approach here is a little hacky, since this edge case applies only to
validate and all of the other evaluateResourceCountExpression callers
don't care about it: we overload the "count" return value as a flag to
allow NodeValidatableResource to allow it to detect this situation and
silently ignore errors in this case.
2018-10-16 18:48:28 -07:00
Martin Atkins
b54342460c core: Check that resource exists in config before evaluating it
We were previously doing this for all of the reference types except this
one. Now we do it for resources and resource instances too, which both
allows us to produce a proper error message when one is missing (rather
than returning an unknown value) and allows us to properly handle the
case where there are no instances yet present in the state (e.g. because
we're in the validate walk) but "count" isn't set, and so a single
unknown value is expected rather than an empty tuple.
2018-10-16 18:48:28 -07:00
Martin Atkins
0c4c7c5a5f core: Context.InitProvider must cache providers using absolute addresses
Previously InitProvider was incorrectly using only the relative address,
which (due to the ambiguity in the string representation of absolute vs.
relative addresses) caused it to always initialize providers in the root
module.

Now we use the absolute address as the key, which then agrees with the
Provider method and ensures that each module gets its own separate
instance of each provider if explicit configuration is present.
2018-10-16 18:48:28 -07:00
Martin Atkins
b8c3b8d45d core: Tolerate missing provider config schema in EvalValidateProvider
This should never happen in real code, but it comes up a lot in test code
where incomplete mock schemas are being used to test with very simple
configurations.
2018-10-16 18:48:28 -07:00
Martin Atkins
e4e3876332 core: EvalValidateProvider must always call provider.Validate
Previously we were skipping all of the validation steps if a provider was
being configured implicitly, and thus had no block in configuration.

This is incorrect, since a provider must still get an opportunity to
configure itself with an empty configuration and possibly reject that
empty configuration with errors.
2018-10-16 18:48:28 -07:00
Martin Atkins
b031e18332 core: Pass ProviderSchema to EvalValidateSelfRef
EvalValidateSelfRef needs schema in order to extract references. It was
previously expecting a *configschema.Block directly, but we weren't
actually passing that in from anywhere except the tests because it's not
available directly in that form during the evaltree for
node_resource_validate.

Instead, we now pass in the whole *ProviderSchema for the associated
provider and have this EvalNode find the schema itself based on the
address. This breaks some of the generality of this node (now only really
works for resource addresses) but that's okay since we have no other
use-case right now anyway.
2018-10-16 18:48:28 -07:00
Martin Atkins
54464e3f93 core: EvalWriteOutput must convert values to string
Our old state format requires all primitive values to be strings. We were
trying to enforce that before, but this didn't work properly because
gocty does not perform automatic type conversions.

Instead, we now convert to string first and then convert the result into
a native Go string afterwards.
2018-10-16 18:48:28 -07:00
Martin Atkins
6a859f1fe3 core: don't use functions in the input-interpolate-var fixture
Using functions here causes this to test more than it needs to, and
requires additional setup in the corresponding tests.
2018-10-16 18:48:28 -07:00
Martin Atkins
0b4ac6d9e3 core: Allow unknown values as variable outputs in EvalWriteOutput
At the moment this must be handled as a special case because we're still
using the old representation of output state, but we do still need to
handle this so that unknown values can properly pass between modules
during validate and plan.
2018-10-16 18:48:28 -07:00
Martin Atkins
d5fda47751 core: Add schema to all of the "Context2Validate" tests
Some of them are still failing for other reasons, but they at least all
now have schema representing their configurations.
2018-10-16 18:48:28 -07:00
Martin Atkins
509c6557dd core: Don't crash EvalValidateProvisioner if no connection block present 2018-10-16 18:48:28 -07:00
Martin Atkins
5b1b564663 core: fix formatting of unexpected diagnostics in context tests 2018-10-16 18:48:28 -07:00
Martin Atkins
7efa010f24 core: Add schema to TestTestContext2Validate_interpolateMap 2018-10-16 18:48:28 -07:00
Martin Atkins
fc7453c073 core: Update tests that use the apply-vars fixture
These now need to include a suitable schema in order to properly process
the configuration.
2018-10-16 18:48:28 -07:00
Martin Atkins
a9a9cdaa15 core: Fix ProcessIgnoreChanges tests
Since our ignoring is now implemented in terms of cty objects and HCL
traversals, rather than flatmap keys, it no longer makes sense to test
for the flatmap detail of keeping the map count in a separate key.

It _does_ make sense to ignore an entire block or map, but that's already
covered by another existing tests for just []string{"resource"} above.
2018-10-16 18:48:28 -07:00
Martin Atkins
39a83b2fd1 core: fix test for buildProviderConfig
This was incorrectly comparing a cty.Value to an hcl.Body. Now we decode
the body first so we can compare two of cty.Value.

Also includes a fix to a stale comment in buildProviderConfig that was no
longer accurate.
2018-10-16 18:48:28 -07:00
Martin Atkins
0a59da418a core: Fix tests for EvalConfigProvider
The interface of this eval node has changed for v0.12, now requiring both
a provider address and the actual provider object.

We also need to give it a working ctx.EvalBlock implementation on the
mock EvalContext, so we just use installSimpleEval here to get our simple
implementation that just knows how to evaluate constant expressions.
2018-10-16 18:48:28 -07:00
Martin Atkins
9833d9991d core: Fix EvalWriteState tests
EvalWriteState now requires a provider address so that it can record the
last-used provider configuration in the state.
2018-10-16 18:48:28 -07:00
Martin Atkins
dc7455887e core: fix tests for debug.go
The plan debug test now needs to provide a schema for the resource types
it uses.
2018-10-16 18:48:28 -07:00
Martin Atkins
0da8fe2087 core: Fix EvalValidateSelfRef tests
We must provide a schema for the synthetic body we use for testing so that
the references can be found within it.
2018-10-16 18:48:28 -07:00
Martin Atkins
4a002bf10b core: EvalValidateSelfRef must catch instances referencing their resources
An instance like aws_instance.foo[0] is not permitted to refer to
aws_instance.foo, since that result contains the individual instance along
with all other instances.
2018-10-16 18:48:28 -07:00
Martin Atkins
fc5a3c5cec core: Fix tests for types in eval_validate.go
A schema is now required for any validation, so these tests now use the
simpleMockProvider function to produce a provider with a simple schema
already configured, and test against that schema.
2018-10-16 18:48:28 -07:00
Martin Atkins
e6049d79b7 core: EvalContextMock code-based mocking of evaluation
EvaluateBlock and EvaluateExpr are often called multiple times in a single
operation, and our usual approach of mocking with static values is a poor
fit for those cases.

To accommodate more complex tests, we allow the test to optionally provide
a callback function to use instead of the static return values.

Since a pretty common need is to just evaluate the given block or
expression in a simple way, we also now have a helper method
installSimpleEval that installs reasonable implementations of these
two methods that can (assuming no other customization) just evaluate
constant expressions, which is sufficient for many tests.
2018-10-16 18:48:28 -07:00
Martin Atkins
8d0bec5728 core: Don't include "type" in connection block schema
The config loader already extracts this during its initial pass and saves
it as a separate field in a configs.Connection value, so requiring it
again here causes confusing errors about this attribute being provided but
not set.
2018-10-16 18:48:28 -07:00
Martin Atkins
f148b26287 core: Fix RefreshGraphBuilder tests
A correct outcome from this test now requires resource type schema to be
available to the graph builder.
2018-10-16 18:48:28 -07:00
Martin Atkins
5e8938de98 core: Remove GraphWalkerPanicwrap, etc
These are some remnants of the shadow graph functionality that was added
to support the graph builder changes in v0.8, but that has since been
removed and so there are no remaining callers for these types and
functions.
2018-10-16 18:48:28 -07:00
Martin Atkins
e1e4762b03 core: Update TestNodeApplyableModuleVariablePath for new address type 2018-10-16 18:48:28 -07:00
Martin Atkins
feff252700 core: fix tests for checkInputVariables
The contract for this function has changed as part of the 0.12
reorganization so that, while before it would accept a partial variables
map if all missing variables were optional, it now expects that the caller
has already collected values for all variables and is just checking them
for type-correctness here.
2018-10-16 18:48:28 -07:00
Martin Atkins
ce41553d56 core: Fix test for NodeDestroyResource.DynamicExpand
Provider addresses are now required for ResourceState, and the string
formatting is now different in the naming of destroy-deposed graph nodes.
2018-10-16 18:48:28 -07:00
Martin Atkins
26e820d829 addrs: Don't return zero-length slice in Reference.Remaining.
Although this rarely matters, making it always be nil when empty makes
deep assertions simpler in tests.

This also includes a minor update to the test in the core package that
first encountered this problem, to improve the quality of its output
on failure.
2018-10-16 18:48:28 -07:00
Martin Atkins
e1bc27ef6c core: ApplyGraphBuilder tests to use the "test" mock provider
These tests now need schema available to run properly, so rather than
crafting a separate schema for each of these we'll just use the one
created by simpleMockProvider, which contains a resource type called
test_object that we'll now use as the basis for all of the tests here.
2018-10-16 18:48:28 -07:00
Martin Atkins
f475b3b931 core: Fix DiffTransformer node addresses
In this codepath we're still using the old "internal" ResourceAddress
parser to parse resource addresses from the diff, but the string being
parsed does not include a module path and so the result is interpreted
as a resource in the root module.

We need to rewrite this to be in the correct module path as part of
converting to the new address representation, in order to get a correct
absolute address.
2018-10-16 18:48:28 -07:00
Martin Atkins
4472cffb35 core: NodeApplyableResourceInstance.References returns its result
A trivial mistake in the rework of this function meant that it was just
discarding its result rather than returning it. It will now return its
result as expected, allowing reference analysis to work for this node
type.
2018-10-16 18:48:28 -07:00
Martin Atkins
5863c581ee core: Fix misleading log message in ProviderTransformer
The final "<resource> provided by <provider>" message was showing the
original selection made by the resource itself rather than the provider
address finally resolved after inheritance, which caused the log to
misrepresent what was actually being created in the graph.
2018-10-16 18:48:28 -07:00
Martin Atkins
f7b8e3b8be core: Fix tests for TestPlanGraphBuilder
Previously this was just stubbing out provider types, but we now need to
include schema for each of the providers and resource types the tests
use in order for the references to be properly detected.

The test fixtures are adjusted slightly here so we can use the
simpleTestSchema as the schema for all of the different blocks in these
tests. The relationships between the resources are still preserved, but
the attributes are renamed to comply with this schema.
2018-10-16 18:48:28 -07:00
Martin Atkins
168354c2e8 core: Tests for scale in and out need to mock EvalExpression
Since expression evaluation is now done via an EvalContext method, and
since these tests use the MockEvalContext, we need to provide a value for
the mock call to return. Previously expression evaluation happened at a
different layer that was not mocked here.
2018-10-16 18:48:28 -07:00
Martin Atkins
bec0f56808 core: Pass components through to the destroy transformers
These transformers both construct temporary graphs using many of the same
transformers used in the apply graph, and properly doing this now requires
access to the providers and provisioners in order to obtain their schemas.

Along with this, we also update the tests here to use the
simpleMockComponentFactory helper to get a mock provider with a schema
already configured, which means we also need to update the test fixtures
and assertions to use the resource type and attributes defined in that
mock factory.
2018-10-16 18:48:28 -07:00
Martin Atkins
1a9ee72ecb core: Additional trace logging in attach resource config transformer 2018-10-16 18:48:28 -07:00
Martin Atkins
71d7a9e480 core: AttachSchemaTransformer fail gracefully with no schema
Having a missing schema is a programming error, but at the time of this
commit we're in the midst of introducing schema all over Terraform and so
there are inevitably some places -- particularly in older unit tests --
where schema isn't yet being provided.

This error allows us to catch those cases and fail gracefully, rather
than panicking further down here when we access t.Components methods.

Also includes some additional logging to aid debugging of this
transformer.
2018-10-16 18:48:28 -07:00
Martin Atkins
b73e4ede5d core: use correct fallback provider address for abstract resource instance
Previously we were attempting to construct a default manually here, and
actually getting it wrong by using the resource type name as a whole
rather than the expected inferrence by prefix.

Now we use the method provided in the addrs package for this purpose,
which implements the standard behavior of shaving off the first
underscore-separated word from the resource type name.
2018-10-16 18:48:28 -07:00
Martin Atkins
f82c192c49 core: mock provider factory functions for testing
Now that any configuration processing requires schema, we need either a
standalone schema or a provider/provisioner configured with one a lot more
often in tests.

To avoid adding loads of extra boilerplate to the tests, these new helper
functions produce objects pre-configured with a schema that should be
useful for a number of different cases, and can be customized further for
more interesting situations.

A lot of our tests can then just use these pre-defined object and
attribute names in their fixtures in situations where the canned schemas
here are good enough.
2018-10-16 18:48:28 -07:00
Martin Atkins
8fabcc0c08 core: Correct test TestOrphanResourceCountTransformer_zeroAndNone
The logic under test has intentionally changed here so that setting count
to any value -- even 1 -- causes Terraform to prefer to keep indexed
instances if both non-indexed and indexed are present.

Previously Terraform treated count = 1 as equivalent to count not set at
all, but we now recognize these as two different situations and treat
count = 1 as the same as any other non-zero count, to ensure that if
count is set conditionally it'll always produce indexed instances, even
if the dynamic expression ends up evaluating to 1.
2018-10-16 18:48:28 -07:00
Martin Atkins
ad6bb4a1d5 core: NodeProvisioner.Name update for new address types
This function was previously checking for a path length greater than one
because the older path format included an always present "root" element
at the start.

We now need to check for a totally-empty list, because otherwise we fail
to add the expected prefix to the front of a path with only one element.

This also includes some adjustments to the related tests and transforms
that do not change behavior but do make the test results easier to
understand and debug.
2018-10-16 18:48:28 -07:00
Martin Atkins
e40e3b9ad8 core: fix inheritence checks in ConfigTreeDependencies
Due to some disagreement about what representation of provider addresses
we were using, the inherited providers map wasn't matching. Now we'll
consistenly use the "compact" form (just the provider name and optional
alias).

Also includes some other tweaks to make this test better-behaved.
2018-10-16 18:48:28 -07:00
Martin Atkins
880d971328 core: TestStateAddModule old-style output
Although the AddModule method now takes a new-style
module address as an argument, internally we're still
shimming it to a []string.

Therefore this test needs to still expect [][]string as
a result, rather tan []addrs.ModuleInstance.
2018-10-16 18:48:28 -07:00
Martin Atkins
3ed396ad5c core: Update ProviderTransformer tests for new ImportTarget interface
Prior to the refactoring to move provider address parsing/selection up
into the frontend, there was some logic here to just-in-time default a
provider config based on the given resource type.

This is now expected to happen at a higher layer, with ImportTarget
expecting an already-valid provider configuration address.

The normal import codepath was already updated with this in mind, but some
of the provider transform tests are using ImportStateTransformer as a
shortcut for getting some resource nodes added to the graph, and so those
tests now need to include a valid provider address in their ImportTarget
values.

Also includes some adjustments to test output to make the tests easier
to debug.
2018-10-16 18:48:28 -07:00
Martin Atkins
b0b1486c46 core: Clarify the MissingProviderTransformer logic
The prior implementation of this function (before the rewrite to use
addrs.AbsProviderConfig values) was relying on some implicit behavior of
our provider address normalization to generate an address in the root
module. Since that wasn't explicit, I introduced a bug here when
introducing the new address type, where I generated an address in the
node's own module, rather than in the root as expected.

This new implementation is functionally equivalent to the prior, but is
written to make the intent more obvious: take _just_ the type from the
node's provider address and create an implicit configuration for it
_in the root module_.

Along with the change in approach, this new implementation also has an
updated documentation comment that better describes its current intent
(previously it was outdated) and hopefully clearer trace logging to
better communicate what it's doing.
2018-10-16 18:48:28 -07:00
Martin Atkins
e4e972db67 core: Fix MissingProviderTransformer matching existing providers
In the change to using addrs types rather than string keys directly here
I incorrectly made this use the _relative_ provider config instead of
the absolute one, causing MissingProviderTransformer to only match
providers defined in the root module (due to ambiguity in the string
representations of these address types).

The rest of this change is improved logging and test output that helped
with debugging this issue.
2018-10-16 18:48:28 -07:00
Martin Atkins
604c347128 core: Update test fixtures for ProviderTransformer tests
Some of these tests were using an outdated form to reference a local
directory as a module. While here, I also updated the provider references
to the new unquoted form, which is preferred from v0.12 onwards.
2018-10-16 18:48:28 -07:00
Martin Atkins
37dfda93ce core: Fix ReferenceTransformer tests
The changes to our GraphNodeReferencable and GraphNodeReferencer
interfaces were not also reflected in our testing structs here, and so
these tests were no longer working.

This updates these implementations to the new required signatures,
adapting the simplified model used in the structs to generate local
variable references, since the reference model no longer uses the
arbitrary strings that this test originally depended on.

We also remove some of the tests here since the functionality they were
testing no longer applies: inter-module dependencies are now handled by
the graph nodes producing extra ReferencableAddrs, and the "backup" form
is no longer used because our new address model is able to distinguish
resources from resource instances without the need for the magical
backup reference forms we previously used.
2018-10-16 18:48:28 -07:00
Martin Atkins
a6ca01f49d core: Fix TargetsTransformer tests
Resolution of dependencies automatically from expressions now requires
a schema to be available. To avoid the need to provide mock schemas for
all of these different resource types, we instead use the depends_on
argument here to mark the dependencies explicitly.
2018-10-16 18:48:28 -07:00
Martin Atkins
17d8ec6b6f core: NodeAbstractResource correct default result from ProvidedBy
Previously we had a bug where we'd use the resource type name instead of
the provider name. Now we use the DefaultProviderConfig helper function
to extract the provider name from the resource type name.
2018-10-16 18:48:28 -07:00
Martin Atkins
c8f186b303 core: TestTransitiveReductionTransformer needs a schema
The ReferenceTransformer can't do its work here unless our resources have
schema attached, since otherwise it doesn't know which attributes to
search to find references.
2018-10-16 18:48:28 -07:00
Martin Atkins
7188cabdaf core: Testing helpers for building provider schemas
Now that a schema is required to decode resource configuration, a lot more
tests than before need access to mock providers in order to get a schema.
These new helpers allow a mock provider with a schema to be constructed
concisely.
2018-10-16 18:48:28 -07:00
Martin Atkins
c58bdc6390 core: fix the tests in variables_test.go
This test required some adaptation for the new variable handling model in
the HCL2-oriented codepaths, where more of the handling is now done in the
"command" package.

The initial reorganization of this test was not correct, so this is a
further revision to adapt the original assertions to the new model as much
as possible, removing a few tests that no longer make sense in the new
world.
2018-10-16 18:48:28 -07:00
Martin Atkins
b33a970b62 core: fix test fixtures for HCL2 syntax idiom
A number of our test fixtures were previously using the non-idiomatic form
of including a single child attribute all on one line with the block
header and bounding braces.

This non-idiomatic form is an error in HCL2, and hclfmt has always "fixed"
it to the expected form of each attribute being on a line of its own, and
so here we just update all the affected test fixtures to canonical form
(using hclfmt), allowing them to be parsed as intended.

Since the these entire files were processed with hclfmt, there are some
other unrelated style changes included in situations where the file
layouts were non-idiomatic in other ways.
2018-10-16 18:46:46 -07:00
Martin Atkins
c82e3ec92f core: even more nil checks to catch missing objects
These are all things that ought to be present in normal use but can end up
being nil in incorrect tests. Test debugging is simpler if these things
return errors gracefully, rather than crashing.
2018-10-16 18:46:46 -07:00
Martin Atkins
26c1584317 core: EvalConfigProvider shouldn't crash if its provider isn't set
While there's no good reason for this to happen in practice, it can arise
in tests if mocks aren't set up quite right, and so we'll catch it and
report it nicely to make test debugging a little easier.
2018-10-16 18:46:46 -07:00
Martin Atkins
7dacb6b6a8 core: Don't crash if EvaluateBlock returns nil during validation
There's actually no good reason for this to happen in the normal case, but
it can happen reasonably easy if a test doesn't properly configure the
MockEvalContext, and so having this check here makes test debugging a
little easier.
2018-10-16 18:46:46 -07:00
Martin Atkins
69d2fa0c15 terraform: fix crasher in TestProcessIgnoreChanges
Since this test case is using t.Run, it must be sure to pass the nested
*testing.T over to the testDiffs function or else calls to t.Fatal will
crash the whole test process, since they would otherwise be called on the
wrong instance of *testing.T.
2018-10-16 18:46:46 -07:00
Martin Atkins
0dd7677d1f core: fail gracefully if provider schemas aren't available
The only reason these cases are arising right now is because we have tests
that haven't yet been updated to properly support schema, but it can't
hurt to add some robustness here to reduce the risk of real crashes.
2018-10-16 18:46:46 -07:00
Martin Atkins
4a21b763aa core: Get tests compiling again
After the refactoring to integrate HCL2 many of the tests were no longer
using correct types, attribute names, etc.

This is a bulk update of all of the tests to make them compile again, with
minimal changes otherwise. Although the tests now compile, many of them
do not yet pass. The tests will be gradually repaired in subsequent
commits, as we continue to complete the refactoring and retrofit work.
2018-10-16 18:46:46 -07:00
Martin Atkins
20f6de735d core: Update MockResourceProvisioner to include GetConfigSchema
This method is now required for the ResourceProvisioner interface.
2018-10-16 18:46:46 -07:00
Martin Atkins
73053eb5ef core: Context.Eval method
Some of the objects that are referencable from expressions are transient
values computed only during a graph walk, and not persisted in state. In
order to support arbitrary evaluation of expressions, such as in the
"terraform console" CLI command, it's necessary to be able to evaluate
these values before we start evaluating.

This new Eval method achieves this by performing a special graph walk that
ignores resources (except for dependency resolution) and just focuses on
evaluating all of these transient values, before returning an evaluation
scope that can then resolve expressions in terms of that result.

This replaces the Context.Interpolator method, which was fraught with
various issues due to it not properly priming the state before evaluating.
2018-10-16 18:46:46 -07:00
Martin Atkins
630e3b4be2 core: evaluation for HCL2-flavored reference expressions
Here we replace the stub implementations in evaluationStateData with real
implementations that are based on their equivalents in the old
Interpolator type.

The behavior here is a little different due to the different semantics
expected under HCL2, but the principle remains the same: the main
references are resolved from the state, using config for validation
in order to produce some helpful error messages.
2018-10-16 18:46:46 -07:00
Martin Atkins
25e3ac56d2 core: Wire back in module input variables to the evaluator
I took some missteps here while doing the initial refactor for HCL2 types.
This restores the map of maps that retains all of the variable values, and
then makes it available to the evaluator.
2018-10-16 18:46:46 -07:00
Martin Atkins
4b5868f653 core: BuiltinEvalContext constructs evaluationStateData itself
Previously our evaluationStateData object was constructed inside
Evaluator.Scope, but this was awkward because all of the fields inside it
need to be populated from BuiltinEvalContext fields, and so the signature
of Evaluator.Scope kept growing new arguments over time.

Instead, we reassign the responsibilities here so that Evaluator.Scope
takes an already-constructed lang.Data, and then teach BuiltinEvalContext
to build this object itself from its own internal values.
2018-10-16 18:46:46 -07:00
Martin Atkins
b0435cd533 core: ProviderConfigTransformer properly locates parent module
Due to a logic error here we were trying to find our our module's parent
as a descendent of itself, rather than as a descendent of the root. It
turns out that we can do this even more simply by just accessing the
Parent field on the given config, avoiding the need to traverse the tree
down from the root at all.

While here, this also switches to using the path.Call helper method rather
than manually slicing the path array, since this better communicates our
intent.
2018-10-16 18:46:46 -07:00
Martin Atkins
d4cfe85361 core: pass InstanceKey to EvaluateBlock
This gives us the value we need to evaluate a "count.index" reference, and
later also the equivalent for the "for_each" argument once implemented.
2018-10-16 18:46:46 -07:00
Martin Atkins
d4285dd27f core: Attach resource and provider config schemas during graph build
This is a little awkward since we need to instantiate the providers much
earlier than before. To avoid a lot of reshuffling here we just spin each
one up and then immediately shut it down again, letting our existing init
functionality during the graph walk still do the main initialization.
2018-10-16 18:46:46 -07:00
Martin Atkins
ca67899827 core: NewLegacyResourceInstanceAddress correct handling of addrs.NoKey
Previously we were setting zero here, but the correct way to represent
"no index" in a *ResourceAddress is to set the index to -1.
2018-10-16 18:46:46 -07:00
Martin Atkins
a28c93c3ce core: render variables, locals and outputs nicely in "terraform graph" 2018-10-16 18:46:46 -07:00
Martin Atkins
226d66bac8 core: include "root" element when constructing legacy ModuleState path
We've not yet adjusted any of the state structs to reflect our new address
types because they are used with encoding/json to produce our state file
format, but the shimming here previously was incorrect because it failed
to include the special "root" string that's always required at element
zero of a module path in the state.
2018-10-16 18:46:46 -07:00
Martin Atkins
c937c06a03 terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.

The three main goals here are:
- Use the configuration models from the "configs" package instead of the
  older models in the "config" package, which is now deprecated and
  preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
  new "lang" package, instead of the Interpolator type and related
  functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
  rather than hand-constructed strings. This is not critical to support
  the above, but was a big help during the implementation of these other
  points since it made it much more explicit what kind of address is
  expected in each context.

Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.

I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-10-16 18:46:46 -07:00
Martin Atkins
a09498a8a3 core: load a provider's schema at initialization
This is currently not very ergonomic due to the API exposed by providers.
We'll smooth this out in a later change to improve the provider API, since
we know we always want the entire schema.
2018-10-16 18:44:26 -07:00
Martin Atkins
c07b0a7806 configs: Re-unify the ManagedResource and DataResource types
Initially the intent here was to tease these apart a little more since
they don't really share much behavior in common in core, but in practice
it'll take a lot of refactoring to tease apart these assumptions in core
right now and so we'll keep these things unified at the configuration
layer in the interests of minimizing disruption at the core layer.

The two types are still kept in separate maps to help reinforce the fact
that they are separate concepts with some behaviors in common, rather than
the same concept.
2018-10-16 18:44:26 -07:00
Martin Atkins
4ed06a9227 terraform: HCL2-flavored module dependency resolver
For the moment this is just a lightly-adapted copy of
ModuleTreeDependencies named ConfigTreeDependencies, with the goal that
the two can live concurrently for the moment while not all callers are yet
updated and then we can drop ModuleTreeDependencies and its helper
functions altogether in a later commit.

This can then be used to make "terraform init" and "terraform providers"
work properly with the HCL2-powered configuration loader.
2018-10-16 18:44:26 -07:00
Martin Atkins
ebafa51723 command: Various updates for the new backend package API
This is a rather-messy, complex change to get the "command" package
building again against the new backend API that was updated for
the new configuration loader.

A lot of this is mechanical rewriting to the new API, but
meta_config.go and meta_backend.go in particular saw some major
changes to interface with the new loader APIs and to deal with
the change in order of steps in the backend API.
2018-10-16 18:44:26 -07:00
Alex Somesan
00822c82c0 Remove a leftover test log file 2018-09-11 17:02:32 +02:00
James Bardin
cfefeec926 walkDestroy is a form of "apply"
When computing the count value, make sure to include walkDestroy with
walkApply, as the former is only a special case of the latter.

When applying a saved plan, the computed count values are lost and we
can no longer query the state for those values. The apply walk was
already considered in the `resourceCountMax` function, but the destroy
walk was not.  This worked when destroying in a single operation
("terraform destroy"), since the state would still be updated with the
latest counts from the plan.
2018-04-10 11:46:29 -04:00
James Bardin
2f8532407a
Merge pull request #17071 from MarkFreebairn/Issue-16360-empty-target-points-to-all-resources-in-state
command: Fix #16360 - Fail fast if -target is specified but empty
2018-04-05 14:19:32 -04:00
James Bardin
060a3bc6d2
Merge pull request #17748 from reancloud/winrm-ntlm
Support NTLM for WinRM communicators.
2018-04-05 09:44:11 -04:00
James Bardin
f25718b065
Merge pull request #17768 from hashicorp/jbardin/unused-destroy-outputs
unused outputs in a destroy should be pruned
2018-04-04 15:20:12 -04:00