Commit Graph

33 Commits

Author SHA1 Message Date
James Bardin
46dbb3dde5 use Dependencies to connect creator and destroyer
The DestroyEdgeTransformer cannot determine ordering from the graph when
the destroyers are from orphaned resources, because there are no
references to resolve. The new stored Dependencies provides what we need
to connect the instances in this case.

We also add the StateDependencies method directly in the
GraphNodeResourceInstance interface, since all instances already
implement this, and we don't need another optional interface to check.

The old code in DestroyEdgeTransformer may no longer be needed in the
long run, but that can be determined separately, since too many of the
tests start with an incomplete state and rely on the Dependencies being
determined from the configuration alone.
2019-11-07 17:49:03 -05:00
James Bardin
42bb4a644c make use of the new state Dependencies
Make use of the new Dependencies field in the instance state.

The inter-instance dependencies will be determined from the complete
reference graph, so that absolute addresses can be stored, rather than
just references within a module. The Dependencies are added to the node
in the same manner as state, i.e. via an "attacher" interface and
transformer.  This is because dependencies are calculated from the graph
itself, and not from the config.
2019-11-07 17:49:03 -05:00
Pam Selle
7d905f6777 Resource for_each 2019-07-22 10:51:16 -04:00
Kristin Laemmert
9869fbc5c4
core: don't panic in NodeAbstractResourceInstance References() ()
* core: don't panic in NodeAbstractResourceInstance References()

It is possible for s.Current to be nil. This was hard to reproduce, so
the root cause is still unknown, but we can guard against the symptom.

* add log statement
2019-05-28 17:27:38 -04:00
Martin Atkins
b1ed146931 core: Attach schemas to nodes created by ResourceCountTransformer
Previously we were only doing this in the case where count wasn't set at
all.
2018-12-07 17:05:36 -08:00
Martin Atkins
168d84b3c4 core: Make resource type schema versions visible to callers
Previously we were fetching these from the provider but then immediately
discarding the version numbers because the schema API had nowhere to put
them.

To avoid a late-breaking change to the internal structure of
terraform.ProviderSchema (which is constructed directly all over the
tests) we're retaining the resource type schemas in a new map alongside
the existing one with the same keys, rather than just switching to
using the providers.Schema struct directly there.

The methods that return resource type schemas now return two arguments,
intentionally creating a little API friction here so each new caller can
be reminded to think about whether they need to do something with the
schema version, though it can be ignored by many callers.

Since this was a breaking change to the Schemas API anyway, this also
fixes another API wart where there was a separate method for fetching
managed vs. data resource types and thus every caller ended up having a
switch statement on "mode". Now we just accept mode as an argument and
do the switch statement within the single SchemaForResourceType method.
2018-11-27 15:53:54 -08:00
Martin Atkins
77e50894d5 core: Don't create self-references in state
Since StateReferences was implemented on NodeAbstractResource rather than
NodeAbstractResourceInstance it wasn't properly detecting references to
the same instance as self-references.

Now that we are using "seen" to filter out duplicates we can also simplify
how we handle these self-references by just pretending we saw them before
we even start the loop.

This change is confirmed by
TestContext2Apply_provisionerMultiSelfRefSingle
2018-10-16 19:14:11 -07:00
Martin Atkins
bd6d3a638a core: Simplify output refs to module call refs in StateReferences
As previously mentioned in a comment here, versions of Terraform prior to
0.12 would store references to module outputs as just references to the
module as a whole in the state. It's not really clear why, but we wanted
to preserve this behavior for 0.12.

The previous implementation actually failed to do so, in spite of the
comment, so this commit fixes it to actually do what the comment
originally claimed.

In a later release we might remove this special case and just depend
directly on outputs where possible, since that'd allow us to produce a
more precise dependency graph for destroy actions, but when we do that
we'll first need to confirm that there isn't a good reason for the
original exception here.
2018-10-16 19:14:11 -07:00
Martin Atkins
86d2716679 core: Normalize results from NodeAbstractResource.StateReferences
The underlying References function includes duplicates and returns refs
in the order they appeared in source (approximately), but after we reduce
to just the raw addresses it's better to dedupe and return in a
predictable order.
2018-10-16 19:14:11 -07:00
Martin Atkins
40f00d8db5 core: StateReferences method should return []addrs.Referenceable
We no longer use strings to represent addresses, so this method was a
leftover outlier from previous refactoring efforts.

At this time the result is not actually being used due to the state type
refactoring, which is a bug we'll address in a subsequent commit.
2018-10-16 19:14:11 -07:00
Martin Atkins
a3403f2766 terraform: Ugly huge change to weave in new State and Plan types
Due to how often the state and plan types are referenced throughout
Terraform, there isn't a great way to switch them out gradually. As a
consequence, this huge commit gets us from the old world to a _compilable_
new world, but still has a large number of known test failures due to
key functionality being stubbed out.

The stubs here are for anything that interacts with providers, since we
now need to do the follow-up work to similarly replace the old
terraform.ResourceProvider interface with its replacement in the new
"providers" package. That work, along with work to fix the remaining
failing tests, will follow in subsequent commits.

The aim here was to replace all references to terraform.State and its
downstream types with states.State, terraform.Plan with plans.Plan,
state.State with statemgr.State, and switch to the new implementations of
the state and plan file formats. However, due to the number of times those
types are used, this also ended up affecting numerous other parts of core
such as terraform.Hook, the backend.Backend interface, and most of the CLI
commands.

Just as with 5861dbf3fc49b19587a31816eb06f511ab861bb4 before, I apologize
in advance to the person who inevitably just found this huge commit while
spelunking through the commit history.
2018-10-16 19:11:09 -07:00
Martin Atkins
479c6b2466 move "configschema" from "config" to "configs"
The "config" package is no longer used and will be removed as part
of the 0.12 release cleanup. Since configschema is part of the
"new world" of configuration modelling, it makes more sense for
it to live as a subdirectory of the newer "configs" package.
2018-10-16 18:50:29 -07:00
Martin Atkins
1761faa29c core: schema attach interfaces are not mutually-exclusive
It was incorrect to use a type switch to detect the optional schema
attachment interfaces, because they are not mutually-exclusive: resource
nodes implement both GraphNodeAttachResourceSchema and
GraphNodeAttachProvisionerSchema.

This fixes a number of test regressions around dependency analysis in
"provisioner" blocks.
2018-10-16 18:49:20 -07:00
James Bardin
a5c3c454ab filter self references in NodeApplyableResourceIns
Don't save self-references in the state dependencies for a resource.
2018-10-16 18:49:20 -07:00
Martin Atkins
dd6b171f62 core: Make provisioner schemas available to plan resource instance nodes
This requires making the "components" object available to the resource
node so it can be used during DynamicExpand. It also involved splitting
the provisioner schema attachment into a separate interface from
GraphNodeProvisionerConsumer so that it can now be handled within
AttachSchemaTransformer, along with all of the other schema attachment
steps.
2018-10-16 18:49:20 -07:00
James Bardin
c6787d0266 update test fixtures
Update test fixtures to work in our new world.
This is mostly changing out attribute names for those in the schema,
adding Providers to states, and updating the test-fixture
configurations.
2018-10-16 18:49:20 -07:00
Martin Atkins
b73e4ede5d core: use correct fallback provider address for abstract resource instance
Previously we were attempting to construct a default manually here, and
actually getting it wrong by using the resource type name as a whole
rather than the expected inferrence by prefix.

Now we use the method provided in the addrs package for this purpose,
which implements the standard behavior of shaving off the first
underscore-separated word from the resource type name.
2018-10-16 18:48:28 -07:00
Martin Atkins
17d8ec6b6f core: NodeAbstractResource correct default result from ProvidedBy
Previously we had a bug where we'd use the resource type name instead of
the provider name. Now we use the DefaultProviderConfig helper function
to extract the provider name from the resource type name.
2018-10-16 18:48:28 -07:00
Martin Atkins
c82e3ec92f core: even more nil checks to catch missing objects
These are all things that ought to be present in normal use but can end up
being nil in incorrect tests. Test debugging is simpler if these things
return errors gracefully, rather than crashing.
2018-10-16 18:46:46 -07:00
Martin Atkins
d4285dd27f core: Attach resource and provider config schemas during graph build
This is a little awkward since we need to instantiate the providers much
earlier than before. To avoid a lot of reshuffling here we just spin each
one up and then immediately shut it down again, letting our existing init
functionality during the graph walk still do the main initialization.
2018-10-16 18:46:46 -07:00
Martin Atkins
c937c06a03 terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.

The three main goals here are:
- Use the configuration models from the "configs" package instead of the
  older models in the "config" package, which is now deprecated and
  preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
  new "lang" package, instead of the Interpolator type and related
  functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
  rather than hand-constructed strings. This is not critical to support
  the above, but was a big help during the implementation of these other
  points since it made it much more explicit what kind of address is
  expected in each context.

Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.

I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-10-16 18:46:46 -07:00
James Bardin
72d4e15c47 ProvidedBy return value is a single string
Clean up ProvidedBy, which doesn't need to be a slice.
2017-11-06 14:27:01 -05:00
James Bardin
a14fd0344c WIP reference providers by full name
This turned out to be a big messy commit, since the way providers are
referenced is tightly coupled throughout the code. That starts to unify
how providers are referenced, using the format output node Name method.

Add a new field to the internal resource data types called
ResolvedProvider. This is set by a new setter method SetProvider when a
resource is connected to a provider during graph creation. This allows
us to later lookup the provider instance a resource is connected to,
without requiring it to have the same module path.

The InitProvider context method now takes 2 arguments, one if the
provider type and the second is the full name of the provider. While the
provider type could still be parsed from the full name, this makes it
more explicit and, and changes to the name format won't effect this
code.
2017-11-02 15:00:06 -04:00
James Bardin
b45b6a5c20 remove duplicates in Dependencies
duplicate entries could end up in "depends_on" in the state, which could
possible lead to erroneous state comparisons. Remove them when walking
the graph, and remove existing duplicates when pruning the state.
2017-04-08 15:37:15 -04:00
Mitchell Hashimoto
a1ec81964b
terraform: destroy ordering needs to handle destroy provisioner edges
This ensures that things aren't destroyed before their values are used.
2017-02-17 14:29:22 -08:00
Mitchell Hashimoto
8ed9bdfedc
terraform: node referenceable name from state shuldn't contain path
Fixes 

I'm **really** surprised this didn't come up earlier.

When only the state is available for a node, the advertised
referenceable name (the name used for dependency connections) included
the module path. This module path is automatically prepended to the
name. This means that probably every non-root resource for state-only
operations (destroys) didn't order properly.

This fixes that by omitting the path properly.

Multiple tests added to verify both graph correctness as well as a
higher level context test.

Will backport to 0.8.x
2017-02-07 20:14:38 -08:00
Mitchell Hashimoto
d59725e9fd
terraform: convert StateDeps to use new structs 2017-01-26 20:47:20 -08:00
Mitchell Hashimoto
9197422881
terraform: new graph nodes implement Dotter 2016-12-02 22:26:40 -05:00
Mitchell Hashimoto
19350d617d
terraform: references can have backups
terraform: more specific resource references

terraform: outputs need to know about the new reference format

terraform: resources w/o a config still have a referencable name
2016-11-08 13:59:30 -08:00
Mitchell Hashimoto
ab4ebcc5c7
terraform: TargetsTransformer on destroy plan
This enables targeting to work properly on planning destroys
2016-10-22 12:12:30 -07:00
Mitchell Hashimoto
aaee4df363
terraform: working on enabling CBD, some cycles 2016-10-19 13:38:53 -07:00
Mitchell Hashimoto
4aa84a2071
terraform: CBD makes the proper edge connections for dependent resources 2016-10-19 13:38:53 -07:00
Mitchell Hashimoto
6622ca001d
terraform: abstract resource nodes 2016-10-19 13:38:53 -07:00