I missed this on my first attempt to write this document. Consequently
we're currently depending on a version of HCL which uses Unicode 9, and
that's significantly lagging behind everything else which is currently on
Unicode 13.
My goal of adding these docs then is to remind us to update HCL to Unicode
15 once we're updating everything else to Unicode 15 with the Go 1.20
release, assuming that the Go team completes that Unicode upgrade as
currently planned.
This opts to inline document these intentional design decisions in the protocol definition as a catch-all for it not being documented elsewhere.
Protocol Buffers files updated via:
```shell
make protobuf
```
There are small typos in:
- docs/planning-behaviors.md
- docs/resource-instance-change-lifecycle.md
- website/README.md
Fixes:
- Should read `exhaustive` rather than `exhastive`.
- Should read `representation` rather than `repesentation`.
- Should read `preferably` rather than `preferrably`.
- Should read `absence` rather than `absense`.
The previous version of this document was produced in haste in order to
support the development of the new provider framework, and so it focused
only on the most important details and left some of the operations totally
unmentioned.
This new version aims to capture the full set of managed-resource-related
provider operations, documenting when Terraform Core will call them and
what the provider ought to do in order to meet Terraform Core's
expectations for a valid response.
This new version does still assume a certain amount of knowledge on the
part of the reader about broadly what Terraform does from a user
perspective and what role providers play in that process. Perhaps a future
revision will include some additional background context as well, but
this is a snapshot of what I had time to do today between other work and
so for now I focused on presenting the remaining operations in a similar
amount of detail to what was here before.
For a while now we've had information equivalent to this in various
internal documents that we've referred to when designing features such as
config-driven refactoring, the "replace" planning option, and so forth.
However, so far we've not put that information in any sort of durable
public place that we can easily find and refer to when having design
discussions on GitHub and similar.
This is therefore an attempt to capture a summary of the three main design
patterns we've identified for planning-related behaviors, with a few
motivating examples of each one, in the hope that this will be a good
reference and some helpful inspiration for future design work.
It's intentionally not totally comprehensive of all planning behaviors
both because that would duplicate the end-user-oriented documentation and
because it would be burdensome to keep updating this document each time we
add anything new which might fit into these categories. However, we might
add a later feature to this document if it illustrates a new take or
different perspective on one of these patterns.
Reference: https://github.com/hashicorp/terraform/issues/30373
This change forward ports the `legacy_type_system` boolean fields in the `ApplyResourceChange.Response` and `PlanResourceChange.Response` messages that existed in protocol version 5, so that existing terraform-plugin-sdk/v2 providers can be muxed with protocol version 6 providers (e.g. terraform-plugin-framework) while also taking advantage of the newer protocol features. This functionality should not be used by any providers or SDKs except those built with terraform-plugin-sdk.
Updated via:
```shell
cp docs/plugin-protocol/tfplugin6.1.proto docs/plugin-protocol/tfplugin6.2.proto
# Copy legacy_type_system fields from tfplugin5.2.proto into ApplyResourceChange.Response and PlanResourceChange
rm internal/tfplugin6/tfplugin6.proto
ln -s ../../docs/plugin-protocol/tfplugin6.2.proto internal/tfplugin6/tfplugin6.proto
go run tools/protobuf-compile/protobuf-compile.go `pwd`
# Updates to internal/plugin6/grpc_provider.go
```
As explained in the changes: The 'enhanced' backend terminology, which
only truly pertains to the 'remote' backend with a single API (Terraform
Cloud/Enterprise's), has been found to be a confusing vestige which need
only be explained in the context of the 'remote' backend.
These changes reorient the explanation(s) of backends to pertain more
directly to their primary purpose, which is storage of state snapshots
(and not implementing operations).
That Terraform operations are still _implemented_ by the literal
`Backend` and `Enhanced` interfaces is inconsequential a user of
Terraform, an internal detail.
Minor version increase to deprecate min_items and max_items in nested
types.
Nested types have MinItems and MaxItems fields that were inherited from
the block implementation, but were never validated by Terraform, and are
not supported by the HCL decoder validations. Mark these fields as
deprecated, indicating that the SDK should handle the required
validation.
Releasing a new major protocol version requires coordination between a
few different projects and codebases, and it's easy to forget a step.
This commit introduces a doc to keep track of these steps, making it
less likely one will be omitted or forgotten.
Previously this is a task that I've just taken on using my own knowledge
of how these parts fit together, but that's not at all a sustainable
situation, and so here I've just documented the steps I've followed in for
previous Unicode version upgrades, and thus which I'd expect to follow
for future Unicode version upgrades too.
I hope this is also a somewhat-interesting overview of how Terraform makes
use of Unicode, even for folks who are just working on the other relevant
features of Terraform and not specifically trying to change Terraform's
Unicode support.
Since I'm the primary maintainer of two of the dependencies involved in
this it seems likely that following this process will still involve me
in _some_ capacity, but hopefully only necessarily in the form of
reviewing PRs and cutting new releases of those libraries.
Update the full-replacement example graph to show the transitive
dependency that is required for the destroy-then-update case. Add
another example describing the case where we reduce the graph to
only an update and replace and the dependency on the destroy node
remains.
* providers.Interface: rename ValidateDataSourceConfig to
ValidateDataResourceConfig
This PR came about after renaming ValidateResourceTypeConfig to
ValidateResourceConfig: I now understand that we'd called it the former
instead of the latter to indicate that the function wasn't necessarily
operating on a resource that actually exists. A possibly-more-accurate
renaming of both functions might then be ValidateManagedResourceConfig
and ValidateDataResourceConfig.
The next commit will update the protocol (v6 only) as well; these are in
separate commits for reviewers and will get squashed together before
merging.
* extend renaming to protov6
* Update Godoc links from godoc.org to pkg.go.dev
* Update reference to renamed GraphNodeResource interface
Ref hashicorp/terraform#24389
* Update dead links; minor formatting adjustments
* Add FIXME item following deprecation of EvalNode
This is the first commit for plugin protocol v6. This is currently
unused (dead) code; future commits will add the necessary conversion
packages, extend configschema, and modify the providers.Interface.
The new plugin protocol includes the following changes:
- A new field has been added to Attribute: NestedType. This will be the
key new feature in plugin protocol v6
- Several massages were renamed for consistency with the verb-noun
pattern seen in _most_ messages.
- The prepared_config has been removed from PrepareProviderConfig
(renamed ValidateProviderConfig), as it has never been used.
- The provisioner service has been removed entirely. This has no impact
on built-in provisioners. 3rd party provisioners are not supported by
the SDK and are not included in this protocol at all.
Although we have in this same directory the protocol buffers schemas for
the static parts of the provider wire protocol, many of the protocol
messages include DynamicValue messages that are presented in a nested
dynamic serialization format.
That format was previously not documented, and was thus defined only by
the implementation. Terraform happens to use a third-party library to
implement parts of this encoding, which means that the rules were even
harder to track down from reading the code.
Regardless of how Terraform happens to implement its serialization and
deserialization of DynamicValue, it's the wire format that is contractual
and so this document is an implementation-agnostic description of the
mapping rules for serializing any Terraform Language value by reference to
a provider-defined schema.
All future changes to the rules described in here must be backward
compatible until protocol major version 6, which is not currently planned.
Implement a new provider_meta block in the terraform block of modules, allowing provider-keyed metadata to be communicated from HCL to provider binaries.
Bundled in this change for minimal protocol version bumping is the addition of markdown support for attribute descriptions and the ability to indicate when an attribute is deprecated, so this information can be shown in the schema dump.
Co-authored-by: Paul Tyng <paul@paultyng.net>
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.
This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.
For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
This is an initial partial description of the plugin protocol focused
mainly on explaining the purpose of the .proto files. In subsequent
updates we will also document the negotiation protocol, etc.
For this first pass the goal was just to publish some information that was
already available in an internal design document so that it's visible to
SDK implementers. It is focused on the .proto versioning strategy because
that was the main topic of the internal design documentation this was
based on.
Add versioned tfplugin proto files to the docs directory, for easier
reference. The latest version starts as a symlink to the current
file used for generated the tfplugin package in ./internal/tfplugin5.
When changing the protocol version, the old file must be copied to
./docs/plugin-protocol/, and a new symlink created for the latest
version.
This doc was originally written in the middle of Terraform v0.12
development while some refactoring was already in progress. Here we update
those parts to refer to where the relevant functionality finally landed,
and also update some things that we did not anticipate changing at the
time but ended up having to be changed during the v0.12 development
anyway.
For a while now we've been gathering some codebase-level docs that felt
out-of-place on the main Terraform website (since they are about the
implementation, not usage) but we had no existing suitable place to put
them.
In order to make this information more available (and, hopefully, more
likely to stay up-to-date as we change things), here we'll establish the
"docs" directory as a place to keep documentation aimed at those who are
working on code changes to the Terraform Core codebase.
User-oriented docs should never appear in this directory. The Terraform
website is always the better place for those. The set of docs here is
rudimentary to start and we'll see if it makes sense to expand and
reorganize it over time based on the experience with having these initial
docs available.
