From 19a994ee7f997349975a6d2518a1137bf3dd4ff3 Mon Sep 17 00:00:00 2001 From: Janos <86970079+janosdebugs@users.noreply.github.com> Date: Thu, 14 Mar 2024 15:05:05 +0100 Subject: [PATCH] Documentation updates for 1.7.0-alpha1 (state encryption) (#1396) Signed-off-by: Janos <86970079+janosdebugs@users.noreply.github.com> --- docs/state_encryption.md | 32 ------------------- internal/encryption/README.md | 14 ++++++++ internal/encryption/method/README.md | 26 +++++++++++++++ internal/encryption/method/config.go | 21 ++++++++++++ internal/encryption/method/descriptor.go | 20 ++++++++++++ internal/encryption/method/method.go | 28 ---------------- website/docs/language/state/encryption.mdx | 1 - .../examples/encryption/configuration.ps1 | 15 +++------ .../examples/encryption/configuration.sh | 15 +++------ .../examples/encryption/configuration.tf | 15 +++------ .../state/examples/encryption/enforce.tf | 11 +++---- .../state/examples/encryption/fallback.tf | 11 ++----- .../encryption/fallback_from_unencrypted.tf | 2 +- .../encryption/fallback_to_unencrypted.tf | 2 +- .../encryption/terraform_remote_state.tf | 4 +-- 15 files changed, 106 insertions(+), 111 deletions(-) create mode 100644 internal/encryption/README.md create mode 100644 internal/encryption/method/README.md create mode 100644 internal/encryption/method/config.go create mode 100644 internal/encryption/method/descriptor.go diff --git a/docs/state_encryption.md b/docs/state_encryption.md index 96e1de2593..e17397143e 100644 --- a/docs/state_encryption.md +++ b/docs/state_encryption.md @@ -307,35 +307,3 @@ type Config struct { Key string `hcl:"key"` } ``` - -### OpenTofu integration - -Currently, the OpenTofu code is in large parts procedural and has globally scoped state. Launching a second instance of OpenTofu is impossible. As it is a much-requested feature to embed OpenTofu as a library, this will need to change in the future. Therefore, the integration of the library should do its best to not introduce more global variables if possible. - -The implementation may avail itself of either a singleton, or choose to pass the Encryption interface along several function calls. Both have benefits and drawbacks. However, singletons make parallelized testing very difficult and should therefore be avoided in tests as much as possible. - -#### Singleton - -A singleton takes an otherwise instance-based struct and stores it in a global variable. As such, it can carry information across otherwise procedural code. - -**Pros:** - -* Simpler access to the singleton from calling code -* Less refactoring as the singleton is either available or not. - -**Cons:** - -* Harder to trace when/where the configuration is initialized. -* Easy to introduce new code paths that access the singleton before it is available. - -#### Passed Instance - -This approach requires changing large parts of the OpenTofu code in order to pass along the `Encryption` object. This can cause additional bugs and seduce the inexperienced coder into introducing a super-object (also often referred to as context) to hold everything. Instead, a more granular approach would be desirable, but may not be possible given the state of the code. - -**Pros:** - -* Easy to trace where a given encryption instance comes from -* Hard to introduce new code paths without passing the correct encryption interface. - -Cons: -* More in-depth refactoring is required / more of the codebase edited in this work diff --git a/internal/encryption/README.md b/internal/encryption/README.md new file mode 100644 index 0000000000..f52dead0ed --- /dev/null +++ b/internal/encryption/README.md @@ -0,0 +1,14 @@ +# OpenTofu State and Plan encryption + +> [!WARNING] +> This file is not an end-user documentation, it is intended for developers. Please follow the user documentation on the OpenTofu website unless you want to work on the encryption code. + +This folder contains the code for state and plan encryption. For a quick example on how to use this package, please take a look at the [example_test.go](example_test.go) file. + +## Structure + +The current folder contains the top level API. It requires a registry for holding the available key providers and encryption methods, which is located in the [registry](registry/) folder. The key providers are located in the [keyprovider](keyprovider/) folder, while the encryption methods are located in the [method](method) folder. You can also find the configuration struct and its related functions in the [config](config) folder. + +## Further reading + +For a detailed design document on state encryption, please read [this document](../../docs/state_encryption.md). \ No newline at end of file diff --git a/internal/encryption/method/README.md b/internal/encryption/method/README.md new file mode 100644 index 0000000000..f67879f001 --- /dev/null +++ b/internal/encryption/method/README.md @@ -0,0 +1,26 @@ +# Encryption methods + +> [!WARNING] +> This file is not an end-user documentation, it is intended for developers. Please follow the user documentation on the OpenTofu website unless you want to work on the encryption code. + +This folder contains the implementations for the encryption methods used in OpenTofu. Encryption methods determine how exactly data is encrypted, but they do not determine what exactly is encrypted. + +## Implementing a method + +When you implement a method, take a look at the [aesgcm](aesgcm) method as a template. + +### Implementing the descriptor + +The descriptor is very simple, you need to implement the [`Descriptor`](descriptor.go) interface in a type. (It does not have to be a struct.) However, make sure that the `ConfigStruct` always returns a struct with `hcl` tags on it. For more information on the `hcl` tags, see the [gohcl documentation](https://godocs.io/github.com/hashicorp/hcl/v2/gohcl). + +### The config struct + +Next, you need to create a config structure. This structure should hold all the fields you expect a user to fill out. **This must be a struct, and you must add `hcl` tags to each field you expect the user to fill out.** + +If the config structure needs input from key providers, it should declare one HCL-tagged field with the type of [`keyprovider.Output`](../keyprovider/output.go) to receive the encryption and decryption key. Note, that the decryption key is not always available. + +Additionally, you must implement the `Build` function described in the [`Config` interface](config.go). You can take a look at [aesgcm/config.go](static/config.go) for an example on implementing this. + +### The method + +The heart of your method is... well, your method. It has the `Encrypt()` and `Decrypt()` methods, which should perform the named tasks. If no decryption key is available, the method should refuse to decrypt data. The method should under no circumstances pass through unencrypted data if it fails to decrypt the data. diff --git a/internal/encryption/method/config.go b/internal/encryption/method/config.go new file mode 100644 index 0000000000..859502ba1a --- /dev/null +++ b/internal/encryption/method/config.go @@ -0,0 +1,21 @@ +// Copyright (c) The OpenTofu Authors +// SPDX-License-Identifier: MPL-2.0 +// Copyright (c) 2023 HashiCorp, Inc. +// SPDX-License-Identifier: MPL-2.0 + +package method + +// Config describes a configuration struct for setting up an encryption Method. You should always implement this +// interface with a struct, and you should tag the fields with HCL tags so the encryption implementation can read +// the .tf code into it. For example: +// +// type MyConfig struct { +// Key string `hcl:"key"` +// } +// +// func (m MyConfig) Build() (Method, error) { ... } +type Config interface { + // Build takes the configuration and builds an encryption method. + // TODO this may be better changed to return hcl.Diagnostics so warnings can be issued? + Build() (Method, error) +} diff --git a/internal/encryption/method/descriptor.go b/internal/encryption/method/descriptor.go new file mode 100644 index 0000000000..ba119ceb68 --- /dev/null +++ b/internal/encryption/method/descriptor.go @@ -0,0 +1,20 @@ +// Copyright (c) The OpenTofu Authors +// SPDX-License-Identifier: MPL-2.0 +// Copyright (c) 2023 HashiCorp, Inc. +// SPDX-License-Identifier: MPL-2.0 + +package method + +// Descriptor contains the details on an encryption method and produces a configuration structure with default values. +type Descriptor interface { + // ID returns the unique identifier used when parsing HCL or JSON configs. + ID() ID + + // ConfigStruct creates a new configuration struct annotated with hcl tags. The Build() receiver on + // this struct must be able to build a Method from the configuration. + // + // Common errors: + // - Returning a struct without a pointer + // - Returning a non-struct + ConfigStruct() Config +} diff --git a/internal/encryption/method/method.go b/internal/encryption/method/method.go index d0e8b5d2eb..89d72a5c54 100644 --- a/internal/encryption/method/method.go +++ b/internal/encryption/method/method.go @@ -5,34 +5,6 @@ package method -// Config describes a configuration struct for setting up an encryption Method. You should always implement this -// interface with a struct, and you should tag the fields with HCL tags so the encryption implementation can read -// the .tf code into it. For example: -// -// type MyConfig struct { -// Key string `hcl:"key"` -// } -// -// func (m MyConfig) Build() (Method, error) { ... } -type Config interface { - // Build takes the configuration and builds an encryption method. - // TODO this may be better changed to return hcl.Diagnostics so warnings can be issued? - Build() (Method, error) -} - -type Descriptor interface { - // ID returns the unique identifier used when parsing HCL or JSON configs. - ID() ID - - // ConfigStruct creates a new configuration struct annotated with hcl tags. The Build() receiver on - // this struct must be able to build a Method from the configuration. - // - // Common errors: - // - Returning a struct without a pointer - // - Returning a non-struct - ConfigStruct() Config -} - // Method is a low-level encryption method interface that is responsible for encrypting a binary blob of data. It should // not try to interpret what kind of data it is encrypting. type Method interface { diff --git a/website/docs/language/state/encryption.mdx b/website/docs/language/state/encryption.mdx index 81bec29be8..bc0b96ecc2 100644 --- a/website/docs/language/state/encryption.mdx +++ b/website/docs/language/state/encryption.mdx @@ -142,7 +142,6 @@ Similar to the initial setup above, migrating to unencrypted state and plan file You can also configure an encryption setup for projects using the `terraform_remote_state` data source. This can be the same encryption setup as your main configuration, but you can also define a separate set of keys and methods. The configuration syntax looks as follows: - {RemoteState} For specific remote states, you can use the following syntax: diff --git a/website/docs/language/state/examples/encryption/configuration.ps1 b/website/docs/language/state/examples/encryption/configuration.ps1 index b649b25008..e8d2dc58cc 100644 --- a/website/docs/language/state/examples/encryption/configuration.ps1 +++ b/website/docs/language/state/examples/encryption/configuration.ps1 @@ -10,22 +10,17 @@ terraform { keys = key_provider.some_key_provider.some_name } - statefile { - # Encryption/decryption for local state files + state { + # Encryption/decryption for state data method = method.some_method.some_method_name } - planfile { - # Encryption/decryption for local plan files + plan { + # Encryption/decryption for plan data method = method.some_method.some_method_name } - backend { - # Encryption/decryption method for backends - method = method.some_method.some_method_name - } - - remote { + remote_state_data_sources { # See below } } diff --git a/website/docs/language/state/examples/encryption/configuration.sh b/website/docs/language/state/examples/encryption/configuration.sh index 9f06b8ec60..68957b0b76 100644 --- a/website/docs/language/state/examples/encryption/configuration.sh +++ b/website/docs/language/state/examples/encryption/configuration.sh @@ -10,22 +10,17 @@ terraform { keys = key_provider.some_key_provider.some_name } - statefile { - # Encryption/decryption for local state files + state { + # Encryption/decryption for state data method = method.some_method.some_method_name } - planfile { - # Encryption/decryption for local plan files + plan { + # Encryption/decryption for plan data method = method.some_method.some_method_name } - backend { - # Encryption/decryption method for backends - method = method.some_method.some_method_name - } - - remote { + remote_state_data_sources { # See below } } diff --git a/website/docs/language/state/examples/encryption/configuration.tf b/website/docs/language/state/examples/encryption/configuration.tf index acb814ca0f..562ec4af34 100644 --- a/website/docs/language/state/examples/encryption/configuration.tf +++ b/website/docs/language/state/examples/encryption/configuration.tf @@ -9,22 +9,17 @@ terraform { keys = key_provider.some_key_provider.some_name } - statefile { - # Encryption/decryption for local state files + state { + # Encryption/decryption for state data method = method.some_method.some_method_name } - planfile { - # Encryption/decryption for local plan files + plan { + # Encryption/decryption for plan data method = method.some_method.some_method_name } - backend { - # Encryption/decryption method for backends - method = method.some_method.some_method_name - } - - remote { + remote_state_data_sources { # See below } } diff --git a/website/docs/language/state/examples/encryption/enforce.tf b/website/docs/language/state/examples/encryption/enforce.tf index b8c2c5d457..76eb55db4a 100644 --- a/website/docs/language/state/examples/encryption/enforce.tf +++ b/website/docs/language/state/examples/encryption/enforce.tf @@ -1,13 +1,10 @@ terraform { encryption { - statefile { - enforce = true + state { + enforced = true } - planfile { - enforce = true - } - backend { - enforce = true + plan { + enforced = true } } } \ No newline at end of file diff --git a/website/docs/language/state/examples/encryption/fallback.tf b/website/docs/language/state/examples/encryption/fallback.tf index 33055a0b45..f8739f5a8a 100644 --- a/website/docs/language/state/examples/encryption/fallback.tf +++ b/website/docs/language/state/examples/encryption/fallback.tf @@ -2,21 +2,14 @@ terraform { encryption { # Methods and key providers here. - statefile { + state { method = method.some_method.new_method fallback { method = method.some_method.old_method } } - planfile { - method = method.some_method.new_method - fallback { - method = method.some_method.old_method - } - } - - backend { + plan { method = method.some_method.new_method fallback { method = method.some_method.old_method diff --git a/website/docs/language/state/examples/encryption/fallback_from_unencrypted.tf b/website/docs/language/state/examples/encryption/fallback_from_unencrypted.tf index 8fa70379a3..3ca8065754 100644 --- a/website/docs/language/state/examples/encryption/fallback_from_unencrypted.tf +++ b/website/docs/language/state/examples/encryption/fallback_from_unencrypted.tf @@ -2,7 +2,7 @@ terraform { encryption { # Methods and key providers here. - statefile { + state { method = method.some_method.new_method fallback { # The empty fallback block allows reading unencrypted state files. diff --git a/website/docs/language/state/examples/encryption/fallback_to_unencrypted.tf b/website/docs/language/state/examples/encryption/fallback_to_unencrypted.tf index 0eb9627549..2ed8e93e1b 100644 --- a/website/docs/language/state/examples/encryption/fallback_to_unencrypted.tf +++ b/website/docs/language/state/examples/encryption/fallback_to_unencrypted.tf @@ -2,7 +2,7 @@ terraform { encryption { # Methods and key providers here. - statefile { + state { # The empty block allows writing unencrypted state files # unless the enforced setting is set to true. fallback { diff --git a/website/docs/language/state/examples/encryption/terraform_remote_state.tf b/website/docs/language/state/examples/encryption/terraform_remote_state.tf index f3673f4db1..ad67265d82 100644 --- a/website/docs/language/state/examples/encryption/terraform_remote_state.tf +++ b/website/docs/language/state/examples/encryption/terraform_remote_state.tf @@ -2,11 +2,11 @@ terraform { encryption { # Key provider and method configuration here - remote { + remote_state_data_sources { default { method = method.my_method.my_name } - terraform_remote_state "my_state" { + remote_state_data_source "my_state" { method = method.my_method.my_other_name } }