From 20fec2304287a035439438e225a6343f163c6a2f Mon Sep 17 00:00:00 2001 From: Christian Mesh Date: Mon, 10 Jun 2024 12:40:59 -0400 Subject: [PATCH 1/7] Add RFC Tofu Version Compatibility Signed-off-by: Christian Mesh --- rfc/20240610-Tofu-Version-Compatibility.md | 109 +++++++++++++++++++++ 1 file changed, 109 insertions(+) create mode 100644 rfc/20240610-Tofu-Version-Compatibility.md diff --git a/rfc/20240610-Tofu-Version-Compatibility.md b/rfc/20240610-Tofu-Version-Compatibility.md new file mode 100644 index 0000000000..683900ce5a --- /dev/null +++ b/rfc/20240610-Tofu-Version-Compatibility.md @@ -0,0 +1,109 @@ +# Tofu Version Compatibility + +Issue: https://github.com/opentofu/opentofu/issues/1708 + +As OpenTofu and Terraform diverge, the concept of "software version" for compatibility becomes murky. + +The "software version" is used in a few locations: +* Feature compatibility with config files in the root project and modules +* Provider compatibility checks (deprecated but still used) +* State compatibility ("terraform_version" field) + +At the time of writing this RFC, OpenTofu 1.7.x has very similar features to Terraform 1.8.x (though not identical). As the "software version" requirements are updated in the previous examples, the exact requirements become murky. A module may require features from "Terraform 1.7.4", but would be perfectly happy with "OpenTofu 1.6.5" (as an example). + +> [!NOTE] +> As we are unable to inspect the Hashicorp Terraform Repository in depth, all version compatibility must be based on Release Notes / Changelog Entries as well as user reports. + +## Proposed Solution + +As we introduce support for `.tofu` files, we open up the ability to add new language features without worrying as much about compatibility in shared modules. If we can differentiate between "OpenTofu Version" and "Terraform Version", the above scenarios become much easier. + +We must also consider "partial knowledge" of version information. Ideally the configuration would clearly define which terraform version and tofu version are required. If that is not the case, we will have to provide a "best guess" to fill in the blanks. + +As the [provider compatibility check is deprecated](https://github.com/hashicorp/terraform-plugin-framework/blob/8a09cef0bb892e2d90c89cad551ce12a959f1a2d/provider/configure.go#L28-L32) and the state's "terraform_version" field is not currently used in OpenTofu, we will not discuss them in this RFC. + +### User Documentation + +Currently the "software version" (both Terraform and Tofu) is defined in the config as `terraform -> required_version`. This represents a valid range of versions that the configuration is compatible with. When OpenTofu is compiled, the "software version" is baked into the binary and used for all version checks (as well as`tofu -version`). + +Example: +```hcl +terraform { + required_version = 1.7.4 +} +``` +This configuration currently requires *both* Terraform 1.7.4 and OpenTofu 1.7.4 which are wildly different things. + +Let's look at a project or module which plans on supporting both tools (for the time being). They will start to introduce `.tofu` files as they start utilizing additional functionality in OpenTofu. How would they go about setting version requirements? + +```hcl +# version.tf +terraform { + required_version = 1.7.4 +} +``` +```hcl +# version.tofu +tofu { + required_version = 1.6.5 +} +``` + +When loaded by: +* Terraform, the `version.tofu` file would be ignored and the "Terraform Required Version" would be 1.7.4. +* OpenTofu, the `version.tofu` file would be loaded instead of `version.tf` and the "OpenTofu Required Version" would be 1.6.5. + + +What happens if only the `version.tf` file exists in this module with no tofu overwrite? This is a common case for modules who have not yet considered explicit OpenTofu migration. + +As a user, I would expect that the "Terraform Required Version" would have an equivalent "OpenTofu Required Version". This is not a 1-1 mapping, but is a reasonable stop-gap solution. + +As this is a not a 1-1 version mapping and a "best effort" solution, I would expect to see a warning: + +```shell +$ tofu init +... +Warning: Using v1.7.x in 'terraform -> required_version' as equivalent to current tofu version 1.6.5! +At file/line: ... +... +``` + +This could also include some information on how to remedy this situation, perhaps linking to the docs. + +### Technical Approach + +OpenTofu internally should know both it's own "software version" and what "terraform version" it is similar to. + +When we update the VERSION file in OpenTofu, we should also include the "terraform version" in the same file, or a similar file. OpenTofu's version package should then be able to supply both versions upon request. + +The "required_version" field is accessed via `sniffCoreVersionRequirements(body)` in internal/configs/parser_config.go and stored in `configs.File.CoreVersionConstraints`. These are then merged into `configs.Module.CoreVersionConstraints` and are checked in `configs.Module.CheckCoreVersionRequirements()`. This check function is called in a variety of locations moving up the stack, but is as high as we need to concern ourselves with for this RFC. + +We have two very similar paths here: +* Split CoreVersionConstraints into two fields: `TofuVersionConstraints` and `TerraformVersionConstraints` +* Add a flag into each VersionConstraint that understands if it is Tofu or Terraform specific. + +Either path is easy to both implement and thoroughly test. + +#### Version Compatibility + +As features don't change much between patch versions of Tofu and Terraform, we can use "Major.Minor.999999" as the "Terraform Version". This means that we are only tracking similarities between Major/Minor releases. + +This can be shown to the user as "Major.Minor.x" and solves the problem of new patch releases in Terraform after a compatible OpenTofu release has already gone out. + +### Open Questions + +Should there be an option (config, CLI, or ENV) to: +* Disable the version checking altogether? +* Treat both Tofu and Terraform version as identical instead of using our equivalent "terraform version" guess? +* Do we show the guessed terraform version in `tofu -version`, maybe with a `-verbose` flag? + +### Future Considerations + +If we run into providers expecting a particular terraform version (even though this is deprecated), we may want to use the "Terraform Version Equivalent" in that API call. + +## Potential Alternatives + +Don't implement this and assume that all module and project authors will adopt the .tofu extension and keep the respective required_versions up to date. + +Implement an option to disable required_version checking. + From d8f25e1f172cfa1cfacada3a9277b96aebaa91af Mon Sep 17 00:00:00 2001 From: Christian Mesh Date: Wed, 12 Jun 2024 10:14:08 -0400 Subject: [PATCH 2/7] Update to reflect most recent conversation Signed-off-by: Christian Mesh --- rfc/20240610-Tofu-Version-Compatibility.md | 45 ++++++++++------------ 1 file changed, 21 insertions(+), 24 deletions(-) diff --git a/rfc/20240610-Tofu-Version-Compatibility.md b/rfc/20240610-Tofu-Version-Compatibility.md index 683900ce5a..4fe673acfe 100644 --- a/rfc/20240610-Tofu-Version-Compatibility.md +++ b/rfc/20240610-Tofu-Version-Compatibility.md @@ -18,8 +18,6 @@ At the time of writing this RFC, OpenTofu 1.7.x has very similar features to Ter As we introduce support for `.tofu` files, we open up the ability to add new language features without worrying as much about compatibility in shared modules. If we can differentiate between "OpenTofu Version" and "Terraform Version", the above scenarios become much easier. -We must also consider "partial knowledge" of version information. Ideally the configuration would clearly define which terraform version and tofu version are required. If that is not the case, we will have to provide a "best guess" to fill in the blanks. - As the [provider compatibility check is deprecated](https://github.com/hashicorp/terraform-plugin-framework/blob/8a09cef0bb892e2d90c89cad551ce12a959f1a2d/provider/configure.go#L28-L32) and the state's "terraform_version" field is not currently used in OpenTofu, we will not discuss them in this RFC. ### User Documentation @@ -29,23 +27,23 @@ Currently the "software version" (both Terraform and Tofu) is defined in the con Example: ```hcl terraform { - required_version = 1.7.4 + required_version = "1.7.4" } ``` -This configuration currently requires *both* Terraform 1.7.4 and OpenTofu 1.7.4 which are wildly different things. +This configuration currently requires either Terraform 1.7.4 or OpenTofu 1.7.4 which are wildly different things. Let's look at a project or module which plans on supporting both tools (for the time being). They will start to introduce `.tofu` files as they start utilizing additional functionality in OpenTofu. How would they go about setting version requirements? ```hcl # version.tf terraform { - required_version = 1.7.4 + required_version = "1.7.4" } ``` ```hcl # version.tofu tofu { - required_version = 1.6.5 + required_version = "1.6.5" } ``` @@ -56,27 +54,27 @@ When loaded by: What happens if only the `version.tf` file exists in this module with no tofu overwrite? This is a common case for modules who have not yet considered explicit OpenTofu migration. -As a user, I would expect that the "Terraform Required Version" would have an equivalent "OpenTofu Required Version". This is not a 1-1 mapping, but is a reasonable stop-gap solution. - -As this is a not a 1-1 version mapping and a "best effort" solution, I would expect to see a warning: +As a user, I would like to know if any incompatible terraform_version has been specified as part of my project's configuration and modules. I would also assume any version requirement under 1.6 would be identical between Terraform and OpenTofu. +```hcl +terraform { + required_version = 1.7.4 +} +``` ```shell -$ tofu init -... -Warning: Using v1.7.x in 'terraform -> required_version' as equivalent to current tofu version 1.6.5! -At file/line: ... -... +$ TF_LOG=debug tofu init +Initializing the backend... +Initializing modules... +Warning: Configuration requests terraform version >= 1.6! + Please update the configuration to specify the required OpenTofu Version. More information is available in the debug log. +Debug: terraform required_version = 1.7.4 in main.tf:421 ``` This could also include some information on how to remedy this situation, perhaps linking to the docs. ### Technical Approach -OpenTofu internally should know both it's own "software version" and what "terraform version" it is similar to. - -When we update the VERSION file in OpenTofu, we should also include the "terraform version" in the same file, or a similar file. OpenTofu's version package should then be able to supply both versions upon request. - -The "required_version" field is accessed via `sniffCoreVersionRequirements(body)` in internal/configs/parser_config.go and stored in `configs.File.CoreVersionConstraints`. These are then merged into `configs.Module.CoreVersionConstraints` and are checked in `configs.Module.CheckCoreVersionRequirements()`. This check function is called in a variety of locations moving up the stack, but is as high as we need to concern ourselves with for this RFC. +The "required_version" field is accessed via `sniffCoreVersionRequirements(body)` in internal/configs/parser_config.go and stored in `configs.File.CoreVersionConstraints`. These are then merged into `configs.Module.CoreVersionConstraints` and are checked in `configs.Module.CheckCoreVersionRequirements()`. This check function is called in a variety of locations moving up the stack, but is as high as we need to concern ourselves with for this RFC. We will need to make sure the warning won't be emitted multiple times due to the spaghettified command package. We have two very similar paths here: * Split CoreVersionConstraints into two fields: `TofuVersionConstraints` and `TerraformVersionConstraints` @@ -84,16 +82,13 @@ We have two very similar paths here: Either path is easy to both implement and thoroughly test. -#### Version Compatibility - -As features don't change much between patch versions of Tofu and Terraform, we can use "Major.Minor.999999" as the "Terraform Version". This means that we are only tracking similarities between Major/Minor releases. - -This can be shown to the user as "Major.Minor.x" and solves the problem of new patch releases in Terraform after a compatible OpenTofu release has already gone out. +The `configs.Module.CheckCoreVersionRequirements()` function is where we will need to emit the warnings and corresponding debug log entries. ### Open Questions Should there be an option (config, CLI, or ENV) to: * Disable the version checking altogether? +* Don't check terraform required_versions * Treat both Tofu and Terraform version as identical instead of using our equivalent "terraform version" guess? * Do we show the guessed terraform version in `tofu -version`, maybe with a `-verbose` flag? @@ -107,3 +102,5 @@ Don't implement this and assume that all module and project authors will adopt t Implement an option to disable required_version checking. +Try to guess the equivalent OpenTofu version for a specified Terraform version. + From d25bd4476988b03591b87de0f65db1c8dbccea41 Mon Sep 17 00:00:00 2001 From: Christian Mesh Date: Fri, 14 Jun 2024 11:26:58 -0400 Subject: [PATCH 3/7] Updates per RFC review process Signed-off-by: Christian Mesh --- rfc/20240610-Tofu-Version-Compatibility.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/rfc/20240610-Tofu-Version-Compatibility.md b/rfc/20240610-Tofu-Version-Compatibility.md index 4fe673acfe..f1cbe5df01 100644 --- a/rfc/20240610-Tofu-Version-Compatibility.md +++ b/rfc/20240610-Tofu-Version-Compatibility.md @@ -69,9 +69,10 @@ Warning: Configuration requests terraform version >= 1.6! Please update the configuration to specify the required OpenTofu Version. More information is available in the debug log. Debug: terraform required_version = 1.7.4 in main.tf:421 ``` - This could also include some information on how to remedy this situation, perhaps linking to the docs. +If an incompatible (>1.5.5) version is detected, OpenTofu will ignore the "Terraform Required Version". + ### Technical Approach The "required_version" field is accessed via `sniffCoreVersionRequirements(body)` in internal/configs/parser_config.go and stored in `configs.File.CoreVersionConstraints`. These are then merged into `configs.Module.CoreVersionConstraints` and are checked in `configs.Module.CheckCoreVersionRequirements()`. This check function is called in a variety of locations moving up the stack, but is as high as we need to concern ourselves with for this RFC. We will need to make sure the warning won't be emitted multiple times due to the spaghettified command package. @@ -84,13 +85,15 @@ Either path is easy to both implement and thoroughly test. The `configs.Module.CheckCoreVersionRequirements()` function is where we will need to emit the warnings and corresponding debug log entries. +As discussed in the `.tofu` RFC, we will want to quickly work on support for this featu + ### Open Questions Should there be an option (config, CLI, or ENV) to: * Disable the version checking altogether? * Don't check terraform required_versions * Treat both Tofu and Terraform version as identical instead of using our equivalent "terraform version" guess? -* Do we show the guessed terraform version in `tofu -version`, maybe with a `-verbose` flag? +* ~~Do we show the guessed terraform version in `tofu -version`, maybe with a `-verbose` flag?~~ This goes against the Technical Steering Committee's strong recommendation against a version compatibility table. ### Future Considerations @@ -102,5 +105,5 @@ Don't implement this and assume that all module and project authors will adopt t Implement an option to disable required_version checking. -Try to guess the equivalent OpenTofu version for a specified Terraform version. +~~Try to guess the equivalent OpenTofu version for a specified Terraform version.~~ This goes against the Technical Steering Committee's strong recommendation against a version compatibility table. From 4647b7f9f5b303734faa8f7faa6b97ec90dc1a23 Mon Sep 17 00:00:00 2001 From: Christian Mesh Date: Wed, 19 Jun 2024 12:38:27 -0400 Subject: [PATCH 4/7] Review feedback Signed-off-by: Christian Mesh --- rfc/20240610-Tofu-Version-Compatibility.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfc/20240610-Tofu-Version-Compatibility.md b/rfc/20240610-Tofu-Version-Compatibility.md index f1cbe5df01..df8cc291f3 100644 --- a/rfc/20240610-Tofu-Version-Compatibility.md +++ b/rfc/20240610-Tofu-Version-Compatibility.md @@ -16,7 +16,7 @@ At the time of writing this RFC, OpenTofu 1.7.x has very similar features to Ter ## Proposed Solution -As we introduce support for `.tofu` files, we open up the ability to add new language features without worrying as much about compatibility in shared modules. If we can differentiate between "OpenTofu Version" and "Terraform Version", the above scenarios become much easier. +As we [introduce support](20240529-OpenTofu-Specific-Code-Override.md) for `.tofu` files, we open up the ability to add new language features without worrying as much about compatibility in shared modules. If we can differentiate between "OpenTofu Version" and "Terraform Version", the above scenarios become much easier. As the [provider compatibility check is deprecated](https://github.com/hashicorp/terraform-plugin-framework/blob/8a09cef0bb892e2d90c89cad551ce12a959f1a2d/provider/configure.go#L28-L32) and the state's "terraform_version" field is not currently used in OpenTofu, we will not discuss them in this RFC. @@ -85,7 +85,7 @@ Either path is easy to both implement and thoroughly test. The `configs.Module.CheckCoreVersionRequirements()` function is where we will need to emit the warnings and corresponding debug log entries. -As discussed in the `.tofu` RFC, we will want to quickly work on support for this featu +As discussed in the `.tofu` RFC, we will want to quickly work on support for this feature. ### Open Questions From 02bfed16a11ebd1345cd9d7e7f7bab4cd32281c8 Mon Sep 17 00:00:00 2001 From: Christian Mesh Date: Fri, 10 Jan 2025 10:44:44 -0500 Subject: [PATCH 5/7] Revise Tofu Version Compatibility RFC As we have received more feedback from the community and now have a better idea of the pace of adoption of the .tofu extension, a clearer view of what users actually need to work around version constraints is available. Signed-off-by: Christian Mesh --- rfc/20240610-Tofu-Version-Compatibility.md | 182 +++++++++++++++------ 1 file changed, 131 insertions(+), 51 deletions(-) diff --git a/rfc/20240610-Tofu-Version-Compatibility.md b/rfc/20240610-Tofu-Version-Compatibility.md index df8cc291f3..75a0e8def2 100644 --- a/rfc/20240610-Tofu-Version-Compatibility.md +++ b/rfc/20240610-Tofu-Version-Compatibility.md @@ -1,6 +1,6 @@ # Tofu Version Compatibility -Issue: https://github.com/opentofu/opentofu/issues/1708 +Issue: https://github.com/opentofu/opentofu/issues/1708 As OpenTofu and Terraform diverge, the concept of "software version" for compatibility becomes murky. @@ -9,69 +9,155 @@ The "software version" is used in a few locations: * Provider compatibility checks (deprecated but still used) * State compatibility ("terraform_version" field) -At the time of writing this RFC, OpenTofu 1.7.x has very similar features to Terraform 1.8.x (though not identical). As the "software version" requirements are updated in the previous examples, the exact requirements become murky. A module may require features from "Terraform 1.7.4", but would be perfectly happy with "OpenTofu 1.6.5" (as an example). +As a concrete example, OpenTofu 1.7.x has very similar features to Terraform 1.8.x (though not identical). As the "software version" requirements are updated in the previous examples, the exact requirements become murky. A module may require features from "Terraform 1.7.4", but might be perfectly happy with "OpenTofu 1.6.5". > [!NOTE] -> As we are unable to inspect the Hashicorp Terraform Repository in depth, all version compatibility must be based on Release Notes / Changelog Entries as well as user reports. +> As we are unable to inspect the HashiCorp Terraform Repository in depth, all version compatibility must be based on Release Notes / Changelog Entries as well as user reports. ## Proposed Solution -As we [introduce support](20240529-OpenTofu-Specific-Code-Override.md) for `.tofu` files, we open up the ability to add new language features without worrying as much about compatibility in shared modules. If we can differentiate between "OpenTofu Version" and "Terraform Version", the above scenarios become much easier. +We aim to address the scenario in which a module may be written for a given Terraform version and has a specific version requirement, while a OpenTofu user may wish to use the module and bypass the "Terraform" version requirement. + +Note: Since the [provider compatibility check is deprecated](https://github.com/hashicorp/terraform-plugin-framework/blob/8a09cef0bb892e2d90c89cad551ce12a959f1a2d/provider/configure.go#L28-L32) and the state's "terraform_version" field is not currently used in OpenTofu, we will consider them out of scope for this RFC. + +First, some definitions: +* A "Terraform Version" is represented by a `terraform { required_version = "requirement" }` block within a `.tf` file. +* A "Tofu Version" is represented by a `terraform { required_version = "requirement" }` block within a `.tofu` file. + +Currently in OpenTofu, the "Terraform Version" and "Tofu Version" are treated identically. As mentioned above, the version requirement maps to different feature sets in the two different projects. + +The only way around this problem as of today is to create a .tofu file for each `terraform {}` block in a project and module. This is the "ideal" scenario as modules can explicitly choose which feature sets they want when used within the two projects. + +An example of the version split is seen [here](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/2771) and "resolves" the original issue that spawned this RFC and discussion. Unfortunately, it took half a year for the module that the user relies upon to update. + +This situation leaves users in the lurch, with the only option available being to fork and modify every module with an incompatible "Terraform Version" constraint. + +The solution proposed in this RFC is to add the ability for a project author to disable "Terraform Version" requirements all together and rely solely on "Tofu Version". As this is a potentially dangerous option and requires users to understand the features in both, it should be an explicit opt-in feature. + +To help with clarity, an alternative to the `terraform{}` block is proposed, the `tofu{}` block. Functionally speaking, it is identical to the `terraform{}` block and will in most cases be treated as such. With this in place, we can amend our definitions above with: A "Tofu Version" is also represented by a `tofu { required_version = "requirement" }` block within a `.tf` or `.tofu` file. We also introduce the term `project{}` block to refer to either interchangeably. + + +From the perspective of a module, root or child, there are two different ways in which "Terraform Version" should be ignored: +1) Disabled for all child modules of the current module. This represents someone who has lots of child modules that have not yet added "Tofu Versions" and they are comfortable with the risk of bypassing the "Terraform Version" in all of them (and their children). +2) Disabled for a specific child module of the current module. This represents someone who is pulling a small number of child modules that have not yet added "Tofu Versions" and they would like to limit the blast radius of ignoring the "Terraform Version" check in this limited area of the current module. + +For for the first scenario, a flag called "bypass_terraform_version_requirement" is added to the `product{}` block. If set to true, all child modules of the current module will only error on "Tofu Versions" and instead produce warnings for "Terraform Versions". + +For the second scenario, a flag called "bypass_terraform_version_requirement" is added to the specific `module{}` block. If set to true, that specific module and all of it's children will only error on "Tofu Versions" and instead produce warnings for "Terraform Versions". -As the [provider compatibility check is deprecated](https://github.com/hashicorp/terraform-plugin-framework/blob/8a09cef0bb892e2d90c89cad551ce12a959f1a2d/provider/configure.go#L28-L32) and the state's "terraform_version" field is not currently used in OpenTofu, we will not discuss them in this RFC. ### User Documentation -Currently the "software version" (both Terraform and Tofu) is defined in the config as `terraform -> required_version`. This represents a valid range of versions that the configuration is compatible with. When OpenTofu is compiled, the "software version" is baked into the binary and used for all version checks (as well as`tofu -version`). +When adopting OpenTofu in a project or module, considerable care should be taken when inspecting project (terraform/tofu) version requirements. Even though these projects have a shared ancestry and may follow similar paths, the specific version numbers can mean very different things. A version represents an arbitrary feature set, which between the two projects will likely diverge further and further over time. -Example: -```hcl -terraform { - required_version = "1.7.4" -} -``` -This configuration currently requires either Terraform 1.7.4 or OpenTofu 1.7.4 which are wildly different things. +The ideal situation is that every child module that is used in a root module declares both a Terraform and OpenTofu version based on the specific feature sets it needs. As of today this is not yet practical to rely upon and can make migrating to OpenTofu more of a headache than is ideal. -Let's look at a project or module which plans on supporting both tools (for the time being). They will start to introduce `.tofu` files as they start utilizing additional functionality in OpenTofu. How would they go about setting version requirements? +To work around this limitation, we introduce the "bypass_terraform_version_requirement" field to both the `teraform{}` and `module{}` blocks. Additionally, for clarity we introduce the `tofu{}` block as a synonym of the `terraform{}` block and refer to them both as the `project{}` block. +Some definitions before we get into behavior: +* "Terraform Version" refers to `terraform{ required_version = "requirement" }` present in a `.tf` file +* "Tofu Version" refers to `terraform{ required_version = "requirement" }` present in a `.tofu` file OR `tofu{ required_version = "requirement" }` in either a `.tf` or `.tofu` file + + +#### Project scoped behavior: + +By setting `bypass_terraform_version_requirement = true` in the `project{}` block, all "Terraform Version" requirements in child modules of the current module will be treated as warnings. + +For example: ```hcl -# version.tf -terraform { - required_version = "1.7.4" -} -``` -```hcl -# version.tofu +# main.tofu + tofu { - required_version = "1.6.5" + required_version = ">=1.5.5" + bypass_terraform_version_requirement = true +} + +module "childA" { + source = "something/i/dont/want/to/fork" + input = "A" +} + +# childB...Y + +module "childZ" { + source = "something/i/dont/want/to/fork" + input = "Z" } ``` -When loaded by: -* Terraform, the `version.tofu` file would be ignored and the "Terraform Required Version" would be 1.7.4. -* OpenTofu, the `version.tofu` file would be loaded instead of `version.tf` and the "OpenTofu Required Version" would be 1.6.5. - - -What happens if only the `version.tf` file exists in this module with no tofu overwrite? This is a common case for modules who have not yet considered explicit OpenTofu migration. - -As a user, I would like to know if any incompatible terraform_version has been specified as part of my project's configuration and modules. I would also assume any version requirement under 1.6 would be identical between Terraform and OpenTofu. - ```hcl +# something/i/dont/want/to/fork/versions.tf terraform { - required_version = 1.7.4 + required_version = ">=1.42.10" } ``` -```shell -$ TF_LOG=debug tofu init + +Running `tofu init` will produce: +``` + Initializing the backend... Initializing modules... -Warning: Configuration requests terraform version >= 1.6! - Please update the configuration to specify the required OpenTofu Version. More information is available in the debug log. -Debug: terraform required_version = 1.7.4 in main.tf:421 -``` -This could also include some information on how to remedy this situation, perhaps linking to the docs. -If an incompatible (>1.5.5) version is detected, OpenTofu will ignore the "Terraform Required Version". +Warning: Ignored terraform version requirement + on: something/i/dont/want/to/fork/versions.tf in module "childA" + + +.... + + +Warning: Ignored terraform version requirement + on: something/i/dont/want/to/fork/versions.tf in module "childZ" + + +``` + + +#### Product scoped behavior: + +By setting `bypass_terraform_version_requirement = true` in a `module{}` block, all "Terraform Version" requirements in the requested module and it's children will treated as warnings. + +For example: +```hcl +# main.tofu + +tofu { + required_version = ">=1.5.5" +} + +module "child" { + source = "something/i/dont/want/to/fork" + bypass_terraform_version_requirement = true +} +``` + +```hcl +# something/i/dont/want/to/fork/versions.tf +terraform { + required_version = ">=1.42.10" +} + +module "inner_child" { + source = "somethingelse/i/dont/want/to/fork" + bypass_terraform_version_requirement = true +} +``` + +Running `tofu init` will produce: +``` + +Initializing the backend... +Initializing modules... + +Warning: Ignored terraform version requirement + on: something/i/dont/want/to/fork/versions.tf in module "child" + + + +Warning: Ignored terraform version requirement + on: somethingelse/i/dont/want/to/fork/versions.tf in module "child.inner_child" + + +``` ### Technical Approach @@ -83,27 +169,21 @@ We have two very similar paths here: Either path is easy to both implement and thoroughly test. -The `configs.Module.CheckCoreVersionRequirements()` function is where we will need to emit the warnings and corresponding debug log entries. +The `configs.Module.CheckCoreVersionRequirements()` function is where we will need to emit the warning diagnostics. -As discussed in the `.tofu` RFC, we will want to quickly work on support for this feature. +Additionally, introducing "bypass_terraform_version_requirement" to the `module{}` block is introducing a new "reserved" field that must not be used by module authors. Given it's unique naming and lack of search results on Github, I believe that it is safe to introduce. ### Open Questions -Should there be an option (config, CLI, or ENV) to: -* Disable the version checking altogether? -* Don't check terraform required_versions -* Treat both Tofu and Terraform version as identical instead of using our equivalent "terraform version" guess? -* ~~Do we show the guessed terraform version in `tofu -version`, maybe with a `-verbose` flag?~~ This goes against the Technical Steering Committee's strong recommendation against a version compatibility table. +Should bypass_terraform_version_requirements ever be the default? ### Future Considerations -If we run into providers expecting a particular terraform version (even though this is deprecated), we may want to use the "Terraform Version Equivalent" in that API call. +If we run into providers expecting a particular terraform version (even though this is deprecated), we may want to use this flag to work around it? ## Potential Alternatives Don't implement this and assume that all module and project authors will adopt the .tofu extension and keep the respective required_versions up to date. -Implement an option to disable required_version checking. - -~~Try to guess the equivalent OpenTofu version for a specified Terraform version.~~ This goes against the Technical Steering Committee's strong recommendation against a version compatibility table. +~~Try to guess the equivalent OpenTofu version for a specified Terraform version.~~ This goes against the Technical Steering Committee's strong recommendation against a version compatibility table and is more confusing for end users. From f09f0cb15acf478d84c71eb1da8ad042b3b8f658 Mon Sep 17 00:00:00 2001 From: Christian Mesh Date: Mon, 13 Jan 2025 09:54:16 -0500 Subject: [PATCH 6/7] Add Open Question from RFC PR discussion Signed-off-by: Christian Mesh --- rfc/20240610-Tofu-Version-Compatibility.md | 45 +++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/rfc/20240610-Tofu-Version-Compatibility.md b/rfc/20240610-Tofu-Version-Compatibility.md index 75a0e8def2..64893845ce 100644 --- a/rfc/20240610-Tofu-Version-Compatibility.md +++ b/rfc/20240610-Tofu-Version-Compatibility.md @@ -175,7 +175,50 @@ Additionally, introducing "bypass_terraform_version_requirement" to the `module{ ### Open Questions -Should bypass_terraform_version_requirements ever be the default? +#### How should existing "Terraform Versions" be treated? + +This RFC proposes that the handling of unsatisfied "Terraform Version" requirements is enabled unless otherwise specified. This keeps consistency with existing functionality and proposes a workaround for those blocked during the migration process. + +@apparentlymart brought up the idea that as "Terraform Version" greater than 1.5.x is effectively meaningless, it could be ignored all together (outside of the root module). + +This idea brings about the question of what is the true utility of "Project Version" and how is it used and relied upon today. In general, it is bumped when new language features or fixes are added. + + +As an example, let's consider what happens when a new feature is introduced. The `required_version` is bumped to the latest version of the project. +```hcl +terraform { + # Updated for extended moved support + required_version = ">=1.9.0" +} +moved { + # Moved now supports migrating between resources (if supported by the provider) + from = some_provider.resource + to = some_provider.other_resource +} +``` +Note that this does not introduce any new configuration options and instead makes the existing functionality more flexible. When run with Terraform 1.9.0, this will behave as expected as the feature required was introduced in that version". When run in previous versions of Terraform, it would have produced an error message detailing that this operation is not supported. OpenTofu 1.9.0 produces the same error message. Eventually this will be supported in OpenTofu and a corresponding .tofu file could be added to ensure compatibility. + +What was the worst case scenario if the `required_version` was ignored by OpenTofu? A user would see a error message detailing why the operation is not valid. A charitable user would look at the release notes, issue tracker, or https://cani.tf for feature support. A uncharitable user would complain about OpenTofu publicly and advocate against it, grumbling about footguns. That could potentially be softened by adding a warning that the required version check is undefined for "Terraform Version" and is being skipped. + + +A more dangerous example is for complex bugs, such as https://github.com/opentofu/opentofu/issues/1616. In that scenario, handling of sensitive attributes was broken in both Terraform and OpenTofu prior to the fork. Terraform seems to fixed the issue in the 1.8 series, and the issue was reported from someone migrating from that version. +```hcl +terraform { + # 1.8.2 and beyond correctly handle sensitive attributes + required_version = ">=1.8.2" +} +resource "random_password" "password1" { + length = 12 + special = false +} +``` + +This bug could not only impact the planning of an apply, but could expose incorrect sensitive attribute detection to third party tools. With the complexity of this project, bugs like these are bound to exist and require fixes over time. "Product Version" is one of the few ways for module authors to require fixes. + +In the case that OpenTofu silently ignored the required_version, a user could be using third party tooling to expose secrets. If the module explicitly was asking for a "Terraform Version" that was unmet, and the user of the module did not know that the safety check was bypassed, they could both be soured against OpenTofu. If a warning were added, clarifying that OpenTofu does not understand "Terraform Versions" past 1.5.x and the requirements should be manually reviewed / module updated, a user would at least have a chance at detecting that something was wrong and deciding to ignore it or not. + + +To make the decision between making "Terraform Version" produce only a warning and the more manual approach of a flag, we must consider those two examples and their outcomes carefully. ### Future Considerations From b200b6f0a615e84a8d03277eb3d5bc8d7f2a95fd Mon Sep 17 00:00:00 2001 From: Christian Mesh Date: Tue, 14 Jan 2025 06:37:36 -0500 Subject: [PATCH 7/7] Update rfc/20240610-Tofu-Version-Compatibility.md Co-authored-by: Oleksandr Levchenkov Signed-off-by: Christian Mesh --- rfc/20240610-Tofu-Version-Compatibility.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfc/20240610-Tofu-Version-Compatibility.md b/rfc/20240610-Tofu-Version-Compatibility.md index 64893845ce..d835da2f92 100644 --- a/rfc/20240610-Tofu-Version-Compatibility.md +++ b/rfc/20240610-Tofu-Version-Compatibility.md @@ -41,7 +41,7 @@ From the perspective of a module, root or child, there are two different ways in 1) Disabled for all child modules of the current module. This represents someone who has lots of child modules that have not yet added "Tofu Versions" and they are comfortable with the risk of bypassing the "Terraform Version" in all of them (and their children). 2) Disabled for a specific child module of the current module. This represents someone who is pulling a small number of child modules that have not yet added "Tofu Versions" and they would like to limit the blast radius of ignoring the "Terraform Version" check in this limited area of the current module. -For for the first scenario, a flag called "bypass_terraform_version_requirement" is added to the `product{}` block. If set to true, all child modules of the current module will only error on "Tofu Versions" and instead produce warnings for "Terraform Versions". +For the first scenario, a flag called "bypass_terraform_version_requirement" is added to the `product{}` block. If set to true, all child modules of the current module will only error on "Tofu Versions" and instead produce warnings for "Terraform Versions". For the second scenario, a flag called "bypass_terraform_version_requirement" is added to the specific `module{}` block. If set to true, that specific module and all of it's children will only error on "Tofu Versions" and instead produce warnings for "Terraform Versions".