# jsonformat This package contains functionality around formatting and displaying the JSON structured output produced by adding the `-json` flag to various Terraform commands. ## Terraform Structured Plan Renderer As of January 2023, this package contains only a single structure: the `Renderer`. The renderer accepts the JSON structured output produced by the `terraform show -json` command and writes it in a human-readable format. Implementation details and decisions for the `Renderer` are discussed in the following sections. ### Implementation There are two subpackages within the `jsonformat` renderer package. The `differ` package compares the `before` and `after` values of the given plan and produces `Diff` objects from the `computed` package. This approach is aimed at ensuring the process by which the plan difference is calculated is separated from the rendering itself. In this way it should be possible to modify the rendering or add new renderer formats without being concerned with the complex diff calculations. #### The `differ` package The `differ` package operates on `Change` objects. These are produced from `jsonplan.Change` objects (which are produced by the `terraform show` command). Each `jsonplan.Change` object represents a single resource within the overall Terraform configuration. The `differ` package will iterate through the `Change` objects and produce a single `Diff` that represents a processed summary of the changes described by the `Change`. You will see that the produced changes are nested so a change to a list attribute will contain a slice of changes, this is discussed in the "[The computed package](#the-computed-package)" section. ##### The `Change` object The `Change` objects contain raw Golang representations of JSON objects (generic `interface{}` fields). These are produced by parsing the `json.RawMessage` objects within the provided changes. The fields the differ cares about from the provided changes are: - `Before`: The value before the proposed change. - `After`: The value after the proposed change. - `Unknown`: If the value is being computed during the change. - `BeforeSensitive`: If the value was sensitive before the change. - `AfterSensitive`: If the value is sensitive after the change. - `ReplacePaths`: If the change is causing the overall resource to be replaced. In addition, the changes define two additional meta fields that they set and manipulate internally: - `BeforeExplicit`: If the value in `Before` is explicit or an implied result due to a change elsewhere. - `AfterExplicit`: If the value in `After` is explicit or an implied result due to a change elsewhere. The actual concrete type of each of the generic fields is determined by the overall schema. The changes are also recursive, this means as we iterate through the `Change` we create relevant child values based on the schema for the given resource. For example, the initial change is always a `block` type which means the `Before` and `After` values will actually be `map[string]interface{}` types mapping each attribute and block to their relevant values. The `Unknown`, `BeforeSensitive`, `AfterSensitive` values will all be either a `map[string]interface{}` which maps each attribute or nested block to their unknown and sensitive status, or it could simply be a `boolean` which generally means the entire block and all children are sensitive or computed. In total, a `Change` can represent the following types: - `Attribute` - `map`: Values will typically be `map[string]interface{}`. - `list`: Values will typically be `[]interface{}`. - `set`: Values will typically be `[]interface{}`. - `object`: Values will typically be `map[string]interface{}`. - `tuple`: Values will typically be `[]interface{}`. - `bool`: Values will typically be a `bool`. - `number`: Values will typically be a `float64`. - `string`: Values will typically be a `string`. - `Block`: Values will typically be `map[string]interface{}`, but they can be split between nested blocks and attributes. - `Output` - Outputs are interesting as we don't have a schema for them, as such they can be any JSON type. - We also use the Output type to represent dynamic attributes, since in both cases we work out the type based on the JSON representation instead of the schema. The `ReplacePaths` field is unique in that it's value doesn't actually change based on the schema - it's always a slice of index slices. An index in this context will either be an integer pointing to a child of a set or a list or a string pointing to the child of a map, object or block. As we iterate through the value we manipulate the outer slice to remove child slices where the index doesn't match and propagate paths that do match onto the children. *Quick note on explicit vs implicit:* In practice, it is only possible to get implicit changes when you manipulate a collection. That is to say child values of a modified collection will insert `nil` entries into the relevant before or after fields of their child changes to represent their values being deleted or created. It is also possible for users to explicitly put null values into their collections, and this behaviour is different to deleting an item in the collection. With the `BeforeExplicit` and `AfterExplicit` values we can tell the difference between whether this value was removed from a collection or this value was set to null in a collection. *Quick note on the go-cty Value and Type objects:* The `Before` and `After` fields are actually go-cty values, but we cannot convert them directly because of the Terraform Cloud redacted endpoint. The redacted endpoint turns sensitive values into strings regardless of their types. Because of this, we cannot just do a direct conversion using the ctyjson package. We would have to iterate through the schema first, find the sensitive values and their mapped types, update the types inside the schema to strings, and then go back and do the overall conversion. This isn't including any of the more complicated parts around what happens if something was sensitive before and isn't sensitive after or vice versa. This would mean the type would need to change between the before and after value. It is in fact just easier to iterate through the values as generic JSON interfaces, and obfuscate the sensitive values as we never need to print them anyway. ##### Iterating through changes The `differ` package will recursively create child `Change` objects for the complex objects. There are two key subtypes of a `Change`: `SliceChange` and `MapChange`. `SliceChange` values are used by list, set, and tuple attributes. `MapChange` values are used by map and object attributes, and blocks. For what it is worth outputs and dynamic types can end up using both, but they're kind of special as the processing for dynamic types works out the type from the JSON struct and then just passes it into the relevant real types for actual processing. The two subtypes implement `GetChild` functions that retrieve a child change for a relevant index (`int` for slice, `string` for map). These functions build an entirely populated `Change` object, and the package will then recursively compute the change for the child (and all other children). When a complex change has all the children changes, it then passes that into the relevant complex diff type. #### The `computed` package A computed `Diff` should contain all the relevant information it needs to render itself. The `Diff` itself contains the action (eg. `Create`, `Delete`, `Update`), and whether this change is causing the overall resource to be replaced (read from the `ReplacePaths` field discussed in the previous section). The actual content of the diffs is passed directly into the internal renderer field. The internal renderer is then an implementation that knows the actual content of the changes and what they represent. For example to instantiate a diff resulting from updating a list of primitives: ```go listDiff := computed.NewDiff(renderers.List([]computed.Diff{ computed.NewDiff(renderers.Primitive(0.0, 0.0, cty.Number), plans.NoOp, false), computed.NewDiff(renderers.Primitive(1.0, nil, cty.Number), plans.Delete, false), computed.NewDiff(renderers.Primitive(nil, 4.0, cty.Number), plans.Create, false), computed.NewDiff(renderers.Primitive(2.0, 2.0, cty.Number), plans.NoOp, false) }, plans.Update, false)) ``` ##### The `RenderHuman` function Currently, there is only one way to render a change, and it is implemented via the `RenderHuman` function. In the future, there may be additional rendering capabilities, but for now the `RenderHuman` function just passes the call directly onto the internal renderer. Rendering the above diff with: `listDiff.RenderHuman(0, RenderOpts{})` would produce: ```text [ 0, - 1 -> null, + 4, 2, ] ``` Note, the render function itself doesn't print out metadata about its own change (eg. there's no `~` symbol in front of the opening bracket). The expectation is that parent changes control how child changes are rendered, so are responsible for deciding on their opening indentation, whether they have a key (as in maps, objects, and blocks), or how the action symbol is displayed. In the above example, the primitive renderer would print out only `1 -> null` while the surrounding list renderer is providing the indentation, the symbol and the line ending commas. ##### Implementing new diff types To implement a new diff type, you must implement the internal Renderer functionality. To do this you create a new implementation of the `computed.DiffRenderer`, make sure it accepts all the data you need, and implement the `RenderHuman` function (and any other additional render functions that may exist). Some changes publish warnings that should be displayed alongside them. If your new change has no warnings you can use the `NoWarningsRenderer` to avoid implementing the additional `Warnings` function. If/when new Renderer types are implemented, additional `Render` like functions will be added. You should implement all of these with your new change type. ##### Implementing new renderer types for changes As of January 2023, there is only a single type of renderer (the human-readable) renderer. As such, the `Diff` structure provides a single `RenderHuman` function. To implement a new renderer: 1. Add a new render function onto the internal `DiffRenderer` interface. 2. Add a new render function onto the `Diff` struct that passes the call onto the internal renderer. 3. Implement the new function on all the existing internal interfaces. Since each internal renderer contains all the information it needs to provide change information about itself, your new Render function should pass in anything it needs. ### New types of Renderer In the future, we may wish to add in different kinds of renderer, such as a compact renderer, or an interactive renderer. To do this, you'll need to modify the Renderer struct or create a new type of Renderer. The logic around creating the `Diff` structures will be shared (ie. calling into the differ package should be consistent across renderers). But when it comes to rendering the changes, I'd expect the `Diff` structures to implement additional functions that allow them to internally organise the data as required and return a relevant object. For the existing human-readable renderer that is simply a string, but for a future interactive renderer it might be a model from an MVC pattern.