Signed-off-by: Ioannis Polyzos <git@ipolyzos.com>
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
Co-authored-by: Christian Mesh <christianmesh1@gmail.com>
Signed-off-by: Ioannis Polyzos <git@ipolyzos.com>
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
Co-authored-by: Christian Mesh <christianmesh1@gmail.com>
One test is removed.
It doesn't seem to be a recommended mode of authentication and I have found no guide on how to configure it.
I've decided to skip this test in the suite, rather than spending more time to make it work.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
All four of these scripts have #! lines calling for them to run in bash,
and they are all marked as executable, so there's no harm in running them
directly and it makes it clearer that we're running them in the shell
specified on the #! line, rather than with whatever "sh' happens to be
on the current system.
(Note that this _was_ still correctly using the #! lines before, because
it's `sh -c` rather than just `sh`, but nonetheless that old way seemed
needlessly confusing.)
When we run many of these together e.g. in the checks.yaml GitHub Actions
workflow, it's hard to tell from the output exactly which command is
producing which subset of the output.
To help clarify that, we'll ask make to print out the command line it's
running before it runs it.
* Add goimports as a check to the Github commit actions
* check diff against main instead of last commit
* goimports should fix the problems
* fix up conditionals and wildcard matching
* specify origin/main in diff
* fetch main branch when checkout
* back to origin main
For now, only check the JSON views package, since this was the instance
that most recently tripped us up. There are a few dozen failures
elsewhere in Terraform which would need to be addressed before expanding
this to other packages.
We have a few different .proto files in this repository that all need to
get recompiled into .pb.go files each time we change them, but we were
previously handling that with some scripts that just assumed that protoc
and the relevant plugins were already installed on the system somewhere,
at the right versions.
In practice we've been constantly flopping between different versions of
these tools due to folks having different versions installed in their
development environments. In particular, the state of the .pb.go files
in the prior commit wasn't reproducible by any single version of the tools
because they've all slightly diverged from one another.
In the interests of being more consistent here and avoiding accidental
inconsistencies, we'll now centralize the protocol buffer compile steps
all into a single tool that knows how to fetch and install the expected
versions of the various tools we need and then run those tools with the
right options to get a stable result.
If we want to upgrade to either a newer protoc or a newer protoc-gen-go
in future then we'll do that in a central location and update all of the
.pb.go files at the same time, so that we're always consistently tracking
the same version of protocol buffers everywhere.
While doing this I attempted to keep as close as possible to the toolchain
we'd most recently used, but since they were not consistent with each
other they've now all changed which version numbers they record at minimum,
and the planproto stub in particular now also has a slightly different
descriptor serialization but is otherwise offering the same API.
This task relied on using `wget` to spider the entire site, which is no longer a
useful way of checking for broken links on a non-production instance of
terraform.io. Also, it didn't include a step for pausing until the server came
online, so the task wouldn't have functioned properly anyway.
Most of the targets in the Makefile have not been used in either CI or
the normal development workflow for some time. Removing them clarifies
that the expected way to build Terraform locally is simple: go install.
Remaining targets:
- fmtcheck, generate: these are used in CI to verify that the code is
correctly formatted and that generate has been run appropriately
- protobuf: referenced in CONTRIBUTING.md as the simplest way to build
the proto files
- website, website-test: used to compile and test the local website in
isolation from the terraform-website repo
Properly exclude files under "./vendor/" path, and find "files" only.
The original method "might" found "directories" with the same naming,
or, might exclude occasionally exclude the go files with "vendor" string
in any part of its path.
It seems that all of the tools we run here are now sufficiently
modules-aware to run without problems in modules mode, and indeed running
_not_ in modules mode was causing problems with locating packages in
mockgen.
The main significant change here is that the package name for the proto
definition is "tfplugin5", which is important because this name is part
of the wire protocol for references to types defined in our package.
Along with that, we also move the generated package into "internal" to
make it explicit that importing the generated Go package from elsewhere is
not the right approach for externally-implemented SDKs, which should
instead vendor the proto definition they are using and generate their
own stubs to ensure that the wire protocol is the only hard dependency
between Terraform Core and plugins.
After this is merged, any provider binaries built against our
helper/schema package will need to be rebuilt so that they use the new
"tfplugin5" package name instead of "proto".
In a future commit we will include more elaborate and organized
documentation on how an external codebase might make use of our RPC
interface definition to implement an SDK, but the primary concern here
is to ensure we have the right wire package name before release.
We're still using vendoring for now until we get _all_ of our tooling
updated, so the main idea here is to force use of the vendor directory
when running tests and building for development so we can quickly find
situations where we forget to run "go mod vendor".
We also setting GO111MODULE=off for installation of tools. Right now this
is the best way to install a tool in GOBIN without also interfering with
go.mod and go.sum, until a better pattern for managing tool dependencies
is devised by the Go team.
Finally, we run "go mod download" before launching "gox" in the main
build process, to prime the local module cache once so that the concurrent
"go build" processes won't race to populate it redundantly. This means
that we'll be producing final builds from the module cache rather than
from vendor as with everything else -- there's currently no way to tell
gox to use -mod=vendor -- but that should be fine in practice since
our go.sum file will ensure that we get the exact sources we expect in
the module cache before building.
Since protoc is not go-gettable, and most development tasks in Terraform
won't involve recompiling protoc files anyway, we'll use a separate
mechanism for these.
This way "go generate" only depends on things we can "go get" in the
"make tools" target.
In a later commit we should also in some way specify a particular version
of protoc to use so that we don't get "flapping" regenerations as
developers work with different versions, but the priority here is just to
make "make generate" minimally usable again to restore the dev workflow
documented in the README.
This also includes some updates that resulted from running "make generate"
and "make protobuf" after those Makefile changes were in place.
The test target will already cover vet, since it's run as part of the
test command now.
Also remove the `go test -i` since it's no longer needed with the new
build cache.
