When configuration is backed by a database, a separate pool of
connections is used for access from the primary database. This
change sets sensible config DB connection limits in order to
reduce database resource usage at scale.
This change addresses a few issues where config stores were not
properly closed when an error was encountered on server startup.
This could result in leaked database connections when dealing with
a database config store.
* Removing supplier concept from the sql store
* Removing other metions to supplier
* Fixing gofmt
* Fixing gofmt
* Renaming NewSqlStore to New
* Fixing tests
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Add support for boolean feature flags.
* Tests
* Use strconv.ParseBool to accept additional values as true.
* Update config/feature_flags.go
Co-authored-by: Christopher Poile <cpoile@gmail.com>
Co-authored-by: Christopher Poile <cpoile@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Basic framework for use of feature flags. Enviroment variable overrides.
* Use Apperr instead of error number increments.
* Undo random viper change.
* Update model/config_test.go
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
MM-27744 disable Zap for unit tests.
Zap has no concept of shutdown or close. Zap is only shutdown when the app exits. Not a problem for console logging, but when creating a new Zap logger that outputs to files on every unit test, that leaves no easy way to clean up until process exit. Depending on what else is running this can exhaust all file handles and cause unit tests to fail.
Zap is now disabled unit tests and uses Logr instead, regardless of config settings. `make test-server` peak file handle usage dropped from ~5K to less than 100.
* Moving diagnostics into a service
* Fixing golint checks
* Fixing tests
* Renaming from diagnostics to telemetry
* Adding missing files
* Initializing telemetry earlier in the server startup
* Fixing tests
* Adding a log for the telemetryID initialization error
* Addressing PR review comments
* Fixing merge problem
* Removing some extra Diagnostics mentions
* Making tests pass
* Fix return nil
A nil error is an interface having nil value but a non-nil type.
This leads to unexpected errors in the code well-documented here:
https://golang.org/doc/faq#nil_error
We fix this by passing an additional parameter to handle it.
Caught using https://github.com/dgryski/semgrep-go
* Bring back auto-generated line
* MM-27525: Obscures the Global Relay SMTP password.
* MM-27525: Desanitize global relay's SMTP password.
* MM-27525: Does not set the fake value if the field is blank.
* Add a config for MM User Limit
* Adding graceful errors for if an administrator invites people passed their user limit
* Including changed vendor files
* Adding unit test
* Fix a bug
* Push up working tests (Thanks Joram)
* Add more cases, clean up logs in code
* One more case
* Refactoring based on PR comments
* Updating i18n
* Some changes based on PR review
* Remove a comment
* Bring back some translations that were somehow removed
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-27178: use FileSettings.Directory in export
* Set empty FileSettings.Directory to default value
* Validate that FileSettings.Directory is non-empty
* Don't set FileSettings.Directory to / in fixConfig
* Run make i18n-extract
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Adds Advanced Logging to server. Advanced Logging is an optional logging capability that allows customers to send log records to any number of destinations.
Supported destinations:
- file
- syslog (with out without TLS)
- raw TCP socket (with out without TLS)
Allows developers to specify discrete log levels as well as the standard trace, debug, info, ... panic. Existing code and logging API usage is unchanged.
Log records are emitted asynchronously to reduce latency to the caller. Supports hot-reloading of logger config, including adding removing targets.
Advanced Logging is configured within config.json via "LogSettings.AdvancedLoggingConfig" which can contain a filespec to another config file, a database DSN, or JSON.
* Fixing behavior of Replicas and SearchReplicas in canary environments
* Trying to fix tests
* Revert "Trying to fix tests"
This reverts commit 3531da9618.
* Revert "Fixing behavior of Replicas and SearchReplicas in canary environments"
This reverts commit 0c05901c84.
* Revert "Disable read/search db replicas in TE/E0 (#14400)"
This reverts commit ef5ac519d9.
* Making the store aware of the license
* Readding the unit tests
* Fixing sqlstor supplier tests
* Adding mutex to ensure license write consistency and fixing tests
* Fixing tests
* Fixing tests
* Shuting down server properly during tests
* Trying to fix tests
* Trying to fix the tests
* Skipping flaky tests
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-23250: improve config.Migrate tests
The config.Migrate tests were not running in isolation, but wrote over the "local" config. The test coverage for both directions was not exhaustive, and lacked coverage for the recently added plugin signature files.
Rewrite the unit tests as such, fixing a minor issue closing the stores after migration.
Fixes: https://mattermost.atlassian.net/browse/MM-23250
* idiomatic err checking
* clarify random file contents
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* Disable read/search db replicas in TE/E0
* fixing tests
* Removing unnecesary text.
* Updating without-license read-replicas config before store initialization
* Reconnecting to database after remove read replicas
With Go 1.14 (https://tip.golang.org/doc/go1.14#go-command),
it now automatically checks for the existence of a vendor directory.
So there is no need to explicitly set the `-mod=vendor` flag.
* Introducing unit (not integration) tests for the app layer
* Initial support for unit tests at the API
* Adding unit tests support to the store layer
* Add unit tests support in commands
* Adding last tests needed for run unit tests properly
* Fixing govet
* Removing some duplication
* Fixing tests
* Fixing tests
* Not compiling test helpers with the main module for api
* Revert "Not compiling test helpers with the main module for api"
This reverts commit 36a199bbe0.
* Fixing tests
* Fixing unit tests
* More consistency between api4/apiteslib.go and app/helper_test.go
* Renaming things to make more obvious the new Setup functions purpose
* Reverting change in go.sum
* Start with empty mock for app layer
* Start with empty mock for api layer
* Start with empty mock for web layer
* Renaming SetupWithStoreMockConfig to SetupConfigWithStoreMock
* Fixing tests on web package
* Removing unnecesary function
* Run all tests fasts in postgres
* Moving postgres config to a file
* Addressing PR review comments
* Testing against mysql in the CI to keep things less changed
* Revert "Testing against mysql in the CI to keep things less changed"
This reverts commit fc940c111d.
* Fixing a test broken in postgres but working in mysql
* Fixing some tests
* Fixing some config tests with postgres
* Fixing the rest of config tests
* Fixing govet error
* Fixing search tests for postgres
While sending the config broadcast message across a cluster, we would
include the environment overrides in the config.
We fix this by exposing a config config store method to remove the overrides,
and then calling that before saving the config across the cluster.
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
