Commit Graph

59 Commits

Author SHA1 Message Date
Ibrahim Serdar Acikgoz
5ea06e51d0 Migrate to stateless app.App (#17542)
* add request context

* move initialialization to server

* use app interface instead of global app functions

* remove app context from webconn

* cleanup

* remove duplicated services

* move context to separate package

* remove finalize init method and move content to NewServer function

* restart workers and schedulers after adding license for tests

* reflect review comments

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-05-11 13:00:44 +03:00
Jesús Espino
35d00b4644 Enable the errorAssertions govet check for mattermost-server code (#17346)
* Enable the errorAssertions govet check for mattermost-server code

* Removing unnecesary change

* Fixing some tests

* Fixing tests

* Fixing more after merge

* Fixing new offending entries

* Fixing small vet checks

* Fixing new cases detected by govet

* Fixing remote_cluster_test errors

* Fixing assertion

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-12 12:51:31 +02:00
Agniva De Sarker
db01f2a91b MM-34002: Improve AddUserToChannel (#17174)
* MM-34002: Improve AddUserToChannel

When we would add a user to a channel, we would
check whether the user is removed from that team or not.

During LDAP sync, this check is not required because the
team member would have just been created. Hence, we
pass a boolean flag to bypass the check.

And with that done, we can freely query the replica.

https://mattermost.atlassian.net/browse/MM-34002

```release-note
NONE
```

* Refactor code

* Rename a struct field

* fix double negative
2021-04-02 14:33:23 +05:30
Agniva De Sarker
4f0cfbe329 MM-33913: Fix read-after-write issues with team member (#17173)
During LDAP sync, we would call AddTeamMember which had a read-after-write issue
where we would create a team member but then immediately after that
query the team member.

The same pattern was found in:
AddTeamMember
AddTeamMembers
AddTeamMemberByToken

To fix this, we just return the inserted team member from AddUserToTeam and use that
instead of query GetTeamMember again.

```release-note
NONE
```

https://mattermost.atlassian.net/browse/MM-33913

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-02 13:20:58 +05:30
Jesús Espino
9cc5089af8 Moving mlog to corelibs (#16915)
* Moving mlog to corelibs

* Regenerating app layers

* Fix golangci-lint problem

* Fixing golangci-lint errors

* Renaming from corelibs to shared

* Renaming from corelibs to shared

* Fixing import

* Fixing merge problems

* Fixing build
2021-03-05 09:18:37 +01:00
Ibrahim Serdar Acikgoz
b3498a1edf [MM-32626] Disable notices for migrations test helper (#17058)
* app/server: make initial product notice fetch blocking

* disable notices on migrations_test helper
2021-03-05 10:57:23 +03:00
Mahmudul Haque
a63dea6c55 unparam lint (#16771)
* fixed: `identifier` is unused lint error

* make saveMultipleMembersT method saveMultipleMembers

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-02-04 11:08:05 +05:30
Madhav Hugar
94c24eea20 Fix empty string comparison issues in the codebase (#16686)
Automatic Merge
2021-01-25 11:15:17 +01:00
Agniva De Sarker
c1dd23a3c8 MM-31063: Change constants to use CamelCase (#16608)
* MM-31063: Change constants to use CamelCase

* store package

* change allcaps to camel case (#16615)

* New tools.mod

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2021-01-04 11:32:29 +05:30
Ibrahim Serdar Acikgoz
8036ba86e7 [MM-31398] upgrade error level to critical in case of panic or os.Exit (#16584)
* upgrade error level to critical in case of panic or os.Exit

* reflect revivew comments

* fix error capitalization
2020-12-23 13:11:35 +03:00
Agniva De Sarker
1a131b54af MM-31064: Simplify indentation (#16565)
* MM-31064: Simplify indentation

Reduce indentation where possible.

```release-note
NONE
```

Command ran to verify:
golangci-lint run --disable-all --enable golint --max-issues-per-linter=10000 --max-same-issues=100000 ./... | grep "block ends with a return state"

https://mattermost.atlassian.net/browse/MM-31064

* incorporate review comments

* enable golint for outdent rule

* fix remaining issues

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-12-21 18:50:47 +03:00
Florent Peterschmitt
349b83f23a implement and use striped LRU cache to lower mutex contention (#15764)
* Implement Striped LRU cache

* ci

* fix

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-12-09 19:25:14 +05:30
Deep Baldha
2d742600f4 MM-30818: Change to idiomatic receiver names for migrations.TestHelper (#16343)
Automatic Merge
2020-12-04 05:45:17 +01:00
Jesús Espino
a74fe05695 Removing supplier concept from the sql store (#16355)
* Removing supplier concept from the sql store

* Removing other metions to supplier

* Fixing gofmt

* Fixing gofmt

* Renaming NewSqlStore to New

* Fixing tests

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-12-02 12:06:23 +01:00
Christopher Speller
1aadd36644 MM-28859 Add feature flag managment system using split.io and remove viper. (#15954)
* Add feature flag managment system using split.io and remove viper.

* Fixing tests.

* Attempt to fix postgres tests.

* Fix watch filepath for advanced logging.

* Review fixes.

* Some error wrapping.

* Remove unessisary store interface.

* Desanitize SplitKey

* Simplify.

* Review feedback.

* Rename split mlog adatper to split logger.

* fsInner

* Style.

* Restore oldcfg test.

* Downgrading non-actionable feature flag errors to warnings.

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-10-29 15:54:39 -07:00
Rodrigo Villablanca
bb4df5a68e ChannelStore migration Part 3 (#15504)
* Migration finished

* Change error var name

* Fix imports

* Fix tests

* Merge with master

* Doing some suggestions

* More suggestions

* Fix i18n

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
2020-10-04 10:12:29 +05:30
Rodrigo Villablanca
f0fc51244c TeamStore migration Part 3 (#15397)
* Migration completed

* TODO
2020-09-16 12:12:10 -03:00
Doug Lauder
1b67322fb9 MM-27744 disable Zap for unit tests. (#15398)
MM-27744 disable Zap for unit tests.

Zap has no concept of shutdown or close. Zap is only shutdown when the app exits. Not a problem for console logging, but when creating a new Zap logger that outputs to files on every unit test, that leaves no easy way to clean up until process exit. Depending on what else is running this can exhaust all file handles and cause unit tests to fail.

Zap is now disabled unit tests and uses Logr instead, regardless of config settings. `make test-server` peak file handle usage dropped from ~5K to less than 100.
2020-09-09 15:25:55 -04:00
Jesús Espino
44079785eb Moving diagnostics into a service (#14832)
* Moving diagnostics into a service

* Fixing golint checks

* Fixing tests

* Renaming from diagnostics to telemetry

* Adding missing files

* Initializing telemetry earlier in the server startup

* Fixing tests

* Adding a log for the telemetryID initialization error

* Addressing PR review comments

* Fixing merge problem

* Removing some extra Diagnostics mentions

* Making tests pass
2020-09-08 20:30:54 +02:00
Agniva De Sarker
78766625df jobstore (#15250)
* Migration completed

* Modify test case

* Update en.json

* Fix layers

* Lint: remove unnecessary use of sprint

* trigger CI

* fix error

* Fixes

* fix test

Co-authored-by: Rodrigo Villablanca <villa061004@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-08-20 19:37:19 +05:30
Rodrigo Villablanca
20e44399c7 SystemStore migration to return plain errors (#14835)
* SystemStore migration to return plain errors

* Fix nilness

* Fix translations

* Fix merge

* Fix layers

* Fix merge errors

* Lint: remove unnecessary use of sprint

* Fix merge errors

* Fix i18n

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
2020-08-13 20:32:57 +05:30
Harrison Healey
504f45b6ec MM-27007 Remove automatic sidebar migration (#15087)
* MM-27007 Add migration of favorited channels to CreateInitialSidebarCategories

* MM-27007 Rewrite migrateFavortitesToSidebarT to use ROW_NUMBER() when available

* MM-27007 Remove automatic sidebar migration

* Remove old i18n strings

* Fix typo

* Address feedback

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-07-23 20:54:29 -03:00
Siyuan Liu
c7f7bef9ec move cache2 package to cache (#14921)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-07-18 13:31:06 +05:30
Harrison Healey
de6a57cdc3 Sidebar V2 Phase 2 (#14967)
* parent 48256721c4 (#14358)

author Eli Yukelzon <reflog@gmail.com> 1585814774 +0300
committer Eli Yukelzon <reflog@gmail.com> 1589111022 +0300

Sidebar caregories implemented

Apply suggestions from code review

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

Update store/sqlstore/channel_store.go

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

Update store/sqlstore/channel_store.go

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

code review suggestions

status messages

edge case

bugs...

timeout reverse

* MM-25126 Add a separate default sorting method for categories (#14575)

* MM-25158 Add user to initial sidebar categories when joining team (#14570)

* MM-25281  Place new categories in the correct position (#14609)

* MM-25277  Return channels that aren't in a category as part of the Channels/Direct Messages categories (#14601)

* MM-25276  Remove categories when leaving a team (#14600)

* Remove categories when leaving a team

* layers

* corrected cleanup function

* lint

* .

* corrected errors in postgres

* .

* MM-25280  Ensure that the "update category order" API call only contains real category IDs and isn't missing any IDs (#14626)

* Ensure that the "update category order" API call only contains real category IDs and isn't missing any IDs

* tests

* correct status code

* MM-25278  Change "update category" API to return 400 when changing unsupported fields (#14599)

* MM-25279  Change category migration to only populate channels in Favorites (#14627)

* MM-25157 Add API to delete custom categories  (#14574)

* MM-25157 Add API to delete custom categories

* get categories fix

* maxorder fix

* Use correct websocket event when deleting category

* Fix tests and remove debug code

* Actually use the right websocket event this time

* test cleanup

* Update test for new category order

Co-authored-by: Eli Yukelzon <reflog@gmail.com>

* MM-24914 Various fixes for sidebar channel handling (#14756)

* Fix checking for channel membership when reordering channels

* Remove unique constraint on SidebarCategories

* Set column sizes for SidebarCategories and SidebarChannels tables

* Allow changing the sorting method for non-DM categories

* Fix nil pointers in error handling

* Fix orphaned channels from other team being returned in Channels category

* Fix non-orphaned channels being duplicated in the Channels category

* Remove unique constraint on SidebarChannels

* Fix category/name of favorites preferences

* Fix testSidebarChannelsMigration

* Rename err to nErr and appErr to err

* Fix channel order returned by GetSidebarCategories on MySQL

* Fix adding/removing favorites preferences

* Remove leftover TODO

* Change SidebarCategoryType enums to use full names (#14786)

* Change SidebarCategoryType enums to use full names

* Fix Channels constant

* Remove leftover debug code

* MM-24914 Fix updateCategory endpoint returning the wrong type (#14795)

* MM-24914 Make some changes to UpdateSidebarCategories (#14806)

* Fix orphaned DMs not always being returned

* MM-24914 Make some changes to UpdateSidebarCategories

* Run updateSidebarCategoryOrderT in a transaction

* Fix deleting SidebarChannels based on order of arguments to UpdateSidebarCategories

* bump for api testing

* bump for api testing

* Change CreateInitialSidebarCategories to return a plain error

* Change MigrateSidebarCategories to return a plain error

* Remove usage of UpdateColumns when updating sidebar categories (#14843)

* Remove usage of UpdateColumns when changing category order

* Add a random test case

* Remove usage of UpdateColumns when updating sidebar categories (#14843)

* Remove usage of UpdateColumns when changing category order

* Add a random test case

* Remove usage of UpdateColumns when updating sidebar categories (#14843)

* Remove usage of UpdateColumns when changing category order

* Add a random test case

* MM-26343 Make CreateInitialSidebarCategories idempotent (#14870)

* Fix bad merge

* Fix another bad merge

* Fix unintentionally removed i18n string

Co-authored-by: Eli Yukelzon <reflog@gmail.com>
2020-07-06 18:20:35 -04:00
Jesús Espino
e2e352e223 Change the App dependency with Server in migrations package (#14804)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-06-29 18:00:16 +02:00
Siyuan Liu
aaea36a24d MM-25115 - make cacheProvider2 accessible to localcachelayer (#14691)
* make cacheProvider2 accessible to localcachelayer

* migrate localcachelayer

* revert test.sh

* workaround gob deserialization issue

* fix web_test.go

* merge conflicts

* resolve merge conflicts

* revert back test workaround

* fix the wrong merge
2020-06-25 17:08:53 +05:30
Jesús Espino
f5eab1271b Removing all FakeApp usages (#14174)
* Removing some other fake apps

* More FakeApp removed

* Removing entirely FakeApp

* Fixing some tests

* Fixing get Cluster id from get plugin status

* Fixing failing tests

* Fixing tests

* Fixing test initialization for web

* Fixing InitServer for server tests

* Fixing InitServer for server tests

* Reverting go.sum and go.mod

* Removing unneded HTMLTemplates function in App layer

* Moving back some functions to its old place to easy the review

* Moving back some functions to its old place to easy the review

* Using the last struct2interface version

* Generating store layers

* Fixing merge problems

* Addressing PR comments

* Small fix

* Fixing app tests build

* Fixing tests

* fixing tests

* Fix tests

* Fixing tests

* Fixing tests

* Fixing tests

* Moving license to server struct

* Adding some fixes to the test compilation

* Fixing cluster and some jobs initialization

* Fixing some license tests compilation problems

* Fixing recursive cache invalidation

* Regenerating app layers

* Fix test compilation

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-06-12 13:43:50 +02:00
Agniva De Sarker
14f7118dde MM-22706: pass along set_online flag in websocket response (#14591)
* MM-22706: pass along set_online flag in websocket response

To let the client know whether a user has created a post without
being online or not, we get the set_online query param and
pass it down to the websocket event being passed down to the client.

With this PR, the "data" field of the `posted` event will contain
a `set_online` boolean field set to true/false depending on the
query_param set_online value set in the createPost call.

* Setting to false for auto responder

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-05-27 17:35:02 +05:30
Shibasis Patel
882b0324b5 MM_22682_Centralize_ID_Validation (#14237)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-05-07 13:27:35 -04:00
Agniva De Sarker
42081a1441 Remove unncessary recover() calls (#14066)
* Remove unncessary recover() calls

A recover which is not called inside a defer will always return nil.
So there's no use of calling it.

And even if we did call it inside a defer, recovering and immediately
panicking does not make sense either. So we just remove it.

* Fixed another instance
2020-03-17 11:04:39 +00:00
Mario de Frutos Dieguez
b162cf92cd MM-21976: Include cache layer to be tested (#13749) 2020-03-04 14:18:03 +01:00
Eli Yukelzon
17523fa5d9 MM-21898 - Part 1: Generate and use an interface instead of *A… (#13840)
* Generate and use an interface instead of *App
2020-02-13 13:26:58 +01:00
Jesús Espino
1d1ab03c38 Passing t to all tests setup functions (#13841)
* Passing t to all tests setup functions

* Fixing build
2020-02-10 19:31:41 +01:00
Andy Librian
3865bc501e golint fix (#13613) 2020-01-20 12:32:20 +01:00
Jesús Espino
a63684fcb5 Consistent license message for all the go files (#13235)
* Consistent license message for all the go files

* Fixing the last set of unconsistencies with the license headers

* Addressing PR review comments

* Fixing busy.go and busy_test.go license header
2019-11-29 12:59:40 +01:00
Miguel de la Cruz
2259b7f2a8 [MM-19948] Set version on module file and internal paths (#13186)
* [MM-19948] Set version on module file and internal paths

* Fixes after merge

* Fix i18n checker error
2019-11-28 14:39:38 +01:00
Joram Wilander
404c49f62f MM-16725 Respect env var overrides when setting config (#11821)
* Respect env var overrides when setting config

* Use strings.NewReader
2019-08-09 11:33:59 -04:00
Allan Guwatudde
15be0c6c79 [MM-16777] Migrate Team.MigrateTeamMembers to Sync by default (#11584)
* Migrate Team.MigrateTeamMembers to Sync by default

* Remove commented out MigrateTeamMembers function

* Remove return nil when no team members in query

* Return nil when no more team members in query

* Remove commented out code lines

* Format MigrateTeamMembers code

* Return some data from MigrateTeamMembers function

* Return data and error from MigrateTeamMembers

* Use result for consistency

* Control loop to allow loop to break in storetest/team_store.go

* Fix test MigrateTeamMembers test in storetest/team_store.go

* Change err to e in storetest/team_store.go
2019-07-10 12:39:24 +02:00
Shobhit Gupta
bd3f72254d Migrate Channel.MigrateChannelMembers to Sync by default (#11277) 2019-06-20 14:37:06 +02:00
Jesús Espino
f934502a56 Migrate Jobs store to sync by default (#11183)
* Migrate Jobs store to sync by default

* Fixing compilation

* Fixing compilation

* Fixing govet
2019-06-15 17:55:06 +02:00
Jesús Espino
f8a89a52fc Migrate System store to Sync by default (#10838) 2019-05-21 18:22:27 +02:00
Jesús Espino
584ec68755 MM-15021: Adding new builtin system schema for Guests (#10601)
* MM-15021: Adding new builtin system schema for Guests

* Fixing tests

* Setting properly the permissions

* Adding guests to sampledata

* Restrict more roles updates in the app layer for guests

* Adding comment to explain that permissions migration must go at the end

* Setting the default guest role for custom scheme during migration

* Fixing import and export

* Creating scheme guest roles on migration

* Fixing tests

* Fixing tests

* Fixing tests
2019-04-30 20:36:21 +02:00
Puneeth Reddy
ec95793b90 MM-15276: Migrate Team.Update to sync by default (#10693)
* MM-15276: Migrate Team.Update to sync by default

* MM-15276: Addressing review comments and change Update func signature similar to other interface Update method

* update store mocks for update fn

* addressing review comments
2019-04-25 15:29:02 +02:00
Jesse Hallam
1e462da2d4 MM-14143 config cleanup final (#10374)
* TestGetLicenseFileFromDisk: avoid using fileutils.FindConfigFile

* config: abstract config-related file access, extend memory store

* simplify config validate to avoid file knowledge

* fix relative file tests

* cluster: fix ConfigChanged event

The old and new configurations were swapped when notifying the enterprise code of configuration changes, creating needless instability in propagating config updates across a cluster.

* config/database: ignore duplicates

* test cleanup

* remove unnecessary Save() in test
2019-03-06 15:06:45 -05:00
Sandeep Sukhani
29060acb45 [MM-13828] Initialize tests in each package with a new temp folder with all test resources (#10261)
* [MM-13828] Running tests from a new temp folder with all test resources

Possible fix for #10132

All packages which have a TestMain and use testlib.MainHelper will have a new current working directory which will have all the test
resources copied.

Note: default.json is copied as config.json as well to make sure tests don't have any impact due to changes in config by devs

* [MM-13828] Added TestMain to remaining packages to use testlib.MainHelper

This makes sure tests from all packages run with same test resources, setup in a new temp folder for each package

* Updated Jenkins file to not not config/default.json

This makes sure CI has same config files as a dev's machine

* [MM-13828] Changes requested from code review

Added accessor methods to testlib.MainHelper for accessing members
Fixed some broken tests due to change in cwd while tests run
Some other code refactoring and improvements

* [MM-13828] Added new factory method with options for creating test main helper and some code refactoring

testlib.NewMainHelperWithOptions supports options to turn on/off test dependencies and environment setup
Some other code refactoring

* Exporting members of testlib.MainHelper to make enterprise tests work

* Fixed gofmt error

* [MM-13828] removed unwanted dependency on plugins directory while setting up test resources

* [MM-13828] Fixed some tests failing due to them being running from temp folder

* [MM-13828] Some code changes suggested in PR review

* Fixed gofmt error
2019-02-19 09:20:11 -05:00
Jesse Hallam
9bf5687311 MM-11262: database config store (#10281)
* vendor github.com/jmoiron/sqlx

* MM-11262: introduce a database store

* revert unnecessary fmt.Errorf

* simplify unit test helper methods

* remote TODO re: retry

* relocate initializeConfigurationsTable for clarity

* factor out a commonStore

* acquire database config lock on close for safety

* add missing header

* fix lock comment
2019-02-15 10:05:29 -04:00
Jesse Hallam
3a71709103 MM-13893: refactor config (#10230)
* refactor utils/config* to config/

* pull validateLdapFilter into app

* clean up Config/GetConfig/GetSanitizedConfig usage

Eliminate app.GetConfig() in favour of just using app.Config() directly,
but expose app.GetSanitizedConfig() for when the old behaviour was
required.

* web: isolate config setup

* TestInvitePeopleProvider: make config explicit

* regenerateClientConfig: avoid racey map access

* integrate watch flag into app.ConfigFile option

* make app.Option return an error

* release.mk: only cp static files from config/

* release.mk: fix cp static files from config/

* api4: TestPlugin cleanup

* s/c/cfg/ for clarity

* fix merge conflict

* testlib: allow customization of testlib driver name
2019-02-12 08:37:54 -05:00
Christopher Speller
8429add371 Cleanup related to context refactor (#9988) 2018-12-17 08:51:46 -08:00
Jesse Hallam
d39d9a5caf Dockerized build updated tests (#9943)
* testlib: introduce and leverage

This doesn't yet factor out the individual test helpers: many packages
still rely on `api4` directly to do this, but now wire up the test store
setup through this package. `app` and `store`, in particular, don't use
`testlib` because of circular dependencies at the moment.

* cmd: command_test.go: use api4 testlib

* cmd: plugin_test.go: remove dependence on test-config.json

* cmd: config_test.go use configured database settings

* ensure test-(te|ee) exit with status code

* test-server: run all tests, deprecating test-te/test-ee

* cmd/mattermost/commands: fix unit tests

Instead of relying on (and modifying) a config.json found in the current path, explicitly create a temporary one from defaults for each test. This was likely the source of various bugs over time, but specifically allows us to override the SqlSettings to point at the configured test database for all tests simultaneously.

* wrap run/check into a test helper

It was insufficient to set a config for each invocation of CheckCommand or RunCommand: some tests relied on the config having changed in a subsequent assertion. Instead, create a new test helper embedding api4.TestHelper. This has the nice advantage of cleaning up all the teardown.

* additional TestConfigGet granularity

* customized config path to avoid default location

* be explicit if the storetest initialization fails

* generate safe coverprofile names in the presence of subtests

* additional TestConfigShow granularity

* fix permission_test.go typo

* fix webhook tests

* actually flag.Parse() to skip database setup on os.Execed tests

* fix recent regression in #9962, not caught by unit tests
2018-12-06 13:19:32 -05:00
Christopher Speller
da265fbaf7 Moving app from singular to being created for every request (#9889)
* Moving app from singular to being created for every request.

* Automatic refactor

* Adding license header

* Feedback fixes
2018-11-28 10:56:21 -08:00