* MM 10658 Change config fields to pointers (#8898)
* Change fields of config structs to pointers and set defaults
MM-10658 https://github.com/mattermost/mattermost-server/issues/8841
* Fix tests that go broken during switching config structs to pointers
MM-10658 https://github.com/mattermost/mattermost-server/issues/8841
* Apply changes of current master while switching config structs to pointers
MM-10658 https://github.com/mattermost/mattermost-server/issues/8841
* Fix new config pointer uses
* Fix app tests
* Fix mail test
* remove debugging statement
* fix TestUpdateConfig
* assign config consistently
* initialize AmazonS3Region in TestS3TestConnection
* initialize fields for TestEmailTest
* fix TestCheckMandatoryS3Fields
* Fix shadowed variables in cmd package
* Fix shadowed variables in plugin package
* Fix shadowed variables in store package
* Fix shadowed variables in web package
* Changes as requested
Signed-off-by: Hanzei <hanzei@mailbox.org>
* Fix build
* Remove unnessary statements
* Use require all the time
* Fix build
* Rename variables according to feedback
* Fix NPE
* Changes as requested
* add query builder to facilitate dynamic SQL construction
* leverage query builder to refactor user store
This is partially setup work for MM-13120, but merged to master to avoid
further conflicts.
* fix unrelated unit test breakage
* documentation tweaks
* Apply suggestions from code review
Co-Authored-By: lieut-data <jesse.hallam@gmail.com>
* prefer comma separated case options to fallthrough
* vendor github.com/Masterminds/squirrel and deps
* switch to using github.com/Masterminds/squirrel
* rm querybuilder
* MM-13664 Added LinkMetadata types
* MM-13664 Use LinkMetadata when populating post metadata
* Fix unused import
* Fix index name on SQLite
* Finish adding unit tests
* Address feedback
* Increase max length of URL column to 2048 characters
* MM-13718 Prevent files from being attached to multiple posts
* Switch back to non-batched AttachToPost
* Change status code when failing to attach a file
* 7494 added the role to the user search filter
* 7494 changed the getUser function to accept the options
* added the role filter for the getAllProfiles method
* 7494 added the Inactive filter for AllProfiles
* 7494 refactored the where clause generation
* 7494 added the roles and inactive filters for inTeam Query
* 7494 fixed the review comments
* Initial models, API, app, and persistence of groups and group syncing.
* Consistent letter casing in ldif.
* Moves group-specific migrations into func.
* Adds API endpoint to retrieve LDAP groups (and associated MM groups) one tree level at a time.
* Adds mattermost group id to SCIMGroup (if available).
* Splits user and group creation so that memberOf works. Returns users from ldap interface.
* Updates method name.
* Returns users IDs instead of User.
* Removes non-essential group data.
* MM-11807: Add GroupFilter to LDAP config. (#9513)
* MM-11807: Add GroupFilter to LDAP config.
* Add diagnostic.
* Adds new config option for using 'memberOf' overlay.
* Adds API endpoint to link a group.
* Removes debug statements.
* Adds unlink group API endpoint.
* Fix to LDAP API. Adds API method to client4 and app.
* Adds some missing app methods. Renames API unexported func.
* Fixes link/unlink API path to accept valid DNs.
* Allow any character for DN portion of path.
* Switches from DN to objectGUID or entryUUID as the remote identifier linking LDAP groups to MM groups.
* Formatting.
* Formatting.
* Setting group name field to an ID for phase 1.
* Adds an LDAP config field to Setting up configuration for local LDAP.
* Changes to LDAP and GroupStore interfaces.
* Draft of nesting groups in API response.
* Removes unnecessary tree models.
* Updates group membershipt create store method to also restore.
* Adds new config to test config.
* Accept AD format length.
* Switches to SetUniqueTogether method.
* Updates revert.
* Tweaks to syncing queries .
* Updates query for pending team and channel memberships.
* Removes old GroupSyncableScanner usage. Some formatting and renaming.
* Fixes bug setting syncable type in selecting paged.
* Adds tests for syncables populator.
* Only add users to teams and channels that are not deleted.
* Renames method.
* Updates test LDAP setup.
* Removes memberof config stuff.
* Renames.
* Updates test data.
* Fix for gofmt.
* Adds missing license.
* Adds missing teardowns.
* Test fix.
* Adds a cycle to the groups test data.
* Changes API to return flat list.
* Removes some unused interface and app methods.
* Returns empty braces if results are empty.
* Adds more LDAP test data.
* Fix for test data error.
* Adds error.
* Moves test groups.
* Adds OU for load test data.
* Moves load test ou creation to load data.
* Adds a new bool flag to SCIMGroups.
* Removes SCIMGroup completely.
* Removes FULL JOIN because it is not supported in MySQL.
* Adds tests for sync queries; renames constant.
* Bad merge fix.
* Vet fix.
* Returning OK on delete ldap group link
* Removes foreign key constraints.
* Adding total to the ldap getAllGroups api endpoint
* Adds get group members page.
* Removes pagination from groups syncables list API.
* Adding syncable check now that foreign key constraint is removes.
* Joins teams and channels to group syncables.
* Adds group member count.
* Adding GetAllChannels and SearchAllChannels for system admins only
* Fix.
* Test fix from pagination removal.
* Orders groupmembers by createat.
* Fixing search of all channels
* Test fix after removing pagination.
* JSON syntax error fix.
* Changing tests (for now) pending investigation.
* Adding GetAllChannels and SearchAllChannels tests for the store
* Adding GetAllChannels and SearchAllChannels API tests
* Omit empty JSON values of group syncables.
* Fixing GetAllChannels and SearchAllChannels tests
* Fixing GetAllChannels and SearchAllChannels store tests
* Fixing GetAllChannels api tests
* Adds 'LDAP groups' feature flag. (#9861)
* Migrate new client functions to idiomatic error handling
* Test fixes.
* Simplification of groups api (#9860)
* Simplification of groups api
* Fixing RequireSyncableType
* Test fix.
* Update api4/group.go
Co-Authored-By: mkraft <martinkraft@gmail.com>
* Update api4/group.go
Co-Authored-By: mkraft <martinkraft@gmail.com>
* Update api4/group.go
Co-Authored-By: mkraft <martinkraft@gmail.com>
* Update api4/group.go
Co-Authored-By: mkraft <martinkraft@gmail.com>
* Update api4/group.go
Co-Authored-By: mkraft <martinkraft@gmail.com>
* Update api4/group.go
Co-Authored-By: mkraft <martinkraft@gmail.com>
* Update api4/group.go
Co-Authored-By: mkraft <martinkraft@gmail.com>
* Update api4/group.go
Co-Authored-By: mkraft <martinkraft@gmail.com>
* Update api4/group.go
Co-Authored-By: mkraft <martinkraft@gmail.com>
* Update api4/group.go
Co-Authored-By: mkraft <martinkraft@gmail.com>
* Fix copy/paste error.
* Fix copy/paste error.
* Adds missing return, changes to correct HTTP status code.
* Adds missing return, changes status codes.
* Check for license.
* Renames variable for new signature.
* Adds client method to get a group.
* Adds client method and tests for PatchGroup.
* Adds more API tests.
* Adds groups API tests.
* Adds client method and tests for getting group syncables.
* Adds tests for patching group teams and channels.
* Update to translations.
* Removes test.
* Fix incorrect conditional.
* Removes unnecessary nil check.
* Removes unnecessary return.
* Updates comment, removes unused variable.
* Uses consistent JSON unmarshal pattern.
* Uses consistent JSON unmarshal pattern.
* Moves const block.
* Switches 'already linked' from error to success response.
* Removes commented-out code.
* Switched to status ok.
* Add parens for readability.
* Fix copy/paste error.
* Unexport some structs.
* Removes repeated validity check.
* Return without attempting commit if there's a rollback.
* Fix incorrect HTTP status code.
* Update store/sqlstore/group_supplier.go
Co-Authored-By: mkraft <martinkraft@gmail.com>
* Adds utility methods for going from groupsyncable to groupteam and groupchannel.
* Fixing george suggestions (#9911)
* Test fix.
* Adds QA data to VC with visualization.
* Fixes typo in graph image.
* Update display name when re-linking in case it has changed in LDAP.
* Adds ability to configure group display name and unique identifier. (#9923)
* Adds ability to configure group display name and unique identifier.
* Adds some configs to confi-ldap make command.
* Fix for move of session.
* Exposes method for use by SAML package.
* Switches GroupSyncableType from int to string.
* Update Jenkins build files.
* Removes unused variable assignment.
* Removes old unnecessary early return.
* Removes unnecessary variable.
* Moves param parsing before license and permissions checks.
* Removes old code.
* Compares agains underlying error rather than error id.
* Switches tests to assertions.
* Adds more assertions.
* Adds missing return.
* Adds space after comma for added legibility.
* Moves a view model to the api package.
* Unexports method.
* Uses id validator function.
* Fix docker-compose flag.
* Typo fix.
* Moves index creation to supplier.
* Removes bad merge.
* Renames parameter.
* Re-adds space.
* Removes unnecessary transaction.
* Escapes the Groups table name with backticks because it is a reserved keyword.
* Fix roles cache bug
* Removing unnecesiary deserializing function
* Switches table name rather than custom SQL everywhere for Postgres without backticks.
* Removes redundant check for sql.ErrNoRows.
* Removes redundant check for sql.ErrNoRows.
* Removes data integrity check and redundant nil conditional.
* Removes redundant check for sql.ErrNoRows.
* Removes unnecessary query.
* Removes ID length validation from persistence tier.
* Makes some supplier methods idempotent.
* Removes some empty switch defaults.
* Renames Group Type field to Source.
* Fix for mistaken field name change.
* Uses IsValidId function.
* Removes comment.
* Changes json key name.
* Removes test because no longer validating user.
* Moves model state validation to app layer.
* Don't create Groups.CanLeave column until phase 2.
* Removes state validation until properties are used in phase 2.
* Removes duplicated check.
* Removes state validation until properties are used in phase 2.
* Removes some tests until phase 2.
* Comment-out a bunch of test related to CanLeave.
* Extra unmarshal validation check. Removes more code for CanLeave.
* Removes tests for CanLeave.
* Explict error msg.
* Rewrite queries.
* Changes index name. Adds index.
* Removes assertion.
* Adds experimental feature flag.
* 6798 added a new api to get the bulk reactions for posts
* 6798 added the permsission check before getting the reactions
* GH-6798 added a new app function for the new endpoint
* 6798 added a store method to get reactions for multiple posts
* 6798 connected the app function with the new store function
* 6798 fixed the review comments
* MM-10856: deduplicate posts with the same pending post id
Leverage a fixed size cache with a window of 30 seconds to deduplicate posts received by a single app server. Clients that duplicate the same pending post id will see one request potentially delayed until the first finishes, after which the same payload should be returned by both. Duplicate posts outside the 30 second window will not be de-duplicated.
Note that the cache is not synchronized between app servers. In an HA cluster consisting of more than one app server, sticky load balancing (e.g. hashing by IP or Session ID) is required to route the users to the same app instance for multiple requests.
Other options considered for this feature:
* adding a column to the `Posts` table: rejected as being too heavyweight
* maintaining a `PendingPostIds` table: similarly rejected for the database impact
* using the pending post id as the post id and relying on the unique constraints on the Post table: rejected for being difficult to show that it's safe to use a client-provided value as the row identifier
* utils/lru: simplify to ttl internally and for new methods
* move seenPendingPostIdsCache to App.Server
* just fail concurrent post requests, vs. trying to wait
* add debug log when create post is deduplicated
* MM-12083: unittest using externally managed database
* cherry-pick Makefile changes from @cpanato
* Jenkins changes (#9915)
* add docker compose
* udpate
* when using minio dont need to set the region
* update
* add wait for it script
* using old minio
* add new jenkins file
* update makefile
* add dockerfile
* rename the docker-compose proj to avoid colision (#9917)
* rename the docker-compose proj to avoid colision
* enable debug
* enable debug to double checkt the branchs and fix docker-compose name (#9919)
* add ee hash to check (#9920)
* fix name (#9921)
* update jenkins file to push from branch and prs
* if a new push comes in stop the running build
* split mysql and postgres variables
* add script to run jenkins-like env in local dev env
* update docker-compose project name to use uuid to make it more randon
* fix DCNAME definition
* update elasticsearch docker image
* revert test
* tidy up stages, and wait for mysql differently
* update docker image and add check for postgres
* checking if is ready
* update docker compose to have a wait for deps
* add readme and rename dockerfile
* fix -unittest setup
* using mm docker image
* restore parallel unit tests at the package level
Spin up a dedicated database for each package under test to avoid races in accessing the same tables.
Simplify the interface for configuring the test database to just a DSN instead of multiple exports for each field.
* try to work around root mysql access in CI
* update local-test-env.sh too
* MYSQL_ROOT_HOST: %
* fix missing quotes
* setting some memory limits for mysql
* revert memory docker compose does not support
* fix env name for postgres
* expose errors in app/export_test.go
* fix test label, better error checking on teardown
* increase query timeout for tests
* fix export_test
* update local dev script
* add configurable mysql root passwd
* api4: improved error handling
* system_store: more logs
* integrate go-junit-report into test-te/test-ee
* add CI_MINIO_HOST and CI_INBUCKET_HOST instead of CI_HOST
* comment re: minio configuration issue
* fix TestStartServerPortUnavailable to pass even when root can bind to :21
* skip TestFindManifest_FolderPermission while running as root
* api4: fix TestGetUsersNotInTeam assertions
This test was relying on data from a previous test run. With the data cleared before each test, the assertions much match reality.
* *testlib: always InitSystemAdmin
Some tests implicitly relied on the basic user having system
administrator privileges because it was the first user created as such.
Eliminate `InitSystemAdmin` and explicitly create the system admin user
instead to avoid this ambiguity going forward.
* *testlib: drop all tables before each test
* api4: split up TestChannelDelete to avoid duplicate InitBasic
* api4: teardown in TestResetPassword, for when this test comes back
* invalidate cache on DropAllTables
This is necessary since the test store persists across tests.
* disable parallel tests
While tests within a package must be explicitly parallelized using `t.Parallel()`, tests across packages are run in parallel by default. This causes problems given that the tests all currently share the same database instance.
Unfortunately, this also means that running the tests is much slower, but we can return to this later.
The searches on Postgres using LIKE with prefix matching will only
use an index if it has the text_pattern_ops flag (forcing C collation).
This drops the old indexes and adds properly optimized ones.
* #132 added UserTermsOfService model
* #132 added UserTermsOfService model
* #132 added logic to save user TOS data in a new table
* #132 Added logic to save and delete user TOS. Updated user TOS action logic
* #132 updated store mocks
* #132 added tests
* #132 removed cache from UserTermsOfService SQL store
* #132 fixed styling and license check
* #132 added message translations in en.json
* #132 fixed save user TOS logic to work second time as well
* #132 removed User.AcceptedTermsOfService colum and migrated accepted TOS data into new table
* #132 fixed formatting
* #132 fixed formatting
* #146 added field 'mandatory' to terms of service
* #146 updated tests
* #146 added getLatestTermsOfService API
* #146 Added tests
* #146 fixed styling
* #146 removed code for managing mandatory/optional TOS
* #146 Added TOS re-acceptance period config
* #146 fixed styling
* #146 removed some code left for debugging
* #146 added TOS re-acceptance period in config
* #146 fixed a json name from service_terms to terms_of_service
* #146 Minor refactoring and added TOS re-acceptance period to diagnistics
* Fixed style
* Updated upgraded script to keep app backward compatible
* MM-11272 Added app.PreparePostForClient
* MM-11272 Added app.PreparePostListForClient
* MM-11272 Added EmojiStore.GetMultipleByName
* MM-11272 Added emojis to PreparePostForClient
* MM-11272 Added unit tests for getting reaction counts
* MM-11272 Added unit tests for TestPreparePostForClient
* MM-11272 Added emojis from reactions to Post.Emojis
* MM-11272 Always update post.UpdateAt when reactions change to bust cache
* Fixed merge conflicts
* Moved post metadata-related code into its own file
* Update store mocks
* Fixed typo
* Add missing license headers
* Updated post metadata tests when custom emojis are disabled
* Fix unreliable unit tests
* Fix inconsistent casing in SQL statements
* Fix blank line
* Invalidate store cache after making changes
* Clear post cache synchronously with reactions
Expire K/V Values
Regenerate Code
pathfix
Update Expiry on Update
Check for Exit Signal
gofmt
Rewrote Go Routine
Remove tempoarily cleanup loop
fix expiretime
TEST: Expired Watchdog as GoRoutine
Check if Srv is nil
Use Scheduler/Worker for Expired Key CleanUp
add license
fix scheduler job type; DoJob Restructuring
Remove unused imports and constants
move db migration from 5.4 to 5.5
* #124 renamed identififers from service terms to terms of service
* #124 renamed identififers from service terms to terms of service
* 124 renamed ServiceTerms model to TermsOfService
* 124 Renamed EnableCustomServiceTerms feature flag to EnableCustomTermsOfService
* 124 Renamed EnableCustomServiceTerms feature flag to EnableCustomTermsOfService
* #124 fixed formatting
* #124 fixed formatting
* #132 renamed table ServiceTerms to TermsOfService
* #124 renamed some missed files from 'service_terms' to 'terms_of_service'
* #124 removed fixed TODOs
* drop migrate of ServiceTerms table, since backporting
* s/ServiceTerms/TermsOfService/ in tests
* s/AcceptedServiceTermsId/AcceptedTermsOfServiceId/
Change the model attribute, even though the column name will eventually be removed.
* s/accepted_service_terms_id/accepted_terms_of_service_id/ to match redux
* s/serviceTerms/termsOfService
* rename column too, and add max size constraint
* s/EnableCustomServiceTerms/EnableCustomTermsOfService
* MM-12067: Add SetDefaultProfileImage to reset the user profile image to a generated one
* Allow to get the default profile image for my user
* Allowing to reset the last update image date to 0
* PR reviews
* #MM-12130 changes for custom service terms
* Fixed styling
* Added getServiceTerms API
* removed unnecessary panic
* removed custom service terms text from flat config
* reverted user sql store as those changes are no longer needed
* added tests
* Updated a config key to be more standard
* Added copyright info
* Loading service terms only if the feature is enabled
* Loading service terms only if the feature is enabled
* removed unused index
* added createservice termns API
* made a param to bool instead of string
* added createservice termns API
* review fixes
* fixed styling
* Minor refactoring
* removed saveConfig and loadConfig magic
* added empty service terms text check to createServiceTerms API
* refactoed some urls to be terms_of_service instead of service_terms
* removed check for support settings
* changed URLs in tests
* removed unused code
* fixed a bug
* added service termd id in conif
* fixed a test
* review fixes
* minor fixes
* Fixed TestCreateServiceTerms
* MM-7188: Cleaning push notification on every read, not only on channel switch
* Removed unnecesary goroutine
* Fixing tests
* Applying suggestion from PR
* MM-11707: Removes edit_others_posts permission from the team_admin role in MakeDefaultRoles().
* MM-11707: Tests fix.
* MM-11707: Update test store.
* MM-11707: Allow to change the permission for edit the others posts on TE
* Fixing tests
