Commit Graph

85 Commits

Author SHA1 Message Date
Ibrahim Serdar Acikgoz
6595b31f23 Replace go-migrate with morph (#19116) 2022-01-08 10:16:07 +02:00
Carlos Tadeu Panato Junior
36008ae412 fix mmctl download script when using as stand alone script (#19143) 2021-12-16 11:31:57 +01:00
Carlos Tadeu Panato Junior
c317254954 build: add mmctl to arm64 packages (#19090)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-12-03 15:37:37 +01:00
Carlos Tadeu Panato Junior
2865ad9221 Revert "DOPS-602: Change how mmctl gets packaged and locally fetched. (#18335)" (#18930)
This reverts commit 5ede63badc.
2021-11-03 14:54:34 +01:00
Elisabeth Kulzer
5ede63badc DOPS-602: Change how mmctl gets packaged and locally fetched. (#18335)
* DOPS-602: Change how mmctl gets packaged and locally fetched.
2021-11-02 14:45:02 +01:00
Agniva De Sarker
acd08f0a48 Fix migration scripts (#18756)
* Fix migration scripts

Set the DB schema version in DB prior to running migration

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=c2525dbb-23eb-4fe7-8fb4-e9a5f503dc8c

```release-note
NONE
```

* Revert schema to 6.0.0 and update CI scripts

* Fix typo

* Add missing default

* Fix extra comma

Co-authored-by: Claudio Costa <cstcld91@gmail.com>
2021-10-19 19:56:09 +05:30
Kyriakos Z
4b85f9ee6a MM-36743: adds last_root_post_at in channels table (#18366)
* MM-36743: adds last_root_post_at in channels table

Channel recency for CRT users should not count replies,
this commit solves that issue by adding a new column to the channels
table: LastRootPostAt.
With that new info CRT users can have recent channels to work as
expected.

* Adds the 'LastRootPostAt' to the SQL scripts

* Fixes tests

* Adds LastRootPostAt migration to v6.1

* Fixes index

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-13 15:20:44 -04:00
Claudio Costa
177680f08c [MM-36472] Fix inconsistencies in Roles columns (#18390)
* Fix inconsistencies in Roles columns

* Add new migrations
2021-10-01 15:31:29 +02:00
Ben Cooke
3b9324b389 [MM-36444] Manage config deprecations for 6.0 (#18035)
* removing deprecated stuff

* removing deprecations and marking others for deprecations

* removing some of the configs

* partially remove ExperimentalChannelOrganization

* fixing merge issues

* moving default permissions to defaultroles function

* removing experimentaltownsquare

* removing old tests and remove legacysidebar config

* fixing test

* removing log

* removing unused variables

* MM-36444: Fix for renamed functions.

* MM-36444: go mod tidy and go mod vendor.

* MM-36444: Vendored go modules.

* MM-36444: Removes UseExperimentalGossip config field from model.

* Fix vendors

* fixing test

Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Martin Kraft <martin@upspin.org>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
2021-09-14 13:22:59 -04:00
Harrison Healey
5ee9c349cb Update Node and NPM minimum versions (#18367) 2021-09-10 12:31:48 -04:00
Claudio Costa
9534f9fb62 Update schemas to include new indexes (#18313)
* Update schemas to include new indexes

* Increase local usability of gitlab scripts. (#18314)

* Increase local usability of gitlab scripts.

* Fix loading/saving SQL dumps

* Add logging to schema tests.

* Fix adding logs dir.

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

Co-authored-by: Elisabeth Kulzer <elikul@elikul.de>
2021-09-01 09:00:49 +02:00
Claudio Costa
ac252f0771 [MM-37621] Update db schema validation scripts for 6.0 (#18223)
* Fix migration scripts

* Update schema validation scripts
2021-08-31 09:50:47 +02:00
Martin Kraft
77dc29308f Fix for test-data make target (#18229)
* Fix for the test-data command.

* Sleep and warn about needing local mode enabled.

* Improved server start synchro.
2021-08-23 09:45:07 -04:00
Martin Kraft
8f01a1b5a1 MM-36448: Removes legacy CLI commands. (#17995)
* MM-36448: Removes legacy CLI commands.

* MM-26448: Update translations.

* MM-36448: Fixes some lint errors.

* MM-36448: Conflict resolution error fix. Lint fixes.

* MM-36448: Removes some more commands.

* MM-36448: Removes unused functions.

* MM-36448: Re-adds config command.

* MM-36448: Re-adds func for use by config.

* MM-36448: Moved structs back.

* MM-36488: Re-adds version.

* MM-36448: Re-added some commands.

* MM-36448: Fix tests.

* MM-36448: Removed unused func.

* MM-36448: Removes test.

* MM-36448: Removes uses of 'config set'.

* MM-36448: Moves some test structs.

* MM-36448: Removes the logs command.

* MM-36448: Re-deleted file after bad merge.

* MM-36448: Deleted test files again.

* MM-36448: Re-delete files.

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-08-18 14:06:28 -04:00
Ben Schumacher
757dc96461 [MM-37772] Idiomatic naming (URL, URI, API) (#18128)
* s/Url/URL/g & s/Uri/URI/g

* s/Api/API/g
2021-08-16 19:46:44 +02:00
Agniva De Sarker
c4b4e1bc38 MM-36271: Bump major version to 6.0 (#17973)
https://mattermost.atlassian.net/browse/MM-36271

```release-note
We bump the major version to 6.0
```
2021-07-22 12:21:47 +05:30
Elisabeth Kulzer
33edd645c7 Lock CI back to master 2021-07-21 16:38:39 +02:00
Ben Schumacher
97ccf0bdf6 [MM-16751] golint model (#17896) 2021-07-21 16:38:39 +02:00
Mattermod
953eebdef4 Lock CI for release-6.0 (#17756)
* Lock CI for release-6.0

* Update download_mmctl_release.sh

Co-authored-by: Mmbot <mmbot@mattermost>
Co-authored-by: Elisabeth Kulzer <elikul@elikul.de>
2021-07-21 16:38:39 +02:00
Claudio Costa
3681cd3688 [MM-32390] Config logic refactor (#17578)
* Replace config generator

* Cleanup

* Some renaming and docs additions to add clarity

* Cleanup logging related methods

* Cleanup emitter

* Fix TestDefaultsGenerator

* Move feature flags synchronization logic out of config package

* Remove unnecessary util functions

* Simplify load/set logic

* Refine semantics and add some test to cover them

* Remove unnecessary deep copies

* Improve logic further

* Fix license header

* Review file store tests

* Fix test

* Fix test

* Avoid additional write during initialization

* More consistent naming

* Update app/feature_flags.go

Co-authored-by: Christopher Speller <crspeller@gmail.com>

* Update config/store.go

Co-authored-by: Christopher Speller <crspeller@gmail.com>

* Update config/store.go

Co-authored-by: Christopher Speller <crspeller@gmail.com>

* Update config/store.go

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

* Move FF synchronizer to its own package

* Remove unidiomatic use of sync.Once

* Add some comments

* Rename function

* More comment

Co-authored-by: Christopher Speller <crspeller@gmail.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2021-05-19 13:30:26 +02:00
Hossein
daec0b3201 All Sections to SubSections (#16917)
* initial

* Revert "initial"

This reverts commit 3d631aeecd.

* [MM-32352] Add Experimental Subsections BACKEND (#16887)

Automatic Merge

* update appiface

* Fix app layers

* Ancillary Permissions on backend (#17061)

Automatic Merge

* [MM-32799] Add About Section (#17015)

* Add About Section

* add mock key

* Update role.go

* Update role.go

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>

* [MM-33437] Fix config access tags for experimental settings (#17111)

Automatic Merge

* [MM-32794] Reporting Sub Section (#17035)

* test

* revert

* add permissions

* add new permission stuff

* add store mock

* fix bad merge

* gofmt fix

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>

* [MM-32343] Environment SubSection (#17054)

* pre-checkout commit

* fix permission for testSiteURL

* pre-merge commit

* increase size of Permissions column in Roles table

* add entry for ENVIRONMENT to testlib/store.go

* use TEXT for Permissions column in Roles table

* use environment subsection permissions for API endpoints

* use subsections permissions for /config/environment

* add suggestions from hahmadia

* update tests to use subsection permissions

* add permissions column back in

* comment out code in upgradeDatabaseToVersion534

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* MM-32351: Add Compliance Subsections (#17023)

* add subsections for compliance sectin

* add to mock functions

* updates for read job

* fixes

* fix test

* update tests

* update tests

* another test fix

* some cleanup

* update mlog

* fix linting

* Fix bad merges

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Hossein <hahmadia@users.noreply.github.com>
Co-authored-by: Hossein Ahmadian-Yazdi <hyazdi1997@gmail.com>

* MM-32347 Site Subsections (#17095)

* Init

* Added migration key in testlib store

* Fix syntax error

* fix bad merge

* fix lint

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* MM-32350 Integrations (#17097)

* implement server subsections

* fix tests

* update test

* go fmt

Co-authored-by: Hossein Ahmadian-Yazdi <hyazdi1997@gmail.com>

* patch forgotten endpoints

* Adding subsection permissions for Authentication (#17087)

* adding new permissions, migrations to do

* permission migrations and ancilary permissions

* running make app-layers

* fixing tests and lint

* adding permissions to saml

* ldap write permissions

* running make app-layers

* fixing conflict

* making app layers

* clean up and fix tests

* change job type

* fix js error, if site url not returned

Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Hossein Ahmadian-Yazdi <hyazdi1997@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* Update permissions_migrations.go

* gofmt

* upgrade to 535

* gofmt

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
Co-authored-by: Anurag Shivarathri <anurag6713@gmail.com>
Co-authored-by: Ben Cooke <benkcooke@gmail.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
2021-04-06 10:39:48 -04:00
Eli Yukelzon
480796a1df MM-33708 - Add MentionCountRoot column to ChannelMembers (#17099)
* added new column for root-only mentions

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-01 14:43:09 +03:00
Eli Yukelzon
3c21eef110 MM-33746 hotfix (#17320)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-01 13:56:39 +03:00
Eli Yukelzon
ab5925c4de MM-33746 Add TotalMsgCountRoot and MsgCountRoot columns (#17150) 2021-03-31 16:51:02 +03:00
Agniva De Sarker
4aac52bced Makefile: Refactor test-server-race (#17196)
test-server-race wasn't using the same set of steps
that the test-server step did. Therefore one test was failing.

Refactored it such that scripts/test.sh can be used to run
normal and race tests as well

```release-note
NONE
```
2021-03-22 23:11:50 +05:30
Claudio Costa
d3a003dcb2 Address outstanding merging issues with cloud branch (#17160) 2021-03-17 10:39:46 +01:00
Martin Raymond Kraft
58dce5930e [MM-31094] Replication Lag (#16888)
* MM-31094: Adds tooling to develop and test using a MySQL instance with replication lag. Adds some lazy lookups to fallback to master if results are not found.

* MM-31094: Removes mysql-read-replica from default docker services.

* MM-31094: Switches (store..SessionStore).Get and (store.TeamStore).GetMember to using context.Context.

* MM-31094: Updates (store.UsersStore).Get to use context.

* MM-31094: Updates (store.PostStore).Get to use context.

* MM-31094: Removes feature flag and config setting.

* MM-31094: Rolls back some master reads.

* MM-31094: Rolls a non-cache read.

* MM-31094: Removes feature flag from the store.

* MM-31094: Removes unused constant and struct field.

* MM-31094: Removes some old feature flag references.

* MM-31094: Fixes some tests.

* MM-31094: App layers fix.

* MM-31094: Fixes mocks.

* MM-31094: Don't reparse flag.

* MM-31094: No reparse.

* MM-31094: Removed unused FeatureFlags field.

* MM-31094: Removes unnecessary feature flags variable declarations.

* MM-31094: Fixes copy-paste error.

* MM-31094: Fixes logical error.

* MM-30194: Removes test method from store.

* Revert "MM-30194: Removes test method from store."

This reverts commit d5a6e8529b.

* MM-31094: Conforming to make's strange syntax.

* MM-31094: Configures helper for read replica with option.

* MM-31094: Adds some missing ctx's.

* MM-31094: WIP

* MM-31094: Updates test names.

* MM-31094: WIP

* MM-31094: Removes unnecessary master reads.

* MM-31094: ID case changes out of scope.

* MM-31094: Removes unused context.

* MM-31094: Switches to a helper. Removes some var naming changes. Fixes a merge error.

* MM-31094: Removes SQLITE db driver ref.

* MM-31094: Layer generate fix.

* MM-31094: Removes unnecessary changes.

* MM-31094: Moves test method.

* MM-31094: Re-add previous fix.

* MM-31094: Removes make command for dev.

* MM-31094: Fix for login.

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-03-12 12:37:30 -05:00
Jyoti Patel
87f553283a [GH-15906][MM-22844] Redesign welcome and verify email. (#16824)
Automatic Merge
2021-02-26 05:22:26 +01:00
John Tzikas
809affd6d1 Create basic make commands for configuring golang-migrate (#16938)
* Create basic make commands for configuring golang-migrate

* Showcase full flow with new migrations

* Apply PR suggestions

* Migrate over team members

* Update mocks

* Fix specs

* Move columns that added after table creation onto separate stmts

* Put back gorp table definitions

* Fix issues with golang-migrate that not tracks underlying db driver

* Help prompt after new migration and consistent checksum for bindata

* Put gorp mapping back

* Apply PR suggestiong

* Close migrations after they run

* Add migration file to bindata check

* Updates needed

* Reset store_test

* Add copyright

* Apply PR suggestions

* Fix new circleci check

* Put back upgrade step for backwards comp

* Add store test to test migration directions

* Apply PR suggestions

* Add go-bindata to tools

* Apply PR suggestios

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-02-23 16:22:31 +02:00
Claudio Costa
6b35b933a8 Revert "MM-27312 Allow uploading 10 files per post (#16235)" (#16908)
This reverts commit 8e2986e5ed.
2021-02-10 20:19:05 +01:00
Diogo Nicolau
8e2986e5ed MM-27312 Allow uploading 10 files per post (#16235)
* Allow uploading 10 files per post

* Increase Post.Fileids column size to fit 10 files

* Add Post.Fileids column changes to DB v531

* Fix current_schema_version

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

* Update store/sqlstore/upgrade.go

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

* Fix schema version

Co-authored-by: Diogo Lima Nicolau <diogo.lima.nicolau@cern.ch>
Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-12-14 09:30:02 +01:00
Agniva De Sarker
fc60a1e8dd MM-30988 - Fix racy test ServerSystemdNotification (#16431)
* MM-30988 - Fix racy test ServerSystemdNotification

The translateFunc is a global variable which was unguarded.
So we convert that into an atomic variable to prevent against races.

https://mattermost.atlassian.net/browse/MM-30988

```release-note
NONE
```

* fix tests

* move call upwards

* use race

* after translations init

* guard with a sync.Once

* missed import

* revert race

* update timeout
2020-12-02 14:22:41 +05:30
Mario de Frutos Dieguez
bb095c2a1a mmctl is not being included in the package (#16105)
It's was including it in the bin folder but when we package we need to include it in the dist/bin so right now the `download_mmctl` script accepts a second parameter to define the path where the binary is going to be stored
2020-10-28 13:59:49 +01:00
Jason Paul Deland
c55f40cae9 MMCTL download migration to S3 location (#15993)
* Update to get latest release from S3

* Update mmctl to download to use S3

* Download MMCTL from S3

* Update download_mmctl_release.sh

* Update Makefile

* Update release.mk

* Update the script called, remove call to sub shell and clean up indentation

* Add quotes around THIS_BRANCH var

* Update script to support overriding the OS for packaging

* Add check for release-<Digit> to better match the branch

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-10-17 11:43:53 -04:00
Jason Paul Deland
e280aa1c63 Update to get latest release from S3 (#15952)
* Update to get latest release from S3

* Update mmctl to download to use S3

* Download MMCTL from S3

* Update download_mmctl_release.sh

* Update Makefile

* Update release.mk

* Update the script called, remove call to sub shell and clean up indentation

* Add quotes around THIS_BRANCH var

* Update script to support overriding the OS for packaging

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-10-16 20:27:50 -04:00
Ashish Bhate
ee76ad435b [MM-28303]: check if mmctl exists before checking on Github and reduce requests to Github (#15387)
Summary:
Every make call would ping Github to find out the mmctl version to download. Each check made 4 requests to Github. So every make execution resulted in 4 requests to Github. This leads to frequent rate-limit errors from Github.
In this PR we check for the mmctl version only if mmctl doesn't already exist. We also print a more helpful error message.
Reduce the number the number of requests to Github from 4 to 2.

Ticket Link:
https://mattermost.atlassian.net/browse/MM-28303
2020-09-24 14:54:57 +05:30
Jason Deland
953899ad95 Updating the mmctl process to set the value once and than use it. Also added the basic auth code from agniva 2020-09-02 18:32:10 -04:00
Jason Deland
cf3cfd2f5e Removing extra latest url code block 2020-09-02 14:52:55 -04:00
Jason Deland
111927b044 Including basic auth changes of for getting mmctl 2020-09-02 14:51:51 -04:00
Jason Paul Deland
907bc2906a Update the get release script for mmctl to get the latest release even if its a pre-release for master builds (#15378)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-02 13:48:28 -04:00
Jason Paul Deland
0da164f70e Update mmtcl download to account for Drat releases (#14363) 2020-08-31 13:13:51 -04:00
Ibrahim Serdar Acikgoz
eaabf96b1b [MM-28080] scripts/test: do not remove existing data directory (#15347)
* scripts/test: do not remove existing data directory

* Update scripts/test.sh

Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
2020-08-27 14:00:47 +03:00
Agniva De Sarker
e1ca3e348f MM-27512: Use an authenticated user to bump up request rate limit (#15251)
* MM-27512: Use an authenticated user to bump up request rate limit

An unauthenticated user can only make 60 requests per hour which means 1
request every minute. This can lead to frequent rate limit errors while
getting the latest release.

We change that to use an authenticated user which is already available
in the CI. This moves us to make 5000 requests per hour.

We also add additional logging in the Makefile targets in case
the command fails again so that it's clear what has happened, and not return
cryptic 404 errors again.

Ideally, we should be able to inspect the output of the curl command, but since
the output value of the entire bash script is fed into the variable, it is a bit
difficult to print debug output.

If this still gives error, then we need to either use a cached artifact somehow
or add additional logging and add a retry logic on top of it.

* fix mistake
2020-08-18 19:42:12 +05:30
Ashish Bhate
7e3d76b9a2 [MM-26886]: check enterprise prereqs before running tests (#15083) 2020-07-22 16:59:12 +05:30
Doug Lauder
b317ee5cf2 MM-25394 session expired push notifications (#14732)
* new job type created that checks for expired mobile sessions and pushes notifications.

* only send session expired notifications if ExtendSessionLengthWithActivity is enabled.

* includes schema change:  field added to Sessions table
2020-06-17 14:47:54 -04:00
Juho Nurminen
37f43a7094 Move Dependency-Check configs to a separate repo (#14497) 2020-05-07 12:05:49 +02:00
Juho Nurminen
cf3ba6661d Fix dependency alerting (#14457)
Automatic Merge
2020-05-04 04:21:47 -07:00
Agniva De Sarker
ac4153df03 Bump minimum Go version required (#14250)
After the auto-vendoring change, minimum Go version
required is 1.14.

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-04-08 14:28:55 +05:30
Juho Nurminen
d1ab17fa7c Update depcheck alert messages (#14163)
Automatic Merge
2020-03-26 07:13:25 -07:00
Juho Nurminen
28a7993ac9 MM-22889: Integrate Dependency-Check to CircleCI (#14023)
* Add basic dependency-check job

* Include summary in alert

* Link to NVD

* Move the alert script to a separate file
2020-03-24 09:19:41 +02:00