IntenseWebs/mattermost
3
0
Fork 0
mirror of https://github.com/mattermost/mattermost.git synced 2025-02-25 18:55:24 -06:00
Code Issues Packages Projects Releases Wiki Activity

MM-51095: Foundation for ESR upgrade scripts (#22448)

Browse Source 
* Add ESR upgrade migration and CI job to verify it

The script was generated as a simple concatenation of migrations in the
interval [54, 101] through:
    files=`for i in $(seq 54 101); do ls mysql/$(printf "%06d" $i)*up.sql; done`
    tail -n +1 $files > ../esrupgrades/esr.5.37-7.8.mysql.up.sql

The CI job runs the migration both through the server and the script,
and for now uploads the dumps generated for manual inspection. An
automatic check for differences is still needed.

* Remove debug print in script

* Fix idx_uploadsessions_type creation

* Ignore tables db_lock and db_migration on dump

* Split workflow in two parallel jobs

* Diff dumps and upload the result

* Add cleanup script

* Use DELIMITER in the script to use mysql CLI

This allows us to remove the complexity of using a different Go script
inside a Docker image.

* Standardize Roles between migrations

Document and cleanup code.

* Upload diff only if it is not empty

* Trigger action only when related files change

* Add a global timeout to the job

* Generalize ESR to ESR upgrade action (#22573)

* Generalize action

* Use logs to ensure migrations are finished

* Add migrations from 5.37 to 6.3

* Remove tables in cleanup script, not through dump

* Add initial-version input to common action

* Add migration from 6.3 to 7.8

* Remove action debug line

* ESR Upgrade: One procedure per table in the v5.37 > v7.8 upgrade script (#22590)

* Squash Users-related migrations in one query

* Squash Drafts-related migrations in one query

* Squash UploadSessions-related migrations in one query

* Squash Threads-related migrations in one query

* Squash Channels-related migrations in one query

* Squash ChannelMembers-related migrations in one query

* Squash Jobs-related migrations in one query

* Squash Sessions-related migrations in one query

* Squash Status-related migrations in one query

* Squash Posts-related migrations in one query

* Squash TeamMembers-related migrations in one query

* Squash Schemes-related migrations in one query

* Squash CommandWebhooks-related migrations in one query

* Squash OAuthApps-related migrations in one query

* Squash Teams-related migrations in one query

* Squash Reactions-related migrations in one query

* Squash PostReminders-related migrations in one query

* Adapt ThreadMemberships migration to unified style

* Adapt LinkMetadata migrations to unified style

* Adapt GroupChannels migration to unified style

* Adapt PluginKVStore migration to unified style

* Adapt UserGroups migration to unified style

* Adapt FileInfo migration to unified style

* Adapt SidebarCategories migration to unified style

* Remove blank line

* Use tabs everywhere

* Wrap every procedure with log statements

* Remove space before parentheses in procedure call

* Remove spurious extra line

* Merge two equal consecutive conditionals

* Avoid the double list of conditions/queries

* Fix variable name

* Remove outdated comment

* Add a preprocess phase with corresponding scripts

* Join all preprocess scripts setting ExpiresAt to 0

This preprocessing is something we should always do, no matter the input
DB, so we can use a common script for all cases instead of repeating the
same code in multiple files.

* Add system-bot if it does not exist

* Cleanup the ProductNoticeViewState table

* Fix SQL

* Move esrupgrades directory under server/

* Update paths in Github action

* Fix trigger path for CI
This commit is contained in:
Alejandro García Montoro
2023-04-20 19:41:36 +02:00
committed by GitHub
parent 3ba419c841
commit 87908bc577
10 changed files with 3422 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
server/scripts/esrupgrades/README.md Normal file
View File

@@ -0,0 +1 @@
A collection of ad-hoc scripts to upgrade between ESRs.

160
server/scripts/esrupgrades/esr.5.37-6.3.mysql.cleanup.sql Normal file
View File

@@ -0,0 +1,160 @@
/* Product notices are controlled externally, via the mattermost/notices repository.
When there is a new notice specified there, the server may have time, right after
the migration and before it is shut down, to download it and modify the
ProductNoticeViewState table, adding a row for all users that have not seen it or
removing old notices that no longer need to be shown. This can happen in the
UpdateProductNotices function that is executed periodically to update the notices
cache. The script will never do this, so we need to remove all rows in that table
to avoid any unwanted diff. */
DELETE FROM ProductNoticeViewState;
/* The script does not update the Systems row that tracks the version, so it is manually updated
here so that it does not show in the diff. */
UPDATE Systems SET Value = '6.3.0' WHERE Name = 'Version';
/* The script does not update the schema_migrations table, which is automatically used by the
migrate library to track the version, so we drop it altogether to avoid spurious errors in
the diff */
DROP TABLE IF EXISTS schema_migrations;
/* Migration 000054_create_crt_channelmembership_count.up sets
ChannelMembers.LastUpdateAt to the results of SELECT ROUND(UNIX_TIMESTAMP(NOW(3))*1000)
which will be different each time the migration is run. Thus, the column will always be
different when comparing the server and script migrations. To bypass this, we update all
rows in ChannelMembers so that they contain the same value for such column. */
UPDATE ChannelMembers SET LastUpdateAt = 1;
/* Migration 000055_create_crt_thread_count_and_unreads.up sets
ThreadMemberships.LastUpdated to the results of SELECT ROUND(UNIX_TIMESTAMP(NOW(3))*1000)
which will be different each time the migration is run. Thus, the column will always be
different when comparing the server and script migrations. To bypass this, we update all
rows in ThreadMemberships so that they contain the same value for such column. */
UPDATE ThreadMemberships SET LastUpdated = 1;
/* The security update check in the server may update the LastSecurityTime system value. To
avoid any spurious difference in the migrations, we update it to a fixed value. */
UPDATE Systems SET Value = 1 WHERE Name = 'LastSecurityTime';
/* The server migration contains an in-app migration that adds new roles for Playbooks:
doPlaybooksRolesCreationMigration, defined in https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/migrations.go#L345-L469
The roles are the ones defined in https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/model/role.go#L874-L929
When this migration finishes, it also adds a new row to the Systems table with the key of the migration.
This in-app migration does not happen in the script, so we remove those rows here. */
DELETE FROM Roles WHERE Name = 'playbook_member';
DELETE FROM Roles WHERE Name = 'playbook_admin';
DELETE FROM Roles WHERE Name = 'run_member';
DELETE FROM Roles WHERE Name = 'run_admin';
DELETE FROM Systems WHERE Name = 'PlaybookRolesCreationMigrationComplete';
/* The server migration contains an in-app migration that add playbooks permissions to certain roles:
getAddPlaybooksPermissions, defined in https://github.com/mattermost/mattermost-server/blob/f9b996934cabf9a8fad5901835e7e9b418917402/app/permissions_migrations.go#L918-L951
The specific roles ('%playbook%') are removed in the procedure below, but the migrations also add a new row to the Systems table marking the migration as complete.
This in-app migration does not happen in the script, so we remove that rows here. */
DELETE FROM Systems WHERE Name = 'playbooks_permissions';
/* The rest of this script defines and executes a procedure to update the Roles table. It performs several changes:
1. Set the UpdateAt column of all rows to a fixed value, so that the server migration changes to this column
do not appear in the diff.
2. Remove the set of specific permissions added in the server migration that is not covered by the script, as
this logic happens all in-app after the normal DB migrations.
3. Set a consistent order in the Permissions column, which is modelled a space-separated string containing each of
the different permissions each role has. This change is the reason why we need a complex procedure, which creates
a temporary table that pairs each single permission to its corresponding ID. So if the Roles table contains two
rows like:
Id: 'abcd'
Permissions: 'view_team read_public_channel invite_user'
Id: 'efgh'
Permissions: 'view_team create_emojis'
then the new temporary table will contain five rows like:
Id: 'abcd'
Permissions: 'view_team'
Id: 'abcd'
Permissions: 'read_public_channel'
Id: 'abcd'
Permissions: 'invite_user'
Id: 'efgh'
Permissions: 'view_team'
Id: 'efgh'
Permissions: 'create_emojis'
*/
DROP PROCEDURE IF EXISTS splitPermissions;
DROP PROCEDURE IF EXISTS sortAndFilterPermissionsInRoles;
DROP TEMPORARY TABLE IF EXISTS temp_roles;
CREATE TEMPORARY TABLE temp_roles(id varchar(26), permission longtext);
DELIMITER //
/* Auxiliary procedure that splits the space-separated permissions string into single rows that are inserted
in the temporary temp_roles table along with their corresponding ID. */
CREATE PROCEDURE splitPermissions(
IN id varchar(26),
IN permissionsString longtext
)
BEGIN
DECLARE idx INT DEFAULT 0;
SELECT TRIM(permissionsString) INTO permissionsString;
SELECT LOCATE(' ', permissionsString) INTO idx;
WHILE idx > 0 DO
INSERT INTO temp_roles SELECT id, TRIM(LEFT(permissionsString, idx));
SELECT SUBSTR(permissionsString, idx+1) INTO permissionsString;
SELECT LOCATE(' ', permissionsString) INTO idx;
END WHILE;
INSERT INTO temp_roles(id, permission) VALUES(id, TRIM(permissionsString));
END; //
/* Main procedure that does update the Roles table */
CREATE PROCEDURE sortAndFilterPermissionsInRoles()
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE rolesId varchar(26) DEFAULT '';
DECLARE rolesPermissions longtext DEFAULT '';
DECLARE cur1 CURSOR FOR SELECT Id, Permissions FROM Roles;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
/* 1. Set a fixed value in the UpdateAt column for all rows in Roles table */
UPDATE Roles SET UpdateAt = 1;
/* Call splitPermissions for every row in the Roles table, thus populating the
temp_roles table. */
OPEN cur1;
read_loop: LOOP
FETCH cur1 INTO rolesId, rolesPermissions;
IF done THEN
LEAVE read_loop;
END IF;
CALL splitPermissions(rolesId, rolesPermissions);
END LOOP;
CLOSE cur1;
/* 2. Filter out the new permissions added by the in-app migrations */
DELETE FROM temp_roles WHERE permission LIKE '%playbook%';
DELETE FROM temp_roles WHERE permission LIKE 'run_create';
DELETE FROM temp_roles WHERE permission LIKE 'run_manage_members';
DELETE FROM temp_roles WHERE permission LIKE 'run_manage_properties';
DELETE FROM temp_roles WHERE permission LIKE 'run_view';
/* Temporarily set to the maximum permitted value, since the call to group_concat
below needs a value bigger than the default */
SET group_concat_max_len = 18446744073709551615;
/* 3. Update the Permissions column in the Roles table with the filtered, sorted permissions,
concatenated again as a space-separated string */
UPDATE
Roles INNER JOIN (
SELECT temp_roles.id as Id, TRIM(group_concat(temp_roles.permission ORDER BY temp_roles.permission SEPARATOR ' ')) as Permissions
FROM Roles JOIN temp_roles ON Roles.Id = temp_roles.id
GROUP BY temp_roles.id
) AS Sorted
ON Roles.Id = Sorted.Id
SET Roles.Permissions = Sorted.Permissions;
/* Reset group_concat_max_len to its default value */
SET group_concat_max_len = 1024;
END; //
DELIMITER ;
CALL sortAndFilterPermissionsInRoles();
DROP TEMPORARY TABLE IF EXISTS temp_roles;

695
server/scripts/esrupgrades/esr.5.37-6.3.mysql.up.sql Normal file
View File

@@ -0,0 +1,695 @@
/* ==> mysql/000054_create_crt_channelmembership_count.up.sql <== */
/* fixCRTChannelMembershipCounts fixes the channel counts, i.e. the total message count,
total root message count, mention count, and mention count in root messages for users
who have viewed the channel after the last post in the channel */
DELIMITER //
CREATE PROCEDURE MigrateCRTChannelMembershipCounts ()
BEGIN
IF(
SELECT
EXISTS (
SELECT
* FROM Systems
WHERE
Name = 'CRTChannelMembershipCountsMigrationComplete') = 0) THEN
UPDATE
ChannelMembers
INNER JOIN Channels ON Channels.Id = ChannelMembers.ChannelId SET
MentionCount = 0, MentionCountRoot = 0, MsgCount = Channels.TotalMsgCount, MsgCountRoot = Channels.TotalMsgCountRoot, LastUpdateAt = (
SELECT
(SELECT ROUND(UNIX_TIMESTAMP(NOW(3))*1000)))
WHERE
ChannelMembers.LastViewedAt >= Channels.LastPostAt;
INSERT INTO Systems
VALUES('CRTChannelMembershipCountsMigrationComplete', 'true');
END IF;
END//
DELIMITER ;
CALL MigrateCRTChannelMembershipCounts ();
DROP PROCEDURE IF EXISTS MigrateCRTChannelMembershipCounts;
/* ==> mysql/000055_create_crt_thread_count_and_unreads.up.sql <== */
/* fixCRTThreadCountsAndUnreads Marks threads as read for users where the last
reply time of the thread is earlier than the time the user viewed the channel.
Marking a thread means setting the mention count to zero and setting the
last viewed at time of the the thread as the last viewed at time
of the channel */
DELIMITER //
CREATE PROCEDURE MigrateCRTThreadCountsAndUnreads ()
BEGIN
IF(SELECT EXISTS(SELECT * FROM Systems WHERE Name = 'CRTThreadCountsAndUnreadsMigrationComplete') = 0) THEN
UPDATE
ThreadMemberships
INNER JOIN (
SELECT
PostId,
UserId,
ChannelMembers.LastViewedAt AS CM_LastViewedAt,
Threads.LastReplyAt
FROM
Threads
INNER JOIN ChannelMembers ON ChannelMembers.ChannelId = Threads.ChannelId
WHERE
Threads.LastReplyAt <= ChannelMembers.LastViewedAt) AS q ON ThreadMemberships.Postid = q.PostId
AND ThreadMemberships.UserId = q.UserId SET LastViewed = q.CM_LastViewedAt + 1, UnreadMentions = 0, LastUpdated = (
SELECT
(SELECT ROUND(UNIX_TIMESTAMP(NOW(3))*1000)));
INSERT INTO Systems
VALUES('CRTThreadCountsAndUnreadsMigrationComplete', 'true');
END IF;
END//
DELIMITER ;
CALL MigrateCRTThreadCountsAndUnreads ();
DROP PROCEDURE IF EXISTS MigrateCRTThreadCountsAndUnreads;
/* ==> mysql/000056_upgrade_channels_v6.0.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Channels'
AND table_schema = DATABASE()
AND index_name = 'idx_channels_team_id_display_name'
) > 0,
'SELECT 1',
'CREATE INDEX idx_channels_team_id_display_name ON Channels(TeamId, DisplayName);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Channels'
AND table_schema = DATABASE()
AND index_name = 'idx_channels_team_id_type'
) > 0,
'SELECT 1',
'CREATE INDEX idx_channels_team_id_type ON Channels(TeamId, Type);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Channels'
AND table_schema = DATABASE()
AND index_name = 'idx_channels_team_id'
) > 0,
'DROP INDEX idx_channels_team_id ON Channels;',
'SELECT 1'
));
PREPARE removeIndexIfExists FROM @preparedStatement;
EXECUTE removeIndexIfExists;
DEALLOCATE PREPARE removeIndexIfExists;
/* ==> mysql/000057_upgrade_command_webhooks_v6.0.up.sql <== */
DELIMITER //
CREATE PROCEDURE MigrateRootId_CommandWebhooks () BEGIN DECLARE ParentId_EXIST INT;
SELECT COUNT(*)
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'CommandWebhooks'
AND table_schema = DATABASE()
AND COLUMN_NAME = 'ParentId' INTO ParentId_EXIST;
IF(ParentId_EXIST > 0) THEN
UPDATE CommandWebhooks SET RootId = ParentId WHERE RootId = '' AND RootId != ParentId;
END IF;
END//
DELIMITER ;
CALL MigrateRootId_CommandWebhooks ();
DROP PROCEDURE IF EXISTS MigrateRootId_CommandWebhooks;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'CommandWebhooks'
AND table_schema = DATABASE()
AND column_name = 'ParentId'
) > 0,
'ALTER TABLE CommandWebhooks DROP COLUMN ParentId;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000058_upgrade_channelmembers_v6.0.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'ChannelMembers'
AND table_schema = DATABASE()
AND column_name = 'NotifyProps'
AND column_type != 'JSON'
) > 0,
'ALTER TABLE ChannelMembers MODIFY COLUMN NotifyProps JSON;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'ChannelMembers'
AND table_schema = DATABASE()
AND index_name = 'idx_channelmembers_user_id'
) > 0,
'DROP INDEX idx_channelmembers_user_id ON ChannelMembers;',
'SELECT 1'
));
PREPARE removeIndexIfExists FROM @preparedStatement;
EXECUTE removeIndexIfExists;
DEALLOCATE PREPARE removeIndexIfExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'ChannelMembers'
AND table_schema = DATABASE()
AND index_name = 'idx_channelmembers_user_id_channel_id_last_viewed_at'
) > 0,
'SELECT 1',
'CREATE INDEX idx_channelmembers_user_id_channel_id_last_viewed_at ON ChannelMembers(UserId, ChannelId, LastViewedAt);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'ChannelMembers'
AND table_schema = DATABASE()
AND index_name = 'idx_channelmembers_channel_id_scheme_guest_user_id'
) > 0,
'SELECT 1',
'CREATE INDEX idx_channelmembers_channel_id_scheme_guest_user_id ON ChannelMembers(ChannelId, SchemeGuest, UserId);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
/* ==> mysql/000059_upgrade_users_v6.0.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Users'
AND table_schema = DATABASE()
AND column_name = 'Props'
AND column_type != 'JSON'
) > 0,
'ALTER TABLE Users MODIFY COLUMN Props JSON;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Users'
AND table_schema = DATABASE()
AND column_name = 'NotifyProps'
AND column_type != 'JSON'
) > 0,
'ALTER TABLE Users MODIFY COLUMN NotifyProps JSON;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Users'
AND table_schema = DATABASE()
AND column_name = 'Timezone'
AND column_default IS NOT NULL
) > 0,
'ALTER TABLE Users ALTER Timezone DROP DEFAULT;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Users'
AND table_schema = DATABASE()
AND column_name = 'Timezone'
AND column_type != 'JSON'
) > 0,
'ALTER TABLE Users MODIFY COLUMN Timezone JSON;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Users'
AND table_schema = DATABASE()
AND column_name = 'Roles'
AND column_type != 'text'
) > 0,
'ALTER TABLE Users MODIFY COLUMN Roles text;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000060_upgrade_jobs_v6.0.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Jobs'
AND table_schema = DATABASE()
AND column_name = 'Data'
AND column_type != 'JSON'
) > 0,
'ALTER TABLE Jobs MODIFY COLUMN Data JSON;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000061_upgrade_link_metadata_v6.0.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'LinkMetadata'
AND table_schema = DATABASE()
AND column_name = 'Data'
AND column_type != 'JSON'
) > 0,
'ALTER TABLE LinkMetadata MODIFY COLUMN Data JSON;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000062_upgrade_sessions_v6.0.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Sessions'
AND table_schema = DATABASE()
AND column_name = 'Props'
AND column_type != 'JSON'
) > 0,
'ALTER TABLE Sessions MODIFY COLUMN Props JSON;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000063_upgrade_threads_v6.0.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Threads'
AND table_schema = DATABASE()
AND column_name = 'Participants'
AND column_type != 'JSON'
) > 0,
'ALTER TABLE Threads MODIFY COLUMN Participants JSON;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Threads'
AND table_schema = DATABASE()
AND index_name = 'idx_threads_channel_id_last_reply_at'
) > 0,
'SELECT 1',
'CREATE INDEX idx_threads_channel_id_last_reply_at ON Threads(ChannelId, LastReplyAt);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Threads'
AND table_schema = DATABASE()
AND index_name = 'idx_threads_channel_id'
) > 0,
'DROP INDEX idx_threads_channel_id ON Threads;',
'SELECT 1'
));
PREPARE removeIndexIfExists FROM @preparedStatement;
EXECUTE removeIndexIfExists;
DEALLOCATE PREPARE removeIndexIfExists;
/* ==> mysql/000064_upgrade_status_v6.0.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Status'
AND table_schema = DATABASE()
AND index_name = 'idx_status_status_dndendtime'
) > 0,
'SELECT 1',
'CREATE INDEX idx_status_status_dndendtime ON Status(Status, DNDEndTime);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Status'
AND table_schema = DATABASE()
AND index_name = 'idx_status_status'
) > 0,
'DROP INDEX idx_status_status ON Status;',
'SELECT 1'
));
PREPARE removeIndexIfExists FROM @preparedStatement;
EXECUTE removeIndexIfExists;
DEALLOCATE PREPARE removeIndexIfExists;
/* ==> mysql/000065_upgrade_groupchannels_v6.0.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'GroupChannels'
AND table_schema = DATABASE()
AND index_name = 'idx_groupchannels_schemeadmin'
) > 0,
'SELECT 1',
'CREATE INDEX idx_groupchannels_schemeadmin ON GroupChannels(SchemeAdmin);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
/* ==> mysql/000066_upgrade_posts_v6.0.up.sql <== */
DELIMITER //
CREATE PROCEDURE MigrateRootId_Posts ()
BEGIN
DECLARE ParentId_EXIST INT;
DECLARE Alter_FileIds INT;
DECLARE Alter_Props INT;
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'Posts'
AND table_schema = DATABASE()
AND COLUMN_NAME = 'ParentId' INTO ParentId_EXIST;
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Posts'
AND table_schema = DATABASE()
AND column_name = 'FileIds'
AND column_type != 'text' INTO Alter_FileIds;
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Posts'
AND table_schema = DATABASE()
AND column_name = 'Props'
AND column_type != 'JSON' INTO Alter_Props;
IF (Alter_Props OR Alter_FileIds) THEN
IF(ParentId_EXIST > 0) THEN
UPDATE Posts SET RootId = ParentId WHERE RootId = '' AND RootId != ParentId;
ALTER TABLE Posts MODIFY COLUMN FileIds text, MODIFY COLUMN Props JSON, DROP COLUMN ParentId;
ELSE
ALTER TABLE Posts MODIFY COLUMN FileIds text, MODIFY COLUMN Props JSON;
END IF;
END IF;
END//
DELIMITER ;
CALL MigrateRootId_Posts ();
DROP PROCEDURE IF EXISTS MigrateRootId_Posts;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Posts'
AND table_schema = DATABASE()
AND index_name = 'idx_posts_root_id_delete_at'
) > 0,
'SELECT 1',
'CREATE INDEX idx_posts_root_id_delete_at ON Posts(RootId, DeleteAt);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Posts'
AND table_schema = DATABASE()
AND index_name = 'idx_posts_root_id'
) > 0,
'DROP INDEX idx_posts_root_id ON Posts;',
'SELECT 1'
));
PREPARE removeIndexIfExists FROM @preparedStatement;
EXECUTE removeIndexIfExists;
DEALLOCATE PREPARE removeIndexIfExists;
/* ==> mysql/000067_upgrade_channelmembers_v6.1.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'ChannelMembers'
AND table_schema = DATABASE()
AND column_name = 'Roles'
AND column_type != 'text'
) > 0,
'ALTER TABLE ChannelMembers MODIFY COLUMN Roles text;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000068_upgrade_teammembers_v6.1.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'TeamMembers'
AND table_schema = DATABASE()
AND column_name = 'Roles'
AND column_type != 'text'
) > 0,
'ALTER TABLE TeamMembers MODIFY COLUMN Roles text;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000069_upgrade_jobs_v6.1.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Jobs'
AND table_schema = DATABASE()
AND index_name = 'idx_jobs_status_type'
) > 0,
'SELECT 1',
'CREATE INDEX idx_jobs_status_type ON Jobs(Status, Type);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
/* ==> mysql/000070_upgrade_cte_v6.1.up.sql <== */
DELIMITER //
CREATE PROCEDURE Migrate_LastRootPostAt ()
BEGIN
DECLARE
LastRootPostAt_EXIST INT;
SELECT
COUNT(*)
FROM
INFORMATION_SCHEMA.COLUMNS
WHERE
TABLE_NAME = 'Channels'
AND table_schema = DATABASE()
AND COLUMN_NAME = 'LastRootPostAt' INTO LastRootPostAt_EXIST;
IF(LastRootPostAt_EXIST = 0) THEN
ALTER TABLE Channels ADD COLUMN LastRootPostAt bigint DEFAULT 0;
UPDATE
Channels
INNER JOIN (
SELECT
Channels.Id channelid,
COALESCE(MAX(Posts.CreateAt), 0) AS lastrootpost
FROM
Channels
LEFT JOIN Posts FORCE INDEX (idx_posts_channel_id_update_at) ON Channels.Id = Posts.ChannelId
WHERE
Posts.RootId = ''
GROUP BY
Channels.Id) AS q ON q.channelid = Channels.Id SET LastRootPostAt = lastrootpost;
END IF;
END//
DELIMITER ;
CALL Migrate_LastRootPostAt ();
DROP PROCEDURE IF EXISTS Migrate_LastRootPostAt;
/* ==> mysql/000071_upgrade_sessions_v6.1.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Sessions'
AND table_schema = DATABASE()
AND column_name = 'Roles'
AND column_type != 'text'
) > 0,
'ALTER TABLE Sessions MODIFY COLUMN Roles text;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000072_upgrade_schemes_v6.3.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Schemes'
AND table_schema = DATABASE()
AND column_name = 'DefaultPlaybookAdminRole'
) > 0,
'SELECT 1',
'ALTER TABLE Schemes ADD COLUMN DefaultPlaybookAdminRole VARCHAR(64) DEFAULT "";'
));
PREPARE alterIfNotExists FROM @preparedStatement;
EXECUTE alterIfNotExists;
DEALLOCATE PREPARE alterIfNotExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Schemes'
AND table_schema = DATABASE()
AND column_name = 'DefaultPlaybookMemberRole'
) > 0,
'SELECT 1',
'ALTER TABLE Schemes ADD COLUMN DefaultPlaybookMemberRole VARCHAR(64) DEFAULT "";'
));
PREPARE alterIfNotExists FROM @preparedStatement;
EXECUTE alterIfNotExists;
DEALLOCATE PREPARE alterIfNotExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Schemes'
AND table_schema = DATABASE()
AND column_name = 'DefaultRunAdminRole'
) > 0,
'SELECT 1',
'ALTER TABLE Schemes ADD COLUMN DefaultRunAdminRole VARCHAR(64) DEFAULT "";'
));
PREPARE alterIfNotExists FROM @preparedStatement;
EXECUTE alterIfNotExists;
DEALLOCATE PREPARE alterIfNotExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Schemes'
AND table_schema = DATABASE()
AND column_name = 'DefaultRunMemberRole'
) > 0,
'SELECT 1',
'ALTER TABLE Schemes ADD COLUMN DefaultRunMemberRole VARCHAR(64) DEFAULT "";'
));
PREPARE alterIfNotExists FROM @preparedStatement;
EXECUTE alterIfNotExists;
DEALLOCATE PREPARE alterIfNotExists;
/* ==> mysql/000073_upgrade_plugin_key_value_store_v6.3.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT Count(*) FROM Information_Schema.Columns
WHERE table_name = 'PluginKeyValueStore'
AND table_schema = DATABASE()
AND column_name = 'PKey'
AND column_type != 'varchar(150)'
) > 0,
'ALTER TABLE PluginKeyValueStore MODIFY COLUMN PKey varchar(150);',
'SELECT 1'
));
PREPARE alterTypeIfExists FROM @preparedStatement;
EXECUTE alterTypeIfExists;
DEALLOCATE PREPARE alterTypeIfExists;
/* ==> mysql/000074_upgrade_users_v6.3.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Users'
AND table_schema = DATABASE()
AND column_name = 'AcceptedTermsOfServiceId'
) > 0,
'ALTER TABLE Users DROP COLUMN AcceptedTermsOfServiceId;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;

199
server/scripts/esrupgrades/esr.5.37-7.8.mysql.cleanup.sql Normal file
View File

@@ -0,0 +1,199 @@
/* Product notices are controlled externally, via the mattermost/notices repository.
When there is a new notice specified there, the server may have time, right after
the migration and before it is shut down, to download it and modify the
ProductNoticeViewState table, adding a row for all users that have not seen it or
removing old notices that no longer need to be shown. This can happen in the
UpdateProductNotices function that is executed periodically to update the notices
cache. The script will never do this, so we need to remove all rows in that table
to avoid any unwanted diff. */
DELETE FROM ProductNoticeViewState;
/* Remove migration-related tables that are only updated through the server to track which
migrations have been applied */
DROP TABLE IF EXISTS db_lock;
DROP TABLE IF EXISTS db_migrations;
/* Migration 000054_create_crt_channelmembership_count.up sets
ChannelMembers.LastUpdateAt to the results of SELECT ROUND(UNIX_TIMESTAMP(NOW(3))*1000)
which will be different each time the migration is run. Thus, the column will always be
different when comparing the server and script migrations. To bypass this, we update all
rows in ChannelMembers so that they contain the same value for such column. */
UPDATE ChannelMembers SET LastUpdateAt = 1;
/* Migration 000055_create_crt_thread_count_and_unreads.up sets
ThreadMemberships.LastUpdated to the results of SELECT ROUND(UNIX_TIMESTAMP(NOW(3))*1000)
which will be different each time the migration is run. Thus, the column will always be
different when comparing the server and script migrations. To bypass this, we update all
rows in ThreadMemberships so that they contain the same value for such column. */
UPDATE ThreadMemberships SET LastUpdated = 1;
/* The security update check in the server may update the LastSecurityTime system value. To
avoid any spurious difference in the migrations, we update it to a fixed value. */
UPDATE Systems SET Value = 1 WHERE Name = 'LastSecurityTime';
/* The server migration may contain a row in the Systems table marking the onboarding as complete.
There are no migrations related to this, so we can simply drop it here. */
DELETE FROM Systems WHERE Name = 'FirstAdminSetupComplete';
/* The server migration contains an in-app migration that adds new roles for Playbooks:
doPlaybooksRolesCreationMigration, defined in https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/migrations.go#L345-L469
The roles are the ones defined in https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/model/role.go#L874-L929
When this migration finishes, it also adds a new row to the Systems table with the key of the migration.
This in-app migration does not happen in the script, so we remove those rows here. */
DELETE FROM Roles WHERE Name = 'playbook_member';
DELETE FROM Roles WHERE Name = 'playbook_admin';
DELETE FROM Roles WHERE Name = 'run_member';
DELETE FROM Roles WHERE Name = 'run_admin';
DELETE FROM Systems WHERE Name = 'PlaybookRolesCreationMigrationComplete';
/* The server migration contains two in-app migrations that add playbooks permissions to certain roles:
getAddPlaybooksPermissions and getPlaybooksPermissionsAddManageRoles, defined in https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/permissions_migrations.go#L1021-L1072
The specific roles ('%playbook%') are removed in the procedure below, but the migrations also add new rows to the Systems table marking the migrations as complete.
These in-app migrations do not happen in the script, so we remove those rows here. */
DELETE FROM Systems WHERE Name = 'playbooks_manage_roles';
DELETE FROM Systems WHERE Name = 'playbooks_permissions';
/* The server migration contains an in-app migration that adds boards permissions to certain roles:
getProductsBoardsPermissions, defined in https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/permissions_migrations.go#L1074-L1093
The specific roles (sysconsole_read_product_boards and sysconsole_write_product_boards) are removed in the procedure below,
but the migrations also adds a new row to the Systems table marking the migrations as complete.
This in-app migration does not happen in the script, so we remove that row here. */
DELETE FROM Systems WHERE Name = 'products_boards';
/* TODO: REVIEW STARTING HERE */
/* The server migration contain an in-app migration that adds Ids to the Teams whose InviteId is an empty string:
doRemainingSchemaMigrations, defined in https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/migrations.go#L515-L540
The migration is not replicated in the script, since it happens in-app, but the server adds a new row to the
Systems table marking the table as complete, which the script doesn't do, so we remove that row here. */
DELETE FROM Systems WHERE Name = 'RemainingSchemaMigrations';
/* The server migration contains three in-app migration that adds a new role and new permissions
related to custom groups. The migrations are:
- doCustomGroupAdminRoleCreationMigration https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/migrations.go#L345-L469
- getAddCustomUserGroupsPermissions https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/permissions_migrations.go#L974-L995
- getAddCustomUserGroupsPermissionRestore https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/permissions_migrations.go#L997-L1019
The specific roles and permissions are removed in the procedure below, but the migrations also
adds a new row to the Roles table for the new role and new rows to the Systems table marking the
migrations as complete.
This in-app migration does not happen in the script, so we remove that row here. */
DELETE FROM Roles WHERE Name = 'system_custom_group_admin';
DELETE FROM Systems WHERE Name = 'CustomGroupAdminRoleCreationMigrationComplete';
DELETE FROM Systems WHERE Name = 'custom_groups_permissions';
DELETE FROM Systems WHERE Name = 'custom_groups_permission_restore';
/* The server migration contains an in-app migration that updates the config, setting ServiceSettings.PostPriority
to true, doPostPriorityConfigDefaultTrueMigration, defined in https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/migrations.go#L542-L560
The migration is not replicated in the script, since it happens in-app, but the server adds a new row to the
Systems table marking the table as complete, which the script doesn't do, so we remove that row here. */
DELETE FROM Systems WHERE Name = 'PostPriorityConfigDefaultTrueMigrationComplete';
/* The rest of this script defines and executes a procedure to update the Roles table. It performs several changes:
1. Set the UpdateAt column of all rows to a fixed value, so that the server migration changes to this column
do not appear in the diff.
2. Remove the set of specific permissions added in the server migration that is not covered by the script, as
this logic happens all in-app after the normal DB migrations.
3. Set a consistent order in the Permissions column, which is modelled a space-separated string containing each of
the different permissions each role has. This change is the reason why we need a complex procedure, which creates
a temporary table that pairs each single permission to its corresponding ID. So if the Roles table contains two
rows like:
Id: 'abcd'
Permissions: 'view_team read_public_channel invite_user'
Id: 'efgh'
Permissions: 'view_team create_emojis'
then the new temporary table will contain five rows like:
Id: 'abcd'
Permissions: 'view_team'
Id: 'abcd'
Permissions: 'read_public_channel'
Id: 'abcd'
Permissions: 'invite_user'
Id: 'efgh'
Permissions: 'view_team'
Id: 'efgh'
Permissions: 'create_emojis'
*/
DROP PROCEDURE IF EXISTS splitPermissions;
DROP PROCEDURE IF EXISTS sortAndFilterPermissionsInRoles;
DROP TEMPORARY TABLE IF EXISTS temp_roles;
CREATE TEMPORARY TABLE temp_roles(id varchar(26), permission longtext);
DELIMITER //
/* Auxiliary procedure that splits the space-separated permissions string into single rows that are inserted
in the temporary temp_roles table along with their corresponding ID. */
CREATE PROCEDURE splitPermissions(
IN id varchar(26),
IN permissionsString longtext
)
BEGIN
DECLARE idx INT DEFAULT 0;
SELECT TRIM(permissionsString) INTO permissionsString;
SELECT LOCATE(' ', permissionsString) INTO idx;
WHILE idx > 0 DO
INSERT INTO temp_roles SELECT id, TRIM(LEFT(permissionsString, idx));
SELECT SUBSTR(permissionsString, idx+1) INTO permissionsString;
SELECT LOCATE(' ', permissionsString) INTO idx;
END WHILE;
INSERT INTO temp_roles(id, permission) VALUES(id, TRIM(permissionsString));
END; //
/* Main procedure that does update the Roles table */
CREATE PROCEDURE sortAndFilterPermissionsInRoles()
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE rolesId varchar(26) DEFAULT '';
DECLARE rolesPermissions longtext DEFAULT '';
DECLARE cur1 CURSOR FOR SELECT Id, Permissions FROM Roles;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
/* 1. Set a fixed value in the UpdateAt column for all rows in Roles table */
UPDATE Roles SET UpdateAt = 1;
/* Call splitPermissions for every row in the Roles table, thus populating the
temp_roles table. */
OPEN cur1;
read_loop: LOOP
FETCH cur1 INTO rolesId, rolesPermissions;
IF done THEN
LEAVE read_loop;
END IF;
CALL splitPermissions(rolesId, rolesPermissions);
END LOOP;
CLOSE cur1;
/* 2. Filter out the new permissions added by the in-app migrations */
DELETE FROM temp_roles WHERE permission LIKE 'sysconsole_read_products_boards';
DELETE FROM temp_roles WHERE permission LIKE 'sysconsole_write_products_boards';
DELETE FROM temp_roles WHERE permission LIKE '%playbook%';
DELETE FROM temp_roles WHERE permission LIKE 'run_create';
DELETE FROM temp_roles WHERE permission LIKE 'run_manage_members';
DELETE FROM temp_roles WHERE permission LIKE 'run_manage_properties';
DELETE FROM temp_roles WHERE permission LIKE 'run_view';
DELETE FROM temp_roles WHERE permission LIKE '%custom_group%';
/* Temporarily set to the maximum permitted value, since the call to group_concat
below needs a value bigger than the default */
SET group_concat_max_len = 18446744073709551615;
/* 3. Update the Permissions column in the Roles table with the filtered, sorted permissions,
concatenated again as a space-separated string */
UPDATE
Roles INNER JOIN (
SELECT temp_roles.id as Id, TRIM(group_concat(temp_roles.permission ORDER BY temp_roles.permission SEPARATOR ' ')) as Permissions
FROM Roles JOIN temp_roles ON Roles.Id = temp_roles.id
GROUP BY temp_roles.id
) AS Sorted
ON Roles.Id = Sorted.Id
SET Roles.Permissions = Sorted.Permissions;
/* Reset group_concat_max_len to its default value */
SET group_concat_max_len = 1024;
END; //
DELIMITER ;
CALL sortAndFilterPermissionsInRoles();
DROP TEMPORARY TABLE IF EXISTS temp_roles;

1385
server/scripts/esrupgrades/esr.5.37-7.8.mysql.up.sql Normal file
View File

File diff suppressed because it is too large Load Diff

168
server/scripts/esrupgrades/esr.6.3-7.8.mysql.cleanup.sql Normal file
View File

@@ -0,0 +1,168 @@
/* Product notices are controlled externally, via the mattermost/notices repository.
When there is a new notice specified there, the server may have time, right after
the migration and before it is shut down, to download it and modify the
ProductNoticeViewState table, adding a row for all users that have not seen it or
removing old notices that no longer need to be shown. This can happen in the
UpdateProductNotices function that is executed periodically to update the notices
cache. The script will never do this, so we need to remove all rows in that table
to avoid any unwanted diff. */
DELETE FROM ProductNoticeViewState;
/* Remove migration-related tables that are only updated through the server to track which
migrations have been applied */
DROP TABLE IF EXISTS db_lock;
DROP TABLE IF EXISTS db_migrations;
/* The security update check in the server may update the LastSecurityTime system value. To
avoid any spurious difference in the migrations, we update it to a fixed value. */
UPDATE Systems SET Value = 1 WHERE Name = 'LastSecurityTime';
/* The server migration may contain a row in the Systems table marking the onboarding as complete.
There are no migrations related to this, so we can simply drop it here. */
DELETE FROM Systems WHERE Name = 'FirstAdminSetupComplete';
/* The server migration contains an in-app migration that add playbooks permissions to certain roles:
getPlaybooksPermissionsAddManageRoles, defined in https://github.com/mattermost/mattermost-server/blob/56a093ceaee6389a01a35b6d4626ef5a9fea4759/app/permissions_migrations.go#L1056-L1072
The specific roles ('%playbook%') are removed in the procedure below, but the migrations also add new rows to the Systems table marking the migrations as complete.
This in-app migration does not happen in the script, so we remove that rows here. */
DELETE FROM Systems WHERE Name = 'playbooks_manage_roles';
/* The server migration contains an in-app migration that adds boards permissions to certain roles:
getProductsBoardsPermissions, defined in https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/permissions_migrations.go#L1074-L1093
The specific roles (sysconsole_read_product_boards and sysconsole_write_product_boards) are removed in the procedure below,
but the migrations also adds a new row to the Systems table marking the migrations as complete.
This in-app migration does not happen in the script, so we remove that row here. */
DELETE FROM Systems WHERE Name = 'products_boards';
/* The server migration contains an in-app migration that adds Ids to the Teams whose InviteId is an empty string:
doRemainingSchemaMigrations, defined in https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/migrations.go#L515-L540
The migration is not replicated in the script, since it happens in-app, but the server adds a new row to the
Systems table marking the table as complete, which the script doesn't do, so we remove that row here. */
DELETE FROM Systems WHERE Name = 'RemainingSchemaMigrations';
/* The server migration contains three in-app migration that adds a new role and new permissions
related to custom groups. The migrations are:
- doCustomGroupAdminRoleCreationMigration https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/migrations.go#L345-L469
- getAddCustomUserGroupsPermissions https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/permissions_migrations.go#L974-L995
- getAddCustomUserGroupsPermissionRestore https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/permissions_migrations.go#L997-L1019
The specific roles and permissions are removed in the procedure below, but the migrations also
adds a new row to the Roles table for the new role and new rows to the Systems table marking the
migrations as complete.
This in-app migration does not happen in the script, so we remove that row here. */
DELETE FROM Roles WHERE Name = 'system_custom_group_admin';
DELETE FROM Systems WHERE Name = 'CustomGroupAdminRoleCreationMigrationComplete';
DELETE FROM Systems WHERE Name = 'custom_groups_permissions';
DELETE FROM Systems WHERE Name = 'custom_groups_permission_restore';
/* The server migration contains an in-app migration that updates the config, setting ServiceSettings.PostPriority
to true, doPostPriorityConfigDefaultTrueMigration, defined in https://github.com/mattermost/mattermost-server/blob/282bd351e3767dcfd8c8340da2e0915197c0dbcb/app/migrations.go#L542-L560
The migration is not replicated in the script, since it happens in-app, but the server adds a new row to the
Systems table marking the table as complete, which the script doesn't do, so we remove that row here. */
DELETE FROM Systems WHERE Name = 'PostPriorityConfigDefaultTrueMigrationComplete';
/* The rest of this script defines and executes a procedure to update the Roles table. It performs several changes:
1. Set the UpdateAt column of all rows to a fixed value, so that the server migration changes to this column
do not appear in the diff.
2. Remove the set of specific permissions added in the server migration that is not covered by the script, as
this logic happens all in-app after the normal DB migrations.
3. Set a consistent order in the Permissions column, which is modelled a space-separated string containing each of
the different permissions each role has. This change is the reason why we need a complex procedure, which creates
a temporary table that pairs each single permission to its corresponding ID. So if the Roles table contains two
rows like:
Id: 'abcd'
Permissions: 'view_team read_public_channel invite_user'
Id: 'efgh'
Permissions: 'view_team create_emojis'
then the new temporary table will contain five rows like:
Id: 'abcd'
Permissions: 'view_team'
Id: 'abcd'
Permissions: 'read_public_channel'
Id: 'abcd'
Permissions: 'invite_user'
Id: 'efgh'
Permissions: 'view_team'
Id: 'efgh'
Permissions: 'create_emojis'
*/
DROP PROCEDURE IF EXISTS splitPermissions;
DROP PROCEDURE IF EXISTS sortAndFilterPermissionsInRoles;
DROP TEMPORARY TABLE IF EXISTS temp_roles;
CREATE TEMPORARY TABLE temp_roles(id varchar(26), permission longtext);
DELIMITER //
/* Auxiliary procedure that splits the space-separated permissions string into single rows that are inserted
in the temporary temp_roles table along with their corresponding ID. */
CREATE PROCEDURE splitPermissions(
IN id varchar(26),
IN permissionsString longtext
)
BEGIN
DECLARE idx INT DEFAULT 0;
SELECT TRIM(permissionsString) INTO permissionsString;
SELECT LOCATE(' ', permissionsString) INTO idx;
WHILE idx > 0 DO
INSERT INTO temp_roles SELECT id, TRIM(LEFT(permissionsString, idx));
SELECT SUBSTR(permissionsString, idx+1) INTO permissionsString;
SELECT LOCATE(' ', permissionsString) INTO idx;
END WHILE;
INSERT INTO temp_roles(id, permission) VALUES(id, TRIM(permissionsString));
END; //
/* Main procedure that does update the Roles table */
CREATE PROCEDURE sortAndFilterPermissionsInRoles()
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE rolesId varchar(26) DEFAULT '';
DECLARE rolesPermissions longtext DEFAULT '';
DECLARE cur1 CURSOR FOR SELECT Id, Permissions FROM Roles;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
/* 1. Set a fixed value in the UpdateAt column for all rows in Roles table */
UPDATE Roles SET UpdateAt = 1;
/* Call splitPermissions for every row in the Roles table, thus populating the
temp_roles table. */
OPEN cur1;
read_loop: LOOP
FETCH cur1 INTO rolesId, rolesPermissions;
IF done THEN
LEAVE read_loop;
END IF;
CALL splitPermissions(rolesId, rolesPermissions);
END LOOP;
CLOSE cur1;
/* 2. Filter out the new permissions added by the in-app migrations */
DELETE FROM temp_roles WHERE permission LIKE 'sysconsole_read_products_boards';
DELETE FROM temp_roles WHERE permission LIKE 'sysconsole_write_products_boards';
DELETE FROM temp_roles WHERE permission LIKE 'playbook_public_manage_roles';
DELETE FROM temp_roles WHERE permission LIKE 'playbook_private_manage_roles';
DELETE FROM temp_roles WHERE permission LIKE '%custom_group%';
/* Temporarily set to the maximum permitted value, since the call to group_concat
below needs a value bigger than the default */
SET group_concat_max_len = 18446744073709551615;
/* 3. Update the Permissions column in the Roles table with the filtered, sorted permissions,
concatenated again as a space-separated string */
UPDATE
Roles INNER JOIN (
SELECT temp_roles.id as Id, TRIM(group_concat(temp_roles.permission ORDER BY temp_roles.permission SEPARATOR ' ')) as Permissions
FROM Roles JOIN temp_roles ON Roles.Id = temp_roles.id
GROUP BY temp_roles.id
) AS Sorted
ON Roles.Id = Sorted.Id
SET Roles.Permissions = Sorted.Permissions;
/* Reset group_concat_max_len to its default value */
SET group_concat_max_len = 1024;
END; //
DELIMITER ;
CALL sortAndFilterPermissionsInRoles();
DROP TEMPORARY TABLE IF EXISTS temp_roles;

599
server/scripts/esrupgrades/esr.6.3-7.8.mysql.up.sql Normal file
View File

@@ -0,0 +1,599 @@
/* ==> mysql/000041_create_upload_sessions.up.sql <== */
/* Release 5.37 was meant to contain the index idx_uploadsessions_type, but a bug prevented that.
This part of the migration #41 adds such index */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'UploadSessions'
AND table_schema = DATABASE()
AND index_name = 'idx_uploadsessions_type'
) > 0,
'SELECT 1',
'CREATE INDEX idx_uploadsessions_type ON UploadSessions(Type);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
/* ==> mysql/000075_alter_upload_sessions_index.up.sql <== */
DELIMITER //
CREATE PROCEDURE AlterIndex()
BEGIN
DECLARE columnName varchar(26) default '';
SELECT IFNULL(GROUP_CONCAT(column_name ORDER BY seq_in_index), '') INTO columnName
FROM information_schema.statistics
WHERE table_schema = DATABASE()
AND table_name = 'UploadSessions'
AND index_name = 'idx_uploadsessions_user_id'
GROUP BY index_name;
IF columnName = 'Type' THEN
DROP INDEX idx_uploadsessions_user_id ON UploadSessions;
CREATE INDEX idx_uploadsessions_user_id ON UploadSessions(UserId);
END IF;
END//
DELIMITER ;
CALL AlterIndex();
DROP PROCEDURE IF EXISTS AlterIndex;
/* ==> mysql/000076_upgrade_lastrootpostat.up.sql <== */
DELIMITER //
CREATE PROCEDURE Migrate_LastRootPostAt_Default ()
BEGIN
IF (
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'Channels'
AND TABLE_SCHEMA = DATABASE()
AND COLUMN_NAME = 'LastRootPostAt'
AND (COLUMN_DEFAULT IS NULL OR COLUMN_DEFAULT != 0)
) = 1 THEN
ALTER TABLE Channels ALTER COLUMN LastRootPostAt SET DEFAULT 0;
END IF;
END//
DELIMITER ;
CALL Migrate_LastRootPostAt_Default ();
DROP PROCEDURE IF EXISTS Migrate_LastRootPostAt_Default;
DELIMITER //
CREATE PROCEDURE Migrate_LastRootPostAt_Fix ()
BEGIN
IF (
SELECT COUNT(*)
FROM Channels
WHERE LastRootPostAt IS NULL
) > 0 THEN
-- fixes migrate cte and sets the LastRootPostAt for channels that don't have it set
UPDATE
Channels
INNER JOIN (
SELECT
Channels.Id channelid,
COALESCE(MAX(Posts.CreateAt), 0) AS lastrootpost
FROM
Channels
LEFT JOIN Posts FORCE INDEX (idx_posts_channel_id_update_at) ON Channels.Id = Posts.ChannelId
WHERE
Posts.RootId = ''
GROUP BY
Channels.Id) AS q ON q.channelid = Channels.Id
SET
LastRootPostAt = lastrootpost
WHERE
LastRootPostAt IS NULL;
-- sets LastRootPostAt to 0, for channels with no posts
UPDATE Channels SET LastRootPostAt=0 WHERE LastRootPostAt IS NULL;
END IF;
END//
DELIMITER ;
CALL Migrate_LastRootPostAt_Fix ();
DROP PROCEDURE IF EXISTS Migrate_LastRootPostAt_Fix;
/* ==> mysql/000077_upgrade_users_v6.5.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Users'
AND table_schema = DATABASE()
AND column_name = 'AcceptedServiceTermsId'
) > 0,
'ALTER TABLE Users DROP COLUMN AcceptedServiceTermsId;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000078_create_oauth_mattermost_app_id.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'OAuthApps'
AND table_schema = DATABASE()
AND column_name = 'MattermostAppID'
) > 0,
'SELECT 1',
'ALTER TABLE OAuthApps ADD COLUMN MattermostAppID varchar(32);'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000079_usergroups_displayname_index.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'UserGroups'
AND table_schema = DATABASE()
AND index_name = 'idx_usergroups_displayname'
) > 0,
'SELECT 1',
'CREATE INDEX idx_usergroups_displayname ON UserGroups(DisplayName);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
/* ==> mysql/000080_posts_createat_id.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Posts'
AND table_schema = DATABASE()
AND index_name = 'idx_posts_create_at_id'
) > 0,
'SELECT 1;',
'CREATE INDEX idx_posts_create_at_id on Posts(CreateAt, Id) LOCK=NONE;'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
/* ==> mysql/000081_threads_deleteat.up.sql <== */
-- Replaced by 000083_threads_threaddeleteat.up.sql
/* ==> mysql/000082_upgrade_oauth_mattermost_app_id.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'OAuthApps'
AND table_schema = DATABASE()
AND column_name = 'MattermostAppID'
) > 0,
'UPDATE OAuthApps SET MattermostAppID = "" WHERE MattermostAppID IS NULL;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'OAuthApps'
AND table_schema = DATABASE()
AND column_name = 'MattermostAppID'
) > 0,
'ALTER TABLE OAuthApps MODIFY MattermostAppID varchar(32) NOT NULL DEFAULT "";',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000083_threads_threaddeleteat.up.sql <== */
-- Drop any existing DeleteAt column from 000081_threads_deleteat.up.sql
SET @preparedStatement = (SELECT IF(
EXISTS(
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Threads'
AND table_schema = DATABASE()
AND column_name = 'DeleteAt'
) > 0,
'ALTER TABLE Threads DROP COLUMN DeleteAt;',
'SELECT 1;'
));
PREPARE removeColumnIfExists FROM @preparedStatement;
EXECUTE removeColumnIfExists;
DEALLOCATE PREPARE removeColumnIfExists;
SET @preparedStatement = (SELECT IF(
NOT EXISTS(
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Threads'
AND table_schema = DATABASE()
AND column_name = 'ThreadDeleteAt'
),
'ALTER TABLE Threads ADD COLUMN ThreadDeleteAt bigint(20);',
'SELECT 1;'
));
PREPARE addColumnIfNotExists FROM @preparedStatement;
EXECUTE addColumnIfNotExists;
DEALLOCATE PREPARE addColumnIfNotExists;
UPDATE Threads, Posts
SET Threads.ThreadDeleteAt = Posts.DeleteAt
WHERE Posts.Id = Threads.PostId
AND Threads.ThreadDeleteAt IS NULL;
/* ==> mysql/000084_recent_searches.up.sql <== */
CREATE TABLE IF NOT EXISTS RecentSearches (
UserId CHAR(26),
SearchPointer int,
Query json,
CreateAt bigint NOT NULL,
PRIMARY KEY (UserId, SearchPointer)
);
/* ==> mysql/000085_fileinfo_add_archived_column.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'FileInfo'
AND table_schema = DATABASE()
AND column_name = 'Archived'
) > 0,
'SELECT 1',
'ALTER TABLE FileInfo ADD COLUMN Archived boolean NOT NULL DEFAULT false;'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000086_add_cloud_limits_archived.up.sql <== */
SET @preparedStatement = (SELECT IF(
NOT EXISTS(
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Teams'
AND table_schema = DATABASE()
AND column_name = 'CloudLimitsArchived'
),
'ALTER TABLE Teams ADD COLUMN CloudLimitsArchived BOOLEAN NOT NULL DEFAULT FALSE;',
'SELECT 1'
));
PREPARE alterIfNotExists FROM @preparedStatement;
EXECUTE alterIfNotExists;
DEALLOCATE PREPARE alterIfNotExists;
/* ==> mysql/000087_sidebar_categories_index.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'SidebarCategories'
AND table_schema = DATABASE()
AND index_name = 'idx_sidebarcategories_userid_teamid'
) > 0,
'SELECT 1;',
'CREATE INDEX idx_sidebarcategories_userid_teamid on SidebarCategories(UserId, TeamId) LOCK=NONE;'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
/* ==> mysql/000088_remaining_migrations.up.sql <== */
DROP TABLE IF EXISTS JobStatuses;
DROP TABLE IF EXISTS PasswordRecovery;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Users'
AND table_schema = DATABASE()
AND column_name = 'ThemeProps'
) > 0,
'INSERT INTO Preferences(UserId, Category, Name, Value) SELECT Id, \'\', \'\', ThemeProps FROM Users WHERE Users.ThemeProps != \'null\'',
'SELECT 1'
));
PREPARE migrateTheme FROM @preparedStatement;
EXECUTE migrateTheme;
DEALLOCATE PREPARE migrateTheme;
-- We have to do this twice because the prepared statement doesn't support multiple SQL queries
-- in a single string.
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Users'
AND table_schema = DATABASE()
AND column_name = 'ThemeProps'
) > 0,
'ALTER TABLE Users DROP COLUMN ThemeProps',
'SELECT 1'
));
PREPARE migrateTheme FROM @preparedStatement;
EXECUTE migrateTheme;
DEALLOCATE PREPARE migrateTheme;
/* ==> mysql/000089_add-channelid-to-reaction.up.sql <== */
SET @preparedStatement = (SELECT IF(
NOT EXISTS(
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Reactions'
AND table_schema = DATABASE()
AND column_name = 'ChannelId'
),
'ALTER TABLE Reactions ADD COLUMN ChannelId varchar(26) NOT NULL DEFAULT "";',
'SELECT 1;'
));
PREPARE addColumnIfNotExists FROM @preparedStatement;
EXECUTE addColumnIfNotExists;
DEALLOCATE PREPARE addColumnIfNotExists;
UPDATE Reactions SET ChannelId = COALESCE((select ChannelId from Posts where Posts.Id = Reactions.PostId), '') WHERE ChannelId="";
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Reactions'
AND table_schema = DATABASE()
AND index_name = 'idx_reactions_channel_id'
) > 0,
'SELECT 1',
'CREATE INDEX idx_reactions_channel_id ON Reactions(ChannelId);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
/* ==> mysql/000090_create_enums.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Channels'
AND table_schema = DATABASE()
AND column_name = 'Type'
AND column_type != 'ENUM("D", "O", "G", "P")'
) > 0,
'ALTER TABLE Channels MODIFY COLUMN Type ENUM("D", "O", "G", "P");',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Teams'
AND table_schema = DATABASE()
AND column_name = 'Type'
AND column_type != 'ENUM("I", "O")'
) > 0,
'ALTER TABLE Teams MODIFY COLUMN Type ENUM("I", "O");',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'UploadSessions'
AND table_schema = DATABASE()
AND column_name = 'Type'
AND column_type != 'ENUM("attachment", "import")'
) > 0,
'ALTER TABLE UploadSessions MODIFY COLUMN Type ENUM("attachment", "import");',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000091_create_post_reminder.up.sql <== */
CREATE TABLE IF NOT EXISTS PostReminders (
PostId varchar(26) NOT NULL,
UserId varchar(26) NOT NULL,
TargetTime bigint,
PRIMARY KEY (PostId, UserId)
);
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'PostReminders'
AND table_schema = DATABASE()
AND index_name = 'idx_postreminders_targettime'
) > 0,
'SELECT 1',
'CREATE INDEX idx_postreminders_targettime ON PostReminders(TargetTime);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
/* ==> mysql/000092_add_createat_to_teammembers.up.sql <== */
SET @preparedStatement = (SELECT IF(
NOT EXISTS(
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'TeamMembers'
AND table_schema = DATABASE()
AND column_name = 'CreateAt'
),
'ALTER TABLE TeamMembers ADD COLUMN CreateAt bigint DEFAULT 0;',
'SELECT 1;'
));
PREPARE addColumnIfNotExists FROM @preparedStatement;
EXECUTE addColumnIfNotExists;
DEALLOCATE PREPARE addColumnIfNotExists;
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'TeamMembers'
AND table_schema = DATABASE()
AND index_name = 'idx_teammembers_create_at'
) > 0,
'SELECT 1',
'CREATE INDEX idx_teammembers_createat ON TeamMembers(CreateAt);'
));
PREPARE createIndexIfNotExists FROM @preparedStatement;
EXECUTE createIndexIfNotExists;
DEALLOCATE PREPARE createIndexIfNotExists;
/* ==> mysql/000093_notify_admin.up.sql <== */
CREATE TABLE IF NOT EXISTS NotifyAdmin (
UserId varchar(26) NOT NULL,
CreateAt bigint(20) DEFAULT NULL,
RequiredPlan varchar(26) NOT NULL,
RequiredFeature varchar(100) NOT NULL,
Trial BOOLEAN NOT NULL,
PRIMARY KEY (UserId, RequiredFeature, RequiredPlan)
);
/* ==> mysql/000094_threads_teamid.up.sql <== */
-- Replaced by 000096_threads_threadteamid.up.sql
/* ==> mysql/000095_remove_posts_parentid.up.sql <== */
-- While upgrading from 5.x to 6.x with manual queries, there is a chance that this
-- migration is skipped. In that case, we need to make sure that the column is dropped.
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Posts'
AND table_schema = DATABASE()
AND column_name = 'ParentId'
) > 0,
'ALTER TABLE Posts DROP COLUMN ParentId;',
'SELECT 1'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000096_threads_threadteamid.up.sql <== */
-- Drop any existing TeamId column from 000094_threads_teamid.up.sql
SET @preparedStatement = (SELECT IF(
EXISTS(
SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE table_name = 'Threads'
AND table_schema = DATABASE()
AND column_name = 'TeamId'
) > 0,
'ALTER TABLE Threads DROP COLUMN TeamId;',
'SELECT 1;'
));
PREPARE removeColumnIfExists FROM @preparedStatement;
EXECUTE removeColumnIfExists;
DEALLOCATE PREPARE removeColumnIfExists;
SET @preparedStatement = (SELECT IF(
NOT EXISTS(
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Threads'
AND table_schema = DATABASE()
AND column_name = 'ThreadTeamId'
),
'ALTER TABLE Threads ADD COLUMN ThreadTeamId varchar(26) DEFAULT NULL;',
'SELECT 1;'
));
PREPARE addColumnIfNotExists FROM @preparedStatement;
EXECUTE addColumnIfNotExists;
DEALLOCATE PREPARE addColumnIfNotExists;
UPDATE Threads, Channels
SET Threads.ThreadTeamId = Channels.TeamId
WHERE Channels.Id = Threads.ChannelId
AND Threads.ThreadTeamId IS NULL;
/* ==> mysql/000097_create_posts_priority.up.sql <== */
CREATE TABLE IF NOT EXISTS PostsPriority (
PostId varchar(26) NOT NULL,
ChannelId varchar(26) NOT NULL,
Priority varchar(32) NOT NULL,
RequestedAck tinyint(1),
PersistentNotifications tinyint(1),
PRIMARY KEY (PostId)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
SET @preparedStatement = (SELECT IF(
NOT EXISTS(
SELECT 1 FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'ChannelMembers'
AND table_schema = DATABASE()
AND column_name = 'UrgentMentionCount'
),
'ALTER TABLE ChannelMembers ADD COLUMN UrgentMentionCount bigint(20);',
'SELECT 1;'
));
PREPARE alterIfNotExists FROM @preparedStatement;
EXECUTE alterIfNotExists;
DEALLOCATE PREPARE alterIfNotExists;
/* ==> mysql/000098_create_post_acknowledgements.up.sql <== */
CREATE TABLE IF NOT EXISTS PostAcknowledgements (
PostId varchar(26) NOT NULL,
UserId varchar(26) NOT NULL,
AcknowledgedAt bigint(20) DEFAULT NULL,
PRIMARY KEY (PostId, UserId)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
/* ==> mysql/000099_create_drafts.up.sql <== */
CREATE TABLE IF NOT EXISTS Drafts (
CreateAt bigint(20) DEFAULT NULL,
UpdateAt bigint(20) DEFAULT NULL,
DeleteAt bigint(20) DEFAULT NULL,
UserId varchar(26) NOT NULL,
ChannelId varchar(26) NOT NULL,
RootId varchar(26) DEFAULT '',
Message text,
Props text,
FileIds text,
PRIMARY KEY (UserId, ChannelId, RootId)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
/* ==> mysql/000100_add_draft_priority_column.up.sql <== */
SET @preparedStatement = (SELECT IF(
(
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'Drafts'
AND table_schema = DATABASE()
AND column_name = 'Priority'
) > 0,
'SELECT 1',
'ALTER TABLE Drafts ADD COLUMN Priority text;'
));
PREPARE alterIfExists FROM @preparedStatement;
EXECUTE alterIfExists;
DEALLOCATE PREPARE alterIfExists;
/* ==> mysql/000101_create_true_up_review_history.up.sql <== */
CREATE TABLE IF NOT EXISTS TrueUpReviewHistory (
DueDate bigint(20),
Completed boolean,
PRIMARY KEY (DueDate)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

23
server/scripts/esrupgrades/esr.common.mysql.preprocess.sql Normal file
View File

@@ -0,0 +1,23 @@
/* The sessions in the DB dump may have expired before the CI tests run, making
the server remove the rows and generating a spurious diff that we want to avoid.
In order to do so, we mark all sessions' ExpiresAt value to 0, so they never expire. */
UPDATE Sessions SET ExpiresAt = 0;
/* The dump may not contain a system-bot user, in which case the server will create
one if it's not shutdown before a job requests it. This situation creates a flaky
tests in which, in rare ocassions, the system-bot is indeed created, generating a
spurious diff. We avoid this by making sure that there is a system-bot user and
corresponding bot */
DELIMITER //
CREATE PROCEDURE AddSystemBotIfNeeded ()
BEGIN
DECLARE CreateSystemBot BOOLEAN;
SELECT COUNT(*) = 0 FROM Users WHERE Username = 'system-bot' INTO CreateSystemBot;
IF CreateSystemBot THEN
/* These values are retrieved from a real system-bot created by a server */
INSERT INTO `Bots` VALUES ('nc7y5x1i8jgr9btabqo5m3579c','','phxrtijfrtfg7k4bwj9nophqyc',0,1681308600015,1681308600015,0);
INSERT INTO `Users` VALUES ('nc7y5x1i8jgr9btabqo5m3579c',1681308600014,1681308600014,0,'system-bot','',NULL,'','system-bot@localhost',0,'','System','','','system_user',0,'{}','{\"push\": \"mention\", \"email\": \"true\", \"channel\": \"true\", \"desktop\": \"mention\", \"comments\": \"never\", \"first_name\": \"false\", \"push_status\": \"away\", \"mention_keys\": \"\", \"push_threads\": \"all\", \"desktop_sound\": \"true\", \"email_threads\": \"all\", \"desktop_threads\": \"all\"}',1681308600014,0,0,'en','{\"manualTimezone\": \"\", \"automaticTimezone\": \"\", \"useAutomaticTimezone\": \"true\"}',0,'',NULL);
END IF;
END//
DELIMITER ;
CALL AddSystemBotIfNeeded();