Commit Graph
136 Commits
Author SHA1 Message Date
Jarek RadoszandLoïc Guitaut a54e3208cb DEV: Hand-pick Rails/WhereNot autofixes (#35117)
We can't enable `Rails/WhereNot` lint/autofix, because it would break
code that uses mini_sql instead of AR (which rubocop, and tbh also we,
can't easily differentiate)

Those are safe because they either:
* are executed in AR model scope definitions
* are clearly chained starting from a AR model
* are less-clearly chained, but still can be traced to a AR model/scope

---------

Co-authored-by: Loïc Guitaut <loic@discourse.org>
2025-10-03 13:29:22 +02:00
Jarek Radosz 60dadd2166 DEV: Remove unnecessary freeze calls (#35064)
All those files have `# frozen_string_literal: true`
2025-09-30 15:36:16 +02:00
Jarek RadoszandLoïc Guitaut e372355fd0 DEV: Clean up scope resolution operators in plugins (#34979)
Co-authored-by: Loïc Guitaut <loic@discourse.org>
2025-09-30 14:36:34 +02:00
Ted Johansson 71ea236a79 FIX: Use a valid value for disabling backups using backup_frequency (#34245)
In #33558 we removed automatic_backups_enabled setting, and instead rely on backup_frequency being blank to disable.

There was a big oversight there with the site setting type system, which will coerce the value to an integer. It also makes it so you can't blank the value out in the UI.

This is a "fix forward" solution where instead of "set to blank to disable" we do "set to 0 to disable". This works along the grain of the site setting type system for a workable fix where we don't have to deal with the irreversible migration in the previous change.

We can potentially go and add in "nullable" to the type system at a later point.
2025-08-12 13:19:56 +08:00
Ted Johansson f8bf51441e DEV: Remove automatic_backups_enabled setting (#33558)
We're removing the automatic_backups_enabled toggle and depending solely on the backup_frequency value. This PR replaces the setting and adds a migration for any forum that has automatic_backups_enabled disabled.
2025-08-11 14:56:38 +08:00
Martin Brennan 7121cfd4ab FEATURE: User preference for editor Markdown monospace font (#34051)
This commit responds to feedback in the Discourse Meta discussion

https://meta.discourse.org/t/monospace-font-in-the-markdown-only-editor/359936

This change introduces a user preference that allows users to choose
whether the Markdown editor uses a monospace font. The default setting
is `true` for new sites, but set to `false` for existing sites to avoid
disrupting current users' experiences.

Admins can change the `default_other_enable_markdown_monospace_font`
site setting to manage this for all users.
2025-08-04 14:56:21 +10:00
Selase Krakani 8e1e54cbc1 FIX: Ensure imported category slug is deduplicate (#34012)
This change ensures the incoming category slug doesn't already exist
2025-08-01 00:06:34 +00:00
Martin Brennan 77211b43ee FEATURE: Enable rich editor for all users (#33699)
We believe the rich editor is a great experience for the
vast majority of sites and users, so we are enabling it
for all sites and all users by default.

This commit does the following:

* Hides the rich_editor site setting and sets it to true by default.
  It can still be overridden by sites that want to disable it
  completely.
* Sets `rich_editor` to true for all sites to enable the rich editor
  everywhere.
* Adds a new `default_composition_mode` site setting and corresponding
user option that defaults to Rich for all users. The other option is
Markdown.
* Changes the rich editor toggle in the composer to use the new
  database-backed user option (`composition_mode`) instead of a local
  storage key/value store. This makes the preference persistent
  across devices.

Existing key/value store settings for the markdown toggle are
kept, the preference will be saved to the user option automatically.
2025-07-28 10:08:50 +10:00
Selase Krakani 6f73f31a45 FIX: Store context for failed uploads (#33546)
This change makes it easier to review failed uploads, especially
downloadable ones, by standardizing `skip_reason` and adding a
`skip_details` column to the uploads DB's `uploads` table for extended
failure information.
2025-07-24 12:05:32 +00:00
Selase Krakani fb9c25b4b8 FIX: Various optimization step fixes (#33523)
1. Ensure `Creating optimized images...` message is only available
during optimization
2. Guard upload lookup for optimization
3. Replace  undefined `stacktrace` method invocation with `backtrace`
2025-07-10 01:53:29 +00:00
Ruben Oussoren be300f960b fix/ Added default values for category color and text_color (#33479) 2025-07-03 20:16:40 -04:00
Ruben Oussoren a697740929 Add category moderation groups and more attributes to generic importer (#32561)
This PR improves the generic bulk import and base scripts by adding
support for several previously unimplemented features and attributes:

- Category Moderation Groups: Implements the import of group-based
moderation settings for categories.
- Group Attributes: Adds import support for public_admission,
public_exit, allow_membership_requests, and assignable_level.
- User Attributes: Adds import support for trust_level and
primary_group_id.
- User Option: Adds import support for the hide_profile_and_presence
user option.
- Multiple Tag Groups per Tag: Updates tag import logic to handle tags
belonging to multiple tag groups.
2025-07-03 17:03:10 -04:00
Gerhard Schlager 249d9514c8 DEV: Import user title in generic_bulk 2025-06-15 15:56:36 +02:00
Gerhard Schlager e371e1b3cd FIX: Skip importing solved topics when plugin isn't installed (#32922) 2025-05-27 22:08:43 +02:00
Selase Krakani 5702cc9250 DEV: Enable migrated_site flag during bulk imports (#32256)
Extend the `import_site_settings` import step to enable the
`migrated_site` flag if possible.

It also includes an escape hatch via the
`SKIP_MIGRATED_SITE_FLAG_UPDATE` flag for cases where you really don't
want to toggle it on.
2025-04-29 22:00:48 +00:00
Ahmed Gagan 1ede31b704 DEV: Update topic solutions import for new table structure (generic_bulk.rb) (#32051) 2025-04-29 10:43:59 +00:00
Selase Krakani 9bfb4c005d FIX: Set last_read_post_number for imported watching topic users (#32229)
Ensure `last_read_post_number` is set to the latest post for imported
topic users with a `watching` notification level, even if they haven't
engaged with the topic
2025-04-10 12:35:51 +00:00
Selase Krakani fb5cdb1da4 DEV: Grant all anniversary badges during imports (#31830)
This change adds an import step to retroactively grant anniversary
badges to users.
2025-03-18 10:45:47 +00:00
Ruben Oussoren a585fc5a24 FIX: Calculate reaction_users_count in the bulk import script (#30964)
`reaction_users_count` should be calculated in the import script instead
of the converter.
2025-02-12 17:29:10 -05:00
Gerhard Schlager eb305e0a98 DEV: Update default config for uploads_importer (#31208) 2025-02-11 16:33:15 +01:00
Gerhard Schlager 9279487079 DEV: Use correct method for creating slugs (#31273)
Empty slugs for topics break Discourse. This makes sure that we always
fall back to "topic" as default. And it also uses the configured slug
generation method instead of always using ASCII.
2025-02-11 16:29:08 +01:00
Ruben Oussoren fba1a483c1 FIX: Avoid loading reactions index when plugin is not installed (#30749)
Adding fix to avoid loading reactions indexes when plugin is not
installed
2025-01-13 17:35:22 -05:00
Ruben OussorenandGerhard Schlager 20a16ea231 Adding Import Support for Discourse Reactions Plugin (#30361)
* Adding Import Support for Discourse Reactions Plugin

---------

Co-authored-by: Gerhard Schlager <gerhard.schlager@discourse.org>
2025-01-07 08:58:26 -05:00
Ruben Oussoren f81bf31993 Fix the import votes for posts table to the new table (#29810) 2024-11-19 10:53:33 -05:00
Jan CernikandRégis Hanol 234133bd3b UX: Split hide_profile_and_presence user option (#29632)
It splits the hide_profile_and_presence user option and the default_hide_profile_and_presence site setting for more granular control. It keeps the option to hide the profile under /u/username/preferences/interface and adds the presence toggle in the quick user menu.

Co-authored-by: Régis Hanol <regis@hanol.fr>
2024-11-12 22:22:58 -03:00
Alan Guo Xiang Tan 57f4176b57 DEV: Bump rubocop_discourse (#29608) 2024-11-06 06:27:49 +08:00
Ruben Oussoren 4b8b7b2fb5 DEV: Add import support for Topic Voting plugin (#29442)
DEV: Add import support for Topic Voting plugin
- Implemented bulk import functionality specifically for topic_voting_votes.
- Updated the method name from import_votes to import_post_voting_votes for improved clarity and maintainability.
- Co-authored-by: Gerhard Schlager <gerhard.schlager@discourse.org>
2024-10-31 08:30:28 -04:00
Martin Brennan 27bdfb6437 FEATURE: Add user preference to disable smart lists (#29434)
Followup 30fdd7738e

Adds a new site setting and corresponding user preference
to disable smart lists. By default they are enabled, because
this is a better experience for most users. A small number of
users would prefer to not have this enabled.

Smart lists automatically append new items to each
list started in the composer when enter is pressed. If
enter is pressed on an empty list item, it is cleared.

This setting will be removed when the new composer is complete.
2024-10-28 14:18:15 +10:00
Alan Guo Xiang Tan 322a3be2db DEV: Remove logical OR assignment of constants (#29201)
Constants should always be only assigned once. The logical OR assignment
of a constant is a relic of the past before we used zeitwerk for
autoloading and had bugs where a file could be loaded twice resulting in
constant redefinition warnings.
2024-10-16 10:09:07 +08:00
Selase Krakani dd34f1927b FIX: Imports of upload-only chat messages (#29162)
The current implementation adds a "note" for chat messages with empty
messages, however chat messages with only uploads  are allowed. This change
allows such messages to be imported.
2024-10-10 15:18:10 +00:00
Selase Krakani 9825bde811 DEV: Generic bulk chat import support (#28697)
* DEV: WIP generic bulk chat import support

This first iteration implements bulk import for:

* direct_messages
* chat_channels
* user_chat_channel_memberships
* chat_threads
* user_chat_thread_memberships
* chat_messages
* chat_reactions
* chat_mentions

* DEV: Refactor raw placeholder interpolation to support chat messages

This change adds support for chat message placeholder interpolation
and switches to using `Chat::Message.cook` for cooking in the interim
instead of hand-cooking chat messages like we currently do for posts

* DEV: Extend upload references import to support chat message uploads

* DEV: Explicity set chat retention

- Set both channel and dm chats to 0
- Add temporary workaround for testing only chat imports

* DEV: Compute channel and thread membership metadata

Compute and set various user channel/thread membership stats and
remove hardcoded test index seed data

* FIX: Fix chat reactions import

Allow multiple reactions on a message by a user
2024-10-08 11:55:30 +00:00
Gerhard Schlager d4379af7f2 FIX: Import script didn't set public attribute of polls (#28864) 2024-10-02 20:02:13 +02:00
Selase Krakani d896f5cb70 DEV: Include post and topic attributes in imported quotes (#27851)
Currently, quotes imported via generic bulk import script do not include
references to the quoted post. This change includes both topic and post attributes
in a quote if the placeholder metadata includes a `post_id`
2024-07-11 16:47:21 +00:00
Gerhard Schlager 7c26d5d084 FIX: Import script was broken after upgrade of sqlite3 gem (#27648) 2024-07-02 17:38:15 +10:00
Selase Krakani f2c4474c1e DEV: Improve user generic bulk importer anonymization (#27307)
* DEV: Improve user generic bulk importer anonymization

Add support for properly anonymizing:
 - email
 - date_of_birth
 - location
 - website
 - bio

* DEV: Remove uneeded anon username check in `import_user_emails`
2024-06-05 11:25:17 +00:00
Loïc Guitaut 2a28cda15c DEV: Update to lastest rubocop-discourse 2024-05-27 18:06:14 +02:00
Selase Krakani 949c70372c DEV: Add support for various fields in generic bulk importer (#27114)
* user_profiles - `location`
* users - `date_of_birth`
* topics - `pinned_at`, `pinned_until`, `pinned_globally`

This also include changes to correctly import PMs. Currently PM topics
are skipped because of a check in `import_users` step which requires `category_id`
to be present.
2024-05-24 13:46:06 +02:00
Selase Krakani 61e12aaebe FEATURE: Extend PM recipient bulk imports (#27063)
* FIX: Support multiple topic allowed user imports

* FEATURE: Add topic allowed groups import support
2024-05-17 13:45:20 +02:00
Ítalo AlvesandGerhard Schlager 73481e8f45 FIX: Add check for existing provider_uids to generic import (#26914)
Co-authored-by: Gerhard Schlager <gerhard.schlager@discourse.org>
2024-05-17 11:36:31 +02:00
Gerhard Schlager 1872047053 DEV: Uploads import script can download files (#26816)
Uploads import script can download files
2024-05-04 22:48:16 +02:00
Gerhard Schlager e3882a0c48 DEV: Add support for user_associated_accounts to import script (#26779) 2024-04-29 19:48:32 +02:00
Gerhard Schlager a538e2f153 DEV: Import script should use case-insensitive check for tag names (#26699) 2024-04-29 19:27:28 +02:00
Gerhard Schlager 4d045bfc61 DEV: Import script should insert more data into user_stats table (#26551)
This SQL tries to insert as much data as possible into the `user_stats` table by either calculating or by approximating stats based on existing. It also fixes an error in the calculation of `reply_count`which mistakenly contained all posts, not just replies.

This change also disables some steps in the `import:ensure_consistency` rake task by setting the `SKIP_USER_STATS` env variable. Otherwise, the rake task will overwrite the calculated data in the `user_stats` table with inaccurate data. I'm not changing or removing the logic from the rake task yet because other bulk import scripts seem to depend on it.
2024-04-11 14:05:21 +02:00
Gerhard Schlager bc98740205 DEV: Improve generic import script (#25972)
* FEATURE: Import into `category_users` table
* FIX: Failed to import `user_options` unless `timezone` was set
* FIX: Prevent reusing original `id` from intermediate DB in `user_fields`
* FEATURE: Order posts by `post_nuber` if available
* FEATURE: Allow `[mention]` placeholder to reference users by"id" or "name" (username)
* FEATURE: Support `[quote]` placeholders in posts
* FEATURE: Support `[link]` placeholders in posts
* FEATURE: Support all kinds of permalinks and remove support for `old_relative_url`
* PERF: Speed up pre-cooking by removing DB lookups
2024-03-05 22:23:36 +01:00
Gerhard Schlager 38ff1a38bd DEV: Improve uploads_importer script (#25971)
* Print instructions when the `sqlite3` gem can't be loaded
* Use `display_filename` instead of `filename` if available
* Support uploading for a multisite
2024-03-05 16:27:45 +01:00
Gerhard Schlager 241bf48497 DEV: Allow rebakes to generate optimized images at the same time
Previously only Sidekiq was allowed to generate more than one optimized image at the same time per machine. This adds an easy mechanism to allow the same in rake tasks and other tools.
2024-01-16 14:33:16 +01:00
Gerhard Schlager dc8c6b8958 DEV: Lots of improvements to the generic_bulk import script
Notable changes:
* Imports a lot more tables from core and plugins
  * site settings
  * uploads with necessary upload references
  * groups and group members
  * user profiles
  * user options
  * user fields & values
  * muted users
  * user notes (plugin)
  * user followers (plugin)
  * user avatars
  * tag groups and tags
  * tag users (notification settings for tags / user)
  * category permissions
  * polls with options and votes
  * post votes (plugin)
  * solutions (plugin)
  * gamification scores (plugin)
  * events (plugin)
  * badges and badge groupings
  * user badges
  * optimized images
  * topic users (notification settings for topics)
  * post custom fields
  * permalinks and permalink normalizations

* It creates the `migration_mappings` table which is used to store the mapping for a handful of imported tables

* Detects duplicate group names and renames them

* Pre-cooking for attachments, images and mentions

* Outputs instructions when gems are missing

* Supports importing uploads from a DB generated by `uploads_importer.rb`

* Checks that all required plugins exists and enables them if needed

* A couple of optimizations and additions in `import.rake`
2023-12-11 16:23:07 +01:00
Gerhard Schlager d725b3ca9e DEV: Add script for preprocessing uploads as part of a migration
This script preprocesses all uploads within a intermediate DB (output of converters) and uploads those files to S3. It does the same for optimized images. This speeds up migrations when you have to run them multiple times, because you only have to preprocess and upload the files once.

This script is very hacky and mostly undocumented for now. That will change in the future.
2023-12-11 16:23:07 +01:00
Jarek Radosz 694b5f108b DEV: Fix various rubocop lints (#24749)
These (21 + 3 from previous PRs) are soon to be enabled in rubocop-discourse:

Capybara/VisibilityMatcher
Lint/DeprecatedOpenSSLConstant
Lint/DisjunctiveAssignmentInConstructor
Lint/EmptyConditionalBody
Lint/EmptyEnsure
Lint/LiteralInInterpolation
Lint/NonLocalExitFromIterator
Lint/ParenthesesAsGroupedExpression
Lint/RedundantCopDisableDirective
Lint/RedundantRequireStatement
Lint/RedundantSafeNavigation
Lint/RedundantStringCoercion
Lint/RedundantWithIndex
Lint/RedundantWithObject
Lint/SafeNavigationChain
Lint/SafeNavigationConsistency
Lint/SelfAssignment
Lint/UnreachableCode
Lint/UselessMethodDefinition
Lint/Void

Previous PRs:
Lint/ShadowedArgument
Lint/DuplicateMethods
Lint/BooleanSymbol
RSpec/SpecFilePathSuffix
2023-12-06 23:25:00 +01:00
Constanza 28f27b2490 DEV: Adding polls, solutions, upload references and other improvements to the Discourse merger script (#23689) 2023-11-16 14:32:53 +01:00