We can use the status of upcoming changes to indicate
whether they are experimental or not, having experimental
in the setting name is redundant.
Migrates the settings and the upcoming change events,
updates code, and updates yaml translation keys.
What is the problem?
Discourse allows admins to mark a tag as a synonym of another tag. For
example, "brunch" can be made a synonym of "lunch". When this happens,
all topics tagged with "brunch" are automatically retagged with "lunch",
and the `tags.target_tag_id` column on the synonym tag record is set to
point to the target tag.
However, several code paths did not account for synonyms:
1. **Search:** `Search#search_tags` and the hashtag advanced filter did
not resolve synonyms. When a user searched using a synonym name (e.g.
`tags:brunch`, `tags:brunch+eggs`, or `#brunch`), no results were
returned because:
- The `tags:` comma path queries `topic_tags` joined with `tags` by
name, but topics are tagged with the target tag "lunch", not the
synonym "brunch".
- The `tags:` plus path aggregates tag names per topic into a
tsvector and matches against the searched name, but the aggregated
names are target tag names, so "brunch" never matches.
- The `#` hashtag path picks the synonym tag's own `id` and queries
`topic_tags` by that ID, but topics store the target tag's ID.
2. **Filter route:** `TopicsFilter#tag_ids_from_tag_names` concatenated
both the synonym's own ID and the target tag ID. For match-all
queries (e.g. `tag:brunch`), this required a topic to have both IDs
in `topic_tags`, which never happens since only the target tag ID is
stored. For negation queries (e.g. `-tag:brunch`), the exclusion
targeted the synonym ID rather than the target, so no topics were
excluded.
What is the solution?
In `Search#search_tags`, add a synonym resolution step before the
existing comma/plus branching logic. It splits the match string into
individual tag names, queries for any that are synonyms via
`Tag.where_name(tag_names).where.not(target_tag_id: nil)`, builds a
name mapping, and replaces synonym names with their target tag names.
The replacement operates on the split array elements rather than using
substring replacement to avoid corrupting tag names that may contain
other tag names as substrings.
In the hashtag advanced filter, pick both `:id` and `:target_tag_id`
from the tag lookup and prefer `target_tag_id` when present, so the
query uses the target tag's ID instead of the synonym's own ID.
In `TopicsFilter#tag_ids_from_tag_names`, replace the transpose/concat
approach with `.map { |id, target_id| target_id || id }` to resolve
each tag to its canonical ID — the target for synonyms, or the tag's
own ID otherwise.
A partial index on `tags.target_tag_id` is added (scoped to
`WHERE target_tag_id IS NOT NULL`) to support efficient synonym lookups.
Unlisted topics (visible=false) should not appear in hot topic
rankings. This change:
- Filters out unlisted topics when updating hot scores
- Excludes unlisted topics from hottest_topic_ids cache
- Applies 50% score reduction to closed topics
- Removes unlisted topics from index
This commit adds several pieces of functionality to help keep admins
in the loop about upcoming changes.
First of all, there is a new initializer on boot that will notify admins
about
newly available upcoming changes, as well as log removed changes and
status movement of existing changes.
* When there is a new upcoming change, we only notify admins about
it when the status is the `promote_upcoming_changes_on_status` - 1,
e.g. if `promote_upcoming_changes_on_status` is `beta` then we only
tell admin about the change once it has reached `alpha`. This means
we may log the `added` event in one deploy, but only actually notify
admins in a subsequent deploy.
* We log removed upcoming changes so we can automatically delete old
site setting data in a future job as needed.
We also now notify admins when upcoming changes are automatically
promoted to enabled based on the site's
`promote_upcoming_changes_on_status`:
<img width="378" height="600" alt="image"
src="https://github.com/user-attachments/assets/4200fbee-9990-4bbc-a378-85946e631e77"
/>
In addition, we now show an indicator in the admin sidebar
if there are new upcoming changes that have been added since
they last visited the upcoming change config page. This data
is stored in a user custom field, because Redis is ephemeral,
and storing in the User table is overkill because 99% of users
are not staff:
<img width="248" height="112" alt="image"
src="https://github.com/user-attachments/assets/4c3d3cf7-ac39-45f8-a2c8-a049cb85b8e9"
/>
Finally, this commit moves both the Track and Promote initializer
logic behind a `DistributedMutex`, we don't want multiple processes
running the same logic here, it needs to be only once.
---------
Co-authored-by: Loïc Guitaut <loic@discourse.org>
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
The top_topics table had 20 indexes on count columns (daily_posts_count,
daily_views_count, etc.) that were never used according to
pg_stat_user_indexes. Only the *_score indexes are queried for /top
pages.
per
https://meta.discourse.org/t/imap-support-for-group-inboxes/160588/39?u=martin
we have been planning to remove IMAP support for a while,
because of its low usage and adoption, high complexity, and maintenance
burden.
This commit removes all IMAP-related code, including models,
jobs, services, and frontend components.
---------
Co-authored-by: Régis Hanol <regis@hanol.fr>
Joining the `topics` table against the `categories` table is a common
thing we do and the missing index on the `category_id` foreign key is
hurting performance on sites with lots of rows in the `topics` and
`categories` tables.
We're adding `slug` to the `Tag` model so that we can move URLs away
from the `/tag/name` convention to `/tag/slug/id`.
This commit only includes adding the column in the database, a migration,
and ActiveRecord validation. For extra info to the reviewer, this is a
list of scenarios for how slugs are generated in categories, in the tag
migration, and tag model:
| scenario | category | tag (migration) | tag (model) |
|-----------------------------------|------------------|------------------|------------------|
| normal ("Turkish Angora") | "turkish-angora" | "turkish-angora" |
"turkish-angora" |
| numeric-only ("123") | "" | "" | "" |
| unicode-only ("猫") | "" | "" | "" |
| unicode mixed ("turkish猫angora") | "turkish-angora" | "turkish-angora"
| "turkish-angora" |
| special chars ("turkish@angora!") | "turkish-angora" |
"turkish-angora" | "turkish-angora" |
| special chars only ("@#$%") | "" | "" | "" |
| conflict | "" | "" | "" |
All empty slugs use slug_for_url → "#{id}-tag" for URLs. This is similar
to how categories work to keep consistent. This PR contains 2
migrations, slug generation and index creation. The migration spec
doesn't allow for "CONCURRENT" index creation, so we are separating them
to make things easier to test.
When dismissing an admin notice, the service would look up the
associated `ProblemCheckTracker` by identifier only. This could find a
tracker with a `NULL` target, which then fails validation on update
since target is now required (as of 8ca5fb706a).
The root issue is that `problem_check_trackers.target` was added with
`null: true` and while we later added a default value of `"__NULL__"`
and a Ruby validation, we never enforced `NOT NULL` at the database
level. This allowed records with `NULL` targets to persist or be
recreated through code paths that explicitly passed nil.
This commit:
- Updates the dismiss service to look up trackers by both identifier AND
target (extracted from the admin notice's details)
- Removes any remaining records with `NULL` targets
- Adds a `NOT NULL` constraint to prevent this from recurring
Ref - https://meta.discourse.org/t/392248
When a user has been destroyed, any claimed reviewables belonging to
that user are partially orphaned, which the review queue doesn't know
how to handle. Ensuring those reviewables are unclaimed (and cleaning up
any orphaned claims on existing sites) ensures the review queue is able
to cleanly handle claimed reviewables.
Currently our automatically generated anchor links for headings in posts
are pretty awful to listen to in screenreaders, because screenreaders
are reading the `name` attribute.
Here's an example from a [Meta
post](https://meta.discourse.org/t/a-new-review-queue-layout-with-all-new-features/388194)...
this "A refreshed layout" heading reads like:
"p-1888119-a-refreshed-layout-1"
This PR adds `aria-label="heading link"` so now this would read like:
"Heading link, same page link, A Refreshed Layout, heading level 1"
I've added a migration so posts with headings will rebake with the
improved markup.
---------
Co-authored-by: Yuriy Kurant <yuriy@discourse.org>
This change adds webhook support for the user anonymization event,
allowing external systems to be notified when a user is anonymized in
Discourse.
Changes:
- Add user_anonymized event type (ID: 310) to WebHookEventType
- Wires up DiscourseEvent listener to trigger webhooks on anonymization
- Add translation string for the event in admin UI
- Add database fixture to seed the event type
- Add comprehensive test coverage
The webhook is triggered when UserAnonymizer.make_anonymous is called
and sends the anonymized user data using WebHookUserSerializer.
Nudged along the path by Claude Code
With reviewables now being able to have multiple actions performed on
them, we don't want to update the final reviewable state until an action
from every action bundle has been performed.
This change makes use of the new `ReviewableActionLog`, logging actions
when they occur, then checking to see if a reviewable has had all
actions performed before finalising it.
As part of the review queue refresh, moderators will be able to perform
individual actions on each part of a reviewable (for example, hiding a
post, and suspending the user). In order to know when a reviewable is
"completed", we need to be able to check that the right number of
actions have been performed on the reviewable.
This change adds a `ReviewableActionLog` model where we can keep track
of the actions that have been performed on each reviewable.
As part of localizing tags, we want to also add locale attribute to
tags.
This commit is standalone so as to not bloat the future ones for review. (we
will be adding slug column + controller changes in a single future PR)
Part 2 of many
- https://github.com/discourse/discourse/pull/35962
- t/153419
Followup 93862b98dd
There still is a slight window between the pre and post deploy migration
for this setting where we can end up with a UserOption row with a null
watched_precedence_over_muted value. This commit adds a safeguard to the
post-migration script to handle that case.
The change_column_null and change_column_default operations for the
watched_precedence_over_muted column have been moved from the
pre-migration
script to the post-migration script. This adjustment ensures that the
column
is only modified after all data transformations are complete, reducing
the
risk of data integrity issues during the migration process.
We have already seen this once in production, where a later failed
migration left the column in a bad state.
This PR introduces part one of the "Upcoming changes" interface for
Discourse admins.
The upcoming changes feature is an enhancement around our existing
site-setting based feature flagging and experiments system. With some
light metadata, we can give admins a much better overview of the current
work we are doing, with ways for them to opt-out in early stages and
opt-in to things that we haven’t yet turned on by default for them.
This system, along with encouraging a more liberal use of site setting
flags for features, experiments, and refactors in the app, should
minimise the problem of breakages and disruptions for all Discourse
users. It is also our intent with this system for it to be easier for
designers to add and remove these changes.
Finally, it also gives us a kind of running changelog that we can use to
communicate with site owners before releases and “What’s new?” updates.
### FOR REVIEWERS
This initial PR is gated behind a hidden `enable_upcoming_changes` site
setting, because there is still more work to do before we reveal this to
admins.
To test the UI, you can add this metadata under any boolean-based site
setting, though upcoming change settings will specifically be hidden:
```
upcoming_change:
status: "alpha" (see UpcomingChanges.statuses.keys)
impact: "feature,staff" (feature|other for the first part, staff|admins|moderators|all_members|developers for the second part)
learn_more_url: "https://some.url"
```
To test the images, add an image under `public/images/upcoming_changes`
with the file name as `SITE_SETTING_NAME.png`
### Interface
Admins can see the following in the interface for upcoming changes:
* The status of the change. Changes can progress along these statuses:
* Pre-Alpha
* Alpha
* Beta
* Stable
* Permanent
* The impact of the change. This is split into Type and Role. Type can
be "Feature" or "Other" for now. Changes may affect the following roles:
* Admins
* Moderators
* Staff
* All members
* The plugin that is making the change
* The groups that are opted-in to the change. Admins can control these
groups for a gradual rollout. If a change is enabled, it is limited to
these groups.
* In some cases, an image related to the change, behind the "Preview"
link
* A link to learn more about the change
Admins can filter the changes by name, description, plugin, status,
impact type, and whether the change is enabled.
### Promotion system
For our hosted customers, we intend to have a status-based
auto-promotion system as changes progress.
For all sites, once a change reaches the Stable status, if an admin
opts-out of that change it will generate an admin problem message that
will be shown on the dashboard.
For self-hosted Discourse admins, changes will only be forcibly enabled
when they reach the Permanent state.
### Notification system
A notification system for upcoming changes so admins can stay informed
will be added in a followup PR.
---------
Co-authored-by: awesomerobot <kris.aubuchon@discourse.org>
We have a setting and a user option called watched_precedence_over_muted
. This is supposed to allow members to see/be notified of topics in
categories they are watching even if those topics also include a tag the
user has muted, or vice-versa if they disable this option.
However, this setting and user option combination doesn't follow our
existing default_ user preference site setting system, where admins can
set a default for all users, but users can change the preference for
themselves. This is leading to a bug in the topic feed logic.
If the site setting watched_precedence_over_muted is set to true, and
the user preference is set to false, the user preference is essentially
ignored. This will lead to users seeing topics with muted tags inside a
watched category feed.
This commit converts watched_precedence_over_muted to a proper default
user preference site setting, which admins can set for all users at
once, then users can override individually.
We recently hid multiple very technical site settings related to image quality. In this PR we add a unified image_quality setting.
Power users can still configure the individual settings (now defaulting to 0) and they will take precedence.
In some pathological cases on the tag intersection route
for topics, when querying across multiple tags, the SQL query
planner can choose a suboptimal plan that results in a
sequential scan of the topic_tags table instead of using
the existing topic_id, tag_id index.
```
-> Parallel Seq Scan on topic_tags t2 (cost=0.00..23043.22 rows=13618 width=4) (actual time=0.031..54.114 rows=10535 loops=3)
Filter: (tag_id = 58)
Rows Removed by Filter: 579415
```
This resulted in a very slow query. Adding this extra
index allows us to hint the query planner to use a more
efficient plan by filtering using the tag ID first _then_ the topic ID.
This adds a new notification's user preference to allow users to control
whether they are notified whenever one of their topic is being linked in
another post.
Internal ref - t/160259
This change ensures that oneboxes, lightboxes are correctly generated
for localized posts.
The `PostLocalizer` was using `PrettyText.cook` directly, which does not
perform all the necessary post-processing steps, such as onebox
generation.
This commit introduces a new `LocalizedCookedPostProcessor` class that
is responsible for post-processing the cooked HTML of translated posts.
This new class reuses the `CookedProcessorMixin` to gain access to the
`post_process_oneboxes` method. The `PostLocalizer` is updated to use
this new processor, ensuring that oneboxes are correctly generated in
the translated content.
---------
Co-authored-by: Nat <natalie.tay@discourse.org>
This commit splits the `moderators_manage_categories_and_groups` setting
into 2 separate settings to allow for more precise control over
categories and groups management by moderators.
Internal topic: t/141392.
This commit allows editing colors of palettes that are installed with
themes. Prior to this commit, editing colors of theme-owned palettes wasn't
allowed because a theme update could override the edits made by admins
and there was no way to revert edits to the original values. With this
commit, all of that is solved by copying the palette when it's first edited
by an admin, and making future updates to the theme update the original
copy only with the ability for admins to revert to the colors in the
original copy at any time.
Internal topic: t/162130.
Replace individual per-locale queries with a single query that
calculates completion progress for all locales at once.
This commit also removes methods off the `base_candidate` (used for
Categories and Topics as well) which are not used any more, as the
progress dashboard only shows progress for posts.
t/162846/4
---------
Co-authored-by: Nat <natalie.tay@discourse.org>
We've had the new floatkit-based autocomplete live in various parts of
Discourse for about a month now, and it's stable enough for us to remove
the site setting letting admins revert to the jquery-based autocomplete
library which is now deprecated.
This was causing restores to fail. The trigger is also unnecessary
because the `topic_id` column is already marked as read-only.
Follow-up to eeeb7d302f
To prepare for the introduction of `CategoryDefaultTimer`, which allows
posts created in certain categories to have a default TopicTimer, we
need to rename the `topic_id` column to `timerable_id`.
---------
Co-authored-by: Alan Guo Xiang Tan <gxtan1990@gmail.com>
We want to allow configuring fields that are both optional and editable to not show up in the signup form.
- Fields that are optional but not editable need to be on the signup form, or they can never be filled in.
- Fields that are not optional need to be on the signup form, or you can never sign up.
This PR extends the current setting
`content_localization_anonymous_language_switcher` to one that can be
shown even for logged in users ->
`content_localization_language_switcher`
When logged in, the preference is saved to the `user` rather than a
cookie. 🍪https://github.com/user-attachments/assets/b6f4d41b-4716-4020-a97d-2c3f90d54576
We're not going to finish/release dual-mode palettes and theme-owned
palettes at this time, so we're cleaning up the code that we've already
merged for these features.
Internal topic: t/161279.
The current impersonation feature works by signing you in as the user you are impersonating. This has the side effect of invalidating your own session and forcing you to log out and in again.
In this experimental implementation you keep your existing session, but DefaultCurrentUserProvider returns the user being impersonated, allowing you to see the site from their perspective.
The change is sponsored by `Rails/AddColumnIndex` rubocop rule:
> (…) add_column does not accept index, but also does not raise an error
for extra keys, so it is possible to mistakenly add the key without
realizing it will not actually add an index.
I opted not to add these "missing" indexes since they've proven not
necessary after all.
Martin Brennan
Alan Guo Xiang Tan
Sam
Jarek RadoszandLoïc Guitaut
Natalie Tay
David Battersby
Gary Pendergast
Sérgio Saquetim
KrisandYuriy Kurant
Mark Doerr
Gabriel Grubba
Ted Johansson
Rafael dos Santos SilvaandNat
Osama Sayegh
Kelv
LincaandAlan Guo Xiang Tan
Bianca Nenciu
Krzysztof Kotlarek