When a post has some replies, and the user click on the button to show them, we would load ALL the replies. This could lead to DoS if there were a very large number of replies.
This adds support for pagination to these post replies.
Internal ref t/129773
FIX: Duplicated parent posts
DEV: Query refactor
When running checks, we look to the existing problem check trackers and try to grab their ProblemCheck classes.
In some cases this is no longer in the problem check repository, e.g. when the check was part of a plugin that has been uninstalled.
In the case where the check was scheduled, this would lead to an error in one of the jobs
his is a new feature that lets admins dismiss notices from the dashboard. This helps with self-service in cases where a notice is "stuck", while we work on provisions to prevent "sticking" in the first place.
Anonymous users are "shadow" users created when an existing real user desires to post anonymously. This feature is off by default, but it can be enabled via the `allow_anonymous_posting` site setting. Those shadow users shouldn't be included in the users directory (`/u`).
In 14cf8eacf1, we added the
`user_search_similar_results` site setting which when enabled will use
trigram matching for similarity search in `UserSearch`. However, we
noted that adding the `index_users_on_username_lower_trgm` index is
causing the PG planner to not use the `index_users_on_username_lower`
index when the `=` operator is used against the `username_lower` column.
Based on the PG mailing list discussion where support for the `=`
operator in gist_trgm_ops was being considered, it stated that "I also have checked that btree_gist is preferred over pg_trgm gist
index for equality search." This is however quite different from reality
on our own PG clusters where the btree index is not preferred leading to
significantly slower queries when the `=` operator is used.
Since the pg_trgm gist index is only used for queries when the `user_search_similar_results` site setting
is enabled, we decided to drop the feature instead as it is hidden and
disabled by default. As such, we can consider it experiemental and drop
it without deprecation.
PG mailing list discussiong: https://www.postgresql.org/message-id/CAPpHfducQ0U8noyb2L3VChsyBMsc5V2Ej2whmEuxmAgHa2jVXg%40mail.gmail.com
Currently, when the custom flag has the same name as the system flag (which is disabled) then it is not displayed. To fix the problem, `custom_` prefix as `name_key` is used to distinguish between the system and the custom flag.
I considered writing a migration to fix existing custom flags name key. However, at the end of migration I would need to run rails code to reset cache `Flag.reset_flag_settings!`. I decided to skip that step as it is a very edge case. If someone has the same flag name as the system flag, then all they have to do is edit the flag and click save.
In addition, I made 2 small fixes:
- edit flag title was missing translation;
- flag form UI was not showing that description is the required field.
The Report model spec was directly adding methods
to the Report class, which was causing errors in the
admin reports controller because it would look for
a translation of the report name (e.g. report_timeout_test)
like so `I18n.t("reports.#{type}.title")`, then get an
error because the translation did not exist.
This is fixed by using `Report.stubs` instead, which is
cleaned up after every test.
This commit adds a new `about_page_hidden_groups` setting to exclude members of specific groups from the admin and moderator lists on the /about page.
Internal topic: t/137717.
### UI changes
All of the UI changes described are gated behind the `use_legacy_pageviews`
site setting.
This commit changes the admin dashboard pageviews report to
use the "Consolidated Pageviews with Browser Detection" report
introduced in 2f2da72747 with
the following changes:
* The report name is changed to "Site traffic"
* The pageview count on the dashboard is counting only using the new method
* The old "Consolidated Pageviews" report is renamed as "Consolidated Legacy Pageviews"
* By default "known crawlers" and "other" sources of pageviews are hidden on the report
When `use_legacy_pageviews` is `true`, we do not show or allow running
the "Site traffic" report for admins. When `use_legacy_pageviews` is `false`,
we do not show or allow running the following legacy reports:
* consolidated_page_views
* consolidated_page_views_browser_detection
* page_view_anon_reqs
* page_view_logged_in_reqs
### Historical data changes
Also part of this change is that, since we introduced our new "Consolidated
Pageviews with Browser Detection" report, some admins are confused at either:
* The lack of data before a certain date , which didn’t exist before
we started collecting it
* Comparing this and the current "Consolidated Pageviews" report data,
which rolls up "Other Pageviews" into "Anonymous Browser" and so it
appears inaccurate
All pageview data in the new report before the date where the _first_
anon or logged in browser pageview was recorded is now hidden.
Currently, categories support designating only 1 group as a moderation group on the category. This commit removes the one group limitation and makes it possible to designate multiple groups as mods on a category.
Internal topic: t/124648.
This commit introduces a new hidden site setting: `group_pm_user_limit`, default to `1000` which will raise an error when attempting to create a PM target a large group.
Prior to this fix we were calling `fetch_stats` which is never checking if we have a cache entry. This call is making a lot of SQL calls, so it's better to use the cache.
Follow-up to e3ae57ea7a
The previous commit added an `after_create` callback that triggers a refresh for the user directory whenever a `User` record is created. Theoretically, this approach should work, however, there's a gotcha in practice, because during a real user registration, when the `User` record is created in the database, it's not marked as active until the user verifies their email address and the user directory excludes inactive users, so the initial directory refresh triggered by the `after_create` callback becomes pointless.
To make a new user appear in the user directory immediately after sign up, we need to trigger a refresh via an `after_save` callback when they verify their email address and become active.
Prior this fix a topic created in the future could generate this exception:
```
Job exception: ERROR: a negative number raised to a non-integer power yields a complex result
```
This fix and spec should ensure we don't consider topics in the future as the rest of the `update_scores` function is doing at other places.
This check was checking the wrong scope, causing problems in certain edge conditions, for example:
1. Admin adds an "on signup" field that isn't editable after signup.
2. Admin adds a "for all users" field.
3. User goes and fills up the "for all users" field from 2.
4. User is now stuck on the required fields page without any fields showing.
With this change, we only consider "for all users" fields when asking if required custom fields are filled in.
### Why?
Before, all flags were static. Therefore, they were stored in class variables and serialized by SiteSerializer. Recently, we added an option for admins to add their own flags or disable existing flags. Therefore, the class variable had to be dropped because it was unsafe for a multisite environment. However, it started causing performance problems.
### Solution
When a new Flag system is used, instead of using PostActionType, we can serialize Flags and use fragment cache for performance reasons.
At the same time, we are still supporting deprecated `replace_flags` API call. When it is used, we fall back to the old solution and the admin cannot add custom flags. In a couple of months, we will be able to drop that API function and clean that code properly. However, because it may still be used, redis cache was introduced to improve performance.
To test backward compatibility you can add this code to any plugin
```ruby
replace_flags do |flag_settings|
flag_settings.add(
4,
:inappropriate,
topic_type: true,
notify_type: true,
auto_action_type: true,
)
flag_settings.add(1001, :trolling, topic_type: true, notify_type: true, auto_action_type: true)
end
```
### Why?
Before, all flags were static. Therefore, they were stored in class variables and serialized by SiteSerializer. Recently, we added an option for admins to add their own flags or disable existing flags. Therefore, the class variable had to be dropped because it was unsafe for a multisite environment. However, it started causing performance problems.
### Solution
When a new Flag system is used, instead of using PostActionType, we can serialize Flags and use fragment cache for performance reasons.
At the same time, we are still supporting deprecated `replace_flags` API call. When it is used, we fall back to the old solution and the admin cannot add custom flags. In a couple of months, we will be able to drop that API function and clean that code properly. However, because it may still be used, redis cache was introduced to improve performance.
To test backward compatibility you can add this code to any plugin
```ruby
replace_flags do |flag_settings|
flag_settings.add(
4,
:inappropriate,
topic_type: true,
notify_type: true,
auto_action_type: true,
)
flag_settings.add(1001, :trolling, topic_type: true, notify_type: true, auto_action_type: true)
end
```
Currently to handle stub topics after merging, there are only options to (1) never delete a stub topic and (2) delete a stub topic after X amount of days. This adds the option to immediately delete a stub topic upon merge.
---------
Co-authored-by: Mark VanLandingham <markvanlan@gmail.com>
Co-authored-by: Renato Atilio <renato@discourse.org>
Following a recent refactor, some methods from `FlagSettings` have been
renamed (`custom_types` -> `additional_message_types`). The
`PostActionType` model was using `custom_types` but when the renaming
was done, it was renamed to `with_additional_message` instead of
`additional_message_types`, which under the right circumstances will
raise an error.
When creating a shared draft, we're recording topic view stats on the draft and then pass those on when the draft is published, conflating the actual view count.
This fixes that by not registering topic views if the topic is a shared draft.
Currently, when adding translation overrides, values aren’t validated
for MF strings. This results in being able to add invalid plural keys or
even strings containing invalid syntax.
This patch addresses this issue by compiling the string when saving an
override if the key is detected as an MF one.
If there’s an error from the compiler, it’s added to the model errors,
which in turn is displayed to the user in the admin UI, helping them to
understand what went wrong.
* FIX: Add post id to the anchor to prevent two identical anchors
We generate anchors for headings in posts. This works fine if there is
only one post in a topic with anchors. The problem comes when you have
two or more posts with the same heading. PrettyText generates anchors
based on the heading text using the raw context of each post, so it is
entirely possible to generate the same anchor for two posts in the same
topic, especially for topics with template replies
Post1:
# heading
context
Post2:
# heading
context
When both posts are on the page at the same time, the anchor will only
work for the first post, according to the [HTML specification](https://html.spec.whatwg.org/multipage/browsing-the-web.html#scroll-to-the-fragment-identifier).
> If there is an a element in the document tree whose root is document
> that has a name attribute whose value is equal to fragment, then
> return the *first* such element in tree order.
This bug is particularly serious in forums with non-Latin languages,
such as Chinese. We do not generate slugs for Chinese, which results in
the heading anchors being completely dependent on their order.
```ruby
[2] pry(main)> PrettyText.cook("# 中文")
=> "<h1><a name=\"h-1\" class=\"anchor\" href=\"#h-1\"></a>中文</h1>"
```
Therefore, the anchors in the two posts must be in exactly the same by
order, causing almost all of the anchors in the second post to be
invalid.
This commit solves this problem by adding the `post_id` to the anchor.
The new anchor generation method will add `p-{post_id}` as a prefix when
post_id is available:
```ruby
[3] pry(main)> PrettyText.cook("# 中文", post_id: 1234)
=> "<h1><a name=\"p-1234-h-1\" class=\"anchor\" href=\"#p-1234-h-1\"></a>中文</h1>"
```
This way we can ensure that each anchor name only appears once on the
same topic. Using post id also prevents the potential possibility of the
same anchor name when splitting/merging topics.
Our old group SMTP SSL option was a checkbox,
but this was not ideal because there are actually
3 different ways SSL can be used when sending
SMTP:
* None
* SSL/TLS
* STARTTLS
We got around this before with specific overrides
for Gmail, but it's not flexible enough and now people
want to use other providers. It's best to be clear,
though it is a technical detail. We provide a way
to test the SMTP settings before saving them so there
should be little chance of messing this up.
This commit also converts GroupEmailSettings to a glimmer
component.
Allow admin to create custom flag which requires an additional message.
I decided to rename the old `custom_flag` into `require_message` as it is more descriptive.
The watch words controller creation function, create_or_update_word(), doesn’t validate the size of the replacement parameter, unlike the word parameter, when creating a replace watched word. So anyone with moderator privileges can create watched words with almost unlimited characters.
Allow admin to create custom flag which requires an additional message.
I decided to rename the old `custom_flag` into `require_message` as it is more descriptive.
This patch upgrades the MessageFormat library to version 3.3.0 from
0.1.5.
Our `I18n.messageFormat` method signature is unchanged, and now uses the
new API under the hood.
We don’t need dedicated locale files for handling pluralization rules
anymore as everything is now included by the library itself.
The compilation of the messages now happens through our
`messageformat-wrapper` gem. It then outputs an ES module that includes
all its needed dependencies.
Most of the changes happen in `JsLocaleHelper` and in the `ExtraLocales`
controller.
A new method called `.output_MF` has been introduced in
`JsLocaleHelper`. It handles all the fetching, compiling and
transpiling to generate the proper MF messages in JS. Overrides and
fallbacks are also handled directly in this method.
The other main change is that now the MF translations are served through
the `ExtraLocales` controller instead of being statically compiled in a
JS file, then having to patch the messages using overrides and
fallbacks. Now the MF translations are just another bundle that is
created on the fly and cached by the client.
Drafts used to be deleted instead of being destroyed. The callbacks that
clean up the upload references were not being called. As a result, the
upload references were not cleaned up and uploads were not deleted
either. This has been partially fixed in 9655bf3e.
This change eliminates a couple of instances where subfolder urls are badly formatted, in most cases we can use Discourse.base_url_no_prefix to prevent adding the subfolder to the base url.
