Commit Graph

4896 Commits

Author SHA1 Message Date
Sam
ffc3da35a6
FIX: handle rapid concurrent SSO attempts more gracefully (#11180)
Rapid concurrent SSO attempts is something that happens quite frequently
in the wild at large enough scale.

When this happens conditions such as adding a user to a group could possibly
fire concurrently causing a user to be added to the same group twice and
erroring out.

To avoid all concurrency issues here we protect with a coarse distributed
mutex. This heavily mitigates the risk around concurrent group additions and
concurrent updates to user related records.
2020-11-10 10:40:41 +00:00
David Taylor
cf21de0e7a
DEV: Migrate Github authentication to ManagedAuthenticator (#11170)
This commit adds an additional find_user_by_email hook to ManagedAuthenticator so that GitHub login can continue to support secondary email addresses

The github_user_infos table will be dropped in a follow-up commit.

This is the last core authenticator to be migrated to ManagedAuthenticator 🎉
2020-11-10 10:09:15 +00:00
Krzysztof Kotlarek
586c8efbd8
FEATURE: the ability to permanently destroy the private message (#11115)
PostDestroyer should accept the option to permanently destroy post from the database. In addition, when the first post is destroyed it destroys the whole topic.

Currently, that feature is limited to private messages and creator of the post. It will be used by discourse-encrypt to explode encrypted private messages.
2020-11-10 15:40:48 +11:00
Arpit Jalan
394f37cf66
FIX: do not use "max_tags_in_filter_list" setting for showing PM tags (#11146)
Ensure we do not respect max_tags_in_filter_list when showing the list of PM tags.

This filter is used on a full page view and there is not point limiting it to a small number.

The expectation is that PM tags are very rarely used, so a hard limit of 1000 should be safe for now.
2020-11-10 08:09:59 +11:00
Guo Xiang Tan
650da7b626 PERF: Update index for category in a background job.
Search indexing can get expensive and there is no need for us to block
the entire request just to wait for index to finish.
2020-11-09 13:51:26 +08:00
Alan Guo Xiang Tan
0e4d966a74
DEV: Remove stale ignored_columns. (#11160) 2020-11-09 15:57:59 +11:00
Bianca Nenciu
0863c36221
FIX: Improve errors when invite to topic fails (#11133)
It used to simply say "not allowed" without giving any hint what the
problem could be. This commit refactors the code and tries to improve
readability.
2020-11-06 16:58:10 +02:00
tshenry
d778d99b55
FIX: limit_topics_per_day should only apply to regular topics (#11127)
PMs were being evaluted by both the limit_topics_per_day and limit_private_messages_per_day rate limiters when it should only be the latter.
2020-11-04 16:23:49 -08:00
Dan Ungureanu
3c51647872
DEV: Remove legacy tag and category routes (#10338)
* DEV: Remove client-side legacy tag and category routes

* DEV: Remove server-side legacy tag routes

* DEV: Refactor ListController#set_category

* FIX: Remove reference to discovery.parentCategory

* FIX: Refactor TagsController#set_category_from_params

* FIX: Build correct canonical URL for tags and categories

* DEV: Fix deprecation notice in Ruby 2.7

* DEV: Replace use of removed legacy tag route

* DEV: Add deprecation notices for old routes and controllers
2020-11-03 16:57:58 +02:00
David Taylor
5140ec9acf
DEV: Cleanup ignored user logic (#11107)
- IgnoredUser records should all now have an expiring_at value. This commit enforces that in the DB, and fixes any corrupt rows
- Changes to the ignored user list are now handled by the `/u/{username}/notification_level` endpoint. This allows setting expiration dates on the ignore. This commit removes the old logic for saving a list of usernames in the user preferences.
- Many specs were calling `IgnoredUser.create`. This commit changes them to use `Fabricate(:ignored_user)` for consistency
2020-11-03 12:38:54 +00:00
Kane York
789e3775df
FIX: Make all email subject vars available in notification subjects (#11064)
A site owner attempting to use both the email_subject site setting and translation overrides for normal post notification
email subjects would find themselves frusturated at the lack of template argument parity.
Make all the variables available for translation overrides by adding the subject variables to the custom interpolation keys list and applying them.

Reported at https://meta.discourse.org/t/customize-subject-format-for-standard-emails/20801/47?u=riking
2020-11-02 20:00:11 -08:00
Arpit Jalan
1476e17c35
FEATURE: new setting to create a linked topic on autoclosing mega topics (#11001)
This commit adds a site setting `auto_close_topics_create_linked_topic`
which when enabled works in conjunction with `auto_close_topics_post_count`
setting and creates a new linked topic for the topic just closed.

The auto-created new topic contains a link for all the previous topics
and the topic titles are appended with `(Part {n})`.

The setting is enabled by default.
2020-11-02 12:18:48 +05:30
Jarek Radosz
2f4a1ff61b
DEV: Update rubocop-discourse from 2.3.2 to 2.4.0 (#11079)
Also fixes whitespace related issues raised by rubocop.
2020-10-30 15:04:29 +01:00
Mark VanLandingham
c750a02f05
DEV: Move reports to concerns for autoloading (#11066) 2020-10-29 11:49:09 -05:00
Martin Brennan
632942e697
FIX: Ensure group SMTP and message builder always uses from address for Reply-To when IMAP is enabled (#11037)
There is a site setting reply_by_email_enabled which when combined with reply_by_email_address creates a Reply-To header in emails in the format "test+%{reply_key}@test.com" along with a PostReplyKey record, so when replying Discourse knows where to route the reply.

However this conflicts with the IMAP implementation. Since we are sending the email for a group via SMTP and from their actual email account, we want all replys to go to that email account as well so the IMAP sync job can pick them up and put them in the correct place. So if the group has IMAP enabled and configured, then the reply-to header will be correct.

This PR also makes a further fix to 64b0b50 by using the correct recipient user for the PostReplyKey record. If the post user is used we encounter this error:

if destination.user_id != user.id && !forwarded_reply_key?(destination, user)
  raise ReplyUserNotMatchingError, "post_reply_key.user_id => #{destination.user_id.inspect}, user.id => #{user.id.inspect}"
end
This is because the user above is found from the from_address, but the destination which is the PostReplyKey is made by the post.user, which will be different people.
2020-10-28 07:01:58 +10:00
Arpit Jalan
c6bf70c870
DEV: annotate models (#11047) 2020-10-27 23:42:33 +05:30
Mark VanLandingham
554a617b3d
DEV: Better custom field preload error (#11044) 2020-10-27 10:05:07 -05:00
Dan Ungureanu
43557143fe
FIX: Reset invite when resending it (#11013)
Resending an invite moved the expire date in the future, but did not
invalidate it. For example, if an invite was sent to an email,
invalidated and then resent, it would still be left invalidated.
2020-10-26 12:26:43 +02:00
jbrw
2bcca46cc5
FEATURE - ImageMagick jpeg quality (#11004)
* FEATURE - Add SiteSettings to control JPEG image quality

`recompress_original_jpg_quality` - the maximum quality of a newly
uploaded file.

`image_preview_jpg_quality` - the maximum quality of OptimizedImages
2020-10-23 12:38:28 -04:00
David Taylor
abb00c3780
FIX: Restore users#topic_tracking_state route to api session_info scope (#10992)
This route was inadvertently removed in 1cec333f, and is required for showing new/unread counts in Discourse mobile apps
2020-10-21 19:44:34 +01:00
David Taylor
b7c680853d DEV: Introduce plugin API to contribute user api key scopes 2020-10-19 10:40:55 +01:00
David Taylor
23e5c605f6 DEV: Add support for allowed parameters in user api key scopes
Initially, this feature is only intended for use in core/plugins, so there is no API for requesting a parameter-scoped key. That may change in future.
2020-10-19 10:40:55 +01:00
David Taylor
1cec333f48 REFACTOR: Introduce RouteMatcher class
This consolidates logic used to match routes in ApiKey, UserApiKey and DefaultCurrentUserProvider. This reduces duplicated logic, and will allow UserApiKeysScope to easily re-use the parameter matching logic from ApiKeyScope
2020-10-19 10:40:55 +01:00
Roman Rizzi
21c53ed249
FEATURE: Topic slow mode. (#10904)
Adds a new slow mode for topics that are heating up. Users will have to wait for a period of time before being able to post again.

We store this interval inside the topics table and track the last time a user posted using the last_posted_at datetime in the TopicUser relation.
2020-10-16 16:24:38 -03:00
Bianca Nenciu
43e52a7dc1
DEV: Remove gifsicle dependency (#10357)
Dependency on gifsicle, allow_animated_avatars and allow_animated_thumbnails
site settings were all removed. Animated GIF images are still allowed, but
the generated optimized images are no longer animated for those (which were
used for avatars and thumbnails).

The added 'animated' is populated by extracting information using FastImage.
This field was used to selectively reoptimize old animations. This process
happens in the background.
2020-10-16 13:41:27 +03:00
Penar Musaraj
5763309953
FEATURE: WCAG compliant color schemes (#10882)
Co-authored-by: Kris <kris.aubuchon@discourse.org>
2020-10-15 14:05:48 -04:00
Penar Musaraj
74de7a49f5
Simplify theme and color scheme seeding (#10872)
Now that we have support for user-selectable color schemes, it makes sense
to simplify seeding and theme updates in the wizard. 

We now:

- seed only one theme, named "Default" (previously "Light")
- seed a user-selectable Dark color scheme
- rename the "Themes" wizard step to "Colors"
- update the default theme's color scheme if a default is set
(a new theme is created if there is no default)
2020-10-14 10:18:02 -04:00
Martin Brennan
c3cede697d
FEATURE: Add weekly bookmark cleanup code (#10899)
When posts or topics are deleted we don't want to immediately delete associated bookmarks, so we have a grace period to recover them and their reminders if the post or topic is un-deleted. This PR adds a task to the Weekly scheduled job to go and delete bookmarks attached to posts or topics deleted > 3 days ago.
2020-10-14 09:38:57 +10:00
Bianca Nenciu
25b8ed740b
DEV: Make site setting type uploaded_image_list use upload IDs (#10401)
It used to be a list of concatenated upload URLs which was prone to
break.
2020-10-13 16:17:06 +03:00
Daniel Waterworth
721ee36425
Replace base_uri with base_path (#10879)
DEV: Replace instances of Discourse.base_uri with Discourse.base_path

This is clearer because the base_uri is actually just a path prefix. This continues the work started in 555f467.
2020-10-09 12:51:24 +01:00
Penar Musaraj
5130b4d674
Ignore disabled out-of-date remote themes (#10870)
No need to alert in admin dashboard about out of date remote themes
that are disabled.
2020-10-08 13:48:16 -04:00
David Taylor
c0293339b8
PERF: Do not enqueue digest emails when attempted recently (#10849)
Previously, Jobs::EnqueueDigestEmails would enqueue a digest job for every user, even if there are no topics to send. The digest job would exit, no email would send, and last_emailed_at would not change. 30 minutes later, Jobs::EnqueueDigestEmails would run again and re-enqueue jobs for the same users.

120fa8ad introduced a temporary mitigation for this issue, by randomly selecting a subset of those users each time.

This commit adds a new `digest_attempted_at` column to the `user_stats` table. This column is updated every time a digest job completes for a user. Using this, we can avoid scheduling digest jobs for the same user every 30 minutes. This also removes the random user selection in 120fa8ad, and instead prioritizes users who had digests attempted the longest time ago.
2020-10-07 15:30:38 +01:00
Martin Brennan
6e2be3e60b
FIX: When admin changes an email for the user the user must confirm the change (#10830)
See https://meta.discourse.org/t/changing-a-users-email/164512 for additional context.

Previously when an admin user changed a user's email we assumed that they would need a password reset too because they likely did not have access to their account. This proved to be incorrect, as there are other reasons a user needs admin to change their email. This PR:

* Changes the admin change email for user flow so the user is sent an email to confirm the change
* We now record who the email change request was requested by
* If the requested by user is admin and not the user we note this in the email sent to the user
* We also make the confirm change email route open to anonymous users, so it can be clicked by the user even if they do not have access to their account. If there is a logged in user we make sure the confirmation matches the current user.
2020-10-07 13:02:24 +10:00
Jarek Radosz
cf44cdb082
FIX: Validate category name on parent change (#10815)
Previously, moving a category into another one, that already had a child category of that name (but with a non-conflicting slug) would cause a 500 error:

```
# PG::UniqueViolation:
#   ERROR:  duplicate key value violates unique constraint "unique_index_categories_on_name"
#   DETAIL:  Key (COALESCE(parent_category_id, '-1'::integer), name)=(5662, Amazing Category 0) already exists.
```

It now returns 422, and shows the same message as when you're renaming a category: "Category Name has already been taken".
2020-10-05 11:50:05 +02:00
Martin Brennan
39b2fb8649
FIX: Invalid URLs could raise exceptions when calling UrlHelper.rails_route_from_url (#10782)
Upload.secure_media_url? raised an exceptions when the URL was invalid,
which was a issue in some situations where secure media URLs must be
removed.

For example, sending digests used PrettyText.strip_secure_media,
which used Upload.secure_media_url? to replace secure media with
placeholders. If the URL was invalid, then an exception would be raised
and left unhandled.

Now instead in UrlHelper.rails_route_from_url we return nil if there is something wrong with the URL.

Co-authored-by: Bianca Nenciu <nenciu.bianca@gmail.com>
2020-09-30 15:20:00 +10:00
Arpit Jalan
025490ecb9
FEATURE: composer option to reload page and force save draft (#10773) 2020-09-29 22:29:03 +05:30
David Taylor
1ba9b34b03
DEV: Move UserApiKey scopes to dedicated table (#10704)
This has no functional impact yet, but it is the first step in adding more granular scopes to UserApiKeys
2020-09-29 10:57:48 +01:00
Krzysztof Kotlarek
e7c72cd1e4
FIX: deprecate whitelist constants (#10716)
Deprecation of:
WHITELISTED_REDIRECT_HOSTNAMES
CUSTOM_INTERPOLATION_KEYS_WHITELIST
WHITELISTED_SVG_ELEMENTS
2020-09-28 13:52:05 +10:00
Osama Sayegh
d745568633
DEV: Stop polluting Class with deprecated settings methods (#10736)
`self.class` here evaluates to `Class` and then we're calling `define_method` on it which means all classes will have those methods defined in them. For example:

```
~/discourse(master*) » rails c
Loading development environment (Rails 6.0.3.3)
[1] pry(main)> Integer.methods
=> [:sqrt,
 :yaml_tag,
 :email_domains_blacklist=,
 :email_domains_whitelist=,
 :unicode_username_character_whitelist=,
 :user_website_domains_whitelist=,
 :whitelisted_link_domains=,
 :email_domains_blacklist,
 :email_domains_whitelist,
 :unicode_username_character_whitelist,
...
...
```

Fix here is to use `self.define_singleton_method`.
2020-09-24 01:58:53 +03:00
Bianca Nenciu
eb891778ff
DEV: Do not translate font names (#10723)
Use the names as provided by discourse-fonts and remove the
translated strings.

It also ensures that the selected font is present in case a font will
be removed in the future.
2020-09-23 12:00:07 +03:00
Krzysztof Kotlarek
0bb51dcbfa
FIX: TL2 promotion message and advance training (#10679)
This is a little bit of refactoring. Core Discourse should have default promotion message for TL2.

In addition, when the Discobot plugin is enabled, the user is invited to advanced training
2020-09-22 10:17:52 +10:00
Krzysztof Kotlarek
c934a0f759
FIX: don't error Topic#similar_to when prepared raw is blank (#10711)
If raw contains incorrect URL, `prepare_data` returns empty string:

https://github.com/discourse/discourse/blob/master/lib/search.rb#L91

Therefore we should not only check if the cooked post is not blank but also if prepared data is not blank.
2020-09-22 07:53:12 +10:00
Daniel Waterworth
6e69c25134 FIX: Use asset_hostnames in rails_multisite
We want to respond to the x-forwarded-for host from the CDN too.
2020-09-17 10:32:43 +01:00
Guo Xiang Tan
b47b640598
FEATURE: Hidden SiteSetting.keep_old_ip_address_count to track IP history. 2020-09-17 12:50:39 +08:00
Martin Brennan
80268357e7
DEV: Change upload verified column to be integer (#10643)
Per review https://review.discourse.org/t/dev-add-verified-to-uploads-and-fill-in-s3-inventory-10406/14180

Change the verified column for Upload to a verified_status integer column, to avoid having NULL as a weird implicit status.
2020-09-17 13:35:29 +10:00
Guo Xiang Tan
49f53bdc50
DEV: Fix another MessageBus::InvalidMessageTarget error.
Follow-up to c2f2b57a9b.
2020-09-15 14:25:10 +08:00
Guo Xiang Tan
f7438fc64b
DEV: Fix MessageBus::InvalidMessageTarget error after upgrading mb.
Follow-up to c2f2b57a9b
2020-09-15 14:15:42 +08:00
Martin Brennan
5268568d23
FEATURE: Remove user topic timers and migrate to bookmarks with reminders (#10474)
This PR removes the user reminder topic timers, because that system has been supplanted and improved by bookmark reminders. The option is removed from the UI and all existing user reminder topic timers are migrated to bookmark reminders.

Migration does this:

* Get all topic_timers with status_type 5 (reminders)
* Gets all bookmarks where the user ID and topic ID match
* Loops through the found topic timers
  * If there is no bookmark for the OP of the topic, then we just create a bookmark with a reminder
  * If there is a bookmark for the OP of the topic and it does **not** have a reminder set, then just 
update it with the topic timer reminder
  * If there is a bookmark for the OP of the topic with a reminder then just discard the topic timer
* Cancels all outstanding user reminder topic timers
* **Trashes (not deletes) all user reminder topic timers**

Notes:

* For now I have left the user reminder topic timer job class in place; this is so the jobs can be cancelled in the migration. It and the specs will be deleted in the next PR.
* At a later date I will write a migration to delete all trashed user topic timers. They are not deleted here in case there are data issues and they need to be recovered.
* A future PR will change the UI of the topic timer modal to make it look more like the bookmark modal.
2020-09-14 11:11:55 +10:00
Guo Xiang Tan
543e972fec
PERF: Remove category_user lookup when loading private messages.
Private messages do not belong to categories so the query is unnecessary
overhead.
2020-09-11 16:26:38 +08:00
Vinoth Kannan
9e4ed03b8f FEATURE: moderators allowed to view groups which members can see.
Currently, if a group's visibility is set to "Group owners, members" then the mods can't view those group pages. The same rule is applied for members visibility setting too.

This reverts commit 7fc7090. And fixed the spec test fails.
2020-09-11 08:20:13 +05:30