Commit Graph

6797 Commits

Author SHA1 Message Date
Sam Saffron
b4bfc27b19 FEATURE: introduce default application level rate limiting by IP
We have tested rate limiting with admin accounts with block rate limiting for
close to 12 months now on meta.discourse.org.

This has resulted in no degradation of services even to admin accounts that
request a lot of info from the site.

The default of 200 requests a minute and 50 per 10 seconds is very generous.
It simply protects against very aggressive clients.

This setting can be disabled or tweaked using:

DISCOURSE_MAX_REQS_PER_IP_MODE and family.

The only big downside here is in cases when a very large number of users tend
to all come from a single IP.

This can be the case on sites accessing Discourse from an internal network
all sharing the same IP via NAT. Or a misconfigured Discourse that is unable
to resolve IP addresses of users due to proxy mis-configuration.
2019-11-18 15:54:50 +11:00
Penar Musaraj
102909edb3 FEATURE: Add support for secure media (#7888)
This PR introduces a new secure media setting. When enabled, it prevent unathorized access to media uploads (files of type image, video and audio). When the `login_required` setting is enabled, then all media uploads will be protected from unauthorized (anonymous) access. When `login_required`is disabled, only media in private messages will be protected from unauthorized access. 

A few notes: 

- the `prevent_anons_from_downloading_files` setting no longer applies to audio and video uploads
- the `secure_media` setting can only be enabled if S3 uploads are already enabled and configured
- upload records have a new column, `secure`, which is a boolean `true/false` of the upload's secure status
- when creating a public post with an upload that has already been uploaded and is marked as secure, the post creator will raise an error
- when enabling or disabling the setting on a site with existing uploads, the rake task `uploads:ensure_correct_acl` should be used to update all uploads' secure status and their ACL on S3
2019-11-18 11:25:42 +10:00
Vinoth Kannan
56b19ba740 UX: instead of total user count display only the count of users going to be affected.
edec922803
2019-11-18 00:09:38 +05:30
Blake Erickson
eb4d8a43e3 DEV: Improve mail-receiver update instructions 2019-11-15 13:57:43 -07:00
Martin Brennan
5c59247c3a Block enabling force 2FA if local logins disabled & vice-versa (#8355) 2019-11-15 17:05:10 +11:00
Gerhard Schlager
f72730703a Update translations 2019-11-14 15:41:42 +01:00
Martin Brennan
e7226a8c84
FEATURE: Allow scoping search to tag (#8345)
* When viewing a tag, the search widget will now show a checkbox to scope the search by tag, which will limit search results to that tag on desktop and mobile
2019-11-14 10:40:26 +10:00
tshenry
9caec39aef
UX: Add CSP documentation link to content_security_policy_script_src setting 2019-11-12 14:57:47 -08:00
Mark VanLandingham
3dd2f2f701
DEV: Remove RSS feed polling in favor of plugin (#8233) 2019-11-12 09:49:02 -06:00
David Taylor
5f927ceeb3
DEV: Display a warning when themes hard-code optimized image links (#8304) 2019-11-12 14:30:19 +00:00
Blake Erickson
15f6f57cdc DEV: Add update message for an outdated mail-receiver
Previous versions of the mail-receiver used query based api credentials,
if we detect this we will show a message in the admin panel to update
the mail receiver.
2019-11-08 17:32:22 -07:00
Martin Brennan
56d3e29a69
FIX: Badge and user title interaction fixes (#8282)
* Fix user title logic when badge name customized
* Fix an issue where a user's title was not considered a badge granted title when the user used a badge for their title and the badge name was customized. this affected the effectiveness of revoke_ungranted_titles! which only operates on badge_granted_titles.
* When a user's title is set now it is considered a badge_granted_title if the badge name OR the badge custom name from TranslationOverride is the same as the title
* When a user's badge is revoked we now also revoke their title if the user's title matches the badge name OR the badge custom name from TranslationOverride
* Add a user history log when the title is revoked to remove confusion about why titles are revoked
* Add granted_title_badge_id to user_profile, now when we set badge_granted_title on a user profile when updating a user's title based on a badge, we also remember which badge matched the title
* When badge name (or custom text) changes update titles of users in a background job
* When the name of a badge changes, or in the case of system badges when their custom translation text changes, then we need to update the title of all corresponding users who have a badge_granted_title and matching granted_title_badge_id. In the case of system badges we need to first get the proper badge ID based on the translation key e.g. badges.regular.name
* Add migration to backfill all granted_title_badge_ids for both normal badge name titles and titles using custom badge text.
2019-11-08 15:34:24 +10:00
Vinoth Kannan
ba5b78a348
FEATURE: support to mute all categories by default. (#8295)
Instead of enabling `suppress_from_latest` setting on many categories now we can enable `mute_all_categories_by_default` site setting. Then users should opt-in to categories for them to appear in the latest and categories pages.
2019-11-08 08:28:11 +05:30
Blake Erickson
2db2003187 DEV: Add deprecation warning of non-header based API auth
This change adds a message to the admin panel if it detects an api
requests that doesn't use the new header based authentication method.

The message is to warn people to switch to header based auth and links
to the api documention topic on meta for more info.
2019-11-07 17:06:13 -07:00
Arpit Jalan
6a417c308f FIX: include onebox default options in development environment 2019-11-07 15:42:53 +05:30
Vinoth Kannan
5d42f052bc Minor copy edit
671f303b53
2019-11-07 11:39:17 +05:30
Sam Saffron
1d1dd2a4d4 PERF: cache static assets in NGINX for longer
Previously our cache would expire any asset that was not accessed for 10
minutes. This is way too short and was never intended. All the assets we
are serving are usually very long living assets like avatars and css files

1 day is a reasonable setting here cause it offers far better protection.
I would consider upping this to a week though longer term.

Maximum disk space of cache was increased as well to 600m. Very unlikely to
ever hit this except on very large sites.

Additionally, this places all the cached assets in nested directories, we
never want cached files to be in one giant directory cause it is inefficient
2019-11-07 12:12:24 +11:00
Gerhard Schlager
ee5b56914c FIX: Use long version of month names in date formats
Otherwise we'd need to add `date.abbr_month_names` to `server.en.yml`
2019-11-06 22:30:51 +01:00
Neil Lalonde
b151963f18 Update translations 2019-11-06 10:43:13 -05:00
Gerhard Schlager
61b1f9c36b FEATURE: Load translation overrides without JS eval 2019-11-05 19:16:38 +01:00
Gerhard Schlager
94a34af702 Update translations 2019-11-05 16:52:48 +01:00
David Taylor
52c5cf33f8
FEATURE: Overhaul of admin API key system (#8284)
- Allow revoking keys without deleting them
- Auto-revoke keys after a period of no use (default 6 months)
- Allow multiple keys per user
- Allow attaching a description to each key, for easier auditing
- Log changes to keys in the staff action log
- Move all key management to one place, and improve the UI
2019-11-05 14:10:23 +00:00
Daniel Waterworth
fa2c06da93 DEV: Change tag routing for categories on the server-side
These are the changes to tags routing that correspond to the category
routing changes of d84c34ad. The new scheme is:

/tags/c/*slug_path/:id/none/:tag_id/ENDPOINT
/tags/c/*slug_path/:id/none/:tag_id
/tags/c/*slug_path/:id/:tag_id/ENDPOINT
/tags/c/*slug_path/:id/:tag_id
2019-11-05 13:24:36 +00:00
Vinoth Kannan
671f303b53
FEATURE: Add welcome message for admins. (#8293) 2019-11-05 18:15:55 +05:30
Daniel Waterworth
f3f04f1643 REFACTOR: Handle no_subcategories option during routing 2019-11-04 10:58:06 +00:00
Arpit Jalan
b7327d2c34 UX: show user email address on "grant admin access" email and UI 2019-11-04 14:47:00 +05:30
Mark VanLandingham
66e0bdc053
FEATURE: Create New Topic button on embed with params (#8280)
* FEATURE: Create New Topic button on embed with params
2019-11-01 14:19:10 -05:00
Vinoth Kannan
72aa26c8c5
FEATURE: New site settings for default tags in user preferences. (#8283) 2019-11-01 12:40:13 +05:30
Neil Lalonde
d777844ed6 FEATURE: categories can require topics have a tag from a tag group
In a category's settings, the Tags tab has two new fields to
specify the number of tags that must be added to a topic
from a tag group. When creating a new topic, an error will be
shown to the user if the requirement isn't met.
2019-10-31 16:10:19 -04:00
Neil Lalonde
091d058ff2 UX: widen tag group name input field and add placeholder text 2019-10-31 16:10:19 -04:00
Daniel Waterworth
d84c34ad75 DEV: Server-side category routing changes
The routes for categories are changing. The scheme that I intend to move
us to is:

/c/*slug_path/(:id)/ENDPOINT
/c/*slug_path/(:id)

This commit adds support for the new scheme to the server side without
dropping support for existing URLs. It is necessary to support existing
URLs for two reasons:

 * This commit does not change any client side routing code,
 * Posts that contain category hashtags that refer to a root category
   are baked into URLs that do not fit this new scheme, (/c/[id]-[slug])
2019-10-31 18:06:22 +00:00
Kris
4434f419c6 Copy edit: rename profile header to profile background 2019-10-31 13:06:59 -04:00
Sam Saffron
c5e67726fd FIX: under some conditions draft would say it was saving when not
This is a major change to draft internals. Previously there were quite a
few cases where the draft system would say "draft saved", when in fact
we just skipped saving.

This commit ensures the draft system deals with draft ownership handover in
a predictable way.

For example:

- Window 1 editing draft
- Window 2 editing same draft at the same time

Previously we would allow window 1 and 2 to just fight on the same draft
each window overwriting the same draft over an over.

This commit introduces an ownership concept where either window 1 or 2 win
and user is prompted on the loser window to reload screen to correct the issue

This also corrects edge cases where a user could have multiple browser windows
open and posts in 1 window, later to post in the second window. Previously
drafts would break in the second window, this corrects it.
2019-10-31 17:15:58 +11:00
Penar Musaraj
580a4a827b Exclude audio/video URLs from search result blurbs
Displays translatable "[audio]" or "[video]" placeholders instead of ugly (and often long) URLs.
2019-10-30 13:07:16 -04:00
Penar Musaraj
965662d215 Prettify server.en.yml 2019-10-30 12:56:25 -04:00
Jarek Radosz
080e899b8c
DEV: Tag group improvements (#8252)
* DEV: Add the actual "tag_groups/new" route

Allows refreshing the "new" page without an error.

* DEV: Prevent attempts to create group tags if tagging is disabled

* DEV: Refactor the tag-groups controller

Gets rid of `selectedItem`, `selected`, and `selectTagGroup` action.

* DEV: Rename tag-groups-show to tag-groups-edit

* DEV: Refactor tag-groups form

* Extracted the tag-groups-form that's used by tag-groups-new and tag-groups-edit
* The model is now a buffered property
* Serialization relies more heavily on RestAdapter now
* Data is sent as JSON
* Payload is now namespaced ("tag_group")

* Update app/assets/javascripts/discourse/controllers/tag-groups-new.js.es6

Co-Authored-By: Joffrey JAFFEUX <j.jaffeux@gmail.com>

* Update app/assets/javascripts/discourse/components/tag-groups-form.js.es6

Co-Authored-By: Joffrey JAFFEUX <j.jaffeux@gmail.com>

* Update app/assets/javascripts/discourse/controllers/tag-groups-edit.js.es6

Co-Authored-By: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2019-10-30 16:57:13 +01:00
Krzysztof Kotlarek
c32bd8ae48 FEATURE: Remove attachments and truncate raw field for incoming emails (#8253)
Adds the settings: 

raw_email_max_length, raw_rejected_email_max_length, delete_rejected_email_after_days. 

These settings control retention of the "raw" emails logs.

raw_email_max_length ensures that if we get incoming email that is huge we will truncate it removing uploads from the raw log.

raw_rejected_email_max_length introduces an even more aggressive truncation for rejected incoming mail. 

delete_rejected_email_after_days controls how many days we will keep rejected emails for (default 90)
2019-10-30 16:54:35 +11:00
Jeff Atwood
6d381b33a4 copyedits on moderator welcome PM 2019-10-28 17:19:42 -07:00
Jeff Wong
64d51ab45e DEV: add UNICORN_BIND_ALL
listen on local interface by default.
2019-10-28 14:11:19 -07:00
Mark VanLandingham
4eb54f08b2
FEATURE: Site setting/UI to allow users to set their primary group (#8244)
* FEATURE: Site setting/ui to allow users to set their primary group

* prettier and remove logic from account template

* added 1 to 43 to make web_hook_user_serializer_spec pass
2019-10-28 12:46:27 -05:00
Leo McArdle
e7ff6809a3 FEATURE: add SES spam header to recognised spam headers (#8254) 2019-10-28 12:46:53 -04:00
Jarek Radosz
5776251cdd
DEV: Remove an obsolete "preferences/about-me" route (#8251)
User's title and bio can be changed on the "preferences/account" page.
2019-10-28 17:29:14 +01:00
Mark VanLandingham
437edfc415
FEATURE: Welcome moderator message - add copy! (#8246) 2019-10-28 08:58:45 -05:00
tshenry
5d1b34e1b9
Minor Copy Change
Context: https://meta.discourse.org/t/email-with-correct-incoming-email-address-in-bcc-is-being-rejected/131003/2
2019-10-24 11:16:31 -07:00
Neil Lalonde
f061aee818 FEATURE: support SCSS in custom email style
In the CSS tab of Admin > Customize > Email Style, SCSS can now be used.
2019-10-23 15:42:37 -04:00
Vinoth Kannan
31577b2131 FEATURE: wiki editors are allowed edit tags for wiki topics.
If a wiki editor's TL is greater than 'min trust level to tag topics' site setting then they can edit the tags for any wiki topic.
2019-10-23 23:35:38 +05:30
Sam Saffron
98d6cee7c7 FIX: various fixes to draft system
- destroyDraft which is called when we cancel a draft is now async,
  removing race conditions when you click "reply" to a post and are
  already editing. We used to trigger double dialogs for cancelling
  drafts which was confusing.

- Remove reply as new topic / reply as pm keys, they are no longer
  used and only caused confustion. For example we used to pop up a
  warning when you are composing a reply and flick to reply as
  new topic

- Remove createTopic key, this was a bug that proliferated. Whenever
  creating a topic via the C shortcut or clicking on new topic on full
  screen search the correct new topic draft key will be used
  consistently

- When abandoning an edit we now say "Are you sure you want to discard
  your changes" (instead of abandon your post which is confusing)
2019-10-21 17:24:06 +11:00
Krzysztof Kotlarek
858cf5836c
FIX: update Redis gem to version 4.1.3
I run our benchmark on commit with hiredis and redis-4.1.3

Results:
type | hidredis | redis 4.1.3 | percent
--- | --- | --- | ---
Categories-50 | 49 | 50 | 102.04%
Categories-75 | 51 | 51 | 100.00%
Categories-90 | 63 | 64 | 101.59%
Categories-99 | 86 | 85 | 98.84%
Home-50 | 55 | 55 | 100.00%
Home-75 | 56 | 57 | 101.79%
Home-90 | 68 | 69 | 101.47%
Home-99 | 102 | 104 | 101.96%
Topic-50 | 36 | 37 | 102.78%
Topic-75 | 37 | 37 | 100.00%
Topic-90 | 47 | 48 | 102.13%
Topic-99 | 60 | 61 | 101.67%
Categories-admin-50 | 124 | 117 | 94.35%
Categories-admin-75 | 130 | 129 | 99.23%
Categories-admin-90 | 147 | 143 | 97.28%
Categories-admin-99 | 204 | 199 | 97.55%
Home-admin-50 | 146 | 148 | 101.37%
Home-admin-75 | 150 | 152 | 101.33%
Home-admin-90 | 169 | 168 | 99.41%
Home-admin-99 | 232 | 223 | 96.12%
Topic-admin-50 | 60 | 61 | 101.67%
Topic-admin-75 | 64 | 63 | 98.44%
Topic-admin-90 | 76 | 73 | 96.05%
Topic-admin-99 | 124 | 94 | 75.81%
Load rails | 2412 | 2360 | 97.84%
rss | 290204 | 295828 | 101.94%
pss | 277948 | 283624 | 102.04%

Redis gem is manipulating Redis config https://github.com/redis/redis-rb/blob/master/lib/redis/client.rb#L95
therefore we cannot pass the frozen config object.

Pass of the copy of the object is protecting original config
2019-10-21 09:59:24 +11:00
tshenry
95114f87ae
Improve spam_hosts copy (#8203)
- Increase size of the reviewable's conversation excerpt to prevent truncation of the new copy
- Remove the `domain` parameter from the `flag_linked_posts_as_spam` method in the user model since it is no longer needed
- Remove the `domain` interpolation variable from all translation files
- Add "All posts from this user that include links should be reviewed." to server.en.yml for added clarity on why the posts entered the queue
2019-10-18 09:31:15 -07:00
Blake Erickson
80d8830c17 DEV: Make max_tags_in_filter_list available clientside
This will allow plugins that use this setting to read the value using
javascript.
2019-10-17 15:22:27 -06:00