IntenseWebs/discourse
3
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-11-30 04:34:13 -06:00
Code Issues Packages Projects Releases Wiki Activity
34,455 Commits 222 Branches 500 Tags 903 MiB
ed1e5ef6cc
Commit Graph

861 Commits

Author SHA1 Message Date
Gerhard Schlager
c29a4dddc1 FEATURE: Support backup uploads/downloads directly to/from S3. 2018-10-11 10:38:43 +08:00
David Taylor
9bf522f227
FEATURE: Mixed case tagging (#6454) 
- By default, behaviour is not changed: tags are made lowercase upon creation and edit.

- If force_lowercase_tags is disabled, then mixed case tags are allowed.

- Tags must remain case-insensitively unique. This is enforced by ActiveRecord and Postgres.

- A migration is added to provide a `UNIQUE` index on `lower(name)`. Migration includes a safety to correct any current tags that do not meet the criteria.

- A `where_name` scope is added to `models/tag.rb`, to allow easy case-insensitive lookups. This is used instead of `Tag.where(name: "blah")`.

- URLs remain lowercase. Mixed case URLs are functional, but have the lowercase equivalent as the canonical.
 2018-10-05 10:23:52 +01:00
Penar Musaraj
9e008047db reset before running docker tests 2018-10-03 10:32:16 -04:00
Neil Lalonde
8af6d81891 FIX: improved category merging in discourse_merger. Use full paths to look for uniqueness instead of category names. 2018-09-20 12:33:58 -04:00
Neil Lalonde
b9891c2641 FIX: error because last_id is nil in discourse_merger script 2018-09-17 09:57:11 -04:00
David Taylor
26bd67a865 DEV: Add travis_fold statements to docker_test 2018-09-12 17:52:58 +01:00
Guo Xiang Tan
71185c13b5
Merge pull request #6377 from tgxworld/remove_tif_tiff 
Drop `tif`, `tiff`, `webp` and `bmp` from supported images.
 2018-09-12 09:32:32 +08:00
Guo Xiang Tan
e1b16e445e Rename FileHelper.is_image? -> FileHelper.is_supported_image?. 2018-09-12 09:22:28 +08:00
Carsten Brandt
921e2213b8 FEATURE: Updated IPB import script 
* IPB import script replace PHP code tags with proper markdown

remove excess newlines in code blocks
decode HTML entities in code blocks
add replacement for list items
proper handling of attachments that are not images
fix typo
improved quote handling
fix code style complaint from travis-ci build
 2018-09-12 11:12:28 +10:00
Neil Lalonde
4653627a40 update plugin-translations.rb script to update .tx/config file in plugins when languages are added or removed 2018-09-10 14:22:45 -04:00
Guo Xiang Tan
434035f167 FIX: Link post to uploads in PostCreator. 
* This ensures that uploads are linked to their post on creation
  instead of a background job which may be delayed if Sidekiq
  is facing difficulties.
 2018-09-06 11:18:11 +08:00
Gerhard Schlager
44922b0c25 zh_TW isn't broken anymore 2018-09-05 00:47:39 +02:00
Guo Xiang Tan
8dc1463ab3 Enable Lint/ShadowingOuterLocalVariable for Rubocop. 2018-09-04 10:16:42 +08:00
Neil Lalonde
15f657309a FEATURE: Zendesk importer that uses its API to get data 2018-08-28 10:21:39 -04:00
Neil Lalonde
30722240e4 add discourse-checklist to plugin-translations.rb 2018-08-23 10:00:27 -04:00
Gerhard Schlager
ac743dab10 Improve mbox import script 
* emails weren't sorted in correct order
* better default regex for splitting mbox files
* output Message-ID if email is skipped because it doesn't have a Date
 2018-08-23 09:46:28 +02:00
Neil Lalonde
3fddbb603c omit zh_TW which breaks the build 2018-08-21 11:17:42 -04:00
Neil Lalonde
0ada6b81c2 DEV: add a way to skip locales with problems that break Discourse and need to be fixed in Transifex 2018-08-21 10:36:48 -04:00
Arpit Jalan
7af0da9498 Fix Vanilla bulk import script 2018-08-16 22:12:26 +05:30
Arpit Jalan
0e04e3990e Improve Vanilla bulk import script 2018-08-16 22:00:26 +05:30
Neil Lalonde
ac3b0f0164 REFACTOR: move remap out of script into a class 2018-08-15 12:37:52 -04:00
Gerhard Schlager
7f4ef3db9e Improve Telligent importer 
* Try multiple filenames and do lots of guessing when searching for attachments
* Unescape HTML in filenames and replace invalid characters in filenames
* Existing permalinks prevented resuming of import
* Prevent duplicate attachments in same post
 2018-08-13 15:28:11 +02:00
Gerhard Schlager
8513605421 Fix the import of avatars and attachments 
This time for real ;-)
 2018-08-12 22:26:07 +02:00
Gerhard Schlager
6d813c2b52 FIX: Importers failed to import avatars 2018-08-12 22:02:17 +02:00
Gerhard Schlager
1794aea939 FEATURE: Add import script for Telligent 2018-08-12 22:01:23 +02:00
Neil Lalonde
f7f24a5399 FIX: discourse_merger: skip collisions on join models when both objects were merged 2018-08-02 16:05:55 -04:00
Mohammad AlTawil
64f533db99 Add display name to user (#6198) 2018-07-31 14:43:16 +10:00
Sam
e4208113a8 improve report and add regular logging 2018-07-27 16:22:14 +10:00
Sam
5e262265a2 update script to provide more mem stats 2018-07-27 12:51:23 +10:00
Godfrey Chan
5affdcbd59 Bump Ruby version in some docs 2018-07-25 14:38:10 -07:00
Vinoth Kannan
1390eb2957 Disable bootstrap mode before start importing 2018-07-25 12:12:26 +05:30
Sam
f0a23d50b4 DEV: add script for testing memory usage in sidekiq 2018-07-24 17:57:02 +10:00
Neil Lalonde
bf7ebecb76 FIX: discourse_merger: many foreign keys were not being updated 2018-07-22 22:05:07 -04:00
Neil Lalonde
4e09206061 FIX: set uploads sequence after copying uplaods in discourse_merger 2018-07-19 11:07:15 -04:00
Régis Hanol
e8e9b5cea4 FIX: clean URLs in SMF1 importer 2018-07-19 13:17:43 +02:00
Régis Hanol
63e5349209 FIX: [img] BBCode tags might have parameters 2018-07-19 13:11:01 +02:00
Régis Hanol
5434cf02a3 FIX: smf1 importer was swallowing some data 2018-07-19 10:29:54 +02:00
Neil Lalonde
def2653fc8 FIX: discourse_merger: copied topic_link records had wrong url, and update all internal links to use new topic URLs in copied posts 2018-07-18 16:45:48 -04:00
Neil Lalonde
24da2940a7 FIX: copy uploads quickly in discourse_merger.rb, and fix user avatar upload id for copied users 2018-07-18 16:42:59 -04:00
Neil Lalonde
dbfa491ee2 FIX: avatars in discourse_merger.rb 2018-07-17 21:40:24 -04:00
Neil Lalonde
f146f94ef6 FIX: errors when copying post_uploads in discourse_merger.rb 2018-07-17 16:47:23 -04:00
Neil Lalonde
04077a7df6 WIP: a fast method of copying uploads in discourse_merger.rb. not working yet. 2018-07-17 16:46:32 -04:00
Neil Lalonde
2786c79354 another check to avoid unique index error in discourse_merger.rb 2018-07-16 13:34:41 -04:00
Neil Lalonde
8d11df6971 FIX: support amazon S3 upload urls in discourse_merger.rb 2018-07-13 16:10:31 -04:00
Neil Lalonde
71814009bd FIX: badges for merged users don't get merged by discourse_merger.rb 2018-07-12 17:43:21 -04:00
Neil Lalonde
cba292cb56 FIX: personal messages not being copied by discourse_merger.rb 2018-07-12 17:41:16 -04:00
Régis Hanol
c818550172 Support custom avatar in SMF1 importer 2018-07-12 17:38:07 +02:00
Régis Hanol
5c4534d895 Update SMF1 import 
- Properly import avatar when they use an external image
- Don't import the same attachment twice
 2018-07-12 16:55:30 +02:00
Neil Lalonde
c33ee13c4c FIX: discourse_merger halts when topic has nil category 2018-06-29 12:21:25 -04:00
Sam
f4f95ce956 correct linting 2018-06-29 16:04:38 +10:00
David Lee
8f43872bff Add Question2Answer import script 2018-06-29 15:48:01 +10:00
Arpit Jalan
c73f98c289 FIX: invert from and to user id in smf1 import script 2018-06-28 12:30:28 +05:30
Gerhard Schlager
fb022098f6 Base importer: Calculate category colors depending on parent category 2018-06-27 20:27:11 +02:00
Vinoth Kannan
652b32484f Assign default value for message template matches 2018-06-26 05:16:03 +05:30
Vinoth Kannan
f3011c709b Extract html content from lithium message template 2018-06-26 05:07:32 +05:30
Neil Lalonde
a1c0d0e6e5 fixes to discourse_merger: failures for Uploads, UserBadges, PostUploads hack 2018-06-21 12:16:05 -04:00
Neil Lalonde
b9cb97df7f add support for badges in discourse_merger 2018-06-19 15:11:48 -04:00
Neil Lalonde
dbcbd8d939 close connections in discourse_merger 2018-06-19 10:34:05 -04:00
Sam
5f64fd0a21 DEV: remove exec_sql and replace with mini_sql 
Introduce new patterns for direct sql that are safe and fast.

MiniSql is not prone to memory bloat that can happen with direct PG usage.
It also has an extremely fast materializer and very a convenient API

- DB.exec(sql, *params) => runs sql returns row count
- DB.query(sql, *params) => runs sql returns usable objects (not a hash)
- DB.query_hash(sql, *params) => runs sql returns an array of hashes
- DB.query_single(sql, *params) => runs sql and returns a flat one dimensional array
- DB.build(sql) => returns a sql builder

See more at: https://github.com/discourse/mini_sql
 2018-06-19 16:13:36 +10:00
OsamaSayegh
91b73e0c2d FIX: remap shouldn't fail silently when an error occurs 2018-06-19 14:49:43 +10:00
Vinoth Kannan
4ffa4a28b0 FIX: duplicate_emails get overridden in new batch of import 2018-06-19 00:21:48 +05:30
Vinoth Kannan
750367007c REFACTOR: Import user visits from lithium database 2018-06-18 20:38:57 +05:30
Vinoth Kannan
ef4a86456b Add attachment folder name in prefix for lithium import 2018-06-18 18:29:14 +05:30
Gerhard Schlager
3f167ae5ce Use short upload URL in import scripts 2018-06-17 22:57:32 +02:00
Gerhard Schlager
88ca838e02 Create avatar from file in base importer 2018-06-17 22:57:31 +02:00
Gerhard Schlager
84d9b2e473 Use correct post id in zendesk importer 2018-06-17 22:57:31 +02:00
Vinoth Kannan
2a0f409b9d Use lowercased email addresses to check duplicates 2018-06-16 20:34:37 +05:30
Vinoth Kannan
ac44374a59 Import user visits from user_log table 2018-06-16 19:10:55 +05:30
Neil Lalonde
20ceadffaf FEATURE: script to merge two discourse sites 2018-06-15 17:13:36 -04:00
Sam
c56bd2ac16 add memory analysis script 2018-06-14 12:18:36 +10:00
discoursehosting
fc973f9363 Improve the VBulletin importer (#5922) 2018-06-12 20:41:21 +02:00
Neil Lalonde
1ba8e8948d FIX: add support for string avatar_type values in PHPBB3 importer 2018-06-07 18:14:11 -04:00
Arpit Jalan
b4e0cddcc9 disable all outgoing emails in base importer 2018-06-07 22:49:38 +05:30
Vinoth Kannan
620a1524cb Use plus addressing email address for duplicates 2018-06-07 19:11:55 +05:30
Arpit Jalan
f9ab3848ed FEATURE: support disabling emails for non-staff users 2018-06-07 18:31:08 +05:30
Guo Xiang Tan
ad5082d969 Make rubocop happy again. 2018-06-07 13:28:18 +08:00
Régis Hanol
127398c68e FIX: import comments of 1st post in SE importer 2018-06-05 18:22:42 +02:00
Régis Hanol
685083491e FEATURE: StackOverflow importer 2018-06-04 16:57:12 +02:00
Régis Hanol
6862194255 extract configuration variables from SMF1 importer 2018-05-30 15:53:57 +02:00
Gerhard Schlager
bf30f74f60 Pulling translations for a new language didn't work 2018-05-29 20:57:32 +02:00
Gerhard Schlager
bdeae17d32 Automatically create locale.js.erb file when adding new locale 2018-05-29 12:58:31 +02:00
Régis Hanol
aeb511e8ff FEATURE: SMF1 importer 2018-05-28 11:02:19 +02:00
Gerhard Schlager
2f0e230dba Adds import script for Zendesk 
It also adds a generic SQLite database that can be used when the data needs some transformation before the actual import.
 2018-05-22 21:55:54 +02:00
Gerhard Schlager
eceeef8413 Imported categories use colors from settings instead of brown 2018-05-22 21:55:54 +02:00
Vinoth Kannan
bb12fa3fdc Migrate user mentions in lithium import 2018-05-21 18:19:22 +05:30
Vinoth Kannan
b229c112f6 FIX: variable name typo 2018-05-21 13:47:30 +05:30
Vinoth Kannan
09151190f9 FIX: Use avatar_dir to import user avatars 2018-05-21 13:43:23 +05:30
Vinoth Kannan
c9c3a83261 Importing lithium post images and attachments 2018-05-21 13:34:52 +05:30
Vinoth Kannan
f3385a74cb Importing lithium topic tags 2018-05-19 11:24:48 +05:30
Vinoth Kannan
ba0dd5889d Improvements in importing the lithium pms 2018-05-18 22:57:15 +05:30
Vinoth Kannan
9f92fdded0 Improvements in lithium topic and post import 2018-05-18 18:53:18 +05:30
Vinoth Kannan
9d4d6276b7 Import user profile fields and avatars 2018-05-18 17:11:20 +05:30
Orlando Del Aguila
3287f47f07 nodebb importer (#5842) 2018-05-18 10:12:40 +02:00
Vinoth Kannan
0639b902dc Import lithium nodes into categories 2018-05-18 01:34:31 +05:30
Vinoth Kannan
ade7c94b43 Import lithium roles into groups 2018-05-16 17:26:15 +05:30
Gerhard Schlager
be891e1c2e docker.rake should pull the latest source 2018-05-15 15:09:49 +02:00
Gerhard Schlager
88217ca0b6 Configure Transifex for local-dates plugin 2018-05-04 10:13:15 +02:00
Joffrey JAFFEUX
78435833a5
FEATURE: discourse-cronos is now a core plugin 2018-05-03 20:15:57 +02:00
Sam
8d0c073d39 also watch LD_PRELOAD in bench 2018-05-03 15:50:45 +10:00
Gerhard Schlager
ef80341806 Update pull_translations script to work with latest Transifex changes 
* supports resources created with Transifex's YML handler version 3
* uses translations-manager gem
* makes sure that the locales supported by translations-manager are not out of sync
* update the lang_map in tx client config before pulling translations
 2018-04-27 19:20:19 +02:00
Régis Hanol
c11e8c9a64 add support for importing attachments in bbPress importer 2018-04-24 11:08:12 -04:00
Neil Lalonde
bbd2120b04 give some feedback that something is happening in downsize_uploads 2018-04-20 16:59:52 -04:00
Neil Lalonde
6e8bf1271f increase purge site settings for imports in case jobs like clean_up_uploads cause a mess 2018-04-09 12:44:59 -04:00
Gerhard Schlager
8412a96acb Do not create empty files when pulling from Transifex 2018-04-06 14:47:25 +02:00
Gerhard Schlager
09c4491f73 Add remaining core plugins to Transifex 2018-04-06 14:15:47 +02:00
Neil Lalonde
17f9c5494d add discourse-chronos to plugin-translations.rb 2018-04-05 16:41:29 -04:00
Gerhard Schlager
f2d00e5eff FEATURE: Use Message-ID for detecting email replies to group 
Ignores the site setting "find_related_post_with_key" and always tries to honor the `In-Reply-To` and `References` header for emails sent to a group.

The senders email address must be included in the `To` or `CC` header of a previous email sent to the group and the `Message-ID` of that email must be included in the current email's `In-Reply-To` or `References` header.
 2018-04-05 11:00:38 +02:00
Miro Michalicka
8d283e78e7 Fix broken queries in Drupal 6 migration. 2018-03-31 17:53:40 +02:00
Gerhard Schlager
cf9965fd54 Add rake task for reordering posts by creation_date 2018-03-29 14:42:41 +02:00
Gerhard Schlager
37fa843efc Allow pulling of Urdu translations from Transifex again 
The translations have been fixed.
 2018-03-27 11:56:40 +02:00
Neil Lalonde
baba45a369 ur translations still break the build, so stop pulling them from Transifex for now 2018-03-26 11:03:14 -04:00
Gerhard Schlager
a0d5e243fd Improve the import script for "Get Satisfaction" 
* Works with the current column layout exported as Excel file
* Tries to fix invalid CSV when it wasn't exported from Excel
* Imports categories
* Imports topics into the correct category
* Allows skipping archived topics
* Allows skipping private topics
* Makes use of the latest features from the base importer
* Some minor fixes and documentation updates
 2018-03-19 14:58:17 +01:00
Régis Hanol
dad1e4b17d don't import data only available in a plugin in phpBB 3 importer 2018-03-17 00:36:12 +01:00
Gerhard Schlager
2834705bd1 Improve base import script 
* Make sure the category description is imported correctly
  (the about topic usually had the wrong excerpt).
* Allow import scripts to mark topics as closed or archived.
* Allow import scripts to store the topic's original id.
  It will be stored in topic_custom_fields as import_topic_id.
 2018-03-16 22:39:06 +01:00
Régis Hanol
bb6ac51607 Update PHPBB3 importer 
- Extract name from profile_fields_data table
- Use 'posts.poster_ip' to track topic views
 2018-03-16 18:32:27 +01:00
Sam
cb7f3c6537 partial upload support for scss 2018-03-14 13:01:52 +11:00
Gerhard Schlager
9b651adadb FIX: mbox importer should ignore emails without date 2018-03-13 13:42:57 +01:00
Sam
8ad7f8b44f updates to theme watcher 2018-03-13 17:39:53 +11:00
Sam
8229cc2ff1 shuffle vars forward 2018-03-12 18:40:14 +11:00
Sam
758b9a7dda FEATURE: prototype of local theme directory watcher 
(note this will be documented a bit late)
 2018-03-12 18:36:06 +11:00
Neil Lalonde
7aa93b84c1 FIX: bulk importers shouldn't insert rows with id less than 1 2018-03-09 14:26:18 -05:00
Neil Lalonde
200c6673f1 FIX: bulk importers wiping all email addresses without warning or errors 2018-03-08 23:36:39 -05:00
Sam
ddcca1d5cc lint 2018-03-08 15:16:12 +11:00
neerajmalve
4b8ff7a057 Update xenforo.rb (#5622) 
#Fix for the error: xenforo.rb: 160: in `gsub!': invalid byte sequence in UTF-8 (ArgumentError)
 2018-03-07 22:57:38 -05:00
Régis Hanol
07bc52dc69 import topic views in phpbb3 2018-03-07 19:27:03 +01:00
Neil Lalonde
1093dacc03 FIX: bulk importers need to create category description topics 2018-03-07 12:10:22 -05:00
Gerhard Schlager
dc32ee5cbf Improvements to mbox import script 
* Ignore errors during indexing and show information about the message causing the problem
* Always activate imported users if they aren't staged
 2018-03-06 11:32:12 +01:00
Régis Hanol
49400337b6 Improve IPBoard3 importer 
- fix encoding issues
- close topics
- don't import queued & not approved topics
- don't import deleted posts
 2018-03-05 10:24:42 +01:00
Neil Lalonde
d7bd69583f fix vanilla bulk importer resume categories and private topics 2018-03-01 15:29:38 -05:00
Régis Hanol
ae9b1e8554 FEATURE: IP.Board 3 importer 2018-03-01 09:43:32 +01:00
Neil Lalonde
e997cc7b77 add more image extensions to downsize_uploads.rb 2018-02-27 22:05:35 -05:00
Neil Lalonde
931caa4335 add a script that can downsize uploads 2018-02-27 16:23:48 -05:00
Régis Hanol
82e68670bf FIX: SimplePress importer wasn't handling increment imports properly 2018-02-24 00:42:17 +01:00
Sam
720e1965e3 FEATURE: add category suppress from latest 
In the past we used suppress_from_homepage, it had mixed semantics
it would remove from category list if category list was on home and
unconditionally remove from latest.

New setting explicitly only removes from latest list but leaves the
category list alond
 2018-02-22 09:56:35 +11:00
Sam
73a492f721 minor changes to discourse bench 
Ruby master is not compatible with bootsnap atm
 2018-02-20 14:41:21 +11:00
Sam
a3c7ee09b6 FIX: ruby bench not working properly 
- Remove thin which is no longer supported
- Bypass admin api rate limiting in profile environment
- Admin password was too short
- Run by default in concurrency 1 mode
- A skip bundle assets flag to speed up local testing
 2018-02-19 11:37:16 +11:00
Sam
5faffeeb82 demo script for profiling middleware stack 2018-02-15 14:36:40 +11:00
Sam
c89b42c488 PERF: only require the rss library if used 
Before:

Total allocated: 257909321 bytes (2514134 objects)
Total retained:  39681579 bytes (343387 objects)

allocated memory by gem
-----------------------------------
  42875979  rss

retained memory by gem
-----------------------------------
   2080188  rss

retained objects by gem
-----------------------------------
     13052  rss

After:

Total allocated: 210562047 bytes (2252030 objects)
Total retained:  37433816 bytes (328635 objects)

----

So, 2 less megabytes on boot and 13000 objects stuck in ruby heaps forever.
 2018-02-15 13:11:33 +11:00
Gerhard Schlager
479f7ed18f Ignore case when removing mailing list name from subject 2018-02-12 21:41:58 +01:00
Régis Hanol
31f5ca6180 improve simple press importer 2018-02-07 17:37:51 +01:00
Neil Lalonde
0edd386b48 FEATURE: Vanilla bulk importer 2018-02-02 16:28:51 -05:00
Arpit Jalan
25ec077eca rename 'min_private_message_{post/title}_length' to 'min_personal_message_{post/title}_length' 2018-02-01 13:25:29 +05:30
Neil Lalonde
421713f1be include saved-searches in plugin-translations.rb 2018-01-31 11:19:21 -05:00
Neil Lalonde
cc830f7119 sorry rubocop 2018-01-30 14:37:15 -05:00
Jay Pfaffman
db17c531d6 Feature: Invision Power Board Importer 2018-01-30 08:44:02 -08:00
Gerhard Schlager
192a0886e2 FIX: BBCode to Markdown conversion in phpBB3 importer was broken 
This fixes the conversion for quotes, code blocks and lists (except for nested lists). It also discourages the usage of the ruby-bbcode-to-md gem.
 2018-01-30 12:50:42 +01:00
Gerhard Schlager
c26db2116c FIX: phpBB3 importer imported some users as anonymous users 2018-01-30 12:50:42 +01:00
Neil Lalonde
a224459960 bulk importer shouldn't try to update primary key sequences to -1 2018-01-19 15:01:00 -05:00
Neil Lalonde
db287ad58e FIX: invalid SQL in vanilla_mysql import script 2018-01-19 11:53:07 -05:00
Gerhard Schlager
6500343431 FIX: mbox importer didn't detected already indexed files 2018-01-17 17:03:53 +01:00