Commit Graph
100 Commits
Author SHA1 Message Date
Vinoth Kannan e12ae453e9 FIX: verify the exitence of s3_object properly without db name 2019-05-29 15:10:36 +05:30
Vinoth Kannan 9a9a06e34b DEV: add option to skip etag verification on 'migrate_to_s3' rake task 2019-05-29 14:16:36 +05:30
Vinoth Kannan b3779dc377 DEV: rename 'uploads:missing' rake task into 'uploads:missing_files'.
To improve the readability.
2019-05-28 23:30:43 +05:30
Vinoth Kannan 338b927f95 DEV: include local uploads while finding missing uploads
in rake task 'posts:missing_uploads'
2019-05-21 22:07:08 +05:30
Vinoth Kannan 7b82850f66 FIX: migrate_to_s3 task should remap attachment links too. 2019-05-21 21:58:11 +05:30
Vinoth Kannan 793915fe6a DEV: should ignore missing post uploads when a user export destroyed 2019-05-21 13:32:36 +05:30
Vinoth Kannan 2941c77abc FIX: skip upload recovery if file not found in s3 2019-05-21 00:06:36 +05:30
Vinoth Kannan 2bfc0cf145 FIX: skip old scheme upload URLs while validating s3 uploads remap 2019-05-20 19:13:41 +05:30
Vinoth Kannan 9f43852609 FIX: remove the unwanted code. 2019-05-20 19:10:21 +05:30
Vinoth Kannan bc0c4b7253 FIX: should not migrate the system uploads to s3 2019-05-20 14:27:34 +05:30
Vinoth Kannan be3fb85a04 DEV: add post migration checks and raise error if failed. 2019-05-20 14:18:28 +05:30
Vinoth Kannan 2a7065c505 FIX: skip uploads without etag in s3 inventory check. 2019-05-20 00:09:52 +05:30
Vinoth Kannan 0e677daaee FIX: include posts with data-orig-src attribute in have_uploads scope query. 2019-05-16 16:39:38 +05:30
Vinoth Kannan 3172172b52 remove unused local variable
ec84c87ddb
2019-05-16 15:39:13 +05:30
Vinoth Kannan ec84c87ddb FIX: skip validation while recovering uploads from s3
TODO: add tests
2019-05-16 15:37:11 +05:30
Vinoth Kannan 56ada8374f DEV: wrap find_missing_uploads method in distributed mutex
And skip posts with deleted topics.\ne8fafbc123170dd1f7d2a8adea4e7810585d3e76
2019-05-16 15:17:53 +05:30
Vinoth Kannan 636b75fa16 REFACTOR: remove duplicate reject loop and implicit return
e8fafbc123
2019-05-16 10:04:04 +05:30
Vinoth Kannan 40328f055e FIX: retrieve original filename from s3 object's content disposition header 2019-05-16 09:47:22 +05:30
Vinoth Kannan be0555cc17 FIX: Add bucket folder path only if not exists 2019-05-15 15:37:40 +05:30
Vinoth Kannan 42b10a646d FIX: return 404 only if upload url also not internal. 2019-05-15 02:06:54 +05:30
Vinoth Kannan 8c07c272f2 make rubocop happy. 2019-05-09 05:25:44 +05:30
Vinoth Kannan 87cd4701b8 FEATURE: option to skip posts with ignored missing uploads 2019-05-09 05:11:15 +05:30
Vinoth Kannan f5a217be92 Fix typo in condition value. 2019-05-07 17:09:08 +05:30
Vinoth Kannan e8fafbc123 List and restore missing post uploads from S3 inventory. 2019-05-04 01:16:20 +05:30
Vinoth Kannan 28547c6f08 revert bundler version change
73418aaf73
2019-05-02 04:37:55 +05:30
Vinoth Kannan 73418aaf73 DEV: Add bucket folder path to inventory id 2019-05-02 04:35:35 +05:30
Vinoth Kannan 92cc57b5da FIX: make 'stats' method compatible with multisite.
a8f410a9c5
2019-05-01 23:34:18 +05:30
Vinoth Kannan 7869a10d18 Revert "FEATURE: Added unlisted topics option to advanced search (#7447)"
This reverts commit 539723f8ff since it is failing the build.
2019-05-01 21:06:20 +05:30
Vinoth Kannan 9c78c18256 Check both site and global setting for s3 enabled value. 2019-05-01 10:49:53 +05:30
Vinoth Kannan 09ae12bd5d PERF: Call web hook events in a new thread (#7451) 2019-04-29 18:50:12 +10:00
Vinoth Kannan c448e014fe Recover missing old scheme uploads from tombstone directory too. 2019-04-29 11:33:57 +05:30
Vinoth Kannan be0ae2e487 FIX: plain text table detection should work even if html content not found. 2019-04-26 11:16:41 +05:30
Vinoth KannanandTim Lange dc2ee78f7a UX: Only do plain text table detection in outside code if HTML pasting enabled
Co-authored-by: Tim Lange <tim.lange@titanhq.de>
PR: https://github.com/discourse/discourse/pull/7436
2019-04-26 09:20:21 +05:30
Vinoth Kannan 8698a10920 Don't raise error if a record already exists. 2019-04-25 21:48:39 +05:30
Vinoth Kannan a65ce81e25 Don't check same upload url twice in the rake task 2019-04-25 20:06:10 +05:30
Vinoth Kannan 84cfe30c1f Use update_all instead of save to select and update particular columns
9ec6ef85b6
2019-04-25 11:56:57 +05:30
Vinoth Kannan 1724c27713 FIX: reload the 'post' model to retrive raw field value. 2019-04-25 02:09:27 +05:30
Vinoth Kannan aed683390c FIX: Don't treat 'upload_patterns' as constant to make the rake task compatible with multisite 2019-04-25 02:06:20 +05:30
Vinoth Kannan b0bb1d95a5 trying to recover old scheme uploads on 'posts:missing_uploads' rake task 2019-04-25 01:16:18 +05:30
Vinoth Kannan 3dae12dec1 reduce unlinked count from missing post upload count 2019-04-19 21:53:00 +05:30
Vinoth Kannan 4fcefc3615 add unlinked missing post upload count in stat 2019-04-19 20:39:21 +05:30
Vinoth Kannan ba6369edc5 Add more missing upload stats in rake task 2019-04-19 20:14:40 +05:30
Vinoth Kannan 48cedf3ebe fix: retrieve posts in order by id.
so we can find starting and ending id of the batch.
2019-04-19 18:30:59 +05:30
Vinoth Kannan ce8d30f667 PERF: use 'delete_all' method instead of 'destroy_all' 2019-04-19 18:00:01 +05:30
Vinoth Kannan 5de483a1b9 PERF: Very post upload's existence by preloaded upload sha1s array 2019-04-19 17:27:16 +05:30
Vinoth Kannan 8e40c35eb8 FIX: 'have_uploads' scope should include all uploads without multisite 'upload_path' prefix 2019-04-15 01:54:55 +05:30
Vinoth Kannan 70fef8e0c3 FIX: change to correct bundled version 2019-04-14 14:46:56 +05:30
Vinoth Kannan 87b53e170b FIX: skip <br> inside <p> if next character is \n 2019-04-14 14:44:54 +05:30
Vinoth Kannan af15ba86a0 Fix the build.
8d5c900142
2019-04-10 19:14:41 +05:30
Vinoth Kannan b0600e52b6 FIX: use 'freeze' method again to fix 'cant modify frozen string' error
8d5c900142
2019-04-10 18:30:59 +05:30
Vinoth Kannan 8d5c900142 DEV: add unique missing uploads index in post custom fields
https://review.discourse.org/t/feature-send-missing-post-uploads-stat-to-prometheus/2609/6?u=vinothkannans
2019-04-10 18:09:35 +05:30
Vinoth Kannan d0fe42e2ef FIX: should look through posts for image markdown
Downloaded onebox images only included in the cooked HTML content. So we have to check 'post.cooked' instead of 'raw'. bfdd0fe64c
2019-04-10 13:52:35 +05:30
Vinoth Kannan 4878ee95e5 DEV: optimize sql query to narrow down the filtering of post with uploads 2019-04-10 12:58:10 +05:30
Vinoth Kannan bfdd0fe64c FIX: empty the missing list on each post loop 2019-04-10 11:20:26 +05:30
Vinoth Kannan 73663c2fdb FIX: should find records by sha1 only in Upload model 2019-04-09 18:48:49 +05:30
Vinoth Kannan f0f659286f UX: move up profile picture field next to the username 2019-04-09 11:46:03 +05:30
Vinoth Kannan f5ac01253a FIX: use new changed method name and remove whitespaces 2019-04-09 02:25:26 +05:30
Vinoth Kannan 914ada1c74 DEV: convert scheduled job EnsurePostUploadsExistence into a rake task 2019-04-09 02:07:35 +05:30
Vinoth Kannan 2ae87a27fa make rubocop happy 2019-04-08 17:03:26 +05:30
Vinoth Kannan ca57e18f42 UX: display signup cta even if 'must_approve_users' setting enabled. 2019-04-08 12:27:56 +05:30
Vinoth Kannan 2c7a50860f DEV: disable missing post uploads check by default. 2019-04-05 02:29:10 +05:30
Vinoth Kannan 64ec4a6a1b fix the EnsurePostUploadsExistence job spec 2019-04-04 21:53:58 +05:30
Vinoth Kannan a385f6c48a REFACTOR: create custom field even when missing post uploads not found 2019-04-04 21:49:44 +05:30
Vinoth Kannan 44e87372da FEATURE: Add scheduled job to ensure s3 uploads existence 2019-04-04 20:37:44 +05:30
Vinoth Kannan 35431a8ddb FIX: set missing count in redis even if zero 2019-04-04 20:05:57 +05:30
Vinoth Kannan df6ef856e6 DEV: save missing s3 uploads count in redis 2019-04-04 19:05:57 +05:30
Vinoth Kannan 3b581de3b8 remove unrelated code
already added in a separate job
2019-04-04 02:26:28 +05:30
Vinoth Kannan b3fb0a7039 FEATURE: ensure consistency of post uploads in cooked content 2019-04-04 02:23:28 +05:30
Vinoth Kannan 904ba266cf SPEC: Add test case in emit_web_hook_event_spec for commit 4c6bfb9 2019-03-31 16:28:40 +05:30
Vinoth Kannan b8bd031648 FIX: Always include custom fields in CategorySerializer
even if it is empty
2019-03-25 07:59:56 +05:30
Vinoth Kannan 1e3cb7575d DEV: Update webhook event attributes even when an error raised 2019-03-21 20:45:21 +05:30
Vinoth Kannan 4c6bfb9b39 DEV: Don't destroy webhook in case of error 2019-03-21 18:34:54 +05:30
Vinoth Kannan 0861ca7764 FIX: Don't push browser state if navigated from root url to home page url and vice versa 2019-03-19 19:22:06 +05:30
Vinoth Kannan daf5a268a7 DEV: Option to preload topic custom fields in Search class 2019-03-17 23:16:09 +05:30
Vinoth Kannan 4477938ea7 DEV: Option to preload category custom fields for site serializer 2019-03-16 17:18:57 +05:30
Vinoth Kannan c773f82ba1 DEV: Added qunit test functions and did minor refactoring
https://github.com/discourse/discourse/commit/135191d1d194b1cd03eb0416b5c4f7bf1a319aaf
2019-03-13 18:38:27 +05:30
Vinoth Kannan d4d67386c9 FIX: change to correct bundled version 2019-03-13 16:43:45 +05:30
Vinoth Kannan 1b454c73ae FIX: 'topic' can have null value 2019-03-13 16:34:47 +05:30
Vinoth Kannan 135191d1d1 DEV: Option to add extra topic status icons in its widget and component
and did some refactoring
2019-03-13 15:17:26 +05:30
Vinoth Kannan 5ba2ef4274 UX: Display error message if no time frame selected in topic status update 2019-03-07 02:17:29 +05:30
Vinoth Kannan 167d85c21f FIX: post & topic destroyed hooks not triggering with tag filter 2019-03-06 22:52:54 +05:30
Vinoth Kannan 1b1f9831b0 Bump onebox version
discourse/onebox@4dd5a62
2019-03-06 11:58:41 +05:30
Vinoth Kannan be436066ff FIX: Prevent default event and stop bubbling up when card popup is closed in mobile 2019-03-01 01:42:17 +05:30
Vinoth Kannan 0cc636f648 UX: Don't add padding bottom for emojis in composer preview 2019-02-28 15:53:21 +05:30
Vinoth Kannan ed0120171c FIX: Hide 'show associated accounts' button if moderators can't see emails
And minor prettier code alignments
2019-02-26 13:49:27 +05:30
Vinoth Kannan 563b953224 DEV: Add 'backfill_etags_' to the method name since it also backfilling the etags 2019-02-19 21:54:35 +05:30
Vinoth Kannan bf2059baf5 SPEC: check enqueued jobs size instead of using mocks 2019-02-19 13:14:06 +05:30
Vinoth Kannan 76696b22fe DEV: call 'enqueue_hooks' method only if active webhooks exist 2019-02-19 12:47:57 +05:30
Vinoth Kannan 27e7f2bee0 DEV: call 'enqueue_hooks' method only if active webhooks exist 2019-02-19 12:43:08 +05:30
Vinoth Kannan f2c3415548 FIX: Should not generate payload until active webhooks are exist 2019-02-19 12:23:42 +05:30
Vinoth Kannan 0472bd4adc FIX: Remove 'backfill_etags' keyword argument from 'uploads:missing' rake task
And etags backfilling code is optimized
2019-02-15 00:34:35 +05:30
Vinoth Kannan b5fbd7385f FIX: run the rake task only for uploads created before a day from inventory date 2019-02-14 17:53:08 +05:30
Vinoth Kannan f7d03b2406 Stop the automatic creation of inventory 'gz' file in fixtures 2019-02-14 14:25:45 +05:30
Vinoth Kannan a9a8855739 DEV: Get only matching records to backfill etags 2019-02-14 06:27:18 +05:30
Vinoth Kannan e2f7db5549 Fix typo 2019-02-14 05:56:30 +05:30
Vinoth Kannan 484bd82278 FIX: Add onceoff job to remove double quotes from s3 etags 2019-02-14 05:19:41 +05:30
Vinoth Kannan 7b5931013a Update rake task to backfill etags from s3 inventory 2019-02-14 05:18:06 +05:30
Vinoth Kannan b8d2549922 FIX: OptimizedImage model doesn't have 'created_at' date column 2019-02-14 03:46:00 +05:30
Vinoth Kannan 426bd810f1 FIX: S3 inventory can have duplicate etags 2019-02-14 03:44:14 +05:30
Vinoth Kannan 1045bbc35b FIX: S3 inventory data can be splitted into multiple csv files 2019-02-14 03:41:52 +05:30