Commit Graph

306 Commits

Author SHA1 Message Date
Chocobozzz
842a15732b Ensure we don't run transcoding after import file 2021-11-23 15:22:07 +01:00
Chocobozzz
4842e518be Fetch publishedAt after wait jobs
To make sure it's published
2021-11-18 16:30:25 +01:00
Chocobozzz
ad5db1044c Add ability to run transcoding jobs 2021-11-18 15:20:57 +01:00
Chocobozzz
b46cf4b920 Add ability to remove hls/webtorrent files 2021-11-18 09:04:30 +01:00
Chocobozzz
1740952b8c Add more logs for video import tests 2021-11-16 08:59:35 +01:00
Chocobozzz
60409162e6 Fix tests cleanup 2021-11-15 09:37:19 +01:00
Chocobozzz
527a52ac42 Add ability to filter out public videos from admin 2021-11-12 14:23:22 +01:00
Chocobozzz
5cf027bdc4 Force video updatedAt update on update 2021-11-10 14:34:02 +01:00
Chocobozzz
d78b51aa4e Fix tests 2021-11-10 14:25:33 +01:00
Chocobozzz
020d3d3d79 Remove resumable cache after upload success 2021-11-10 09:43:33 +01:00
Florian CUNY
8dd754c767 Added 144p encoding (#4492)
* Added 144p encoding

Implements https://github.com/Chocobozzz/PeerTube/issues/4428

* Fixed typo in core-utils

* Increased BitPerPixel for 144p

* Disabled 144p by default in test.yaml

* Another try at fixing tests

* Fixed test in video-transcoder (api-3)

* Fixed test in video-imports (api-4)

* Fixed test in live-constraints (api-2)

* Tried to fix tests in api-3 again

* Revert "Tried to fix tests in api-3 again"

This reverts commit 266e1143fa37f333d149c2c2791c7bd33621ac14.

* Fixed test in config.ts (api-2)

* Try to fix test in video-hls.ts (api-3)

* Fixed test in video-transcoder.ts (api-3)

* Fix tests

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-11-05 10:23:02 +01:00
Chocobozzz
d324756edb Add ability to filter by file type 2021-11-03 11:32:41 +01:00
Chocobozzz
3c10840fa9 Add video file size info in admin videos list 2021-10-29 11:48:21 +02:00
Chocobozzz
2760b454a7 Deprecate filter video query
Introduce include and isLocal instead
2021-10-29 11:48:21 +02:00
Rigel Kent
276250f0a3 prevent multiple post-process triggering of upload-resumable (#4175)
* prevent multiple post-process triggering of upload-resumable

* switch from 409 to 503 for upload being processed

* Improve resumable upload check

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-10-25 17:42:20 +02:00
Chocobozzz
5480933b7f Add peertube short link import test 2021-10-22 14:25:32 +02:00
Chocobozzz
62549e6c98 Rewrite youtube-dl import
Use python3 binary
Allows to use a custom youtube-dl release URL
Allows to use yt-dlp (youtube-dl fork)
Remove proxy config from configuration to use HTTP_PROXY and HTTPS_PROXY
env variables
2021-10-22 10:25:24 +02:00
Chocobozzz
27c3c9456d Increase some timeouts 2021-10-18 10:23:45 +02:00
Chocobozzz
179bfea5e0 Fix import tests 2021-10-14 09:36:55 +02:00
Chocobozzz
3b3490e41a Increase HDR timeout import 2021-10-13 09:18:48 +02:00
Chocobozzz
9f430a53be Fix bitrate tests 2021-10-12 09:18:54 +02:00
Chocobozzz
790c2837dd Fix silent 500 after resumable upload 2021-09-09 09:35:30 +02:00
Chocobozzz
fb72d2e1c2 Fix infohash with object storage 2021-09-07 15:16:26 +02:00
Chocobozzz
e3c9ea7226 Add peertube import test 2021-08-27 16:21:19 +02:00
Chocobozzz
41fb13c330 esModuleInterop to true 2021-08-27 15:12:22 +02:00
Chocobozzz
851675c559 Rename jobs list command 2021-08-26 10:01:42 +02:00
Chocobozzz
cbb1d46b91 Fix hls tests 2021-08-18 10:02:48 +02:00
Jelle Besseling
0305db28c9 Add support for saving video files to object storage (#4290)
* Add support for saving video files to object storage

* Add support for custom url generation on s3 stored files

Uses two config keys to support url generation that doesn't directly go
to (compatible s3). Can be used to generate urls to any cache server or
CDN.

* Upload files to s3 concurrently and delete originals afterwards

* Only publish after move to object storage is complete

* Use base url instead of url template

* Fix mistyped config field

* Add rudenmentary way to download before transcode

* Implement Chocobozzz suggestions

https://github.com/Chocobozzz/PeerTube/pull/4290#issuecomment-891670478

The remarks in question:
    Try to use objectStorage prefix instead of s3 prefix for your function/variables/config names
    Prefer to use a tree for the config: s3.streaming_playlists_bucket -> object_storage.streaming_playlists.bucket
    Use uppercase for config: S3.STREAMING_PLAYLISTS_BUCKETINFO.bucket -> OBJECT_STORAGE.STREAMING_PLAYLISTS.BUCKET (maybe BUCKET_NAME instead of BUCKET)
    I suggest to rename moveJobsRunning to pendingMovingJobs (or better, create a dedicated videoJobInfo table with a pendingMove & videoId columns so we could also use this table to track pending transcoding jobs)
    https://github.com/Chocobozzz/PeerTube/pull/4290/files#diff-3e26d41ca4bda1de8e1747af70ca2af642abcc1e9e0bfb94239ff2165acfbde5R19 uses a string instead of an integer
    I think we should store the origin object storage URL in fileUrl, without base_url injection. Instead, inject the base_url at "runtime" so admins can easily change this configuration without running a script to update DB URLs

* Import correct function

* Support multipart upload

* Remove import of node 15.0 module stream/promises

* Extend maximum upload job length

Using the same value as for redundancy downloading seems logical

* Use dynamic part size for really large uploads

Also adds very small part size for local testing

* Fix decreasePendingMove query

* Resolve various PR comments

* Move to object storage after optimize

* Make upload size configurable and increase default

* Prune webtorrent files that are stored in object storage

* Move files after transcoding jobs

* Fix federation

* Add video path manager

* Support move to external storage job in client

* Fix live object storage tests

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-08-17 08:26:20 +02:00
Chocobozzz
679c12e69c Improve target bitrate calculation 2021-08-06 14:13:26 +02:00
Chocobozzz
764b1a14fc Use random names for VOD HLS playlists 2021-07-26 11:29:31 +02:00
Chocobozzz
83903cb65d Generate random uuid for video files 2021-07-26 11:29:31 +02:00
Chocobozzz
fea11cf25c Fix tests with node 16 2021-07-22 14:58:25 +02:00
Chocobozzz
c63830f154 Rename captions commands 2021-07-21 13:58:35 +02:00
Chocobozzz
4c7e60bc17 Reorganize imports 2021-07-20 15:27:19 +02:00
Chocobozzz
c0e8b12e7f Refactor requests 2021-07-20 15:27:18 +02:00
Chocobozzz
59bbcced37 Centralize test URLs 2021-07-20 15:27:18 +02:00
Chocobozzz
254d3579f5 Use an object to represent a server 2021-07-20 15:27:18 +02:00
Chocobozzz
89d241a79c Shorter server command names 2021-07-20 15:27:18 +02:00
Chocobozzz
d23dd9fbfc Introduce videos command 2021-07-20 15:27:18 +02:00
Chocobozzz
7926c5f9b3 Introduce user command 2021-07-20 15:27:18 +02:00
Chocobozzz
41d1d07501 Introduce login command 2021-07-20 15:27:18 +02:00
Chocobozzz
6c5065a011 Introduce server commands 2021-07-20 15:27:18 +02:00
Chocobozzz
0d8ecb7592 Fix NSFW tests 2021-07-20 15:27:18 +02:00
Chocobozzz
dd0ebb7151 Introduce notifications command 2021-07-20 15:27:18 +02:00
Chocobozzz
9293139fde Introduce sql command 2021-07-20 15:27:18 +02:00
Chocobozzz
12edc1495a Introduce comments command 2021-07-20 15:27:18 +02:00
Chocobozzz
a54618880c Introduce channels command 2021-07-20 15:27:18 +02:00
Chocobozzz
57f879a540 Introduce streaming playlists command 2021-07-20 15:27:18 +02:00
Chocobozzz
6910f20f11 Introduce import command 2021-07-20 15:27:18 +02:00
Chocobozzz
313228e9c3 Introduce history command 2021-07-20 15:27:18 +02:00