Commit Graph

299 Commits

Author SHA1 Message Date
kontrollanten
e1ab52d7ec Add migrate-to-object-storage script (#4481)
* add migrate-to-object-storage-script

closes #4467

* add migrate-to-unique-playlist-filenames script

* fix(migrate-to-unique-playlist-filenames): update master/segments256

run updateMasterHLSPlaylist and updateSha256VODSegments after
file rename.

* Improve move to object storage scripts

* PR remarks

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-11-09 11:05:35 +01:00
Chocobozzz
df1db951c5 Support RTMPS 2021-11-05 11:38:17 +01:00
Florian CUNY
8dd754c767 Added 144p encoding (#4492)
* Added 144p encoding

Implements https://github.com/Chocobozzz/PeerTube/issues/4428

* Fixed typo in core-utils

* Increased BitPerPixel for 144p

* Disabled 144p by default in test.yaml

* Another try at fixing tests

* Fixed test in video-transcoder (api-3)

* Fixed test in video-imports (api-4)

* Fixed test in live-constraints (api-2)

* Tried to fix tests in api-3 again

* Revert "Tried to fix tests in api-3 again"

This reverts commit 266e1143fa37f333d149c2c2791c7bd33621ac14.

* Fixed test in config.ts (api-2)

* Try to fix test in video-hls.ts (api-3)

* Fixed test in video-transcoder.ts (api-3)

* Fix tests

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-11-05 10:23:02 +01:00
Chocobozzz
2760b454a7 Deprecate filter video query
Introduce include and isLocal instead
2021-10-29 11:48:21 +02:00
Florian CUNY
754b6f5f41 Made the video channels limit (per user) server-wide configurable (#4491)
* Made the video channels limit (per user) server-wide configurable

Implements https://github.com/Chocobozzz/PeerTube/issues/3092

Also added a "quota bar" in the account's settings page

* Fixed lint errors

* Another pass at fixing lint errors

* Applied code suggestions

* Removed 'video channels quota'
2021-10-26 16:42:10 +02:00
Chocobozzz
605450a6f7 Fix dependency security alert 2021-10-26 16:04:09 +02:00
Chocobozzz
d1bfbdeb20 Random listen for mocked servers 2021-10-22 14:31:38 +02:00
Chocobozzz
ab4001aade Provide origin URL to client and fix remote share 2021-10-22 14:11:58 +02:00
Chocobozzz
62549e6c98 Rewrite youtube-dl import
Use python3 binary
Allows to use a custom youtube-dl release URL
Allows to use yt-dlp (youtube-dl fork)
Remove proxy config from configuration to use HTTP_PROXY and HTTPS_PROXY
env variables
2021-10-22 10:25:24 +02:00
Chocobozzz
64553e8809 Add ability to filter logs by tags 2021-10-20 14:33:38 +02:00
Chocobozzz
978c87e7f5 Add channel filters for my videos/followers 2021-10-20 09:25:44 +02:00
Chocobozzz
906f46d084 Split check user params tests 2021-10-20 09:25:44 +02:00
Chocobozzz
4beda9e12a Add ability to view my followers 2021-10-20 09:25:44 +02:00
Chocobozzz
cf0c8ee588 Fix admin edition disabling feature 2021-10-14 11:35:43 +02:00
Chocobozzz
179bfea5e0 Fix import tests 2021-10-14 09:36:55 +02:00
Chocobozzz
3c25d37aef Try to fix mock server ports 2021-10-12 13:30:41 +02:00
Chocobozzz
9da443f374 Remove unused comments 2021-10-08 11:22:22 +02:00
Chocobozzz
8de589b953 Fix tests 2021-09-09 09:55:20 +02:00
Chocobozzz
790c2837dd Fix silent 500 after resumable upload 2021-09-09 09:35:30 +02:00
Chocobozzz
fb72d2e1c2 Fix infohash with object storage 2021-09-07 15:16:26 +02:00
Chocobozzz
70430c2796 Wait mock server termination 2021-09-06 08:13:11 +02:00
Chocobozzz
fbcd44d233 Add debug messages to webtorrent tests 2021-09-02 09:55:48 +02:00
Chocobozzz
e3c9ea7226 Add peertube import test 2021-08-27 16:21:19 +02:00
Chocobozzz
41fb13c330 esModuleInterop to true 2021-08-27 15:12:22 +02:00
Chocobozzz
706a450fe6 Fix test event leak 2021-08-26 11:11:52 +02:00
Chocobozzz
851675c559 Rename jobs list command 2021-08-26 10:01:42 +02:00
smilekison
8729a87024 Support proxies for PeerTube (#4346)
* Updated with latest code

* Updated Support proxies for PeerTube

* Support Proxies for PeerTube (Updated with change request)

* Cleanup proxy PR

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-08-25 15:08:37 +02:00
Chocobozzz
63436fc5ea Fix uploading empty tags 2021-08-18 11:12:21 +02:00
Chocobozzz
1f6125be8b Optimize torrent URL update 2021-08-18 09:14:51 +02:00
Jelle Besseling
0305db28c9 Add support for saving video files to object storage (#4290)
* Add support for saving video files to object storage

* Add support for custom url generation on s3 stored files

Uses two config keys to support url generation that doesn't directly go
to (compatible s3). Can be used to generate urls to any cache server or
CDN.

* Upload files to s3 concurrently and delete originals afterwards

* Only publish after move to object storage is complete

* Use base url instead of url template

* Fix mistyped config field

* Add rudenmentary way to download before transcode

* Implement Chocobozzz suggestions

https://github.com/Chocobozzz/PeerTube/pull/4290#issuecomment-891670478

The remarks in question:
    Try to use objectStorage prefix instead of s3 prefix for your function/variables/config names
    Prefer to use a tree for the config: s3.streaming_playlists_bucket -> object_storage.streaming_playlists.bucket
    Use uppercase for config: S3.STREAMING_PLAYLISTS_BUCKETINFO.bucket -> OBJECT_STORAGE.STREAMING_PLAYLISTS.BUCKET (maybe BUCKET_NAME instead of BUCKET)
    I suggest to rename moveJobsRunning to pendingMovingJobs (or better, create a dedicated videoJobInfo table with a pendingMove & videoId columns so we could also use this table to track pending transcoding jobs)
    https://github.com/Chocobozzz/PeerTube/pull/4290/files#diff-3e26d41ca4bda1de8e1747af70ca2af642abcc1e9e0bfb94239ff2165acfbde5R19 uses a string instead of an integer
    I think we should store the origin object storage URL in fileUrl, without base_url injection. Instead, inject the base_url at "runtime" so admins can easily change this configuration without running a script to update DB URLs

* Import correct function

* Support multipart upload

* Remove import of node 15.0 module stream/promises

* Extend maximum upload job length

Using the same value as for redundancy downloading seems logical

* Use dynamic part size for really large uploads

Also adds very small part size for local testing

* Fix decreasePendingMove query

* Resolve various PR comments

* Move to object storage after optimize

* Make upload size configurable and increase default

* Prune webtorrent files that are stored in object storage

* Move files after transcoding jobs

* Fix federation

* Add video path manager

* Support move to external storage job in client

* Fix live object storage tests

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-08-17 08:26:20 +02:00
Chocobozzz
a1c63fe1a2 Fix send rtmp command 2021-08-06 15:06:47 +02:00
Chocobozzz
679c12e69c Improve target bitrate calculation 2021-08-06 14:13:26 +02:00
Chocobozzz
c826f34a45 Limit live bitrate 2021-08-06 10:39:40 +02:00
Chocobozzz
b033851fb5 Search channels against handles and not names 2021-07-29 10:27:24 +02:00
Chocobozzz
29837f8885 Add ability to search by host in server 2021-07-27 17:10:52 +02:00
Chocobozzz
15a7eafb89 Refactor video links builders 2021-07-26 15:04:37 +02:00
Chocobozzz
764b1a14fc Use random names for VOD HLS playlists 2021-07-26 11:29:31 +02:00
Chocobozzz
83903cb65d Generate random uuid for video files 2021-07-26 11:29:31 +02:00
Chocobozzz
2e980ed30d Simplify createServer args 2021-07-22 12:08:02 +02:00
Chocobozzz
7a397c7f15 Upgrade server dependencies 2021-07-22 12:01:58 +02:00
Chocobozzz
c63830f154 Rename captions commands 2021-07-21 13:58:35 +02:00
Chocobozzz
4d029ef8ec Add ability for instances to follow any actor 2021-07-21 13:35:31 +02:00
Chocobozzz
4c7e60bc17 Reorganize imports 2021-07-20 15:27:19 +02:00
Chocobozzz
c0e8b12e7f Refactor requests 2021-07-20 15:27:18 +02:00
Chocobozzz
08642a765e Fix server run 2021-07-20 15:27:18 +02:00
Chocobozzz
59bbcced37 Centralize test URLs 2021-07-20 15:27:18 +02:00
Chocobozzz
254d3579f5 Use an object to represent a server 2021-07-20 15:27:18 +02:00
Chocobozzz
89d241a79c Shorter server command names 2021-07-20 15:27:18 +02:00
Chocobozzz
d23dd9fbfc Introduce videos command 2021-07-20 15:27:18 +02:00
Chocobozzz
7926c5f9b3 Introduce user command 2021-07-20 15:27:18 +02:00