Commit Graph

160 Commits

Author SHA1 Message Date
Chocobozzz
ce8d0b5aae
Remove deprecated scheduler 2023-07-28 16:22:27 +02:00
Chocobozzz
97583d0023
Add more rate limits 2023-07-26 08:37:50 +02:00
Chocobozzz
f162d32da0
Support lazy download thumbnails 2023-06-29 10:19:33 +02:00
Chocobozzz
d8f39b126d
Add storyboard support 2023-06-29 10:16:55 +02:00
Chocobozzz
0c9668f779 Implement remote runner jobs in server
Move ffmpeg functions to @shared
2023-05-09 08:57:34 +02:00
Chocobozzz
4765348107
Add x-powered-by header 2023-02-27 09:22:59 +01:00
Chocobozzz
5bfe684e76
Move webfinger controller in well known router 2023-02-15 16:36:04 +01:00
Chocobozzz
5847724474
Add tracker opentelemtry metrics 2023-01-05 10:19:51 +01:00
Chocobozzz
9ab330b90d Use private ACL for private videos in s3 2022-10-24 14:48:24 +02:00
Chocobozzz
9d4c60dccc
Add ability for plugins to register ws routes 2022-10-11 11:11:04 +02:00
Chocobozzz
a3e5f804ad
Encrypt OTP secret 2022-10-10 11:12:23 +02:00
Chocobozzz
cfd57d2ca0
Live supports object storage
* Sync live files (segments, master playlist, resolution playlist,
   segment sha file) into object storage
 * Automatically delete them when the live ends
 * Segment sha file is now a file on disk, and not stored in memory
   anymore
2022-10-04 10:03:17 +02:00
Chocobozzz
4404a7c467
Prevent job queue to be started before plugins 2022-09-14 11:45:18 +02:00
Chocobozzz
37a04703a4
Add activeRequests info on db acquire timeout
Can help us to debug sequelize acquire timeout error
2022-09-08 10:19:21 +02:00
Florent
2a491182e4
Channel sync (#5135)
* Add external channel URL for channel update / creation (#754)

* Disallow synchronisation if user has no video quota (#754)

* More constraints serverside (#754)

* Disable sync if server configuration does not allow HTTP import (#754)

* Working version synchronizing videos with a job (#754)

TODO: refactoring, too much code duplication

* More logs and try/catch (#754)

* Fix eslint error (#754)

* WIP: support synchronization time change (#754)

* New frontend #754

* WIP: Create sync front (#754)

* Enhance UI, sync creation form (#754)

* Warning message when HTTP upload is disallowed

* More consistent names (#754)

* Binding Front with API (#754)

* Add a /me API (#754)

* Improve list UI (#754)

* Implement creation and deletion routes (#754)

* Lint (#754)

* Lint again (#754)

* WIP: UI for triggering import existing videos (#754)

* Implement jobs for syncing and importing channels

* Don't sync videos before sync creation + avoid concurrency issue (#754)

* Cleanup (#754)

* Cleanup: OpenAPI + API rework (#754)

* Remove dead code (#754)

* Eslint (#754)

* Revert the mess with whitespaces in constants.ts (#754)

* Some fixes after rebase (#754)

* Several fixes after PR remarks (#754)

* Front + API: Rename video-channels-sync to video-channel-syncs (#754)

* Allow enabling channel sync through UI (#754)

* getChannelInfo (#754)

* Minor fixes: openapi + model + sql (#754)

* Simplified API validators (#754)

* Rename MChannelSync to MChannelSyncChannel (#754)

* Add command for VideoChannelSync (#754)

* Use synchronization.enabled config (#754)

* Check parameters test + some fixes (#754)

* Fix conflict mistake (#754)

* Restrict access to video channel sync list API (#754)

* Start adding unit test for synchronization (#754)

* Continue testing (#754)

* Tests finished + convertion of job to scheduler (#754)

* Add lastSyncAt field (#754)

* Fix externalRemoteUrl sort + creation date not well formatted (#754)

* Small fix (#754)

* Factorize addYoutubeDLImport and buildVideo (#754)

* Check duplicates on channel not on users (#754)

* factorize thumbnail generation (#754)

* Fetch error should return status 400 (#754)

* Separate video-channel-import and video-channel-sync-latest (#754)

* Bump DB migration version after rebase (#754)

* Prettier states in UI table (#754)

* Add DefaultScope in VideoChannelSyncModel (#754)

* Fix audit logs (#754)

* Ensure user can upload when importing channel + minor fixes (#754)

* Mark synchronization as failed on exception + typos (#754)

* Change REST API for importing videos into channel (#754)

* Add option for fully synchronize a chnanel (#754)

* Return a whole sync object on creation to avoid tricks in Front (#754)

* Various remarks (#754)

* Single quotes by default (#754)

* Rename synchronization to video_channel_synchronization

* Add check.latest_videos_count and max_per_user options (#754)

* Better channel rendering in list #754

* Allow sorting with channel name and state (#754)

* Add missing tests for channel imports (#754)

* Prefer using a parent job for channel sync

* Styling

* Client styling

Co-authored-by: Chocobozzz <me@florianbigard.com>
2022-08-10 09:53:39 +02:00
Chocobozzz
5a921e7b74
Move to bullmq 2022-08-09 09:18:07 +02:00
Chocobozzz
c795e19663
Automatically rebuild native modules on ABI change 2022-08-03 15:08:36 +02:00
Chocobozzz
ea8107bff8
Split static router 2022-07-13 11:16:29 +02:00
Chocobozzz
dea0df90d5
Start Redis earlier 2022-07-13 10:00:17 +02:00
Chocobozzz
9452d4fd33
/!\ Use a dedicated config file for development
It means you have to replace NODE_ENV=test to NODE_ENV=dev if you use it
npm run dev:* commands are already updated
2022-07-06 16:12:17 +02:00
Chocobozzz
630d0a1bf5
Introduce experimental telemetry 2022-07-06 15:13:55 +02:00
Chocobozzz
38f57175e7
Update tsconfig 2022-06-03 16:17:28 +02:00
Chocobozzz
b211106695 Support video views/viewers stats in server
* Add "currentTime" and "event" body params to view endpoint
 * Merge watching and view endpoints
 * Introduce WatchAction AP activity
 * Add tables to store viewer information of local videos
 * Add endpoints to fetch video views/viewers stats of local videos
 * Refactor views/viewers handlers
 * Support "views" and "viewers" counters for both VOD and live videos
2022-04-15 09:49:35 +02:00
Chocobozzz
c729caf6cc Add basic video editor support 2022-02-28 10:42:19 +01:00
Chocobozzz
f8360396ff
Stop using tsconfig register
Prefer to replace paths at compile time
2022-01-06 08:26:14 +01:00
Chocobozzz
9270bd3a7e
Migrate to --enable-source-maps option 2022-01-03 16:37:16 +01:00
Chocobozzz
5a63748877
Node 10 is not supported anymore 2021-12-24 13:43:59 +01:00
Chocobozzz
a9cd881bf8
Add benchmark startup option for server 2021-11-12 11:08:30 +01:00
Chocobozzz
51353d9a03 Refactor video views
Introduce viewers attribute for live videos
Count views for live videos
Reduce delay to see the viewer update for lives
Add ability to configure video views buffer interval and view ip
expiration
2021-11-09 15:00:31 +01:00
Chocobozzz
df1db951c5
Support RTMPS 2021-11-05 11:38:17 +01:00
Chocobozzz
41fb13c330
esModuleInterop to true 2021-08-27 15:12:22 +02:00
Chocobozzz
e19fdf5775
Register plugins when peertube server is ready 2021-07-21 15:56:49 +02:00
Chocobozzz
c0e8b12e7f
Refactor requests 2021-07-20 15:27:18 +02:00
Johan Fleury
943dc18254 Trim access log messages to prevent double line break 2021-07-12 15:40:27 +02:00
Chocobozzz
8cc6120118
Upgrade server dependencies 2021-06-25 17:39:27 +02:00
Chocobozzz
8ebf2a5d5d
Refactor live manager 2021-06-17 08:47:03 +02:00
Chocobozzz
e030bfb59d
Refactor server errors handler 2021-06-02 18:15:41 +02:00
Rigel Kent
1cfbdd30d9 refactor deprecated body-parser usage 2021-06-02 16:57:07 +02:00
Rigel Kent
76148b27f7 refactor API errors to standard error format 2021-06-02 16:57:07 +02:00
Chocobozzz
2539932e16
Instance homepage support (#4007)
* Prepare homepage parsers

* Add ability to update instance hompage

* Add ability to set homepage as landing page

* Add homepage preview in admin

* Dynamically update left menu for homepage

* Inject home content in homepage

* Add videos list and channel miniature custom markup

* Remove unused elements in markup service
2021-05-27 15:59:55 +02:00
kontrollanten
f6d6e7f861
Resumable video uploads (#3933)
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-05-10 11:13:41 +02:00
Chocobozzz
8155db669b
Dissociate frameguard from csp 2021-04-12 15:33:54 +02:00
Chocobozzz
32a18cbf33
Add new plugin/peertube version notifs 2021-03-24 18:18:41 +01:00
Chocobozzz
ae71acca14
Check ffmepg version on startup 2021-03-24 18:18:40 +01:00
Chocobozzz
452b3bea08
Introduce tags to the logger
That could help to filter unneeded logs and/or select particular
components
2021-03-24 18:18:40 +01:00
Chocobozzz
90a8bd305d Dissociate video file names and video uuid 2021-02-18 13:38:09 +01:00
Chocobozzz
ba5a8d89bb
Update server dependencies 2021-02-03 09:45:08 +01:00
Chocobozzz
78d62f4d18
Fix and add skipping ping log tests 2021-01-13 09:38:19 +01:00
Johan Fleury
12c1e38df2 feat: allow administrator to disable logging of ping requests 2021-01-13 09:14:43 +01:00
Chocobozzz
c2b823826d
Log RTMP server error (address already in use) 2021-01-11 09:33:14 +01:00