Commit Graph

3865 Commits

Author SHA1 Message Date
Chocobozzz
18f13f43ec Fix "Correctly save transaction with retries" 2024-07-24 15:20:03 +02:00
Chocobozzz
89e3951587 Correctly save transaction with retries 2024-07-23 16:38:28 +02:00
Chocobozzz
300676f62b Don't replace caption on transcription 2024-07-12 16:20:26 +02:00
Chocobozzz
e99e4aa93c Run transcription after studio 2024-07-11 11:29:46 +02:00
Chocobozzz
c5fa9fe70e More robust updateTorrentMetadata fn 2024-07-11 08:07:30 +02:00
Chocobozzz
c5de5ef1d0 Fix registering unamed settings 2024-07-10 12:12:07 +02:00
Chocobozzz
471bf5bbce Don't block video files on transcription 2024-07-04 11:18:04 +02:00
Chocobozzz
8ab6f23a00 Don't block video update on storyboard generation 2024-07-03 15:40:10 +02:00
Chocobozzz
f5104fb234 Don't transcribe/encode videos with invalid state 2024-07-03 09:09:20 +02:00
Chocobozzz
2d26eff129 Don't create pendingTranscription twice 2024-07-03 08:52:05 +02:00
Chocobozzz
f81bc3d3dc Relax client log rate limiting 2024-07-02 09:43:07 +02:00
Chocobozzz
01b3396c80 Better view handling with looped videos 2024-07-02 09:41:33 +02:00
Chocobozzz
ebcf3329f5 More robust transcription error job handler 2024-07-01 14:38:19 +02:00
Chocobozzz
b45ed3c05a Do not run transcription/transcoding on lives 2024-07-01 11:15:09 +02:00
Chocobozzz
3a71086e35 Correctly handle transcription conflicts 2024-06-28 14:42:19 +02:00
Grant
4bc29f6d3f Add Express req, res to filter:api.search.videos.local.list.params 2024-06-28 09:20:39 +02:00
Chocobozzz
fd4831e502 Metadata to know if the caption is auto generated 2024-06-28 08:44:59 +02:00
Chocobozzz
1bfb791e05 Integrate transcription in PeerTube 2024-06-28 08:44:58 +02:00
lutangar
ef14cf4a5c feat(transcription): groundwork
chore: fiddling around some more

chore: add ctranslate2 and timestamped

chore: add performance markers

chore: refactor test

chore: change worflow name

chore: ensure Python3

chore(duration): convert to chai/mocha syntahx

chore(transcription): add individual tests for others transcribers

chore(transcription): implement formats test of all implementations

Also compare result of other implementation to the reference implementation

chore(transcription): add more test case with other language and models size and local model

chore(test): wip ctranslate 2 adapat

chore(transcription): wip transcript file and benchmark

chore(test): clean a bit

chore(test): clean a bit

chore(test): refacto timestamed spec

chore(test): update workflow

chore(test): fix glob expansion with sh

chore(test): extract some hw info

chore(test): fix async tests

chore(benchmark): add model info

feat(transcription): allow use of a local mode in timestamped-whisper

feat(transcription): extract run and profiling info in own value object

feat(transcription): extract run concept in own class an run more bench

chore(transcription): somplify run object only a uuid is now needed and add more benchmark scenario

docs(transcription): creates own package readme

docs(transcription): add local model usage

docs(transcription): update README

fix(transcription): use fr video for better comparison

chore(transcription): make openai comparison passed

docs(timestamped): clea

chore(transcription): change transcribers transcribe method signature

Introduce whisper builtin model.

fix(transcription): activate language detection

Forbid transcript creation without a language.
Add `languageDetection` flag to an engine and some assertions.

Fix an issue in `whisper-ctranslate2` :
https://github.com/Softcatala/whisper-ctranslate2/pull/93

chore(transcription): use PeerTube time helpers instead of custom ones

Update existing time function to output an integer number of seconds and add a ms human-readable time formatter with hints of tests.

chore(transcription): use PeerTube UUID helpers

chore(transcription): enable CER evaluation

Thanks to this recent fix in Jiwer <3
https://github.com/jitsi/jiwer/issues/873

chore(jiwer): creates JiWer package

I'm not very happy with the TranscriptFileEvaluator constructor... suggestions ?

chore(JiWer): add usage in README

docs(jiwer): update JiWer readme

chore(transcription): use FunMOOC video in fixtures

chore(transcription): add proper english video fixture

chore(transcription): use os tmp directory where relevant

chore(transcription): fix jiwer cli test reference.txt

chore(transcription): move benchmark out of tests

chore(transcription): remove transcription workflow

docs(transcription): add benchmark info

fix(transcription): use ms precision in other transcribers

chore(transcription): simplify most of the tests

chore(transcription): remove slashes when building path with join

chore(transcription): make fromPath method async

chore(transcription): assert path to model is a directory for CTranslate2 transcriber

chore(transcription): ctranslate2 assertion

chore(transcription): ctranslate2 assertion

chore(transcription): add preinstall script for Python dependencies

chore(transcription): add download and unzip utils functions

chore(transcription): add download and unzip utils functions

chore(transcription): download & unzip models fixtures

chore(transcription): zip

chore(transcription): raise download file test timeout

chore(transcription): simplify download file test

chore(transcription): add transcriptions test to CI

chore(transcription): raise test preconditions timeout

chore(transcription): run preinstall scripts before running ci

chore(transcription): create dedicated tmp folder for transcriber tests

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): use short video for local model test

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): setup verbosity based on NODE_ENV value
2024-06-28 08:43:40 +02:00
Chocobozzz
43e186ef44 Add views tag to middlewares too 2024-06-27 10:51:36 +02:00
Chocobozzz
9772280e99 Upgrade to angular 18 & vite 2024-06-26 08:33:54 +02:00
Chocobozzz
bc8c853a53 Fix lint 2024-06-24 09:48:57 +02:00
Chocobozzz
5412465df4 Fix build 2024-06-24 09:20:32 +02:00
Chocobozzz
209043e2d4 Fix legacy upload req timeout 2024-06-24 09:10:11 +02:00
Chocobozzz
05d84f659d Fix loading actor involved in video 2024-06-21 16:05:40 +02:00
Chocobozzz
985e79f61e Fix lint and tests 2024-06-21 15:28:09 +02:00
Chocobozzz
a722194809 Update server dependencies 2024-06-21 14:39:50 +02:00
Chocobozzz
990266f149 Owner must not be able to approve its own comment 2024-06-21 10:27:03 +02:00
kontrollanten
ab22399da0 feat(plugins/filter): add req to api.video.get.result 2024-06-21 07:01:02 +02:00
Chocobozzz
802601c241 Fix channel update federation 2024-06-20 08:48:24 +02:00
Chocobozzz
0d0a965f8a Fix detecting account actor 2024-06-19 17:37:27 +02:00
Chocobozzz
346be1d478 Support Service AP actors 2024-06-19 17:08:59 +02:00
kontrollanten
ca26687e1f feat: config option object_storage.max_request_attempts (#6418)
* feat: config option object_storage.max_attempts

Backblaze recommends to have a high amount of attempts since they've
designed their architecture so that it will return 5xx errors to
indicate that the client should do a new attempt.

https://www.backblaze.com/blog/b2-503-500-server-error/

closes #6415

* Rephrase comment

---------

Co-authored-by: Chocobozzz <me@florianbigard.com>
2024-06-11 13:57:18 +02:00
Chocobozzz
633039c1fa Add concurrency to delete remote files 2024-06-06 11:33:09 +02:00
Chocobozzz
585050821a Also prune storyboards and actor images 2024-06-05 15:47:37 +02:00
Chocobozzz
2b189131fa Add house-keeping script 2024-06-05 15:35:58 +02:00
Chocobozzz
3427330611 Support update object storage urls 2024-06-05 09:01:40 +02:00
Chocobozzz
96b9748585 Support CLI move of original video file 2024-06-04 09:08:24 +02:00
Chocobozzz
54c140c800 Support object storage in prune script
Also prune original files and user exports
2024-06-03 16:37:44 +02:00
Chocobozzz
568a1b1e85 Fix segmentsSha256Url in export 2024-06-03 09:44:12 +02:00
Chocobozzz
7c9f07e140 Improve Video AP compatibility
Compat with text/html descriptions
Compat with SPDX for licences
Compat with missing sensitive attribute
Compat with missing tag attribute
Compat with missing video file magnet URI
Compat with missing streaming playlist segmentsSha256Url
Compat with optional comments/likes/dislikes/shares URI in video object

Add more debug logs when the object is not valid
2024-06-03 09:36:29 +02:00
Chocobozzz
1e3a5b25c3 Plan to have multiple sizes in the future 2024-05-31 09:02:34 +02:00
Gergo Bogdan
3fbe849183 #5618 - added configuration fields for Thumbnails and Previews, if no config is provided, the previous, default values are configured 2024-05-31 08:57:23 +02:00
Chocobozzz
6a4db88e5c Handle 410 HTTP response code for AP objects 2024-05-30 10:28:20 +02:00
Chocobozzz
671c6c1f96 Fix auto tags migration 2024-05-29 15:06:08 +02:00
Chocobozzz
29329d6c45 Implement auto tag on comments and videos
* Comments and videos can be automatically tagged using core rules or
   watched word lists
 * These tags can be used to automatically filter videos and comments
 * Introduce a new video comment policy where comments must be approved
   first
 * Comments may have to be approved if the user auto block them using
   core rules or watched word lists
 * Implement FEP-5624 to federate reply control policies
2024-05-29 15:03:14 +02:00
Chocobozzz
b65d639695 Don't include banners in account API 2024-05-29 09:00:43 +02:00
Chocobozzz
e2c354878f Fix "reading 'isBlacklisted' of undefined" 2024-05-29 08:57:13 +02:00
Chocobozzz
4ea659d569 Handle videos with FPS < 1 2024-05-29 08:56:53 +02:00
Chocobozzz
b7e07bde8c Update search REST API doc 2024-05-28 09:05:35 +02:00