Commit Graph

172 Commits

Author SHA1 Message Date
Chocobozzz
af11f812ac
Fix client stats 2024-07-03 15:59:26 +02:00
Chocobozzz
91302ea12a
Fix CI stats task 2024-07-02 14:06:52 +02:00
Chocobozzz
45d22afca6
Add manual run to docker ci build 2024-07-01 10:53:19 +02:00
Chocobozzz
dcfd83350a
Faster CI docker build 2024-07-01 10:50:29 +02:00
Chocobozzz
1bfb791e05
Integrate transcription in PeerTube 2024-06-28 08:44:58 +02:00
lutangar
ef14cf4a5c
feat(transcription): groundwork
chore: fiddling around some more

chore: add ctranslate2 and timestamped

chore: add performance markers

chore: refactor test

chore: change worflow name

chore: ensure Python3

chore(duration): convert to chai/mocha syntahx

chore(transcription): add individual tests for others transcribers

chore(transcription): implement formats test of all implementations

Also compare result of other implementation to the reference implementation

chore(transcription): add more test case with other language and models size and local model

chore(test): wip ctranslate 2 adapat

chore(transcription): wip transcript file and benchmark

chore(test): clean a bit

chore(test): clean a bit

chore(test): refacto timestamed spec

chore(test): update workflow

chore(test): fix glob expansion with sh

chore(test): extract some hw info

chore(test): fix async tests

chore(benchmark): add model info

feat(transcription): allow use of a local mode in timestamped-whisper

feat(transcription): extract run and profiling info in own value object

feat(transcription): extract run concept in own class an run more bench

chore(transcription): somplify run object only a uuid is now needed and add more benchmark scenario

docs(transcription): creates own package readme

docs(transcription): add local model usage

docs(transcription): update README

fix(transcription): use fr video for better comparison

chore(transcription): make openai comparison passed

docs(timestamped): clea

chore(transcription): change transcribers transcribe method signature

Introduce whisper builtin model.

fix(transcription): activate language detection

Forbid transcript creation without a language.
Add `languageDetection` flag to an engine and some assertions.

Fix an issue in `whisper-ctranslate2` :
https://github.com/Softcatala/whisper-ctranslate2/pull/93

chore(transcription): use PeerTube time helpers instead of custom ones

Update existing time function to output an integer number of seconds and add a ms human-readable time formatter with hints of tests.

chore(transcription): use PeerTube UUID helpers

chore(transcription): enable CER evaluation

Thanks to this recent fix in Jiwer <3
https://github.com/jitsi/jiwer/issues/873

chore(jiwer): creates JiWer package

I'm not very happy with the TranscriptFileEvaluator constructor... suggestions ?

chore(JiWer): add usage in README

docs(jiwer): update JiWer readme

chore(transcription): use FunMOOC video in fixtures

chore(transcription): add proper english video fixture

chore(transcription): use os tmp directory where relevant

chore(transcription): fix jiwer cli test reference.txt

chore(transcription): move benchmark out of tests

chore(transcription): remove transcription workflow

docs(transcription): add benchmark info

fix(transcription): use ms precision in other transcribers

chore(transcription): simplify most of the tests

chore(transcription): remove slashes when building path with join

chore(transcription): make fromPath method async

chore(transcription): assert path to model is a directory for CTranslate2 transcriber

chore(transcription): ctranslate2 assertion

chore(transcription): ctranslate2 assertion

chore(transcription): add preinstall script for Python dependencies

chore(transcription): add download and unzip utils functions

chore(transcription): add download and unzip utils functions

chore(transcription): download & unzip models fixtures

chore(transcription): zip

chore(transcription): raise download file test timeout

chore(transcription): simplify download file test

chore(transcription): add transcriptions test to CI

chore(transcription): raise test preconditions timeout

chore(transcription): run preinstall scripts before running ci

chore(transcription): create dedicated tmp folder for transcriber tests

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): use short video for local model test

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): setup verbosity based on NODE_ENV value
2024-06-28 08:43:40 +02:00
Chocobozzz
2728810f60
Remove bundlewatch
CI fails, our projects generates too many chunks unfortunately
2024-06-26 14:20:45 +02:00
Chocobozzz
9772280e99
Upgrade to angular 18 & vite 2024-06-26 08:33:54 +02:00
Chocobozzz
ca90b8f651
Update github actions 2024-02-06 07:34:58 +01:00
Chocobozzz
ee8f377515
Update codeql ci 2023-11-08 10:53:32 +01:00
Chocobozzz
678ae8abbd
Breaking: only support node 18
Node 16 is not supported anymore

See https://nodejs.org/fr/blog/announcements/nodejs16-eol
2023-10-11 13:59:24 +02:00
Chocobozzz
5a3d0650c9
server/server -> server/core 2023-10-04 15:13:25 +02:00
Yehuda Deutsch
f7882ca3eb Switch from debian bullseye to bookworm 2023-08-17 15:18:21 +02:00
Chocobozzz
a5bde7ad60
Fix benchmark script 2023-08-17 15:03:27 +02:00
Chocobozzz
3a4992633e
Migrate server to ESM
Sorry for the very big commit that may lead to git log issues and merge
conflicts, but it's a major step forward:

 * Server can be faster at startup because imports() are async and we can
   easily lazy import big modules
 * Angular doesn't seem to support ES import (with .js extension), so we
   had to correctly organize peertube into a monorepo:
    * Use yarn workspace feature
    * Use typescript reference projects for dependencies
    * Shared projects have been moved into "packages", each one is now a
      node module (with a dedicated package.json/tsconfig.json)
    * server/tools have been moved into apps/ and is now a dedicated app
      bundled and published on NPM so users don't have to build peertube
      cli tools manually
    * server/tests have been moved into packages/ so we don't compile
      them every time we want to run the server
 * Use isolatedModule option:
   * Had to move from const enum to const
     (https://www.typescriptlang.org/docs/handbook/enums.html#objects-vs-enums)
   * Had to explictely specify "type" imports when used in decorators
 * Prefer tsx (that uses esbuild under the hood) instead of ts-node to
   load typescript files (tests with mocha or scripts):
     * To reduce test complexity as esbuild doesn't support decorator
       metadata, we only test server files that do not import server
       models
     * We still build tests files into js files for a faster CI
 * Remove unmaintained peertube CLI import script
 * Removed some barrels to speed up execution (less imports)
2023-08-11 15:02:33 +02:00
Chocobozzz
c37e305342
Fix CI tests 2023-06-29 10:19:05 +02:00
Chocobozzz
902b16af03
Breaking: remove node 14 support 2023-05-09 10:02:54 +02:00
Chocobozzz
21609258de Increase timeouts 2023-05-09 08:57:34 +02:00
Chocobozzz
3a0c2a77b1 Enable external plugins to test the PR 2023-05-09 08:57:34 +02:00
Chocobozzz
1772b383de Add peertube runner cli 2023-05-09 08:57:34 +02:00
Chocobozzz
18c53ef949 Fix activitypub doc URL 2023-05-09 08:57:34 +02:00
Mats Blomdahl
907bcfa02c
Misc wording fixes, while learning how to test/contribute (#5727)
* Misc wording fixes, while learning how to test/contribute

- Remove duplicate discrimination topic in Code of Conduct
- Correct capitalization of some brands (PeerTube, GitHub)
- Replace `npm` with `yarn`, for additional consistency

* Revert `yarn` to `npm run` per PR #5727 comment

* Remove note on how to start Redis on Windows/WSL
2023-04-05 08:51:38 +02:00
Chocobozzz
00ee545c24
Remove $ for bash commands
It prevents easy copy
2023-04-04 09:18:06 +02:00
Chocobozzz
ebd61437c1
Revert matrix for docker build 2023-03-27 09:00:16 +02:00
Chocobozzz
43fe59ae86
Fix docker ci matrix 2023-03-23 08:55:27 +01:00
Chocobozzz
d6fcbe8d6d
Try to fix docker build timeout 2023-03-22 09:14:20 +01:00
Chocobozzz
11c834bdba
Add server code new feature walkthrough 2023-03-16 14:15:33 +01:00
Chocobozzz
ca829e9a83
Deprecate node 14 2023-03-15 09:38:24 +01:00
Chocobozzz
f44cd95c01
Update doc anchor links 2023-02-23 09:46:43 +01:00
Chocobozzz
c5c95361e6
Move to new documentation links 2023-02-22 16:15:14 +01:00
Chocobozzz
efaf379789
Fix dev instance following test instances 2023-02-17 10:28:33 +01:00
Chocobozzz
f8deb7982c
Fix redoc cli command 2023-02-17 10:10:22 +01:00
Chocobozzz
f11d8a8380
Update contributing guide 2023-02-17 09:14:00 +01:00
Chocobozzz
29ff788963
Add contributing guide for embed 2023-02-16 16:47:13 +01:00
Chocobozzz
1f26c0b846
Add openapi doc guide in documentation 2023-02-16 16:41:32 +01:00
Chocobozzz
2fcd3cbf02
Update contributing guide 2023-02-09 12:16:42 +01:00
Chocobozzz
0daaab0c0e
Try to fix github api rate limit 2022-12-29 10:24:06 +01:00
Chocobozzz
a47bbfcc7f
Fix docker ci 2022-11-02 14:09:54 +01:00
Chocobozzz
3a58ab452d
Upgrade docker action dep 2022-10-31 09:24:31 +01:00
Chocobozzz
823cc35bb5
Revert "Use node 16 for tests"
This reverts commit 1f45f8b00d.
2022-10-28 17:26:26 +02:00
Chocobozzz
1f45f8b00d
Use node 16 for tests 2022-10-28 17:12:33 +02:00
Chocobozzz
9ab330b90d Use private ACL for private videos in s3 2022-10-24 14:48:24 +02:00
Chocobozzz
8c13fd744f
Update upluad artifacts 2022-10-11 14:36:19 +02:00
Chocobozzz
36305301cb
Set AKISMET_KEY as env 2022-09-28 08:25:46 +02:00
Jelle Besseling
d342d752b3 Add info about running redis server 2022-09-19 08:53:37 +02:00
Chocobozzz
14b3e8f445
Fix test after hooks 2022-07-13 11:34:48 +02:00
Chocobozzz
edd7b1256b
Add info on dev env variables 2022-07-07 08:27:27 +02:00
Chocobozzz
9452d4fd33
/!\ Use a dedicated config file for development
It means you have to replace NODE_ENV=test to NODE_ENV=dev if you use it
npm run dev:* commands are already updated
2022-07-06 16:12:17 +02:00
Chocobozzz
388a31d241
Upgrade CI actions 2022-06-27 15:28:32 +02:00
Chocobozzz
936ce6e563
Remove all checkboxes in issue templates 2022-06-14 09:04:14 +02:00