From e755a63a1baaa1ddc94abd7f26f6708e8cb869a9 Mon Sep 17 00:00:00 2001 From: Benjamin Bouvier Date: Sat, 6 Jan 2018 18:10:30 +0100 Subject: [PATCH] Reword docs (#176) * Reword documentation; * Wrap lines at 80 characters in the documentation. * Wrap CONTRIBUTING.md too; * Tweaks to CONTRIBUTING.md wording; * Move development section to CONTRIBUTING.md; * Explain database creation in CONTRIBUTING.md; --- .github/CONTRIBUTING.md | 74 +++++++++++++++++++-- ARCHITECTURE.md | 46 ++++++++----- FAQ.md | 23 ++++--- README.md | 141 ++++++++++++++++++++++++---------------- 4 files changed, 196 insertions(+), 88 deletions(-) diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 3764eb7c4..989087728 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -7,30 +7,90 @@ Interesting in contributing? Awesome :) * [Give your feedback](#give-your-feedback) * [Develop on the Server side](#develop-on-the-server-side) * [Develop on the Client side](#develop-on-the-client-side) + * [Get started with development](#get-started-with-development) * [Write documentation](#write-documentation) ## Give your feedback -Even if you are not a developer, or you do not want to develop on PeerTube you can give a feedback on potential bugs, features that you are interested in, user interace, design, decentralized architecture... +You don't need to know how to code to start contributing to PeerTube! Other +contributions are very valuable too, among which: you can test the software and +report bugs, you can give feedback on potential bugs, features that you are +interested in, user interace, design, decentralized architecture... +## Development + ## Develop on the Server side -The server is a web server developed with [NodeJS](https://nodejs.org)/[Express](http://expressjs.com). +The server is a web server developed with +[NodeJS](https://nodejs.org)/[Express](http://expressjs.com). -Newcommer? You can find a documentation of the server code/architecture [here](https://github.com/Chocobozzz/PeerTube/blob/master/support/doc/server/code.md). +Newcomer? You can find a documentation of the server code/architecture +[here](https://github.com/Chocobozzz/PeerTube/blob/master/support/doc/server/code.md). -Do not hesitate to talk about the functionnality you want to develop by creating an issue :) +Don't hesitate to talk about features you want to develop by creating an issue +before you start working on them :). ## Develop on the Client side -The client is a web application developed with [TypeScript](https://www.typescriptlang.org/)/[Angular2](https://angular.io/) +The client is a web application developed with +[TypeScript](https://www.typescriptlang.org/)/[Angular2](https://angular.io/). -Newcommer? You can find a documentation of the server code/architecture [here](https://github.com/Chocobozzz/PeerTube/blob/master/support/doc/client/code.md). +Newcomer? You can find a documentation of the server code/architecture +[here](https://github.com/Chocobozzz/PeerTube/blob/master/support/doc/client/code.md). + + +## Get started with development + +In this mode, the server will run requests between instances more quickly, the +video durations are limited to a few seconds. + +### Prerequisites + +First, make sure that you have followed the steps to install the dependencies. + +Then, create a postgres database and user with the values set in the +`config/default.yaml` file. For instance, if you do not change the values +there, the following commands would create a new database called `peertube_dev` +and a postgres user called `peertube` with password `peertube`: + +```bash + # sudo -u postgres createuser -P peertube + Enter password for new role: peertube + # sudo -u postgres createdb -O peertube peertube_dev +``` + +### Server side + +To develop on the server-side: + +```bash + $ npm run dev:server +``` + +Then, the server will listen on `localhost:9000`. When server source files +change, these are automatically recompiled and the server will automatically +restart. + +### Client side + +To develop on the client side: + +```bash + $ npm run dev:client +``` + +The API will listen on `localhost:9000` and the frontend on `localhost:3000`. +Client files are automatically compiled on change, and the web browser will +reload them automatically thanks to hot module replacement. + +**Username**: *root*
+**Password**: *test* ## Write documentation -You can help to write the documentation of the REST API, code, architecture, demonstrations... +You can help to write the documentation of the REST API, code, architecture, +demonstrations... diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index ac62fa4b7..ecc4d2d40 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -2,25 +2,39 @@ ## Vocabulary - - **Fediverse:** several servers following each others - - **Instance:** a server which runs PeerTube in the fediverse - - **Origin instance:** the instance on which the video was uploaded and which is seeding (WebSeed protocol) the video - - **Following:** the action of a PeerTube instance which will follow another instance (subscribe to its videos) + - **Fediverse:** several servers following each others. + - **Instance:** a server which runs PeerTube in the fediverse. + - **Origin instance:** the instance on which the video was uploaded and which + is seeding (through the WebSeed protocol) the video. + - **Following:** the action of a PeerTube instance which will follow another + instance (subscribe to its videos). ## Base ### Communications - * All the communications between the instances are signed with [Linked Data Signatures](https://w3c-dvcg.github.io/ld-signatures/) with the private key of the account that made the action - * We use the [ActivityPub](https://www.w3.org/TR/activitypub/) protocol (only server-server for now). Object models could be found in [shared/models/activitypub directory](https://github.com/Chocobozzz/PeerTube/tree/develop/shared/models/activitypub). - * All the requests are retried several times if they fail - * The maximum of points would be defined + * All the communication between the instances are signed with [Linked Data + Signatures](https://w3c-dvcg.github.io/ld-signatures/) with the private key + of the account that authored the action. + * We use the [ActivityPub](https://www.w3.org/TR/activitypub/) protocol (only + server-server for now). Object models could be found in + [shared/models/activitypub + directory](https://github.com/Chocobozzz/PeerTube/tree/develop/shared/models/activitypub). + * All the requests are retried several times if they fail. + * The maximum of points would be defined. ### Instance - * An instance has a websocket tracker which is responsible for all the video uploaded in it - * An instance has an administrator that can follow other instances - * An instance can be configured to follow back automatically - * An instance can blacklist other instances (only used in "follow back" mode) - * An instance cannot choose which other instance follow it, but it can decide to **reject all** followers - * After having uploaded a video, the instance seeds it (WebSeed protocol) - * If a user wants to watch a video, he asks its instance the magnet URI and the frontend adds the torrent (with WebTorrent), creates the HTML5 video player and streams the file into it - * A user watching a video seeds it too (BitTorrent) so another user who is watching the same video can get the data from the origin server and the user 1 (etc) + * An instance has a websocket tracker which is responsible for all the video + uploaded in it. + * An instance has an administrator that can follow other instances. + * An instance can be configured to follow back automatically. + * An instance can blacklist other instances (only used in "follow back" + mode). + * An instance cannot choose which other instances follow it, but it can + decide to **reject all** followers. + * After having uploaded a video, the instance seeds it (WebSeed protocol). + * If a user wants to watch a video, they ask its instance the magnet URI and + the frontend adds the torrent (with WebTorrent), creates the HTML5 video + player and streams the file into it. + * A user watching a video seeds it too (BitTorrent). Thus another user who is + watching the same video can get the data from the origin server and other + users watching it. diff --git a/FAQ.md b/FAQ.md index 7e2b3339c..06c86ce10 100644 --- a/FAQ.md +++ b/FAQ.md @@ -2,25 +2,31 @@ ## If nobody watches a video, is it seeded? -Yes, the origin server always seeds videos uploaded on it through [Webseed](http://www.bittorrent.org/beps/bep_0019.html). +Yes, the origin server always seeds videos uploaded on it thanks to +[Webseed](http://www.bittorrent.org/beps/bep_0019.html). ## What is WebSeed? -It is a BitTorrent extension that allow a server to seed a file through HTTP. It just need to serve statically a file, and then the clients will request chunks with a Content-Range HTTP header. +It is a BitTorrent extension that allows a server to seed a file through HTTP. +It just needs to statically serve a file, then the clients will request chunks +with a `Content-Range` HTTP header. -## If a client requests each chunk of a video through HTTP, the server be overloaded! +## If a client requests each chunk of a video through HTTP, will the server be overloaded? -Not really. Reverse proxies like Nginx handle very well requests of static files. In my tests it can send chunks at 10MB/s without consuming more than 5% of CPU on a very small VPS. +Not really. Reverse proxies like Nginx handle very well requests of static +files. In my tests, it can send chunks at 10MB/s without consuming more than 5% +of CPU on a very small VPS. -## An index of all videos of servers you follow won't be too large for small servers? +## Will an index of all the videos of servers you follow be too large for small servers? -No, 1000000 videos will represent around 2GB on PostgreSQL. It is acceptable for a video platform. +In our benchmarks, 1,000,000 videos takes around 2GB of storage on PostgreSQL. +We think it is acceptable for a video platform. -## What kind of videos can I upload? +## What codecs can I use for the videos I want to upload? WEBM, MP4 or OGV videos. @@ -29,4 +35,5 @@ WEBM, MP4 or OGV videos. If you already have followers, you can't. -If you don't: update your configuration and run `NODE_ENV=production npm run update-host` to update the torrent files. +If you don't have any followers, update your configuration and run +`NODE_ENV=production npm run update-host` to update the torrent files. diff --git a/README.md b/README.md index 077d6b8f0..094d81c4a 100644 --- a/README.md +++ b/README.md @@ -3,10 +3,14 @@

-Federated (ActivityPub) video streaming platform using P2P (BitTorrent) directly in the web browser with WebTorrent. +Federated (ActivityPub) video streaming platform using P2P (BitTorrent) +directly in the web browser with WebTorrent.

-**PeerTube is sponsored by [Framasoft](https://framatube.org/#en), a non-profit that promotes, spreads and develops free-libre software. If you want to support this project, please [consider donating them](https://soutenir.framasoft.org/en/).** +**PeerTube is sponsored by [Framasoft](https://framatube.org/#en), a non-profit +that promotes, spreads and develops free culture in general, and free-libre +software in particular. If you want to support this project, please [consider +donating them](https://soutenir.framasoft.org/en/).**

Client @@ -58,20 +62,28 @@ Federated (ActivityPub) video streaming platform using P2P (BitTorrent) directly ## Demonstration -Want to see in action? +Want to see it in action? * [Demo server](http://peertube.cpy.re) - * [Video](https://peertube.cpy.re/videos/watch/f78a97f8-a142-4ce1-a5bd-154bf9386504) to see how the "decentralization feature" looks like - * Experimental demo servers that share videos (they are in the same network): [peertube2](http://peertube2.cpy.re), [peertube3](http://peertube3.cpy.re). Since I do experiments with them, sometimes they might not work correctly. + * [Video](https://peertube.cpy.re/videos/watch/f78a97f8-a142-4ce1-a5bd-154bf9386504) + to see how the "decentralization feature" looks like + * Experimental demo servers that share videos (they are in the same + network): [peertube2](http://peertube2.cpy.re), + [peertube3](http://peertube3.cpy.re). Since I do experiments with them, + sometimes they might not work correctly. ## Why -We can't build a FOSS video streaming alternatives to YouTube, Dailymotion, Vimeo... with a centralized software. One organization alone cannot have enough money to pay bandwidth and video storage of its server. +We can't build a FOSS video streaming alternatives to YouTube, Dailymotion, +Vimeo... with a centralized software. One organization alone may not have +enough money to pay for bandwidth and video storage of its servers. -So we need to have a decentralized network (as [Diaspora](https://github.com/diaspora/diaspora) for example). -But it's not enough because one video could become famous and overload the server. -It's the reason why we need to use a P2P protocol to limit the server load. -Thanks to [WebTorrent](https://github.com/feross/webtorrent), we can make P2P (thus bittorrent) inside the web browser right now. +So we need to have a decentralized network of servers seeding videos (as +[Diaspora](https://github.com/diaspora/diaspora) for example). But it's not +enough because one video could become famous and overload the server. It's the +reason why we need to use a P2P protocol to limit the server load. Thanks to +[WebTorrent](https://github.com/feross/webtorrent), we can make P2P (thus +bittorrent) inside the web browser, as of today. ## Features @@ -111,7 +123,8 @@ Thanks to [WebTorrent](https://github.com/feross/webtorrent), we can make P2P (t ## Installation -See [wiki](https://github.com/Chocobozzz/PeerTube/wiki) for complete installation commands. +See [wiki](https://github.com/Chocobozzz/PeerTube/wiki) for complete +installation commands. ### Front compatibility @@ -128,12 +141,16 @@ See [wiki](https://github.com/Chocobozzz/PeerTube/wiki) for complete installatio #### Debian - 1. Install NodeJS 8.x (current LTS): [https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions](https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions) - 2. Install yarn: [https://yarnpkg.com/en/docs/install](https://yarnpkg.com/en/docs/install) + 1. Install NodeJS 8.x (current LTS): + [https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions](https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions) + 2. Install yarn: + [https://yarnpkg.com/en/docs/install](https://yarnpkg.com/en/docs/install) 4. Run: +```bash $ apt-get update $ apt-get install ffmpeg postgresql openssl +``` #### Ubuntu 16.04 @@ -141,112 +158,122 @@ See [wiki](https://github.com/Chocobozzz/PeerTube/wiki) for complete installatio 2. Install yarn: (same as Debian) 3. Run: +```bash $ apt-get update $ apt-get install ffmpeg postgresql openssl +``` #### Arch Linux - 1. Run: + 1. Run: +```bash $ pacman -S nodejs yarn ffmpeg postgresql openssl +``` -#### Other distribution... (PR welcome) +#### Other distributions +Feel free to update this README file in a pull request! -### Sources +### Build from the sources +```bash $ git clone -b master https://github.com/Chocobozzz/PeerTube $ cd PeerTube $ yarn install $ npm run build +``` ## Usage ### Production -If you want to run PeerTube for production (bad idea for now :) ): +If you want to run PeerTube in production (which might be a bad idea for now :) ): +```bash $ cp config/production.yaml.example config/production.yaml +``` -Then edit the `config/production.yaml` file according to your webserver configuration. Keys set in this file will override those of `config/default.yml`. +Then edit the `config/production.yaml` file according to your webserver +configuration. Keys set in this file will override those of +`config/default.yml`. -Finally, run the server with the `production` `NODE_ENV` variable set. +Finally, run the server with the `NODE_ENV` environment variable set to +`production`: +```bash $ NODE_ENV=production npm start +``` -The administrator password is automatically generated and can be found in the logs. You can set another password with: +The administrator password is automatically generated and can be found in the +logs. You can set another password with: +```bash $ NODE_ENV=production npm run reset-password -- -u root +``` **Nginx template** (reverse proxy): https://github.com/Chocobozzz/PeerTube/tree/master/support/nginx
**Systemd template**: https://github.com/Chocobozzz/PeerTube/tree/master/support/systemd You can check the application (CORS headers, tracker websocket...) by running: +```bash $ NODE_ENV=production npm run check +``` ### Upgrade -The following commands will upgrade the source (according to your current branch), upgrade node modules and rebuild client application: +The following commands will upgrade the source (according to your current +branch), upgrade node modules and rebuild client application: +```bash # systemctl stop peertube $ npm run upgrade-peertube # systemctl start peertube +``` -### Development - -In this mode, the server will run requests between instances more quickly, the video durations are limited to a few seconds. - -To develop on the server-side (server files are automatically compiled when we modify them and the server restarts automatically too): - - $ npm run dev:server - -The server (with the client) will listen on `localhost:9000`. - - -To develop on the client side (client files are automatically compiled when we modify them): - - $ npm run dev:client - -The API will listen on `localhost:9000` and the frontend on `localhost:3000` (with hot module replacement, you don't need to refresh the web browser). - -**Username**: *root*
-**Password**: *test* - -### Test with 3 fresh nodes +### Test with three fresh nodes +```bash $ npm run clean:server:test $ npm run play +``` -Then you will get access to the three nodes at `http://localhost:900{1,2,3}` with the `root` as username and `test{1,2,3}` for the password. +Then you will get access to the three nodes at `http://localhost:900{1,2,3}` +with the `root` as username and `test{1,2,3}` for the password. ### Other commands -To print all available command run: +To print all available commands, run: +```bash $ npm run help +``` ## Contributing -See the [contributing guide](https://github.com/Chocobozzz/PeerTube/blob/master/.github/CONTRIBUTING.md). - -See the [server code documentation](https://github.com/Chocobozzz/PeerTube/blob/master/support/doc/server/code.md). - -See the [client code documentation](https://github.com/Chocobozzz/PeerTube/blob/master/support/doc/client/code.md). - +See the [contributing +guide](https://github.com/Chocobozzz/PeerTube/blob/master/.github/CONTRIBUTING.md) +to see how to contribute to PeerTube. Spoiler alert: you don't need to be a +coder to help! ## Architecture -See [ARCHITECTURE.md](https://github.com/Chocobozzz/PeerTube/blob/master/ARCHITECTURE.md) for a more detailed explication. +See [ARCHITECTURE.md](https://github.com/Chocobozzz/PeerTube/blob/master/ARCHITECTURE.md) for a more detailed explanation. ### Backend - * The backend is a REST API - * Servers communicate with each others with [Activity Pub](https://www.w3.org/TR/activitypub/) - * Each server has its own users who query it (search videos, where the torrent URI of this specific video is...) - * If a user upload a video, the server seeds it and sends the video information (name, short description, torrent URI...) its followers - * A server is a tracker responsible for all the videos uploaded in it - * Even if nobody watches a video, it is seeded by the server (through [WebSeed protocol](http://www.bittorrent.org/beps/bep_0019.html)) where the video was uploaded + * The backend is a REST API. + * Servers communicate with each others with [Activity + Pub](https://www.w3.org/TR/activitypub/). + * Each server has its own users who query it (search videos, query where the + torrent URI of this specific video is...). + * If a user uploads a video, the server seeds it and sends its followers some + metadata (name, short description, torrent URI...). + * A server is a tracker responsible for all the videos uploaded in it. + * Even if nobody watches a video, it is seeded by the server (through + [WebSeed protocol](http://www.bittorrent.org/beps/bep_0019.html)) where the + video was uploaded. Here are some simple schemes: