Merge pull request #196 from simplex-chat/v1

v1

README.md

@@ -2,7 +2,7 @@
 
 # SimpleX Chat
 
-**The world's most private and secure chat** - open-source, decentralized, and without global identities of any kind.
+SimpleX - the most private and secure open-source chat and applications platform - now with double-ratchet E2E encryption.
 
 [![GitHub build](https://github.com/simplex-chat/simplex-chat/workflows/build/badge.svg)](https://github.com/simplex-chat/simplex-chat/actions?query=workflow%3Abuild)
 [![GitHub downloads](https://img.shields.io/github/downloads/simplex-chat/simplex-chat/total)](https://github.com/simplex-chat/simplex-chat/releases)
@@ -10,13 +10,11 @@
 [![Follow on Twitter](https://img.shields.io/twitter/follow/SimpleXChat?style=social)](https://twitter.com/simplexchat)
 [![Join on Reddit](https://img.shields.io/reddit/subreddit-subscribers/SimpleXChat?style=social)](https://www.reddit.com/r/SimpleXChat)
 
-SimpleX chat prototype is a thin terminal UI on top of [SimpleXMQ](https://github.com/simplex-chat/simplexmq) message broker that uses [SMP protocols](https://github.com/simplex-chat/simplexmq/blob/master/protocol). The motivation for SimpleX chat is [presented here](./simplex.md). See [simplex.chat](https://simplex.chat) website for chat demo and the explanations of the system and how SMP protocol works.
+SimpleX Chat is a terminal (command line) UI using [SimpleXMQ](https://github.com/simplex-chat/simplexmq) message broker.
 
-**NEW in v0.5.4: [messages persistence](#access-chat-history)**
+See [SimpleX overview](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md) for more information on platform objectives and technical design.
 
-**NEW in v0.5.0: [user contact addresses](#user-contact-addresses-alpha)**
-
-**Please note**: v0.5.0 of SimpleX Chat works with the same database, but the connection links are not compatible with the prior versions - please ask all your contacts to upgrade!
+**v1.0.0 is released: [read announcement here](https://github.com/simplex-chat/simplex-chat/blob/master/blog/20220112-simplex-chat-v1-released.md)**
 
 ### :zap: Quick installation
 
@@ -26,22 +24,6 @@ curl -o- https://raw.githubusercontent.com/simplex-chat/simplex-chat/master/inst
 
 Once the chat client is installed, simply run `simplex-chat` from your terminal.
 
-### :wave: Welcome
-
-**We are building the world's most private and secure chat**. If you would like to support it, you can do so in the following ways:
-
-- 🌟  **Star it on GitHub** - it helps us raise the visibility of the project.
-
-- **Install the chat and try it out** - if you spot a bug, please [raise an issue](https://github.com/simplex-chat/simplex-chat/issues).
-
-- :speech_balloon:  **Spread the word** - terminal chat is an [early-stage product](#disclaimer) while we stabilize the protocol - you can invite your friends for some fun chat inside your terminal. We're using it right inside our IDEs as we are coding it 👨‍💻
-
-- **Make a donation** via [opencollective](https://opencollective.com/simplex-chat) - every donation helps, however large or small!
-
-- **Make a contribution to the project** - we're constantly moving the project forward and there are always lots of things to do!
-
-We appreciate all the help from our contributors, thank you!
-
 ![simplex-chat](./images/connection.gif)
 
 ## Table of contents
@@ -52,7 +34,6 @@ We appreciate all the help from our contributors, thank you!
 - [Installation](#🚀-installation)
   - [Download chat client](#download-chat-client)
     - [Linux and MacOS](#linux-and-macos)
-      - [Troubleshooting on Unix](#troubleshooting-on-unix)
     - [Windows](#windows)
   - [Build from source](#build-from-source)
     - [Using Docker](#using-docker)
@@ -62,26 +43,26 @@ We appreciate all the help from our contributors, thank you!
   - [How to use SimpleX chat](#how-to-use-simplex-chat)
   - [Groups](#groups)
   - [Sending files](#sending-files)
-  - [User contact addresses](#user-contact-addresses-alpha)
+  - [User contact addresses](#user-contact-addresses)
   - [Access chat history](#access-chat-history)
 - [Roadmap](#Roadmap)
 - [License](#license)
 
 ## Disclaimer
 
-This is WIP implementation of SimpleX Chat that implements a new network topology for asynchronous communication combining the advantages and avoiding the disadvantages of federated and P2P networks.
+SimpleX Chat implements a new network topology for asynchronous communication combining the advantages and avoiding the disadvantages of federated and P2P networks.
 
-If you expect software to be reliable most of the time, then this is probably not ready for you yet. We use it ourselves for terminal chat and it seems to work most of the time - we would really appreciate if you try SimpleX Chat and give us your feedback!
+[SimpleXMQ security model](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md) had many improvements in v1.0.0; the implementation has not been audited yet.
 
-> :warning: **Please note:** The main differentiation of SimpleX network is the approach to internet message routing rather than encryption; for that reason no sufficient attention was paid to either TCP transport level encryption or to E2E encryption protocols - they are implemented in an ad hoc way based on RSA and AES algorithms. See [SMP protocol](https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#appendix-a) on TCP transport encryption protocol (AEAD-GCM scheme, with an AES key negotiation based on RSA key hash known to the client in advance) and [this section](https://github.com/simplex-chat/simplexmq/blob/master/rfcs/2021-01-26-crypto.md#e2e-encryption) on E2E encryption protocol (an ad hoc hybrid scheme a la PGP). These protocols will change in a consumer ready version to something more robust.
+We use SimpleX Chat all the time, but you may find some bugs. We would really appreciate if you use it and let us know anything that needs to be fixed or improved.
 
 ## Network topology
 
-SimpleX is a decentralized client-server network that uses redundant, disposable nodes to asynchronously pass messages via message queues, providing receiver and sender anonymity.
+SimpleX is a client-server network that uses redundant, disposable nodes to asynchronously pass messages via message queues, providing receiver and sender anonymity.
 
 Unlike P2P networks, all messages are passed through one or several (for redundancy) servers, that do not even need to have persistence (in fact, the current [SMP server implementation](https://github.com/simplex-chat/simplexmq#smp-server) uses in-memory message storage, persisting only the queue records) - it provides better metadata protection than P2P designs, as no global participant ID is required, and avoids many [problems of P2P networks](https://github.com/simplex-chat/simplex-chat/blob/master/simplex.md#comparison-with-p2p-messaging-protocols).
 
-Unlike federated networks, the participating server nodes **do not have records of the users**, **do not communicate with each other**, **do not store messages** after they are delivered to the recipients, and there is no way to discover the full list of participating servers. SimpleX network avoids the problem of metadata visibility that federated networks suffer from and better protects the network, as servers do not communicate with each other. Each server node provides unidirectional "dumb pipes" to the users, that do authorization without authentication, having no knowledge of the the users or their contacts. Each queue is assigned two RSA keys - one for receiver and one for sender - and each queue access is authorized with a signature created using a respective key's private counterpart.
+Unlike federated networks, the participating server nodes **do not have records of the users**, **do not communicate with each other**, **do not store messages** after they are delivered to the recipients, and there is no way to discover the full list of participating servers. SimpleX network avoids the problem of metadata visibility that federated networks have and better protects the network, as servers do not communicate with each other. Each server node provides unidirectional "dumb pipes" to the users, that do authorization without authentication, having no knowledge of the the users or their contacts. Each queue is assigned two Ed448 keys - one for receiver and one for sender - and each queue access is authorized with a signature created using a respective key's private counterpart.
 
 The routing of messages relies on the knowledge of client devices how user contacts and groups map at any given moment of time to these disposable queues on server nodes.
 
@@ -95,13 +76,15 @@ The routing of messages relies on the knowledge of client devices how user conta
 - Auto-populated recipient name - just type your messages to reply to the sender once the connection is established.
 - Demo SMP servers available and pre-configured in the app - or you can [deploy your own server](https://github.com/simplex-chat/simplexmq#using-smp-server-and-smp-agent).
 - No global identity or any names visible to the server(s), ensuring full privacy of your contacts and conversations.
-- E2E encryption, with RSA public key that has to be passed out-of-band (see [How to use SimpleX chat](#how-to-use-simplex-chat)).
-- Message signing and verification with automatically generated RSA keys.
+- Two layers of E2E encryption (double-ratchet for duplex connections, using X3DH key agreement with ephemeral Curve448 keys, and NaCl crypto_box for SMP queues, using Curve25519 keys) and out-of-band passing of recipient keys (see [How to use SimpleX chat](#how-to-use-simplex-chat)).
 - Message integrity validation (via including the digests of the previous messages).
-- Authentication of each command/message by SMP servers with automatically generated RSA key pairs.
-- TCP transport encryption using SMP transport protocol.
+- Authentication of each command/message by SMP servers with automatically generated Ed448 keys.
+- TLS 1.2 transport encryption.
+- Additional encryption of messages from SMP server to recipient to reduce traffic correlation.
 
-RSA keys are not used as identity, they are randomly generated for each contact.
+Public keys involved in key exchange are not used as identity, they are randomly generated for each contact.
+
+See [Encryption Primitives Used](https://github.com/simplex-chat/simplexmq/blob/master/protocol/overview-tjr.md#encryption-primitives-used) for technical details.
 
 <a name="🚀-installation"></a>
 
@@ -134,30 +117,6 @@ mv <binary> ~/.local/bin/simplex-chat
 
 On MacOS you also need to [allow Gatekeeper to run it](https://support.apple.com/en-us/HT202491).
 
-##### Troubleshooting on Unix
-
-If you get `simplex-chat: command not found` when executing the downloaded binary, you need to add the directory containing it to the [`PATH` variable](https://man7.org/linux/man-pages/man7/environ.7.html) (find "PATH" in page). To modify `PATH` for future sessions, put `PATH="$PATH:/path/to/dir"` in `~/.profile`, or in `~/.bash_profile` if that's what you have. See [this answer](https://unix.stackexchange.com/a/26059) for the detailed explanation on the appropriate place to define environment variables for `bash` and other shells.
-
-For example, if you followed the previous instructions, open `~/.profile` for editing:
-
-```sh
-vi ~/.profile
-```
-
-And add the following line to the end:
-
-```sh
-PATH="$PATH:$HOME/.local/bin"
-```
-
-Note that this will not automatically update your `PATH` for the remainder of the session. To do this, you should run:
-
-```sh
-source ~/.profile
-```
-
-Or restart your terminal to start a new session.
-
 #### Windows
 
 ```sh
@@ -198,9 +157,9 @@ $ stack install
 
 ### Running the chat client
 
-To start the chat client, run `simplex-chat` from the terminal. If you get `simplex-chat: command not found`, see [Troubleshooting on Unix](#troubleshooting-on-unix).
+To start the chat client, run `simplex-chat` from the terminal.
 
-By default, app data directory is created in the home directory (`~/.simplex`, or `%APPDATA%/simplex` on Windows), and two SQLite database files `simplex.chat.db` and `simplex.agent.db` are initialized in it.
+By default, app data directory is created in the home directory (`~/.simplex`, or `%APPDATA%/simplex` on Windows), and two SQLite database files `simplex_v1_chat.db` and `simplex_v1_agent.db` are initialized in it.
 
 To specify a different file path prefix for the database files use `-d` command line option:
 
@@ -208,17 +167,17 @@ To specify a different file path prefix for the database files use `-d` command
 $ simplex-chat -d alice
 ```
 
-Running above, for example, would create `alice.chat.db` and `alice.agent.db` database files in current directory.
+Running above, for example, would create `alice_v1_chat.db` and `alice_v1_agent.db` database files in current directory.
 
-Default SMP servers are hosted on Linode (London, UK and Fremont, CA) - they are [pre-configured in the app](https://github.com/simplex-chat/simplex-chat/blob/master/src/Simplex/Chat/Options.hs#L40). Base-64 encoded string after server host is the transport key digest.
+Three default SMP servers are hosted on Linode - they are [pre-configured in the app](https://github.com/simplex-chat/simplex-chat/blob/master/src/Simplex/Chat/Options.hs#L42).
 
 If you deployed your own SMP server(s) you can configure client via `-s` option:
 
 ```shell
-$ simplex-chat -s smp.example.com:5223#KXNE1m2E1m0lm92WGKet9CL6+lO742Vy5G6nsrkvgs8=
+$ simplex-chat -s smp://LcJUMfVhwD8yxjAiSaDzzGF3-kLG4Uh0Fl_ZIjrRwjI=@smp.example.com
 ```
 
-The base-64 encoded string in server address is the digest of RSA transport handshake key that the server will generate on the first run and output its digest.
+Base64url encoded string preceding the server address is the server's online certificate fingerprint. It is generated on server initialization and validated by client during TLS handshake.
 
 You can still talk to people using default or any other server - it only affects the location of the message queue when you initiate the connection (and the reply queue can be on another server, as set by the other party's client).
 
@@ -238,7 +197,7 @@ Once you've set up your local profile, enter `/c` (for `/connect`) to create a n
 
 You are able to create multiple invitations by entering `/connect` multiple times and sending these invitations to the corresponding contacts you'd like to connect with.
 
-The invitation has the format `smp::<server>::<queue_id>::<rsa_public_key_for_this_queue_only>`. The invitation can only be used once and even if this is intercepted, the attacker would not be able to use it to send you the messages via this queue once your contact confirms that the connection is established.
+The invitation can only be used once and even if this is intercepted, the attacker would not be able to use it to send you the messages via this queue once your contact confirms that the connection is established. See agent protocol for explanation of [invitation format](https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md#connection-request).
 
 The contact who received the invitation should enter `/c <invitation>` to accept the connection. This establishes the connection, and both parties are notified.
 
@@ -262,7 +221,7 @@ You can send a file to your contact with `/f @<contact> <file_path>` - the recip
 
 You can send files to a group with `/f #<group> <file_path>`.
 
-### User contact addresses (alpha)
+### User contact addresses
 
 As an alternative to one-time invitation links, you can create a long-term address with `/ad` (for `/address`). The created address can then be shared via any channel, and used by other users as a link to make a contact request with `/c <user_contact_address>`.
 
@@ -272,47 +231,28 @@ User address is "long-term" in a sense that it is a multiple-use connection link
 
 Use `/help address` for other commands.
 
-> :warning: **Please note:** This is an "alpha" feature - at the moment there is nothing to prevent someone who has obtained this address from spamming you with connection requests; countermeasures will be added soon!  (In the short term, you can simply delete the long-term address you created if it starts getting abused.)
-
 ![simplex-chat](./images/user-addresses.gif)
 
 ### Access chat history
 
 SimpleX chat stores all your contacts and conversations in a local SQLite database, making it private and portable by design, owned and controlled by user.
 
-You can view and search your chat history by querying your database:
+You can view and search your chat history by querying your database. Run the below script to create message views in your database.
 
-```
-sqlite3 ~/.simplex/simplex.chat.db
+```sh
+curl -o- https://raw.githubusercontent.com/simplex-chat/simplex-chat/master/message_views.sql | sqlite3 ~/.simplex/simplex_v1_chat.db
 ```
 
-Now you can run queries against `direct_messages`, `group_messages` and `all_messages` (or their simpler alternatives `direct_messages_plain`, `group_messages_plain` and `all_messages_plain`), for example:
+Open SQLite Command Line Shell:
 
-```sql
--- you can put these or your preferred settings into ~/.sqliterc to persist across sqlite3 client sessions
-.mode column
-.headers on
-
--- simple views into direct, group and all_messages with user's messages deduplicated for group and all_messages
--- only 'x.msg.new' ("new message") chat events - filters out service events
--- msg_sent is 0 for received, 1 for sent
-select * from direct_messages_plain;
-select * from group_messages_plain;
-select * from all_messages_plain;
-
--- query other details of your chat history with regular SQL
-select * from direct_messages where msg_sent = 1 and chat_msg_event = 'x.file'; -- files you offered for sending
-select * from direct_messages where msg_sent = 0 and contact = 'catherine' and msg_body like '%cats%'; -- everything catherine sent related to cats
-select * from group_messages where group_name = 'team' and contact = 'alice'; -- all correspondence with alice in #team
-
--- aggregate your chat data
-select contact_or_group, num_messages from (
-  select contact as contact_or_group, count(1) as num_messages from direct_messages_plain group by contact
-  union
-  select group_name as contact_or_group, count(1) as num_messages from group_messages_plain group by group_name
-) order by num_messages desc;
+```sh
+sqlite3 ~/.simplex/simplex_v1_chat.db
 ```
 
+See [Message queries](./message_queries.md) for examples.
+
+> **Please note:** SQLite foreign key constraints are disabled by default, and must be **[enabled separately for each database connection](https://sqlite.org/foreignkeys.html#fk_enable)**. The latter can be achieved by running `PRAGMA foreign_keys = ON;` command on an open database connection. By running data altering queries without enabling foreign keys prior to that, you may risk putting your database in an inconsistent state.
+
 **Convenience queries**
 
 Get all messages from today (`chat_dt` is in UTC):
@@ -327,8 +267,6 @@ Get overnight messages in the morning:
 select * from all_messages_plain where chat_dt > datetime('now', '-15 hours') order by chat_dt;
 ```
 
-> **Please note:** SQLite foreign key constraints are disabled by default, and must be **[enabled separately for each database connection](https://sqlite.org/foreignkeys.html#fk_enable)**. The latter can be achieved by running `PRAGMA foreign_keys = ON;` command on an open database connection. By running data altering queries without enabling foreign keys prior to that, you may risk putting your database in an inconsistent state.
-
 ## Roadmap
 
 1. Mobile and desktop apps (in progress).

apps/simplex-chat/Main.hs

@@ -22,6 +22,6 @@ welcomeGetOpts = do
   appDir <- getAppUserDataDirectory "simplex"
   opts@ChatOpts {dbFile} <- getChatOpts appDir
   putStrLn $ "SimpleX Chat v" ++ versionNumber
-  putStrLn $ "db: " <> dbFile <> ".chat.db, " <> dbFile <> ".agent.db"
+  putStrLn $ "db: " <> dbFile <> "_chat.db, " <> dbFile <> "_agent.db"
   putStrLn "type \"/help\" or \"/h\" for usage info"
   pure opts

images/simplex-chat-logo.svg

@@ -1,5 +1,6 @@
 <svg width="830" height="212" viewBox="0 0 830 212" fill="none" xmlns="http://www.w3.org/2000/svg">
-<g clip-path="url(#clip0_102_125)">
+<g clip-path="url(#clip0_203_128)">
+<rect width="830" height="212" fill="white"/>
 <path fill-rule="evenodd" clip-rule="evenodd" d="M166.17 130.508C167.62 130.508 169.051 130.643 170.462 130.912C171.873 131.182 173.226 131.528 174.519 131.952C175.813 132.375 177.018 132.856 178.136 133.395C179.253 133.934 180.282 134.473 181.223 135.012L176.989 142.863L176.931 142.799C176.764 142.635 176.372 142.368 175.754 141.997C175.049 141.574 174.167 141.131 173.108 140.669C172.05 140.208 170.893 139.803 169.639 139.457C168.385 139.111 167.13 138.937 165.876 138.937C162.426 138.937 160.701 140.073 160.701 142.344C160.701 143.037 160.887 143.614 161.26 144.076C161.632 144.538 162.181 144.951 162.906 145.317C163.632 145.683 164.543 146.019 165.641 146.327C166.738 146.635 168.012 146.982 169.463 147.367C171.462 147.905 173.265 148.492 174.872 149.127C176.479 149.763 177.842 150.552 178.959 151.495C180.076 152.438 180.938 153.583 181.546 154.93C182.154 156.277 182.457 157.893 182.457 159.779C182.457 162.089 182.016 164.042 181.134 165.639C180.252 167.237 179.086 168.526 177.636 169.508C176.185 170.489 174.519 171.201 172.638 171.644C170.756 172.086 168.816 172.308 166.817 172.308C165.288 172.308 163.72 172.192 162.113 171.961C160.505 171.73 158.937 171.394 157.408 170.951C155.88 170.508 154.4 169.979 152.969 169.363C151.538 168.747 150.215 168.035 149 167.227L153.234 158.971L153.309 159.051C153.523 159.258 154.008 159.597 154.762 160.068C155.625 160.607 156.693 161.146 157.967 161.685C159.241 162.223 160.662 162.705 162.23 163.128C163.798 163.551 165.386 163.763 166.993 163.763C170.403 163.763 172.109 162.743 172.109 160.703C172.109 159.933 171.854 159.298 171.344 158.798C170.835 158.298 170.129 157.845 169.227 157.441C168.326 157.037 167.258 156.662 166.023 156.315C164.788 155.969 163.445 155.584 161.995 155.161C160.074 154.583 158.408 153.958 156.997 153.284C155.586 152.611 154.419 151.831 153.498 150.946C152.577 150.061 151.891 149.041 151.44 147.886C150.989 146.732 150.764 145.384 150.764 143.845C150.764 141.689 151.176 139.784 151.999 138.129C152.822 136.474 153.939 135.079 155.35 133.943C156.762 132.808 158.398 131.952 160.26 131.374C162.122 130.797 164.092 130.508 166.17 130.508ZM204.642 130.855V171.846H194.999V130.855H204.642ZM230.002 130.855L240.821 152.274L251.7 130.855H262.107V171.846H252.464V147.309L243.409 165.149H238.234L229.179 147.309V171.846H219.536V130.855H230.002ZM326.725 130.855V163.532H346.952V171.846H317.082V130.855H326.725ZM386.039 130.855V139.168H366.87V147.136H383.335V154.814H366.87V163.532H386.569V171.846H357.227V130.855H386.039ZM293.668 130.855L294.328 130.87C296.072 130.949 297.685 131.349 299.166 132.067C300.832 132.875 302.262 133.934 303.458 135.242C304.654 136.551 305.594 138.033 306.281 139.688C306.967 141.343 307.31 143.017 307.31 144.711C307.31 146.52 306.986 148.252 306.339 149.907C305.692 151.562 304.791 153.044 303.634 154.352C302.478 155.661 301.077 156.7 299.43 157.47C297.967 158.154 296.364 158.534 294.622 158.61L293.962 158.625H285.495V171.846H275.851V130.855H293.668ZM293.021 139.168H285.495V150.369H293.374C294.511 150.369 295.491 149.888 296.314 148.925C297.137 147.963 297.549 146.558 297.549 144.711C297.549 143.749 297.421 142.921 297.166 142.228C296.912 141.535 296.569 140.958 296.137 140.496C295.706 140.034 295.216 139.698 294.667 139.486C294.119 139.274 293.57 139.168 293.021 139.168Z" fill="#062D56"/>
 <path fill-rule="evenodd" clip-rule="evenodd" d="M420.98 159.789L436.927 175.75L445.055 167.615L445.052 167.613L461.309 151.345L445.361 135.386L445.364 135.385L437.39 127.405L421.134 143.674L421.133 143.673L437.389 127.405L421.444 111.444L429.571 103.309L445.517 119.27L461.774 103L469.747 110.98L453.49 127.25L469.436 143.21L485.693 126.941L493.667 134.921L477.409 151.19L493.358 167.152L485.23 175.286L469.282 159.325L453.026 175.594L468.975 191.556L460.847 199.691L444.899 183.729L428.645 200L420.672 192.02L436.926 175.749L420.978 159.79L420.98 159.789Z" fill="#07B4B9"/>
 <path fill-rule="evenodd" clip-rule="evenodd" d="M404.957 127.482L420.902 143.442L437.159 127.173L445.132 135.153L428.876 151.422L444.823 167.383L436.696 175.518L420.748 159.557L404.493 175.827L396.52 167.847L412.775 151.577L396.829 135.617L404.957 127.482Z" fill="#062D56"/>
@@ -7,27 +8,23 @@
 <path d="M588.156 130.777V172.106H576.817V155.924H562.976V172.106H551.637V130.777H562.976V146.028H576.817V130.777H588.156Z" fill="#FFC513"/>
 <path d="M614.092 130.777H626.769L639.912 172.106H628.398L625.897 164.015H614.906L612.464 172.106H600.892L614.092 130.777ZM624.094 156.448L620.431 141.895L616.593 156.448H624.094Z" fill="#FFC513"/>
 <path d="M681 140.673H669.137V172.106H657.797V140.673H645.876V130.777H681V140.673Z" fill="#FFC513"/>
-<path d="M74.2686 88.7447C74.3977 89.2443 74.5471 89.7263 74.7166 90.1906C74.8948 90.6404 75.1516 91.2743 75.487 92.0924C76.077 93.4644 76.4817 94.5744 76.7009 95.4226C76.7459 95.5969 76.7381 95.7104 76.6774 95.7633C76.6167 95.8162 76.5398 95.8546 76.4469 95.8786C76.3539 95.9027 76.29 95.9192 76.2552 95.9282C76.0925 95.9702 75.9602 95.9858 75.8583 95.975C75.7533 95.9525 75.6508 95.8675 75.5506 95.7198L73.9596 91.0742C73.8017 90.6068 73.5075 89.6603 73.0771 88.2347C72.7276 88.9447 72.3479 89.5138 71.938 89.942C71.5251 90.3585 71.0631 90.6329 70.5519 90.765C69.8199 90.9541 68.9544 90.6263 67.9552 89.7814C68.0351 89.9467 68.1125 90.1746 68.1876 90.465C68.2627 90.7555 68.3062 90.9239 68.3182 90.9704L69.7735 96.6737C69.8036 96.7899 69.7637 96.8993 69.6539 97.0021C69.5527 97.0902 69.4382 97.1508 69.3104 97.1838C69.1478 97.2258 69.0166 97.1977 68.9169 97.0996C68.8257 96.9868 68.7357 96.8303 68.6468 96.6302L68.0163 94.1904C67.9202 93.8186 67.8237 93.3973 67.7269 92.9266C67.63 92.4559 67.5634 92.126 67.5269 91.9371C67.3749 91.1089 67.2132 90.3636 67.0421 89.7013C66.991 89.5038 66.9082 89.3269 66.7934 89.1707C66.6757 89.0028 66.5185 88.8265 66.3221 88.6418C66.1256 88.4571 65.9715 88.2924 65.8598 88.1478C65.7596 88.0001 65.6855 87.8333 65.6375 87.6475C65.5805 87.4267 65.6623 87.2878 65.883 87.2308C66.2316 87.1407 66.5493 87.1949 66.8361 87.3935C67.1199 87.5804 67.4541 87.8906 67.8388 88.3242C68.1647 88.6738 68.439 88.9437 68.6617 89.1341C68.8844 89.3244 69.146 89.4737 69.4467 89.5819C69.7443 89.6785 70.0732 89.6802 70.4334 89.5872C70.9622 89.4753 71.3666 89.2654 71.6465 88.9576C71.9233 88.6382 72.1069 88.2933 72.1971 87.9229C72.2873 87.5526 72.3537 87.114 72.3962 86.6072C72.4366 86.1878 72.4895 85.8891 72.555 85.711C72.6175 85.5213 72.7288 85.4244 72.8889 85.4202C73.2314 85.3069 73.4883 85.5814 73.6594 86.2436C73.8035 86.8013 73.9043 87.2152 73.9618 87.4855L74.2686 88.7447ZM82.1485 94.3866C81.463 94.5637 80.8024 94.5733 80.1667 94.4153C79.5395 94.2428 78.9986 93.924 78.5438 93.459C78.0976 92.9794 77.7829 92.3852 77.5997 91.6765C77.2995 90.5147 77.4998 89.5395 78.2009 88.751C78.9135 87.9596 79.8914 87.4032 81.1345 87.0819C81.7038 86.9348 82.2685 86.9376 82.8285 87.0903C83.4002 87.24 83.9046 87.5373 84.3418 87.982C84.779 88.4268 85.0861 88.9919 85.2633 89.6774C85.4555 90.421 85.4319 91.1211 85.1926 91.7779C84.9503 92.423 84.5545 92.9777 84.0052 93.4419C83.4645 93.8915 82.8456 94.2064 82.1485 94.3866ZM81.7001 93.4427C82.7806 93.1635 83.5366 92.7078 83.9682 92.0757C84.3998 91.4436 84.4939 90.6571 84.2507 89.716C84.0856 89.077 83.7222 88.5822 83.1606 88.2315C82.6107 87.8779 81.9465 87.8016 81.168 88.0028C80.6452 88.1379 80.1421 88.3733 79.6587 88.7089C79.1753 89.0446 78.8104 89.4549 78.5639 89.94C78.3174 90.4251 78.2663 90.9465 78.4104 91.5042C78.5305 91.9689 78.7593 92.3746 79.0967 92.7212C79.4312 93.0561 79.8274 93.2946 80.2855 93.4365C80.7522 93.5638 81.2237 93.5658 81.7001 93.4427ZM91.7555 91.9038C90.8261 92.144 90.0476 92.1531 89.42 91.931C88.7895 91.6973 88.2504 91.2418 87.8026 90.5642C87.3518 89.8751 86.9222 88.9079 86.5135 87.6625L85.247 84.8479C85.0894 84.43 84.9174 84.0282 84.731 83.6426C84.5446 83.257 84.4289 82.977 84.3838 82.8028C84.3208 82.5588 84.344 82.3607 84.4533 82.2085C84.5597 82.0446 84.787 81.893 85.1352 81.7534C85.355 82.3163 85.5141 82.74 85.6124 83.0244C86.0314 84.1184 86.3597 84.9569 86.5971 85.54C86.8432 86.1086 87.12 86.7001 87.4276 87.3147C88.2201 86.6885 89.0288 86.2688 89.8537 86.0556C90.5973 85.8634 91.3001 85.8491 91.9621 86.0127C92.621 86.1647 93.1853 86.4774 93.6547 86.951C94.1211 87.413 94.4429 87.9868 94.6201 88.6722C94.8272 89.4739 94.626 90.158 94.0164 90.7246C93.4154 91.2765 92.6617 91.6696 91.7555 91.9038ZM93.6531 88.9593C93.6437 88.875 93.6195 88.7573 93.5805 88.6063C93.3673 87.7814 92.9235 87.2392 92.2491 86.9796C91.5864 86.7171 90.738 86.7195 89.7039 86.9867C89.0649 87.1518 88.5984 87.4088 88.3043 87.7574C88.0188 88.0915 87.9316 88.4735 88.0427 88.9033C88.2979 89.8909 88.7513 90.5422 89.4028 90.8572C90.066 91.1692 90.9552 91.181 92.0706 90.8928C92.605 90.7547 93.0208 90.5171 93.3179 90.18C93.615 89.843 93.7268 89.4361 93.6531 88.9593ZM97.4424 90.4341C97.1752 90.5031 96.8597 90.1695 96.4959 89.4331C96.129 88.6851 95.8488 88.0324 95.6552 87.4751C95.4616 86.9179 95.165 86.034 94.7654 84.8234C94.7028 84.6289 94.6309 84.3748 94.5498 84.0611C94.4477 83.6661 94.4062 83.3856 94.4253 83.2195C94.456 83.0505 94.5468 82.9464 94.6979 82.9074C94.8257 82.8744 94.9319 82.8779 95.0166 82.918C95.11 82.9434 95.2007 83.0067 95.2888 83.1079L95.4516 83.8095C95.6437 84.5531 95.8602 85.247 96.1011 85.8912C96.3536 86.5324 96.7052 87.3897 97.1561 88.463C97.3752 88.927 97.5427 89.3113 97.6586 89.616C97.7239 89.7726 97.7369 89.9428 97.6976 90.1265C97.6553 90.2985 97.5702 90.4011 97.4424 90.4341ZM94.5296 80.9616C94.274 81.0276 94.0568 80.9784 93.878 80.814C93.7107 80.6465 93.5896 80.4175 93.5145 80.127C93.4815 79.9992 93.4633 79.9048 93.4599 79.8437C93.4002 79.6608 93.4808 79.5408 93.7015 79.4838C93.8642 79Line truncated
-<path d="M707.823 89.2408C708.791 89.4194 709.565 89.422 710.146 89.2486C710.727 89.0752 711.066 88.7289 711.162 88.2096C711.247 87.7494 711.16 87.3247 710.903 86.9355C710.647 86.5344 710.305 86.1723 709.877 85.8492C709.448 85.5261 708.856 85.13 708.101 84.6611C708.035 84.6245 707.875 84.528 707.624 84.3717C706.597 83.7185 705.824 83.1122 705.305 82.5527C704.798 81.9953 704.61 81.3626 704.74 80.6546C704.886 79.8639 705.309 79.3257 706.009 79.04C706.708 78.7542 707.548 78.7017 708.527 78.8825C709.247 79.0153 709.89 79.2317 710.457 79.5316C711.038 79.8219 711.476 80.159 711.772 80.543C712.079 80.9291 712.195 81.3228 712.121 81.724C711.966 81.7687 711.824 81.779 711.694 81.7551C711.553 81.7289 711.411 81.6722 711.268 81.5849C711.139 81.4879 710.948 81.3366 710.693 81.1309C710.278 80.7982 709.884 80.524 709.509 80.3083C709.136 80.0808 708.678 79.917 708.135 79.8168C707.45 79.6905 706.874 79.7061 706.406 79.8637C705.937 80.0213 705.664 80.3125 705.585 80.7374C705.524 81.0678 705.611 81.3948 705.844 81.7185C706.089 82.0444 706.41 82.3538 706.807 82.6468C707.219 82.9302 707.77 83.2821 708.461 83.7026C709.338 84.2426 710.053 84.7286 710.608 85.1605C711.163 85.5924 711.607 86.0953 711.94 86.6694C712.288 87.2338 712.399 87.8523 712.275 88.525C712.129 89.3156 711.638 89.8595 710.802 90.1568C709.969 90.4422 708.991 90.4814 707.87 90.2745C706.985 90.1111 706.136 89.8202 705.322 89.4015C704.509 88.9829 703.842 88.4633 703.322 87.8426C702.804 87.2101 702.529 86.5187 702.497 85.7683C702.556 85.4497 702.694 85.1946 702.913 85.0031C703.133 84.7997 703.373 84.722 703.633 84.77C703.491 85.537 703.583 86.2311 703.908 86.8524C704.246 87.464 704.756 87.973 705.438 88.3795C706.131 88.7881 706.926 89.0752 707.823 89.2408ZM716.135 91.4706C716.013 91.4724 715.934 91.47 715.899 91.4635C715.533 91.396 715.329 91.0777 715.288 90.5088C715.258 89.942 715.334 89.1689 715.515 88.1894C715.589 87.7882 715.716 87.1649 715.896 86.3196C716.016 85.8048 716.099 85.4175 716.147 85.1579C714.696 84.89 714.001 84.5908 714.062 84.2604C714.081 84.1542 714.108 84.0738 714.143 84.0191C714.189 83.9667 714.265 83.9502 714.372 83.9698L716.324 84.202C716.419 83.6216 716.506 82.9848 716.585 82.2917C716.651 81.6694 716.726 81.0975 716.81 80.5761C716.834 80.4463 716.903 80.3676 717.018 80.34C717.135 80.3006 717.253 80.2917 717.371 80.3135C717.513 80.3397 717.616 80.4382 717.683 80.609C717.751 80.768 717.771 80.9242 717.742 81.0776L717.268 84.3398L718.725 84.6818C718.953 84.6996 719.134 84.7757 719.268 84.9102C719.414 85.0469 719.47 85.2403 719.436 85.4903C719.168 85.5507 718.885 85.5656 718.585 85.5347C718.286 85.5038 717.788 85.4242 717.092 85.2957C716.889 86.3931 716.609 88.2084 716.252 90.7416C716.204 91.0012 716.17 91.1841 716.15 91.2903C716.145 91.3869 716.14 91.447 716.135 91.4706ZM723.781 92.8818C722.554 92.6553 721.525 92.2457 720.695 91.6532C719.879 91.051 719.571 90.2071 719.771 89.1215C719.843 88.732 720.031 88.3762 720.334 88.0539C720.638 87.7317 721.04 87.5009 721.542 87.3617C722.046 87.2106 722.628 87.1961 723.289 87.318C723.796 87.4117 724.223 87.5148 724.568 87.6273C724.913 87.7399 725.296 87.9448 725.717 88.2422C726.138 88.5396 726.554 88.9641 726.965 89.5158C726.926 88.6667 726.775 87.9005 726.511 87.2172C726.246 86.5338 725.909 85.9834 725.498 85.5658C725.1 85.1387 724.678 84.8837 724.229 84.8009C724.017 84.7617 723.809 84.766 723.605 84.8138C723.414 84.8519 723.177 84.9178 722.891 85.0115C722.561 85.1458 722.325 85.2 722.184 85.1738C722.125 85.1629 722.075 85.1354 722.034 85.0913C722.007 85.0376 722 84.9812 722.01 84.9222C722.098 84.4502 722.372 84.1225 722.833 83.9391C723.305 83.7579 723.84 83.7346 724.438 83.8693C725.255 84.069 725.945 84.5624 726.508 85.3496C727.072 86.1249 727.46 87.0324 727.671 88.0719C727.893 89.1136 727.919 90.1006 727.747 91.0329C727.718 91.1863 727.666 91.4024 727.59 91.6813L727.03 91.5413C726.75 90.5499 726.29 89.8 725.652 89.2917C725.027 88.7738 724.361 88.4495 723.653 88.3188C722.745 88.1511 722.07 88.167 721.631 88.3665C721.191 88.5661 720.924 88.9254 720.828 89.4447C720.728 89.9875 720.919 90.4742 721.4 90.9048C721.882 91.3355 722.548 91.6292 723.398 91.786C723.87 91.8731 724.252 91.9192 724.544 91.9243C724.838 91.9176 725.147 91.8953 725.471 91.8575C725.734 91.8207 725.963 91.802 726.159 91.8014C726.354 91.8009 726.599 91.8278 726.894 91.8823C727.225 91.9433 727.427 92.005 727.501 92.0674C727.586 92.1321 727.614 92.247 727.583 92.4122C727.562 92.5302 727.293 92.6331 726.776 92.7208C726.26 92.8085 725.687 92.8676 725.059 92.8981C724.431 92.9286 724.005 92.9232 723.781 92.8818ZM730.349 94.0941C730.039 93.0607 729.86 91.8807 729.812 90.5539C729.776 89.2294 729.844 88.1009 730.016 87.1687C730.084 86.8029 730.21 86.5821 730.395 86.5064C730.5 86.4648 730.576 86.4484 730.624 86.4571C730.706 86.4723 730.753 86.5481 730.765 86.6845C730.788 86.823 730.792 87.0677 730.776 87.4187L730.746 88.0722C730.994 87.4589 731.334 87.0032 731.768 86.7049C732.215 86.3969 732.71 86.2931 733.253 86.3933C733.855 86.5044 734.281 86.7051 Line truncated
-<g clip-path="url(#clip1_102_125)">
-<path d="M50.2387 113.916C51.5671 118.787 52.9042 123.656 54.217 128.532C54.4939 129.532 54.7099 130.548 54.8637 131.573C54.9677 132.185 54.9384 132.81 54.7782 133.405C54.2694 135.013 53.218 136.484 51.6927 137.272C50.4728 137.931 49.1963 138.477 47.8783 138.902C43.4092 140.248 38.9148 141.51 34.4204 142.77C33.5125 143.025 32.5645 143.14 31.6317 143.302C29.0311 143.753 27.2459 142.42 25.9291 140.347C25.2721 139.313 24.8719 138.104 24.402 136.955C22.76 132.941 21.7331 128.738 20.6475 124.554C19.7529 121.106 18.8513 117.66 17.9428 114.216C16.8974 110.271 15.7934 106.341 14.8014 102.383C14.4677 101.044 14.2788 99.6725 14.2382 98.2944C14.1638 95.748 15.4549 94.0751 17.6632 92.9546C22.7913 90.3528 28.3037 89.0112 33.8653 87.8106C35.8336 87.3856 37.7655 87.6206 39.6755 88.2534C41.0227 88.6893 42.1762 89.6047 42.9237 90.8313C43.5318 91.8328 44.0289 92.8992 44.4057 94.0107C45.5605 97.5243 46.6626 101.055 47.7123 104.604C48.6203 107.688 49.4176 110.806 50.2638 113.909L50.2387 113.916ZM26.7266 133.343C34.5239 130.44 42.4832 127.998 50.5654 126.029C50.4746 125.604 50.4178 125.269 50.3307 124.942C48.2244 117.022 45.8658 108.955 43.751 101.037C43.6454 100.636 43.5105 100.244 43.3469 99.8638C42.5917 98.1226 42.0916 96.5259 41.2979 94.8028C40.9462 94.0573 40.5475 93.3355 40.1042 92.6419C39.8742 92.2762 39.5675 91.9667 39.2066 91.7359C38.8458 91.505 38.4398 91.3587 38.0183 91.3077C36.7789 91.0867 35.5109 91.129 34.2941 91.4318C33.0151 91.755 31.7446 92.1182 30.4581 92.404C26.5757 93.2663 22.8809 94.6486 19.2827 96.3317C18.1372 96.8675 17.6623 97.7137 17.7232 98.9381C17.7446 99.421 17.8053 99.9016 17.9045 100.375C18.0941 101.253 18.3141 102.126 18.5447 102.994C20.4968 110.346 22.4518 117.698 24.4094 125.049C24.9778 127.188 25.53 129.331 26.1069 131.467C26.2676 132.057 26.4896 132.631 26.7266 133.343ZM51.0585 128.369L27.3375 135.146C27.9084 136.627 28.2194 138.053 29.0065 139.274C29.6111 140.212 30.4854 140.677 31.6243 140.387C32.0501 140.279 32.4828 140.198 32.9069 140.084C34.2782 139.716 35.6471 139.34 37.0163 138.964C40.7003 137.953 44.3919 136.969 48.0621 135.912C49.0317 135.625 49.9624 135.22 50.8336 134.704C51.6378 134.233 52.3048 133.449 52.1344 132.438C51.9128 131.123 51.4611 129.845 51.0585 128.369L51.0585 128.369Z" fill="black"/>
-<path d="M42.0733 133.993C42.3621 135.063 41.6259 136.263 40.4726 136.602C40.2212 136.677 39.9567 136.7 39.6951 136.669C39.4336 136.638 39.1803 136.553 38.9505 136.42C38.7208 136.286 38.5193 136.107 38.3582 135.893C38.1971 135.679 38.0797 135.435 38.0131 135.175C37.8791 134.615 37.9575 134.026 38.2326 133.527C38.5078 133.029 38.9592 132.657 39.4959 132.488C40.5094 132.197 41.7903 132.946 42.0733 133.993Z" fill="black"/>
-</g>
-<g clip-path="url(#clip2_102_125)">
-<path d="M672.405 92.9004C669.025 95.1338 665.72 97.3143 662.421 99.5034C662.178 99.6642 661.987 99.9025 661.753 100.078C661.21 100.483 660.728 100.284 660.645 99.6037C660.614 99.3331 660.641 99.0584 660.725 98.7992C661.119 97.6115 661.499 96.4158 661.963 95.2557C662.843 93.0589 663.778 90.8844 664.69 88.701C664.786 88.4735 664.888 88.2494 664.969 88.0646C662.844 85.8817 660.841 83.6623 658.891 81.3933C658.629 81.0828 658.412 80.7363 658.245 80.364C658.033 79.8962 658.213 79.5992 658.713 79.5585C659.207 79.5159 659.703 79.5077 660.198 79.534C662.087 79.644 663.974 79.7853 665.863 79.8948C666.603 79.9377 667.346 79.9277 668.127 79.9426C668.535 79.051 668.934 78.1872 669.325 77.3199C670.355 75.038 671.384 72.7553 672.411 70.4716C672.447 70.3879 672.488 70.3064 672.533 70.2278C672.821 69.7566 673.314 69.7368 673.618 70.2045C673.745 70.4196 673.845 70.6497 673.917 70.8896C674.536 72.6991 675.12 74.5213 675.774 76.3177C676.154 77.363 676.642 78.3682 677.092 79.4171C677.307 79.4283 677.484 79.4411 677.661 79.4468C679.867 79.5109 682.074 79.5617 684.279 79.6426C685.335 79.6813 686.388 79.7836 687.443 79.842C687.788 79.8611 688.07 79.9635 688.205 80.3004C688.271 80.4611 688.283 80.6398 688.238 80.808C688.193 80.9762 688.093 81.1244 687.956 81.229C687.226 81.8752 686.498 82.5244 685.74 83.1356C683.729 84.7581 681.703 86.3605 679.684 87.973C679.493 88.1257 679.313 88.2925 679.134 88.4472C679.148 88.5539 679.169 88.6593 679.199 88.7625C680.5 92.0385 681.234 95.4611 681.748 98.9407C681.842 99.4326 681.908 99.9295 681.947 100.429C681.977 101.226 681.708 101.323 680.949 101.406C679.888 101.524 679.278 100.928 678.641 100.273C676.983 98.5648 675.324 96.8567 673.897 94.9361C673.41 94.2812 672.932 93.6206 672.405 92.9004ZM663.126 97.2731L663.218 97.3681C663.349 97.2842 663.484 97.2053 663.611 97.1156C666.312 95.2108 669.013 93.3061 671.715 91.4013C672.386 90.9297 672.744 90.9542 673.326 91.5249C673.717 91.9034 674.083 92.3072 674.422 92.7341C675.995 94.7409 677.791 96.5368 679.558 98.3612C679.629 98.4352 679.714 98.4956 679.857 98.616C679.843 98.4448 679.845 98.3571 679.828 98.2734C679.223 95.2592 678.606 92.2473 677.662 89.3186C677.411 88.5397 677.519 87.9485 678.2 87.3957C680.452 85.5668 682.662 83.6842 684.886 81.8219C685.069 81.6689 685.251 81.5146 685.539 81.2715C685.158 81.2515 684.934 81.2373 684.711 81.2282C682.279 81.1347 679.848 81.0069 677.416 80.9655C676.368 80.9476 675.735 80.4654 675.329 79.5557C675.32 79.5349 675.312 79.5136 675.304 79.4923C674.925 78.4442 674.519 77.4055 674.177 76.3454C673.82 75.2413 673.528 74.1149 673.202 72.9999C673.14 72.788 673.052 72.5841 672.932 72.2584C672.61 72.9622 672.313 73.542 672.073 74.145C671.315 76.0498 670.562 77.9572 669.837 79.8754C669.37 81.1125 668.432 81.5985 667.246 81.453C665.48 81.237 663.708 81.0725 661.938 80.8978C661.452 80.8499 660.964 80.8435 660.391 80.8135C660.97 81.4019 661.511 81.9417 662.04 82.4923C663.453 83.9618 664.869 85.4293 666.266 86.9151C666.489 87.171 666.668 87.4634 666.794 87.7799C666.974 88.192 666.889 88.5117 666.502 88.7099C666.218 88.855 666.124 89.0748 666.026 89.3276C665.502 90.6879 664.96 92.0407 664.46 93.4088C663.993 94.6891 663.569 95.9848 663.126 97.2731L663.126 97.2731Z" fill="black"/>
-</g>
-<path fill-rule="evenodd" clip-rule="evenodd" d="M791.167 53.3673C791.326 53.0064 791.422 52.6039 791.518 52.2019C792.147 50.1081 792.91 47.6871 792.823 45.4506C792.683 42.501 791.542 40.611 789.873 39.4849C787.523 37.902 783.944 37.972 780.845 39.3138C777.762 40.634 775.131 43.1357 774.427 46.1266C773.927 48.1646 774.327 50.4734 776.142 52.8328C778.183 55.5191 781.188 56.5453 784.451 56.5412C785.706 56.5604 786.918 56.3783 788.178 56.1263C787.673 57.2531 787.161 58.3898 786.639 59.5404C784.819 63.2306 782.513 66.8382 779.999 70.0899C776.733 74.4322 772.864 78.0725 768.298 81.018C767.912 81.3151 767.795 81.8791 768.055 82.2902C768.315 82.7006 768.87 82.815 769.317 82.5602C774.057 79.4943 778.104 75.7283 781.49 71.2105C784.01 67.8183 786.448 64.1509 788.274 60.3208C789.014 58.7227 789.735 57.1511 790.384 55.5531C793.05 54.7073 795.549 53.4481 797.409 52.1951C809.632 43.6615 814.721 24.3303 816.38 10.1046C816.447 9.61311 816.113 9.17791 815.612 9.11671C815.11 9.05613 814.676 9.42381 814.608 9.91533C812.996 23.6796 808.207 42.4453 796.361 50.6941C794.968 51.664 793.134 52.6156 791.167 53.3673ZM788.975 54.0508C789.305 53.3094 789.571 52.5303 789.774 51.7105C790.304 49.7597 791.086 47.5763 791.035 45.5484C790.937 43.3265 790.14 41.8557 788.904 41.0216C786.926 39.6876 784.065 39.8973 781.584 41.0001C779.027 42.0815 776.757 44.0604 776.214 46.5555C775.841 48.1461 776.173 49.8986 777.551 51.7083C779.268 53.9455 781.791 54.7508 784.439 54.7192C785.965 54.7384 787.501 54.4808 788.975 54.0508Z" fill="black"/>
+<path d="M77.4107 75.5793C77.7477 75.4922 78.1818 75.7332 78.7131 76.3024C79.2444 76.8716 79.7432 77.5545 80.2093 78.3512C80.6725 79.1363 80.9581 79.7379 81.0662 80.1562C81.2314 80.7952 81.2406 81.4063 81.0938 81.9896C80.9557 82.5583 80.6744 83.0524 80.2499 83.4719C79.8224 83.8799 79.2891 84.1664 78.6501 84.3316C77.7671 84.5598 76.9387 84.5197 76.165 84.2115C75.4029 83.9003 74.7506 83.4864 74.208 82.9697C73.6625 82.4414 73.0738 81.7941 72.442 81.0278L72.0629 80.5681L73.1877 84.7765C73.2537 85.0321 73.2718 85.2939 73.2419 85.5619C73.2206 85.8153 73.1054 85.969 72.8963 86.023C72.6872 86.0771 72.5061 85.9999 72.3531 85.7916C72.1971 85.5716 72.0891 85.3455 72.029 85.1131C71.8369 84.3695 71.6187 83.2856 71.3746 81.8614C71.1305 80.4372 70.9198 79.3823 70.7427 78.6969C70.7007 78.5342 70.6455 78.3687 70.5772 78.2005C70.509 78.0322 70.46 77.9147 70.4304 77.848C70.2912 77.5493 70.1901 77.278 70.127 77.034L70.109 76.9643C70.1439 76.9553 70.2424 76.905 70.4047 76.8135C70.5756 76.7074 70.6785 76.6498 70.7133 76.6408L70.8876 76.5958C71.8425 78.4684 72.9675 80.1359 74.2627 81.5984C75.5664 83.0463 76.9735 83.575 78.4838 83.1847C79.1926 83.0015 79.673 82.6543 79.925 82.1429C80.1857 81.617 80.223 80.9938 80.0368 80.2735C79.8777 79.6577 79.6333 79.1198 79.3037 78.6596C78.9711 78.1878 78.5097 77.6254 77.9195 76.9723L77.2102 76.1702C77.1604 76.1211 77.1279 76.0676 77.1129 76.0095C77.0889 75.9165 77.1042 75.832 77.1589 75.7559C77.2106 75.6682 77.2946 75.6093 77.4107 75.5793ZM73.1877 84.7765L73.1787 84.7416L73.1877 84.7765ZM87.3228 82.2947C86.6373 82.4719 85.9767 82.4815 85.341 82.3235C84.7139 82.151 84.1729 81.8322 83.7181 81.3672C83.2719 80.8876 82.9572 80.2934 82.774 79.5847C82.4738 78.4229 82.6742 77.4477 83.3752 76.6592C84.0878 75.8677 85.0657 75.3114 86.3088 74.9901C86.8781 74.843 87.4428 74.8458 88.0028 74.9985C88.5745 75.1482 89.0789 75.4455 89.5161 75.8902C89.9533 76.335 90.2605 76.9001 90.4376 77.5856C90.6298 78.3292 90.6062 79.0293 90.3669 79.6861C90.1246 80.3312 89.7288 80.8859 89.1795 81.3501C88.6388 81.7997 88.0199 82.1146 87.3228 82.2947ZM86.8744 81.3509C87.9549 81.0717 88.7109 80.616 89.1425 79.9839C89.5741 79.3518 89.6683 78.5653 89.425 77.6242C89.2599 76.9852 88.8965 76.4903 88.3349 76.1397C87.785 75.7861 87.1208 75.7098 86.3423 75.911C85.8195 76.0461 85.3164 76.2815 84.833 76.6171C84.3496 76.9528 83.9847 77.3631 83.7382 77.8482C83.4918 78.3333 83.4406 78.8547 83.5847 79.4124C83.7048 79.8771 83.9336 80.2828 84.271 80.6294C84.6055 80.9643 85.0018 81.2028 85.4598 81.3447C85.9265 81.4719 86.398 81.474 86.8744 81.3509ZM100.48 78.7829C99.8296 78.951 99.2694 78.9657 98.7997 78.8268C98.3386 78.6732 97.9577 78.3503 97.6569 77.8579C97.4645 78.5521 97.1808 79.1088 96.8057 79.528C96.4277 79.9355 95.9773 80.2069 95.4544 80.342C94.5482 80.5762 93.767 80.431 93.1109 79.9065C92.4548 79.382 91.9391 78.3936 91.5638 76.9413C91.3866 76.2558 91.3062 75.6569 91.3225 75.1445C91.3505 74.6291 91.4877 74.1289 91.7341 73.6438C91.9118 73.6599 92.0316 73.7157 92.0934 73.8112C92.1639 73.8922 92.2232 74.0256 92.2712 74.2115C92.3193 74.3974 92.3441 74.5892 92.3457 74.7872C92.3588 74.9821 92.3641 75.0984 92.3615 75.1363C92.3703 75.5059 92.3859 75.8303 92.4085 76.1095C92.4397 76.3741 92.5049 76.6981 92.6039 77.0815C92.8201 77.918 93.1541 78.563 93.6059 79.0163C94.0664 79.4551 94.5987 79.5964 95.2028 79.4403C95.8883 79.2631 96.3657 78.9043 96.635 78.3637C96.9129 77.8085 96.9648 77.1939 96.7906 76.5201L96.2093 74.774C96.1912 74.7043 96.2197 74.6226 96.2946 74.5288C96.3812 74.4321 96.4884 74.3672 96.6162 74.3342C96.7788 74.2922 96.9177 74.374 97.0329 74.5798C97.148 74.7855 97.2904 75.1205 97.4599 75.5849C97.6828 76.1594 97.8844 76.6278 98.0648 76.9902C98.2538 77.338 98.5137 77.6241 98.8444 77.8485C99.172 78.0613 99.5798 78.1046 100.068 77.9785C100.66 77.8254 101.048 77.5516 101.231 77.1572C101.411 76.7512 101.432 76.281 101.294 75.7466C101.171 75.2702 100.949 74.7947 100.628 74.3199C100.316 73.8305 99.9883 73.4503 99.6456 73.1795C99.607 73.0779 99.6367 72.9773 99.7349 72.8775C99.8447 72.7748 99.968 72.7243 100.105 72.7262C100.326 72.7186 100.566 72.8549 100.824 73.1352C101.094 73.4125 101.344 73.7816 101.575 74.2425C101.817 74.7005 102.001 75.1734 102.128 75.6614C102.317 76.3934 102.287 77.0455 102.038 77.618C101.801 78.1874 101.282 78.5757 100.48 78.7829ZM116.274 74.7013C115.623 74.8694 115.063 74.8841 114.593 74.7452C114.132 74.5916 113.751 74.2687 113.451 73.7763C113.258 74.4705 112.974 75.0272 112.599 75.4464C112.221 75.8539 111.771 76.1253 111.248 76.2604C110.342 76.4946 109.561 76.3494 108.905 75.8249C108.248 75.3004 107.733 74.312 107.357 72.8597C107.18 72.1742 107.1 71.5753 107.116 71.0629C107.144 70.5475 107.281 70.0473 107.528 69.5622C107.705 69.5783 107.825 69.6341 107.887 69.7296C107.958 69.8106 108.017 69.944 108.065 70.1299C108.113 70.3158 108.138 70.5076 108.139 70.7056C108.152 70.9005 108.158 71.0168 108.155 71.0547C108.164 71.4243 108.18 71.7487 108.202 72.0279C108.233 72.2925 108.298 72.6165 108.398 72.9999C108.614 73.8364 108.9Line truncated
+<path d="M671.576 77.0878C671.483 77.5952 671.415 78.0952 671.373 78.5878C671.345 79.0707 671.31 79.7538 671.269 80.637C671.225 82.1299 671.124 83.307 670.965 84.1685C670.932 84.3455 670.877 84.4451 670.8 84.4675C670.723 84.4898 670.637 84.4923 670.542 84.4748C670.448 84.4574 670.383 84.4454 670.348 84.4389C670.182 84.4084 670.056 84.3667 669.968 84.3139C669.882 84.2493 669.825 84.1289 669.797 83.9528L670.314 79.0695C670.368 78.5792 670.5 77.5969 670.711 76.1227C670.095 76.6191 669.511 76.975 668.958 77.1903C668.408 77.3938 667.874 77.4477 667.354 77.3518C666.611 77.2146 665.964 76.5523 665.415 75.3648C665.417 75.5483 665.392 75.7876 665.337 76.0826C665.283 76.3776 665.251 76.5488 665.242 76.596L664.156 82.3809C664.135 82.4989 664.052 82.5813 663.909 82.6282C663.78 82.6654 663.651 82.672 663.521 82.6481C663.356 82.6176 663.249 82.5368 663.2 82.4057C663.165 82.265 663.149 82.0852 663.153 81.8663L663.61 79.3881C663.68 79.0105 663.77 78.5879 663.881 78.1202C663.992 77.6525 664.071 77.3254 664.117 77.1387C664.329 76.3236 664.496 75.5797 664.621 74.907C664.658 74.7064 664.657 74.5111 664.619 74.321C664.583 74.1191 664.515 73.893 664.415 73.6427C664.314 73.3923 664.244 73.178 664.204 72.9997C664.175 72.8236 664.179 72.6412 664.213 72.4524C664.255 72.2281 664.388 72.1367 664.612 72.1781C664.966 72.2435 665.231 72.4266 665.407 72.7276C665.586 73.0168 665.758 73.4391 665.924 73.9944C666.072 74.4489 666.207 74.8093 666.328 75.0758C666.45 75.3423 666.624 75.588 666.851 75.813C667.08 76.0261 667.378 76.1664 667.744 76.234C668.27 76.3556 668.725 76.3359 669.109 76.1748C669.495 76.002 669.807 75.7667 670.045 75.4689C670.283 75.1712 670.528 74.8015 670.78 74.36C670.994 73.9967 671.168 73.7482 671.302 73.6144C671.439 73.4688 671.581 73.4278 671.728 73.4915C672.086 73.5333 672.203 73.8905 672.079 74.5631C671.974 75.1295 671.891 75.5474 671.829 75.8166L671.576 77.0878ZM676.341 85.5269C675.645 85.3984 675.042 85.1284 674.532 84.7171C674.036 84.2961 673.68 83.7789 673.464 83.1654C673.262 82.5424 673.227 81.8709 673.36 81.1511C673.578 79.971 674.171 79.1714 675.139 78.7521C676.119 78.3351 677.24 78.2431 678.503 78.4761C679.081 78.5829 679.592 78.8236 680.035 79.1983C680.491 79.5752 680.823 80.0574 681.031 80.6451C681.24 81.2328 681.28 81.8747 681.152 82.5709C681.012 83.3262 680.696 83.9511 680.202 84.4456C679.71 84.9284 679.117 85.2644 678.423 85.4536C677.743 85.6331 677.049 85.6576 676.341 85.5269ZM676.333 84.482C677.43 84.6846 678.308 84.5903 678.966 84.1992C679.624 83.8082 680.041 83.1347 680.217 82.1788C680.337 81.5298 680.216 80.9279 679.855 80.3731C679.506 79.8205 678.936 79.4712 678.145 79.3253C677.614 79.2272 677.058 79.2284 676.479 79.3289C675.899 79.4293 675.395 79.6474 674.967 79.9833C674.539 80.3192 674.272 80.7703 674.168 81.3368C674.081 81.8088 674.117 82.2731 674.277 82.7297C674.439 83.1745 674.697 83.5578 675.053 83.8797C675.422 84.1919 675.849 84.3927 676.333 84.482ZM686.099 87.328C685.155 87.1537 684.445 86.8336 683.97 86.3676C683.497 85.8898 683.2 85.2493 683.08 84.4461C682.962 83.6311 682.98 82.5729 683.135 81.2714L683.174 78.1852C683.207 77.7398 683.221 77.303 683.215 76.8748C683.208 76.4465 683.221 76.1438 683.254 75.9668C683.3 75.719 683.404 75.5492 683.568 75.4573C683.733 75.3536 684.003 75.312 684.378 75.3323C684.34 75.9354 684.305 76.3866 684.274 76.686C684.193 77.8546 684.137 78.7534 684.106 79.3823C684.09 80.0015 684.091 80.6546 684.111 81.3416C685.093 81.1081 686.004 81.0687 686.842 81.2233C687.597 81.3627 688.24 81.6462 688.771 82.0737C689.305 82.4894 689.684 83.011 689.91 83.6384C690.138 84.2541 690.188 84.91 690.059 85.6062C689.909 86.4205 689.438 86.9559 688.646 87.2124C687.869 87.4594 687.02 87.4979 686.099 87.328ZM689.062 85.4587C689.089 85.3782 689.116 85.2613 689.145 85.1079C689.299 84.2701 689.126 83.5913 688.624 83.0715C688.133 82.5539 687.363 82.1982 686.313 82.0044C685.664 81.8846 685.133 81.9207 684.719 82.1128C684.319 82.2953 684.079 82.6048 683.998 83.0414C683.813 84.0445 683.949 84.8262 684.407 85.3867C684.877 85.9493 685.678 86.3351 686.811 86.5442C687.354 86.6444 687.831 86.6044 688.243 86.4241C688.654 86.2438 688.927 85.922 689.062 85.4587ZM691.875 88.3941C691.604 88.344 691.459 87.9084 691.439 87.0873C691.422 86.2543 691.443 85.5444 691.503 84.9575C691.562 84.3705 691.666 83.444 691.815 82.1779C691.84 81.9751 691.882 81.7144 691.941 81.3958C692.015 80.9945 692.095 80.7227 692.183 80.5801C692.282 80.4398 692.408 80.3838 692.562 80.4121C692.691 80.4361 692.786 80.4841 692.846 80.5562C692.92 80.6186 692.976 80.7143 693.013 80.8432L692.864 81.5479C692.725 82.3032 692.629 83.0237 692.575 83.7094C692.534 84.3973 692.491 85.3229 692.447 86.4862C692.45 86.9993 692.44 87.4184 692.416 87.7435C692.409 87.9131 692.349 88.0729 692.236 88.2228C692.125 88.361 692.005 88.4181 691.875 88.3941ZM693.23 78.5769C692.97 78.529 692.794 78.3927 692.701 78.1682C692.62 77.9458 692.607 77.6871 692.661 77.3921C692.685 77.2623 692.709 77.1689 692.731 77.1121C692.754 76.9211 692.878 76.8463 693.102 76.8877C6Line truncated
+<path fill-rule="evenodd" clip-rule="evenodd" d="M791.167 53.3674C791.326 53.0066 791.422 52.604 791.518 52.2021C792.147 50.1082 792.91 47.6873 792.823 45.4507C792.683 42.5012 791.542 40.6111 789.873 39.4851C787.523 37.9021 783.944 37.9722 780.845 39.3139C777.762 40.6341 775.131 43.1358 774.427 46.1268C773.927 48.1647 774.327 50.4736 776.142 52.8329C778.183 55.5192 781.188 56.5454 784.451 56.5413C785.706 56.5605 786.918 56.3784 788.178 56.1264C787.673 57.2532 787.161 58.3899 786.639 59.5406C784.819 63.2308 782.513 66.8383 779.999 70.09C776.733 74.4323 772.864 78.0726 768.298 81.0181C767.912 81.3152 767.795 81.8793 768.055 82.2903C768.315 82.7007 768.87 82.8151 769.317 82.5603C774.057 79.4944 778.104 75.7284 781.49 71.2106C784.01 67.8184 786.448 64.151 788.274 60.3209C789.014 58.7229 789.735 57.1513 790.384 55.5532C793.05 54.7074 795.549 53.4482 797.409 52.1953C809.632 43.6616 814.721 24.3304 816.38 10.1047C816.447 9.61323 816.113 9.17804 815.612 9.11684C815.11 9.05625 814.676 9.42394 814.608 9.91545C812.996 23.6797 808.207 42.4454 796.361 50.6942C794.968 51.6641 793.134 52.6157 791.167 53.3674ZM788.975 54.0509C789.305 53.3096 789.571 52.5304 789.774 51.7106C790.304 49.7598 791.086 47.5764 791.035 45.5485C790.937 43.3267 790.14 41.8558 788.904 41.0217C786.926 39.6877 784.065 39.8974 781.584 41.0003C779.027 42.0816 776.757 44.0605 776.214 46.5556C775.841 48.1462 776.173 49.8987 777.551 51.7084C779.268 53.9457 781.791 54.7509 784.439 54.7193C785.965 54.7385 787.501 54.4809 788.975 54.0509Z" fill="black"/>
 <path fill-rule="evenodd" clip-rule="evenodd" d="M815.723 9.48371C815.881 9.91195 815.964 10.4476 816.047 10.9839C816.231 12.0312 816.327 13.0738 816.577 13.7609C818.043 17.9391 819.853 22.9313 822.575 26.4691C822.856 26.8501 823.434 26.9309 823.827 26.6227C824.221 26.3152 824.327 25.7667 823.985 25.3434C821.416 21.978 819.735 17.1922 818.271 13.1429C818.04 12.4275 817.915 11.1644 817.742 10.1016C817.564 9.17823 817.31 8.36459 816.866 7.82661C816.635 7.50454 815.998 7.24641 815.153 7.68357C813.987 8.32455 811.682 10.7482 811.439 10.9691C808.986 13.2133 806.754 15.1371 803.26 15.2131C802.786 15.2439 802.402 15.67 802.356 16.1302C802.372 16.6327 802.813 17.045 803.287 17.0136C807.25 16.9135 809.921 14.8772 812.693 12.3023C812.827 12.1083 814.193 10.6518 815.341 9.77581C815.486 9.696 815.579 9.56168 815.723 9.48371Z" fill="black"/>
+<path fill-rule="evenodd" clip-rule="evenodd" d="M33.207 105.722C32.9094 105.935 32.8553 106.156 33.0075 106.539C33.1129 106.805 33.2555 107.002 33.7514 107.566C34.0661 107.923 34.1592 108.135 34.0949 108.345C34.0678 108.433 34.0007 108.533 33.9425 108.571C33.8853 108.608 33.1674 108.866 32.3472 109.143C30.6567 109.713 30.2182 109.872 29.3283 110.233C27.0025 111.179 25.7044 112.028 25.547 112.706C25.4319 113.202 25.8276 113.528 26.9509 113.863C27.4398 114.008 27.5944 114.107 27.6597 114.317C27.7533 114.618 27.6452 114.775 26.4098 116.128C24.9114 117.77 23.6679 119.387 23.0195 120.536C22.5704 121.331 22.3594 122.073 22.476 122.447C22.623 122.918 22.9836 122.966 24.4293 122.704C24.8216 122.633 25.0454 122.658 25.1639 122.786C25.2929 122.926 25.2904 123.28 25.1563 123.878C25.0954 124.149 24.9902 124.617 24.9225 124.918C24.3941 127.262 24.1401 129.44 24.2214 130.927C24.2536 131.516 24.4322 132.144 24.6239 132.343C24.9534 132.685 25.4387 132.559 26.1013 131.961C26.7369 131.387 26.9132 131.293 27.1907 131.377C27.4323 131.45 27.515 131.596 27.7005 132.277C28.0759 133.655 29.1282 136.451 29.725 137.655C30.5054 139.231 31.302 140.086 31.8308 139.917C32.1486 139.815 32.4364 139.35 32.671 138.559C32.8281 138.029 32.9308 137.865 33.1481 137.797C33.4363 137.707 33.6263 137.825 34.4926 138.629C36.1468 140.164 37.7216 141.423 38.9235 142.171C40.1957 142.963 41.1754 143.214 41.5172 142.836C41.7092 142.624 41.7454 142.176 41.6218 141.541C41.4396 140.604 41.4466 140.4 41.6662 140.255C41.839 140.142 42.1861 140.178 43.2683 140.424C45.764 140.992 47.7183 141.257 49.3035 141.243C50.2497 141.235 50.9143 141.085 51.1824 140.819C51.5443 140.461 51.4037 140.034 50.6415 139.176C50.237 138.72 50.1341 138.508 50.2052 138.274C50.2341 138.179 50.3029 138.078 50.3724 138.028C50.451 137.972 50.8913 137.818 51.6882 137.568C55.5188 136.369 58.0613 135.122 58.6324 134.161C58.8681 133.765 58.8096 133.514 58.4137 133.224C58.1494 133.03 57.9362 132.939 57.34 132.763C56.8215 132.609 56.7129 132.54 56.6428 132.315C56.5449 132.001 56.6239 131.878 57.6264 130.786C60.6043 127.541 62.1183 125.141 61.8141 124.147C61.7694 124.001 61.7367 123.967 61.551 123.869C61.2904 123.733 60.9377 123.724 60.3637 123.839C59.6344 123.985 59.3834 124 59.2261 123.907C59.144 123.859 59.0722 123.781 59.0537 123.72C58.9951 123.529 59.083 122.938 59.3415 121.786C59.8961 119.313 60.1537 117.268 60.0891 115.849C60.0672 115.368 60.0436 115.218 59.9312 114.845C59.8103 114.444 59.7864 114.394 59.6505 114.265C59.328 113.958 58.9116 114.056 58.29 114.585C57.6719 115.111 57.536 115.215 57.4212 115.252C57.2755 115.298 57.0998 115.274 56.9552 115.189C56.8575 115.132 56.8293 115.057 56.3167 113.497C55.0143 109.533 53.9462 107.383 53.0022 106.825C52.6682 106.628 52.4715 106.638 52.2399 106.864C52.0163 107.083 51.8922 107.303 51.7302 107.768C51.4727 108.508 51.3154 108.774 51.0947 108.842C50.8329 108.924 50.6759 108.815 49.5103 107.738C47.335 105.729 45.2748 104.231 44.0945 103.802C43.3415 103.528 42.933 103.546 42.7389 103.864C42.5543 104.167 42.5489 104.523 42.7166 105.325C42.7684 105.572 42.8139 105.847 42.8176 105.936C42.8258 106.13 42.7084 106.37 42.5869 106.407C42.3804 106.469 41.865 106.398 40.7761 106.155C38.1567 105.572 35.9636 105.302 34.5427 105.39C34.0234 105.422 33.4168 105.573 33.207 105.722ZM44.7981 117.181C45.1396 117.262 45.432 117.417 45.6995 117.657L45.9122 117.847L47.1317 121.757L48.3512 125.666L48.2836 125.951C48.1979 126.312 48.0589 126.577 47.8106 126.853L47.6146 127.07L43.6941 128.293L39.7736 129.516L39.5423 129.455C39.1202 129.343 38.9602 129.265 38.6769 129.033L38.3962 128.802L37.168 124.865L35.9397 120.927L36.0397 120.578C36.1481 120.2 36.265 120 36.5323 119.736L36.7046 119.566L37.8196 119.195C38.4329 118.99 40.194 118.439 41.7333 117.97L44.5319 117.117L44.7981 117.181Z" fill="white" stroke="black"/>
+<path fill-rule="evenodd" clip-rule="evenodd" d="M36.9116 94.0916C36.614 94.3039 36.5599 94.5253 36.7121 94.9088C36.8175 95.1743 36.9601 95.371 37.456 95.9349C37.7706 96.2927 37.8638 96.5039 37.7995 96.714C37.7724 96.8026 37.7052 96.9023 37.6471 96.9403C37.5899 96.9777 36.872 97.2349 36.0518 97.5118C34.3613 98.0825 33.9228 98.2409 33.0328 98.6027C30.7071 99.5479 29.409 100.397 29.2516 101.076C29.1365 101.572 29.5322 101.898 30.6555 102.232C31.1443 102.377 31.299 102.476 31.3643 102.686C31.4579 102.987 31.3498 103.144 30.1144 104.498C28.6159 106.14 27.3725 107.756 26.7241 108.905C26.275 109.701 26.064 110.443 26.1806 110.816C26.3276 111.288 26.6882 111.335 28.1338 111.073C28.5262 111.002 28.75 111.027 28.8685 111.156C28.9975 111.295 28.995 111.649 28.8609 112.247C28.8 112.518 28.6948 112.986 28.627 113.287C28.0987 115.631 27.8447 117.809 27.926 119.297C27.9582 119.886 28.1368 120.514 28.3285 120.712C28.658 121.054 29.1433 120.928 29.8059 120.33C30.4415 119.756 30.6178 119.662 30.8953 119.746C31.1368 119.819 31.2196 119.965 31.4051 120.646C31.7805 122.024 32.8328 124.82 33.4296 126.025C34.21 127.6 35.0066 128.456 35.5354 128.286C35.8532 128.185 36.141 127.719 36.3756 126.928C36.5327 126.398 36.6354 126.234 36.8527 126.167C37.1409 126.077 37.3309 126.194 38.1972 126.998C39.8514 128.533 41.4262 129.792 42.6281 130.54C43.9003 131.332 44.88 131.583 45.2218 131.205C45.4138 130.993 45.45 130.545 45.3264 129.91C45.1442 128.974 45.1512 128.769 45.3708 128.625C45.5436 128.511 45.8907 128.548 46.9729 128.794C49.4686 129.361 51.4229 129.627 53.0081 129.613C53.9543 129.604 54.6189 129.454 54.887 129.189C55.2489 128.83 55.1083 128.403 54.3461 127.545C53.9416 127.09 53.8387 126.877 53.9098 126.643C53.9387 126.548 54.0075 126.447 54.077 126.397C54.1556 126.341 54.5959 126.187 55.3927 125.938C59.2234 124.739 61.7659 123.491 62.337 122.531C62.5727 122.134 62.5141 121.883 62.1183 121.593C61.854 121.4 61.6408 121.308 61.0446 121.132C60.5261 120.979 60.4175 120.909 60.3474 120.684C60.2495 120.37 60.3285 120.247 61.331 119.155C64.3089 115.911 65.8229 113.511 65.5187 112.517C65.474 112.37 65.4413 112.336 65.2556 112.239C64.995 112.102 64.6423 112.093 64.0683 112.208C63.339 112.354 63.088 112.37 62.9307 112.277C62.8486 112.228 62.7768 112.15 62.7582 112.089C62.6997 111.898 62.7876 111.307 63.0461 110.155C63.6006 107.682 63.8583 105.637 63.7937 104.218C63.7718 103.737 63.7482 103.587 63.6358 103.214C63.5149 102.813 63.491 102.764 63.3551 102.634C63.0326 102.328 62.6162 102.426 61.9946 102.954C61.3765 103.48 61.2406 103.585 61.1258 103.621C60.9801 103.667 60.8044 103.644 60.6597 103.558C60.5621 103.501 60.5339 103.426 60.0213 101.866C58.7189 97.9018 57.6508 95.752 56.7068 95.1944C56.3727 94.9972 56.1761 95.0073 55.9445 95.2336C55.7209 95.4521 55.5968 95.6721 55.4348 96.1376C55.1773 96.877 55.02 97.1428 54.7992 97.2117C54.5375 97.2934 54.3805 97.1839 53.2149 96.1073C51.0396 94.0982 48.9794 92.6007 47.7991 92.171C47.0461 91.8968 46.6376 91.9158 46.4435 92.2336C46.2589 92.536 46.2535 92.8923 46.4212 93.6939C46.473 93.9416 46.5185 94.2166 46.5222 94.3051C46.5304 94.4997 46.413 94.7395 46.2915 94.7762C46.0849 94.8386 45.5696 94.7669 44.4807 94.5244C41.8613 93.9408 39.6682 93.6715 38.2473 93.759C37.728 93.7909 37.1214 93.942 36.9116 94.0916ZM48.5027 105.55C48.8442 105.632 49.1366 105.786 49.4041 106.026L49.6168 106.217L50.8363 110.126L52.0558 114.036L51.9882 114.32C51.9025 114.681 51.7635 114.946 51.5152 115.222L51.3192 115.439L47.3987 116.662L43.4782 117.885L43.2469 117.824C42.8248 117.713 42.6648 117.635 42.3815 117.402L42.1008 117.172L40.8726 113.234L39.6443 109.297L39.7443 108.947C39.8527 108.569 39.9696 108.369 40.2369 108.106L40.4092 107.936L41.5242 107.564C42.1375 107.359 43.8986 106.808 45.4379 106.339L48.2365 105.486L48.5027 105.55Z" fill="white" stroke="black"/>
+<g clip-path="url(#clip1_203_128)">
+<path d="M790.439 140.186C789.297 145.104 788.165 150.025 787.007 154.94C786.777 155.951 786.485 156.948 786.134 157.924C785.935 158.512 785.613 159.048 785.189 159.496C783.979 160.67 782.355 161.467 780.639 161.437C779.252 161.438 777.87 161.313 776.508 161.062C771.935 160.127 767.38 159.106 762.825 158.083C761.905 157.876 761.016 157.528 760.118 157.228C757.615 156.391 756.676 154.371 756.5 151.922C756.412 150.699 756.633 149.445 756.765 148.211C757.223 143.898 758.313 139.712 759.342 135.513C760.191 132.053 761.032 128.592 761.866 125.129C762.817 121.161 763.709 117.177 764.714 113.222C765.055 111.885 765.539 110.588 766.157 109.356C767.3 107.079 769.23 106.219 771.705 106.28C777.454 106.422 782.943 107.856 788.409 109.438C790.343 109.997 791.932 111.121 793.313 112.584C794.293 113.606 794.874 114.959 794.95 116.394C795.01 117.564 794.942 118.739 794.746 119.896C794.096 123.537 793.391 127.168 792.632 130.79C791.968 133.936 791.191 137.058 790.464 140.192L790.439 140.186ZM760.525 146.134C768.766 147.278 776.931 148.904 784.98 151.004C785.102 150.587 785.211 150.265 785.289 149.936C787.192 141.965 788.942 133.745 790.837 125.771C790.934 125.368 791.001 124.959 791.038 124.546C791.199 122.655 791.516 121.012 791.635 119.119C791.679 118.296 791.67 117.471 791.609 116.65C791.58 116.219 791.457 115.801 791.249 115.427C791.041 115.052 790.753 114.731 790.406 114.486C789.419 113.704 788.283 113.139 787.068 112.829C785.789 112.507 784.498 112.224 783.23 111.865C779.403 110.782 775.495 110.246 771.529 110.021C770.267 109.949 769.447 110.469 768.92 111.576C768.71 112.011 768.535 112.463 768.398 112.927C768.148 113.79 767.928 114.662 767.719 115.536C765.95 122.934 764.183 130.334 762.419 137.734C761.905 139.886 761.374 142.035 760.869 144.189C760.73 144.784 760.654 145.395 760.525 146.134ZM784.304 153.298L760.207 148.011C760.007 149.586 759.605 150.988 759.718 152.437C759.805 153.549 760.354 154.374 761.494 154.659C761.921 154.766 762.34 154.9 762.767 155C764.149 155.327 765.533 155.645 766.916 155.963C770.639 156.821 774.355 157.707 778.088 158.517C779.078 158.724 780.089 158.809 781.101 158.768C782.032 158.735 782.991 158.361 783.321 157.391C783.75 156.127 783.958 154.788 784.304 153.298Z" fill="black" fill-opacity="0.3"/>
+<path d="M774.347 153.986C774.093 155.065 772.876 155.772 771.7 155.523C771.443 155.471 771.199 155.366 770.984 155.214C770.768 155.063 770.585 154.868 770.446 154.641C770.307 154.415 770.215 154.162 770.175 153.897C770.134 153.632 770.147 153.361 770.212 153.101C770.36 152.544 770.708 152.063 771.187 151.755C771.665 151.446 772.239 151.333 772.792 151.439C773.822 151.664 774.594 152.93 774.347 153.986Z" fill="black" fill-opacity="0.3"/>
+</g>
 </g>
 <defs>
-<clipPath id="clip0_102_125">
+<clipPath id="clip0_203_128">
 <rect width="830" height="212" fill="white"/>
 </clipPath>
-<clipPath id="clip1_102_125">
-<rect width="31.6846" height="53.6201" fill="white" transform="translate(12 93.4165) rotate(-15.2539)"/>
-</clipPath>
-<clipPath id="clip2_102_125">
-<rect width="30.3062" height="31.2838" fill="white" transform="translate(658.634 69) rotate(2.99317)"/>
+<clipPath id="clip1_203_128">
+<rect width="31.6846" height="53.6201" fill="white" transform="translate(767.122 104) rotate(13.0653)"/>
 </clipPath>
 </defs>
 </svg>

install.sh

@@ -1,7 +1,8 @@
 set -eu
 
 APP_NAME="simplex-chat"
-TARGET_DIR="$HOME/.local/bin"
+BIN_DIR="$HOME/.local/bin"
+BIN_PATH="$BIN_DIR/$APP_NAME"
 PLATFORM="$(uname)"
 
 if [ $PLATFORM == "Darwin" ]; then
@@ -10,39 +11,88 @@ elif [ $PLATFORM == "Linux" ]; then
 	PLATFORM="ubuntu-20_04-x86-64"
 else
 	echo "Scripted installation on your platform is not supported."
-	echo "See compiled binaries in the latest release: https://github.com/simplex-chat/simplex-chat/releases/latest"
+	echo "See compiled binaries in the latest release: https://github.com/$APP_NAME/$APP_NAME/releases/latest"
 	exit 1
 fi
 
-[ ! -d $TARGET_DIR ] && mkdir -p $TARGET_DIR
+# / Prepare to upgrade from v0 to v1
+
+# Determine path of chat binary
+if [[ -n "$(which $APP_NAME)" ]]; then
+	binary=$(which $APP_NAME)
+elif [[ -f "$BIN_PATH" ]]; then
+	binary=$BIN_PATH
+else
+	binary=""
+fi
+
+# If chat binary not found, check v0 initial migration and offer to abort or continue
+if [[ -z $binary ]]; then
+	agent_db="$HOME/.simplex/simplex.agent.db"
+	if [[ \
+		-f "$agent_db" && \
+		$(echo "select * from migrations;" | sqlite3 $agent_db | grep 20210101_initial) \
+	]]; then
+		echo "Warning: found SimpleX Chat database, the current version is not backwards compatible."
+		echo "If you continue, the current version will be installed as $APP_NAME with a clean database, the old database will be preserved."
+		while true; do
+			read -p "Please choose to (a)bort or (c)ontinue: " yn
+			case $yn in
+					[Aa]* ) exit 1 ;;
+					[Cc]* ) break ;;
+					* ) echo "Please answer 'a' or 'c'."
+			esac
+		done
+	fi
+# If chat binary found, check version and offer to abort or continue, on continue rename chat binary
+elif [[ ! $($binary -h | grep v1) ]]; then
+	echo "Warning: found a previous version of SimpleX Chat, the current version is not backwards compatible."
+	echo "If you continue, it will be renamed to $APP_NAME-v0, and the new version will be installed as $APP_NAME with a clean database."
+	while true; do
+		read -p "Please choose (a)bort or (c)ontinue: " yn
+		case $yn in
+				[Aa]* ) exit 1 ;;
+				[Cc]* )
+					binary_v0="$binary-v0"
+					mv ${binary} ${binary_v0}
+					echo "Renamed $binary into $binary_v0"
+					break
+					;;
+				* ) echo "Please answer 'a' or 'c'."
+		esac
+	done
+fi
+# Prepare to upgrade from v0 to v1 /
+
+[ ! -d $BIN_DIR ] && mkdir -p $BIN_DIR
 
 if [ -n "$(command -v curl)" ]; then
-	curl -L -o $TARGET_DIR/$APP_NAME "https://github.com/$APP_NAME/$APP_NAME/releases/latest/download/$APP_NAME-$PLATFORM"
+	curl -L -o $BIN_PATH "https://github.com/$APP_NAME/$APP_NAME/releases/latest/download/$APP_NAME-$PLATFORM"
 elif [ -n "$(command -v wget)" ]; then
-	wget -O $TARGET_DIR/$APP_NAME "https://github.com/$APP_NAME/$APP_NAME/releases/latest/download/$APP_NAME-$PLATFORM"
+	wget -O $BIN_PATH "https://github.com/$APP_NAME/$APP_NAME/releases/latest/download/$APP_NAME-$PLATFORM"
 else
-	echo "Cannot download simplex-chat - please install curl or wget"
+	echo "Cannot download $APP_NAME - please install curl or wget"
 	exit 1
 fi
 
-chmod +x $TARGET_DIR/$APP_NAME
+chmod +x $BIN_PATH
 
 echo "$APP_NAME installed sucesfully!"
 
-if [ -z "$(command -v simplex-chat)" ]; then
+if [ -z "$(command -v $APP_NAME)" ]; then
 	if [ -n "$($SHELL -c 'echo $ZSH_VERSION')" ]; then
 		SHELL_FILE="$HOME/.zshrc"
 	elif [ -n "$($SHELL -c 'echo $BASH_VERSION')" ]; then
 		SHELL_FILE="$HOME/.bashrc"
 	else
-		echo "Unknown shell - cannot add simplex-chat folder to PATH"
-		echo "Please add $TARGET_DIR to PATH variable"
-		echo "Or you can run simplex-chat via full path: $TARGET_DIR/simplex-chat"
+		echo "Unknown shell - cannot add $APP_NAME folder to PATH"
+		echo "Please add $BIN_DIR to PATH variable"
+		echo "Or you can run $APP_NAME via full path: $BIN_PATH"
 	fi
 	if [ -n "$SHELL_FILE" ]; then
-		echo "export PATH=\$PATH:$TARGET_DIR" >> $SHELL_FILE
-		echo "Source your $SHELL_FILE or open a new shell and type simplex-chat to run it"
+		echo "export PATH=\$PATH:$BIN_DIR" >> $SHELL_FILE
+		echo "Source your $SHELL_FILE or open a new shell and type $APP_NAME to run it"
 	fi
 else
-	echo "Type simplex-chat in your terminal to run it"
+	echo "Type $APP_NAME in your terminal to run it"
 fi

message_queries.md

@@ -0,0 +1,39 @@
+# Message queries
+
+You can run queries against `direct_messages`, `group_messages` and `all_messages` (or their simpler alternatives `direct_messages_plain`, `group_messages_plain` and `all_messages_plain`), for example:
+
+```sql
+-- you can put these or your preferred settings into ~/.sqliterc
+-- to persist across sqlite3 client sessions
+.mode column
+.headers on
+.nullvalue NULL
+
+-- simple views into direct, group and all_messages
+-- with user's messages deduplicated for group and all_messages;
+-- only 'x.msg.new' ("new message") chat events - filters out service events;
+-- msg_sent is 0 for received, 1 for sent
+select * from direct_messages_plain;
+select * from group_messages_plain;
+select * from all_messages_plain;
+
+-- query other details of your chat history with regular SQL, for example:
+-- files you offered for sending
+select * from direct_messages where msg_sent = 1 and chat_msg_event = 'x.file';
+-- everything catherine sent related to cats
+select * from direct_messages where msg_sent = 0 and contact = 'catherine' and msg_body like '%cats%'; 
+-- all correspondence with alice in #team
+select * from group_messages where group_name = 'team' and contact = 'alice';
+
+-- aggregate your chat data
+select contact_or_group, num_messages from (
+  select
+    contact as contact_or_group, count(1) as num_messages
+    from direct_messages_plain group by contact
+  union
+  select
+    group_name as contact_or_group, count(1) as num_messages
+    from group_messages_plain group by group_name
+)
+order by num_messages desc;
+```

message_views.sql

@@ -0,0 +1,110 @@
+CREATE VIEW direct_messages AS
+SELECT
+  ct.local_display_name AS contact,
+  m.message_id AS message_id,
+  m.msg_sent AS msg_sent,
+  m.chat_msg_event AS chat_msg_event,
+  m.msg_body AS msg_body,
+  md.msg_delivery_id AS delivery_id,
+  datetime(md.chat_ts) AS chat_dt,
+  md.agent_msg_meta AS msg_meta,
+  mde.delivery_status AS delivery_status,
+  datetime(mde.created_at) AS delivery_status_dt
+FROM messages m
+JOIN msg_deliveries md ON md.message_id = m.message_id
+JOIN (
+  SELECT msg_delivery_id, MAX(created_at) MaxDate
+  FROM msg_delivery_events
+  GROUP BY msg_delivery_id
+) MaxDates ON MaxDates.msg_delivery_id = md.msg_delivery_id
+JOIN msg_delivery_events mde ON mde.msg_delivery_id = MaxDates.msg_delivery_id
+                            AND mde.created_at = MaxDates.MaxDate
+JOIN connections c ON c.connection_id = md.connection_id
+JOIN contacts ct ON ct.contact_id = c.contact_id
+ORDER BY chat_dt DESC;
+
+CREATE VIEW direct_messages_plain AS
+SELECT
+  dm.contact AS contact,
+  dm.msg_sent AS msg_sent,
+  dm.msg_body AS msg_body,
+  dm.chat_dt AS chat_dt
+FROM direct_messages dm
+WHERE dm.chat_msg_event = 'x.msg.new';
+
+CREATE VIEW group_messages AS
+SELECT
+  g.local_display_name AS group_name,
+  gm.local_display_name AS contact,
+  m.message_id AS message_id,
+  m.msg_sent AS msg_sent,
+  m.chat_msg_event AS chat_msg_event,
+  m.msg_body AS msg_body,
+  md.msg_delivery_id AS delivery_id,
+  datetime(md.chat_ts) AS chat_dt,
+  md.agent_msg_meta AS msg_meta,
+  mde.delivery_status AS delivery_status,
+  datetime(mde.created_at) AS delivery_status_dt
+FROM messages m
+JOIN msg_deliveries md ON md.message_id = m.message_id
+JOIN (
+  SELECT msg_delivery_id, MAX(created_at) MaxDate
+  FROM msg_delivery_events
+  GROUP BY msg_delivery_id
+) MaxDates ON MaxDates.msg_delivery_id = md.msg_delivery_id
+JOIN msg_delivery_events mde ON mde.msg_delivery_id = MaxDates.msg_delivery_id
+                            AND mde.created_at = MaxDates.MaxDate
+JOIN connections c ON c.connection_id = md.connection_id
+JOIN group_members gm ON gm.group_member_id = c.group_member_id
+JOIN groups g ON g.group_id = gm.group_id
+ORDER BY chat_dt DESC;
+
+CREATE VIEW group_messages_plain AS
+SELECT
+  gm.group_name AS group_name,
+  (CASE WHEN gm.msg_sent = 0 THEN gm.contact ELSE gm.group_name END) AS contact,
+  gm.msg_sent AS msg_sent,
+  gm.msg_body AS msg_body,
+  gm.chat_dt AS chat_dt
+FROM group_messages gm
+JOIN (
+  SELECT message_id, MIN(delivery_id) MinDeliveryId
+  FROM group_messages
+  GROUP BY message_id
+) Deduplicated ON Deduplicated.message_id = gm.message_id
+              AND Deduplicated.MinDeliveryId = gm.delivery_id
+WHERE gm.chat_msg_event = 'x.msg.new';
+
+CREATE VIEW all_messages (
+  group_name,
+  contact,
+  message_id,
+  msg_sent,
+  chat_msg_event,
+  msg_body,
+  delivery_id,
+  chat_dt,
+  msg_meta,
+  delivery_status,
+  delivery_status_dt
+) AS
+  SELECT * FROM (
+    SELECT NULL AS group_name, * FROM direct_messages
+    UNION
+    SELECT * FROM group_messages
+  )
+  ORDER BY chat_dt DESC;
+
+CREATE VIEW all_messages_plain (
+  group_name,
+  contact,
+  msg_sent,
+  msg_body,
+  chat_dt
+) AS
+  SELECT * FROM (
+    SELECT NULL AS group_name, * FROM direct_messages_plain
+    UNION
+    SELECT * FROM group_messages_plain
+  )
+  ORDER BY chat_dt DESC;

migrations/20211205_user_contacts.sql

@@ -1,29 +0,0 @@
-CREATE TABLE user_contact_links (
-  user_contact_link_id INTEGER PRIMARY KEY,
-  conn_req_contact BLOB NOT NULL,
-  local_display_name TEXT NOT NULL DEFAULT '',
-  created_at TEXT NOT NULL DEFAULT (datetime('now')),
-  user_id INTEGER NOT NULL REFERENCES users,
-  UNIQUE (user_id, local_display_name)
-);
-
-CREATE TABLE contact_requests (
-  contact_request_id INTEGER PRIMARY KEY,
-  user_contact_link_id INTEGER NOT NULL REFERENCES user_contact_links
-    ON UPDATE CASCADE ON DELETE CASCADE,
-  agent_invitation_id  BLOB NOT NULL,
-  contact_profile_id INTEGER REFERENCES contact_profiles
-    DEFERRABLE INITIALLY DEFERRED, -- NULL if it's an incognito profile
-  local_display_name TEXT NOT NULL,
-  created_at TEXT NOT NULL DEFAULT (datetime('now')),
-  user_id INTEGER NOT NULL REFERENCES users,
-  FOREIGN KEY (user_id, local_display_name)
-    REFERENCES display_names (user_id, local_display_name)
-    ON UPDATE CASCADE
-    DEFERRABLE INITIALLY DEFERRED,
-  UNIQUE (user_id, local_display_name),
-  UNIQUE (user_id, contact_profile_id)
-);
-
-ALTER TABLE connections ADD user_contact_link_id INTEGER
-REFERENCES user_contact_links ON DELETE RESTRICT;

migrations/20211229_messages.sql

@@ -1,200 +0,0 @@
-DROP TABLE event_body_parts;
-DROP TABLE contact_profile_events;
-DROP TABLE group_profile_events;
-DROP TABLE group_event_parents;
-DROP TABLE group_events;
-DROP TABLE message_events;
-DROP TABLE message_content;
-DROP TABLE events;
-DROP TABLE messages;
-
--- all message events as received or sent, append only
--- maps to message deliveries as one-to-many for group messages
-CREATE TABLE messages (
-  message_id INTEGER PRIMARY KEY,
-  msg_sent INTEGER NOT NULL, -- 0 for received, 1 for sent
-  chat_msg_event TEXT NOT NULL, -- message event type (the constructor of ChatMsgEvent)
-  msg_body BLOB, -- agent message body as received or sent
-  created_at TEXT NOT NULL DEFAULT (datetime('now'))
-);
-
--- message deliveries communicated with the agent, append only
-CREATE TABLE msg_deliveries (
-  msg_delivery_id INTEGER PRIMARY KEY,
-  message_id INTEGER NOT NULL REFERENCES messages ON DELETE CASCADE, -- non UNIQUE for group messages
-  connection_id INTEGER NOT NULL REFERENCES connections ON DELETE CASCADE,
-  agent_msg_id INTEGER, -- internal agent message ID (NULL while pending)
-  agent_msg_meta TEXT, -- JSON with timestamps etc. sent in MSG, NULL for sent
-  chat_ts TEXT NOT NULL DEFAULT (datetime('now')), -- broker_ts for received, created_at for sent
-  UNIQUE (connection_id, agent_msg_id)
-);
-
--- TODO recovery for received messages with "rcv_agent" status - acknowledge to agent
--- changes of messagy delivery status, append only
-CREATE TABLE msg_delivery_events (
-  msg_delivery_event_id INTEGER PRIMARY KEY,
-  msg_delivery_id INTEGER NOT NULL REFERENCES msg_deliveries ON DELETE CASCADE, -- non UNIQUE for multiple events per msg delivery
-  delivery_status TEXT NOT NULL, -- see MsgDeliveryStatus for allowed values
-  created_at TEXT NOT NULL DEFAULT (datetime('now')),
-  UNIQUE (msg_delivery_id, delivery_status)
-);
-
-CREATE VIEW direct_messages AS
-SELECT
-  ct.local_display_name AS contact,
-  m.message_id AS message_id,
-  m.msg_sent AS msg_sent,
-  m.chat_msg_event AS chat_msg_event,
-  m.msg_body AS msg_body,
-  md.msg_delivery_id AS delivery_id,
-  datetime(md.chat_ts) AS chat_dt,
-  md.agent_msg_meta AS msg_meta,
-  mde.delivery_status AS delivery_status,
-  datetime(mde.created_at) AS delivery_status_dt
-FROM messages m
-JOIN msg_deliveries md ON md.message_id = m.message_id
-JOIN (
-  SELECT msg_delivery_id, MAX(created_at) MaxDate
-  FROM msg_delivery_events
-  GROUP BY msg_delivery_id
-) MaxDates ON MaxDates.msg_delivery_id = md.msg_delivery_id
-JOIN msg_delivery_events mde ON mde.msg_delivery_id = MaxDates.msg_delivery_id
-                            AND mde.created_at = MaxDates.MaxDate
-JOIN connections c ON c.connection_id = md.connection_id
-JOIN contacts ct ON ct.contact_id = c.contact_id
-ORDER BY chat_dt DESC;
-
-CREATE VIEW direct_messages_plain AS
-SELECT
-  dm.contact AS contact,
-  dm.msg_sent AS msg_sent,
-  dm.msg_body AS msg_body,
-  dm.chat_dt AS chat_dt
-FROM direct_messages dm
-WHERE dm.chat_msg_event = 'x.msg.new';
-
-CREATE VIEW group_messages AS
-SELECT
-  g.local_display_name AS group_name,
-  gm.local_display_name AS contact,
-  m.message_id AS message_id,
-  m.msg_sent AS msg_sent,
-  m.chat_msg_event AS chat_msg_event,
-  m.msg_body AS msg_body,
-  md.msg_delivery_id AS delivery_id,
-  datetime(md.chat_ts) AS chat_dt,
-  md.agent_msg_meta AS msg_meta,
-  mde.delivery_status AS delivery_status,
-  datetime(mde.created_at) AS delivery_status_dt
-FROM messages m
-JOIN msg_deliveries md ON md.message_id = m.message_id
-JOIN (
-  SELECT msg_delivery_id, MAX(created_at) MaxDate
-  FROM msg_delivery_events
-  GROUP BY msg_delivery_id
-) MaxDates ON MaxDates.msg_delivery_id = md.msg_delivery_id
-JOIN msg_delivery_events mde ON mde.msg_delivery_id = MaxDates.msg_delivery_id
-                            AND mde.created_at = MaxDates.MaxDate
-JOIN connections c ON c.connection_id = md.connection_id
-JOIN group_members gm ON gm.group_member_id = c.group_member_id
-JOIN groups g ON g.group_id = gm.group_id
-ORDER BY chat_dt DESC;
-
-CREATE VIEW group_messages_plain AS
-SELECT
-  gm.group_name AS group_name,
-  (CASE WHEN gm.msg_sent = 0 THEN gm.contact ELSE gm.group_name END) AS contact,
-  gm.msg_sent AS msg_sent,
-  gm.msg_body AS msg_body,
-  gm.chat_dt AS chat_dt
-FROM group_messages gm
-JOIN (
-  SELECT message_id, MIN(delivery_id) MinDeliveryId
-  FROM group_messages
-  GROUP BY message_id
-) Deduplicated ON Deduplicated.message_id = gm.message_id
-              AND Deduplicated.MinDeliveryId = gm.delivery_id
-WHERE gm.chat_msg_event = 'x.msg.new';
-
-CREATE VIEW all_messages (
-  group_name,
-  contact,
-  message_id,
-  msg_sent,
-  chat_msg_event,
-  msg_body,
-  delivery_id,
-  chat_dt,
-  msg_meta,
-  delivery_status,
-  delivery_status_dt
-) AS
-  SELECT * FROM (
-    SELECT NULL AS group_name, * FROM direct_messages
-    UNION
-    SELECT * FROM group_messages
-  )
-  ORDER BY chat_dt DESC;
-
-CREATE VIEW all_messages_plain (
-  group_name,
-  contact,
-  msg_sent,
-  msg_body,
-  chat_dt
-) AS
-  SELECT * FROM (
-    SELECT NULL AS group_name, * FROM direct_messages_plain
-    UNION
-    SELECT * FROM group_messages_plain
-  )
-  ORDER BY chat_dt DESC;
-
--- TODO group message parents and chat items not to be implemented in current scope
-
--- CREATE TABLE group_message_parents (
---   group_message_parent_id INTEGER PRIMARY KEY,
---   message_id INTEGER NOT NULL REFERENCES group_messages (event_id),
---   parent_group_member_id INTEGER REFERENCES group_members (group_member_id), -- can be NULL if parent_member_id is incorrect
---   parent_member_id BLOB, -- shared member ID, unique per group
---   parent_message_id INTEGER REFERENCES messages (message_id) ON DELETE CASCADE, -- can be NULL if received message references another message that's not received yet
---   parent_chat_msg_id INTEGER NOT NULL,
---   parent_msg_body_hash BLOB NOT NULL
--- );
-
--- CREATE INDEX group_event_parents_parent_chat_event_id_index
---   ON group_message_parents (parent_member_id, parent_chat_msg_id);
-
--- CREATE TABLE chat_items ( -- mutable chat_items presented to user
---   chat_item_id INTEGER PRIMARY KEY,
---   chat_msg_id INTEGER NOT NULL, -- sent as part of the message that created the item
---   item_deleted INTEGER NOT NULL, -- 1 for deleted
---   item_type TEXT NOT NULL,
---   item_text TEXT NOT NULL, -- textual representation
---   item_props TEXT NOT NULL -- JSON
--- );
-
--- CREATE TABLE direct_chat_items (
---   chat_item_id INTEGER NOT NULL UNIQUE REFERENCES chat_items ON DELETE CASCADE,
---   contact_id INTEGER NOT NULL REFERENCES contacts ON DELETE RESTRICT,
---   item_sent INTEGER -- 1 for sent, 0 for received
--- );
-
--- CREATE TABLE group_chat_items (
---   chat_item_id INTEGER NOT NULL UNIQUE REFERENCES chat_items ON DELETE CASCADE,
---   group_member_id INTEGER REFERENCES group_members ON DELETE RESTRICT, -- NULL for sent
---   group_id INTEGER NOT NULL REFERENCES groups ON DELETE RESTRICT
--- );
-
--- CREATE TABLE chat_item_content (
---   chat_item_content_id INTEGER PRIMARY KEY,
---   chat_item_id INTEGER NOT NULL REFERENCES chat_items ON DELETE CASCADE,
---   content_type TEXT NOT NULL,
---   content_size INTEGER NOT NULL,
---   content BLOB NOT NULL
--- );
-
--- CREATE TABLE chat_item_messages (
---   message_id INTEGER NOT NULL UNIQUE REFERENCES messages,
---   chat_item_id INTEGER NOT NULL REFERENCES chat_items
--- );

migrations/20220101_initial.sql

@@ -15,13 +15,13 @@ CREATE TABLE users (
   active_user INTEGER NOT NULL DEFAULT 0, -- 1 for active user
   FOREIGN KEY (user_id, local_display_name)
     REFERENCES display_names (user_id, local_display_name)
-    ON DELETE RESTRICT
+    ON DELETE CASCADE
     ON UPDATE CASCADE
     DEFERRABLE INITIALLY DEFERRED
 );
 
 CREATE TABLE display_names (
-  user_id INTEGER NOT NULL REFERENCES users,
+  user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE,
   local_display_name TEXT NOT NULL,
   ldn_base TEXT NOT NULL,
   ldn_suffix INTEGER NOT NULL DEFAULT 0,
@@ -31,15 +31,15 @@ CREATE TABLE display_names (
 
 CREATE TABLE contacts (
   contact_id INTEGER PRIMARY KEY,
-  contact_profile_id INTEGER REFERENCES contact_profiles, -- NULL if it's an incognito profile
-  user_id INTEGER NOT NULL REFERENCES users,
+  contact_profile_id INTEGER REFERENCES contact_profiles ON DELETE SET NULL, -- NULL if it's an incognito profile
+  user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE,
   local_display_name TEXT NOT NULL,
   is_user INTEGER NOT NULL DEFAULT 0, -- 1 if this contact is a user
   via_group INTEGER REFERENCES groups (group_id) ON DELETE SET NULL,
   created_at TEXT NOT NULL DEFAULT (datetime('now')),
   FOREIGN KEY (user_id, local_display_name)
     REFERENCES display_names (user_id, local_display_name)
-    ON DELETE RESTRICT
+    ON DELETE CASCADE
     ON UPDATE CASCADE,
   UNIQUE (user_id, local_display_name),
   UNIQUE (user_id, contact_profile_id)
@@ -49,7 +49,7 @@ CREATE TABLE sent_probes (
   sent_probe_id INTEGER PRIMARY KEY,
   contact_id INTEGER NOT NULL UNIQUE REFERENCES contacts ON DELETE CASCADE,
   probe BLOB NOT NULL,
-  user_id INTEGER NOT NULL REFERENCES users,
+  user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE,
   UNIQUE (user_id, probe)
 );
 
@@ -57,7 +57,7 @@ CREATE TABLE sent_probe_hashes (
   sent_probe_hash_id INTEGER PRIMARY KEY,
   sent_probe_id INTEGER NOT NULL REFERENCES sent_probes ON DELETE CASCADE,
   contact_id INTEGER NOT NULL REFERENCES contacts ON DELETE CASCADE,
-  user_id INTEGER NOT NULL REFERENCES users,
+  user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE,
   UNIQUE (sent_probe_id, contact_id)
 );
 
@@ -66,7 +66,7 @@ CREATE TABLE received_probes (
   contact_id INTEGER NOT NULL REFERENCES contacts ON DELETE CASCADE,
   probe BLOB,
   probe_hash BLOB NOT NULL,
-  user_id INTEGER NOT NULL REFERENCES users
+  user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE
 );
 
 CREATE TABLE known_servers(
@@ -74,7 +74,7 @@ CREATE TABLE known_servers(
   host TEXT NOT NULL,
   port TEXT NOT NULL,
   key_hash BLOB,
-  user_id INTEGER NOT NULL REFERENCES users,
+  user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE,
   UNIQUE (user_id, host, port)
 ) WITHOUT ROWID;
 
@@ -87,35 +87,38 @@ CREATE TABLE group_profiles ( -- shared group profiles
 
 CREATE TABLE groups (
   group_id INTEGER PRIMARY KEY, -- local group ID
-  user_id INTEGER NOT NULL REFERENCES users,
+  user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE,
   local_display_name TEXT NOT NULL, -- local group name without spaces
-  group_profile_id INTEGER REFERENCES group_profiles, -- shared group profile
-  inv_queue_info BLOB,
+  group_profile_id INTEGER REFERENCES group_profiles ON DELETE SET NULL, -- shared group profile
+  inv_queue_info BLOB, -- received
   FOREIGN KEY (user_id, local_display_name)
     REFERENCES display_names (user_id, local_display_name)
-    ON DELETE RESTRICT
+    ON DELETE CASCADE
     ON UPDATE CASCADE,
   UNIQUE (user_id, local_display_name),
   UNIQUE (user_id, group_profile_id)
 );
 
+CREATE INDEX idx_groups_inv_queue_info ON groups (inv_queue_info);
+
 CREATE TABLE group_members ( -- group members, excluding the local user
   group_member_id INTEGER PRIMARY KEY,
-  group_id INTEGER NOT NULL REFERENCES groups ON DELETE RESTRICT,
+  group_id INTEGER NOT NULL REFERENCES groups ON DELETE CASCADE,
   member_id BLOB NOT NULL, -- shared member ID, unique per group
   member_role TEXT NOT NULL, -- owner, admin, member
   member_category TEXT NOT NULL, -- see GroupMemberCategory
   member_status TEXT NOT NULL, -- see GroupMemberStatus
-  invited_by INTEGER REFERENCES contacts (contact_id) ON DELETE RESTRICT, -- NULL for the members who joined before the current user and for the group creator
-  group_queue_info BLOB,
-  direct_queue_info BLOB,
-  user_id INTEGER NOT NULL REFERENCES users,
+  invited_by INTEGER REFERENCES contacts (contact_id) ON DELETE SET NULL, -- NULL for the members who joined before the current user and for the group creator
+  sent_inv_queue_info BLOB, -- sent
+  group_queue_info BLOB, -- received
+  direct_queue_info BLOB, -- received
+  user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE,
   local_display_name TEXT NOT NULL, -- should be the same as contact
-  contact_profile_id INTEGER NOT NULL REFERENCES contact_profiles ON DELETE RESTRICT,
-  contact_id INTEGER REFERENCES contacts ON DELETE RESTRICT,
+  contact_profile_id INTEGER NOT NULL REFERENCES contact_profiles ON DELETE CASCADE,
+  contact_id INTEGER REFERENCES contacts ON DELETE CASCADE,
   FOREIGN KEY (user_id, local_display_name)
     REFERENCES display_names (user_id, local_display_name)
-    ON DELETE RESTRICT
+    ON DELETE CASCADE
     ON UPDATE CASCADE,
   UNIQUE (group_id, member_id)
 );
@@ -132,28 +135,28 @@ CREATE TABLE group_member_intros (
 
 CREATE TABLE files (
   file_id INTEGER PRIMARY KEY,
-  contact_id INTEGER REFERENCES contacts ON DELETE RESTRICT,
-  group_id INTEGER REFERENCES groups ON DELETE RESTRICT,
+  contact_id INTEGER REFERENCES contacts ON DELETE CASCADE,
+  group_id INTEGER REFERENCES groups ON DELETE CASCADE,
   file_name TEXT NOT NULL,
   file_path TEXT,
   file_size INTEGER NOT NULL,
   chunk_size INTEGER NOT NULL,
   created_at TEXT NOT NULL DEFAULT (datetime('now')),
-  user_id INTEGER NOT NULL REFERENCES users
+  user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE
 );
 
 CREATE TABLE snd_files (
-  file_id INTEGER NOT NULL REFERENCES files ON DELETE RESTRICT,
-  connection_id INTEGER NOT NULL REFERENCES connections ON DELETE RESTRICT,
+  file_id INTEGER NOT NULL REFERENCES files ON DELETE CASCADE,
+  connection_id INTEGER NOT NULL REFERENCES connections ON DELETE CASCADE,
   file_status TEXT NOT NULL, -- new, accepted, connected, completed
-  group_member_id INTEGER REFERENCES group_members ON DELETE RESTRICT,
+  group_member_id INTEGER REFERENCES group_members ON DELETE CASCADE,
   PRIMARY KEY (file_id, connection_id)
 ) WITHOUT ROWID;
 
 CREATE TABLE rcv_files (
-  file_id INTEGER PRIMARY KEY REFERENCES files ON DELETE RESTRICT,
+  file_id INTEGER PRIMARY KEY REFERENCES files ON DELETE CASCADE,
   file_status TEXT NOT NULL, -- new, accepted, connected, completed
-  group_member_id INTEGER REFERENCES group_members ON DELETE RESTRICT,
+  group_member_id INTEGER REFERENCES group_members ON DELETE CASCADE,
   file_queue_info BLOB
 );
 
@@ -168,7 +171,7 @@ CREATE TABLE snd_file_chunks (
 ) WITHOUT ROWID;
 
 CREATE TABLE rcv_file_chunks (
-  file_id INTEGER NOT NULL REFERENCES rcv_files,
+  file_id INTEGER NOT NULL REFERENCES rcv_files ON DELETE CASCADE,
   chunk_number INTEGER NOT NULL,
   chunk_agent_msg_id INTEGER NOT NULL,
   chunk_stored INTEGER NOT NULL DEFAULT 0, -- 0 (received), 1 (appended to file)
@@ -179,102 +182,78 @@ CREATE TABLE connections ( -- all SMP agent connections
   connection_id INTEGER PRIMARY KEY,
   agent_conn_id BLOB NOT NULL UNIQUE,
   conn_level INTEGER NOT NULL DEFAULT 0,
-  via_contact INTEGER REFERENCES contacts (contact_id),
+  via_contact INTEGER REFERENCES contacts (contact_id) ON DELETE SET NULL,
   conn_status TEXT NOT NULL,
   conn_type TEXT NOT NULL, -- contact, member, rcv_file, snd_file
-  contact_id INTEGER REFERENCES contacts ON DELETE RESTRICT,
-  group_member_id INTEGER REFERENCES group_members ON DELETE RESTRICT,
+  user_contact_link_id INTEGER REFERENCES user_contact_links ON DELETE CASCADE,
+  contact_id INTEGER REFERENCES contacts ON DELETE CASCADE,
+  group_member_id INTEGER REFERENCES group_members ON DELETE CASCADE,
   snd_file_id INTEGER,
-  rcv_file_id INTEGER REFERENCES rcv_files (file_id) ON DELETE RESTRICT,
+  rcv_file_id INTEGER REFERENCES rcv_files (file_id) ON DELETE CASCADE,
   created_at TEXT NOT NULL DEFAULT (datetime('now')),
-  user_id INTEGER NOT NULL REFERENCES users,
+  user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE,
   FOREIGN KEY (snd_file_id, connection_id)
     REFERENCES snd_files (file_id, connection_id)
-    ON DELETE RESTRICT
+    ON DELETE CASCADE
     DEFERRABLE INITIALLY DEFERRED
 );
 
--- PLEASE NOTE: all tables below were unused and are removed in the migration 20211227_messages.sql
+CREATE TABLE user_contact_links (
+  user_contact_link_id INTEGER PRIMARY KEY,
+  conn_req_contact BLOB NOT NULL,
+  local_display_name TEXT NOT NULL DEFAULT '',
+  created_at TEXT NOT NULL DEFAULT (datetime('now')),
+  user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE,
+  UNIQUE (user_id, local_display_name)
+);
 
-CREATE TABLE events ( -- messages received by the agent, append only
-  event_id INTEGER PRIMARY KEY,
-  agent_msg_id INTEGER NOT NULL, -- internal message ID
-  external_msg_id INTEGER NOT NULL, -- external message ID (sent or received)
-  agent_meta TEXT NOT NULL, -- JSON with timestamps etc. sent in MSG
-  connection_id INTEGER NOT NULL REFERENCES connections,
-  received INTEGER NOT NULL, -- 0 for received, 1 for sent
-  chat_event_id INTEGER,
-  continuation_of INTEGER, -- references chat_event_id, but can be incorrect
-  event_type TEXT NOT NULL, -- event type - see protocol/types.ts
-  event_encoding INTEGER NOT NULL, -- format of event_body: 0 - binary, 1 - text utf8, 2 - JSON (utf8)
-  content_type TEXT NOT NULL, -- content type - see protocol/types.ts
-  event_body BLOB, -- agent message body as sent
-  event_hash BLOB NOT NULL,
-  integrity TEXT NOT NULL DEFAULT '',
+CREATE TABLE contact_requests (
+  contact_request_id INTEGER PRIMARY KEY,
+  user_contact_link_id INTEGER NOT NULL REFERENCES user_contact_links
+    ON UPDATE CASCADE ON DELETE CASCADE,
+  agent_invitation_id  BLOB NOT NULL,
+  contact_profile_id INTEGER REFERENCES contact_profiles
+    ON DELETE SET NULL -- NULL if it's an incognito profile
+    DEFERRABLE INITIALLY DEFERRED,
+  local_display_name TEXT NOT NULL,
+  created_at TEXT NOT NULL DEFAULT (datetime('now')),
+  user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE,
+  FOREIGN KEY (user_id, local_display_name)
+    REFERENCES display_names (user_id, local_display_name)
+    ON UPDATE CASCADE
+    ON DELETE CASCADE
+    DEFERRABLE INITIALLY DEFERRED,
+  UNIQUE (user_id, local_display_name),
+  UNIQUE (user_id, contact_profile_id)
+);
+
+-- all message events as received or sent, append only
+-- maps to message deliveries as one-to-many for group messages
+CREATE TABLE messages (
+  message_id INTEGER PRIMARY KEY,
+  msg_sent INTEGER NOT NULL, -- 0 for received, 1 for sent
+  chat_msg_event TEXT NOT NULL, -- message event type (the constructor of ChatMsgEvent)
+  msg_body BLOB, -- agent message body as received or sent
   created_at TEXT NOT NULL DEFAULT (datetime('now'))
 );
 
-CREATE INDEX events_external_msg_id_index ON events (connection_id, external_msg_id);
-
-CREATE TABLE event_body_parts (
-  event_body_part_id INTEGER PRIMARY KEY,
-  event_id REFERENCES events,
-  full_size INTEGER NOT NULL,
-  part_status TEXT, -- full, partial
-  content_type TEXT NOT NULL,
-  event_part BLOB
+-- message deliveries communicated with the agent, append only
+CREATE TABLE msg_deliveries (
+  msg_delivery_id INTEGER PRIMARY KEY,
+  message_id INTEGER NOT NULL REFERENCES messages ON DELETE CASCADE, -- non UNIQUE for group messages
+  connection_id INTEGER NOT NULL REFERENCES connections ON DELETE CASCADE,
+  agent_msg_id INTEGER, -- internal agent message ID (NULL while pending)
+  agent_msg_meta TEXT, -- JSON with timestamps etc. sent in MSG, NULL for sent
+  chat_ts TEXT NOT NULL DEFAULT (datetime('now')), -- broker_ts for received, created_at for sent
+  UNIQUE (connection_id, agent_msg_id)
 );
 
-CREATE TABLE contact_profile_events (
-  event_id INTEGER NOT NULL UNIQUE REFERENCES events,
-  contact_profile_id INTEGER NOT NULL REFERENCES contact_profiles
-);
-
-CREATE TABLE group_profile_events (
-  event_id INTEGER NOT NULL UNIQUE REFERENCES events,
-  group_profile_id INTEGER NOT NULL REFERENCES group_profiles
-);
-
-CREATE TABLE group_events (
-  event_id INTEGER NOT NULL UNIQUE REFERENCES events,
-  group_id INTEGER NOT NULL REFERENCES groups ON DELETE RESTRICT,
-  group_member_id INTEGER REFERENCES group_members -- NULL for current user
-);
-
-CREATE TABLE group_event_parents (
-  group_event_parent_id INTEGER PRIMARY KEY,
-  event_id INTEGER NOT NULL REFERENCES group_events (event_id),
-  parent_group_member_id INTEGER REFERENCES group_members (group_member_id), -- can be NULL if parent_member_id is incorrect
-  parent_member_id BLOB, -- shared member ID, unique per group
-  parent_event_id INTEGER REFERENCES events (event_id) ON DELETE CASCADE, -- this can be NULL if received event references another event that's not received yet
-  parent_chat_event_id INTEGER NOT NULL,
-  parent_event_hash BLOB NOT NULL
-);
-
-CREATE INDEX group_event_parents_parent_chat_event_id_index
-  ON group_event_parents (parent_member_id, parent_chat_event_id);
-
-CREATE TABLE messages ( -- mutable messages presented to user
-  message_id INTEGER PRIMARY KEY,
-  contact_id INTEGER NOT NULL REFERENCES contacts ON DELETE RESTRICT, -- 1 for sent messages
-  group_id INTEGER REFERENCES groups ON DELETE RESTRICT, -- NULL for direct messages
-  deleted INTEGER NOT NULL, -- 1 for deleted
-  msg_type TEXT NOT NULL,
-  content_type TEXT NOT NULL,
-  msg_text TEXT NOT NULL, -- textual representation
-  msg_props TEXT NOT NULL -- JSON
-);
-
-CREATE TABLE message_content (
-  message_content_id INTEGER PRIMARY KEY,
-  message_id INTEGER REFERENCES messages ON DELETE CASCADE,
-  content_type TEXT NOT NULL,
-  content_size INTEGER, -- full expected content size
-  content_status TEXT, -- empty, part, full
-  content BLOB NOT NULL
-);
-
-CREATE TABLE message_events (
-  event_id INTEGER NOT NULL UNIQUE REFERENCES events,
-  message_id INTEGER NOT NULL REFERENCES messages
+-- TODO recovery for received messages with "rcv_agent" status - acknowledge to agent
+-- changes of messagy delivery status, append only
+CREATE TABLE msg_delivery_events (
+  msg_delivery_event_id INTEGER PRIMARY KEY,
+  msg_delivery_id INTEGER NOT NULL REFERENCES msg_deliveries ON DELETE CASCADE, -- non UNIQUE for multiple events per msg delivery
+  delivery_status TEXT NOT NULL, -- see MsgDeliveryStatus for allowed values
+  created_at TEXT NOT NULL DEFAULT (datetime('now')),
+  UNIQUE (msg_delivery_id, delivery_status)
 );

migrations/20220106_group_members_inv_queue_info.sql

@@ -1,3 +0,0 @@
-ALTER TABLE group_members ADD inv_queue_info BLOB;
-
-CREATE INDEX idx_groups_inv_queue_info ON groups (inv_queue_info);

package.yaml

@@ -1,5 +1,5 @@
 name: simplex-chat
-version: 0.5.5
+version: 1.0.0
 #synopsis:
 #description:
 homepage: https://github.com/simplex-chat/simplex-chat#readme
@@ -29,7 +29,7 @@ dependencies:
   - optparse-applicative >= 0.15 && < 0.17
   - process == 1.6.*
   - simple-logger == 0.1.*
-  - simplexmq >= 0.5.2 && < 0.6
+  - simplexmq >= 1.0 && < 1.1
   - sqlite-simple == 0.4.*
   - stm == 2.5.*
   - terminal == 0.2.*
@@ -37,6 +37,7 @@ dependencies:
   - time == 1.9.*
   - unliftio == 0.2.*
   - unliftio-core == 0.2.*
+  - unordered-containers == 0.2.*
 
 library:
   source-dirs: src

simplex-chat.cabal

@@ -5,7 +5,7 @@ cabal-version: 1.12
 -- see: https://github.com/sol/hpack
 
 name:           simplex-chat
-version:        0.5.5
+version:        1.0.0
 category:       Web, System, Services, Cryptography
 homepage:       https://github.com/simplex-chat/simplex-chat#readme
 author:         Evgeny Poberezkin
@@ -56,7 +56,7 @@ library
     , optparse-applicative >=0.15 && <0.17
     , process ==1.6.*
     , simple-logger ==0.1.*
-    , simplexmq >=0.5.2 && <0.6
+    , simplexmq ==1.0.*
     , sqlite-simple ==0.4.*
     , stm ==2.5.*
     , terminal ==0.2.*
@@ -64,6 +64,7 @@ library
     , time ==1.9.*
     , unliftio ==0.2.*
     , unliftio-core ==0.2.*
+    , unordered-containers ==0.2.*
   default-language: Haskell2010
 
 executable simplex-chat
@@ -92,7 +93,7 @@ executable simplex-chat
     , process ==1.6.*
     , simple-logger ==0.1.*
     , simplex-chat
-    , simplexmq >=0.5.2 && <0.6
+    , simplexmq ==1.0.*
     , sqlite-simple ==0.4.*
     , stm ==2.5.*
     , terminal ==0.2.*
@@ -100,6 +101,7 @@ executable simplex-chat
     , time ==1.9.*
     , unliftio ==0.2.*
     , unliftio-core ==0.2.*
+    , unordered-containers ==0.2.*
   default-language: Haskell2010
 
 test-suite simplex-chat-test
@@ -136,7 +138,7 @@ test-suite simplex-chat-test
     , process ==1.6.*
     , simple-logger ==0.1.*
     , simplex-chat
-    , simplexmq >=0.5.2 && <0.6
+    , simplexmq ==1.0.*
     , sqlite-simple ==0.4.*
     , stm ==2.5.*
     , terminal ==0.2.*
@@ -144,4 +146,5 @@ test-suite simplex-chat-test
     , time ==1.9.*
     , unliftio ==0.2.*
     , unliftio-core ==0.2.*
+    , unordered-containers ==0.2.*
   default-language: Haskell2010

src/Simplex/Chat.hs

@@ -9,6 +9,7 @@
 {-# LANGUAGE RecordWildCards #-}
 {-# LANGUAGE ScopedTypeVariables #-}
 {-# LANGUAGE TupleSections #-}
+{-# LANGUAGE TypeApplications #-}
 
 module Simplex.Chat where
 
@@ -34,6 +35,7 @@ import Data.Maybe (isJust, mapMaybe)
 import Data.Text (Text)
 import qualified Data.Text as T
 import Data.Text.Encoding (encodeUtf8)
+import Data.Word (Word32)
 import Simplex.Chat.Controller
 import Simplex.Chat.Help
 import Simplex.Chat.Input
@@ -50,10 +52,12 @@ import Simplex.Messaging.Agent
 import Simplex.Messaging.Agent.Env.SQLite (AgentConfig (..), defaultAgentConfig)
 import Simplex.Messaging.Agent.Protocol
 import qualified Simplex.Messaging.Crypto as C
+import Simplex.Messaging.Encoding
+import Simplex.Messaging.Encoding.String
 import Simplex.Messaging.Parsers (parseAll)
 import Simplex.Messaging.Protocol (MsgBody)
 import qualified Simplex.Messaging.Protocol as SMP
-import Simplex.Messaging.Util (bshow, raceAny_, tryError)
+import Simplex.Messaging.Util (raceAny_, tryError)
 import System.Exit (exitFailure, exitSuccess)
 import System.FilePath (combine, splitExtensions, takeFileName)
 import System.IO (Handle, IOMode (..), SeekMode (..), hFlush, openFile, stdout)
@@ -73,9 +77,8 @@ data ChatCommand
   | MarkdownHelp
   | Welcome
   | AddContact
-  | Connect (Maybe AConnectionRequest)
+  | Connect (Maybe AConnectionRequestUri)
   | ConnectAdmin
-  | SendAdminWelcome ContactName
   | DeleteContact ContactName
   | ListContacts
   | CreateMyAddress
@@ -117,7 +120,7 @@ defaultChatConfig =
           },
       dbPoolSize = 1,
       tbqSize = 16,
-      fileChunkSize = 7050
+      fileChunkSize = 15780
     }
 
 logCfg :: LogConfig
@@ -138,7 +141,7 @@ newChatController config@ChatConfig {agentConfig = cfg, dbPoolSize, tbqSize} Cha
   chatStore <- createStore f dbPoolSize
   currentUser <- newTVarIO =<< getCreateActiveUser chatStore
   chatTerminal <- newChatTerminal t
-  smpAgent <- getSMPAgentClient cfg {dbFile = dbFile <> ".agent.db", smpServers}
+  smpAgent <- getSMPAgentClient cfg {dbFile = dbFile <> "_agent.db", smpServers}
   idsDrg <- newTVarIO =<< drgNew
   inputQ <- newTBQueueIO tbqSize
   notifyQ <- newTBQueueIO tbqSize
@@ -150,10 +153,10 @@ newChatController config@ChatConfig {agentConfig = cfg, dbPoolSize, tbqSize} Cha
 runSimplexChat :: ChatController -> IO ()
 runSimplexChat = runReaderT $ do
   user <- readTVarIO =<< asks currentUser
-  whenM (asks firstTime) . printToView . chatWelcome user $ Onboarding 0 0 0 0 0
+  whenM (asks firstTime) . printToView $ chatWelcome user
   race_ runTerminalInput runChatController
 
-runChatController :: (MonadUnliftIO m, MonadReader ChatController m) => m ()
+runChatController :: (MonadUnliftIO m, MonadReader ChatController m, MonadFail m) => m ()
 runChatController =
   raceAny_
     [ inputSubscriber,
@@ -167,7 +170,7 @@ withLock lock =
     (void . atomically $ takeTMVar lock)
     (atomically $ putTMVar lock ())
 
-inputSubscriber :: (MonadUnliftIO m, MonadReader ChatController m) => m ()
+inputSubscriber :: (MonadUnliftIO m, MonadReader ChatController m, MonadFail m) => m ()
 inputSubscriber = do
   q <- asks inputQ
   l <- asks chatLock
@@ -184,7 +187,6 @@ inputSubscriber = do
               SendGroupMessage g msg -> showSentGroupMessage g msg
               SendFile c f -> showSentFileInvitation c f
               SendGroupFile g f -> showSentGroupFileInvitation g f
-              SendAdminWelcome c -> forM_ adminWelcomeMessages $ showSentMessage c
               _ -> printToView [plain s]
             user <- readTVarIO =<< asks currentUser
             withAgentLock a . withLock l . void . runExceptT $
@@ -197,9 +199,7 @@ processChatCommand user@User {userId, profile} = \case
   GroupsHelp -> printToView groupsHelpInfo
   MyAddressHelp -> printToView myAddressHelpInfo
   MarkdownHelp -> printToView markdownInfo
-  Welcome -> do
-    ob <- withStore (`getOnboarding` userId)
-    printToView $ chatWelcome user ob
+  Welcome -> printToView $ chatWelcome user
   AddContact -> do
     (connId, cReq) <- withAgent (`createConnection` SCMInvitation)
     withStore $ \st -> createDirectConnection st userId connId
@@ -208,7 +208,6 @@ processChatCommand user@User {userId, profile} = \case
   Connect (Just (ACR SCMContact cReq)) -> connect cReq (XContact profile Nothing) >> showSentInvitation
   Connect Nothing -> showInvalidConnReq
   ConnectAdmin -> connect adminContactReq (XContact profile Nothing) >> showSentInvitation
-  SendAdminWelcome cName -> forM_ adminWelcomeMessages $ sendMessageCmd cName
   DeleteContact cName ->
     withStore (\st -> getContactGroupNames st userId cName) >>= \case
       [] -> do
@@ -259,7 +258,7 @@ processChatCommand user@User {userId, profile} = \case
     unless (memberActive membership) $ chatError CEGroupMemberNotActive
     let sendInvitation memberId cReq = do
           sendDirectMessage (contactConn contact) $
-            XGrpInv $ GroupInvitation (userMemberId, userRole) (memberId, memRole) cReq groupProfile
+            XGrpInv $ GroupInvitation (MemberIdRole userMemberId userRole) (MemberIdRole memberId memRole) cReq groupProfile
           showSentGroupInvitation gName cName
           setActive $ ActiveG gName
     case contactMember contact members of
@@ -276,7 +275,7 @@ processChatCommand user@User {userId, profile} = \case
         | otherwise -> chatError (CEGroupDuplicateMember cName)
   JoinGroup gName -> do
     ReceivedGroupInvitation {fromMember, userMember, connRequest} <- withStore $ \st -> getGroupInvitation st user gName
-    agentConnId <- withAgent $ \a -> joinConnection a connRequest . directMessage . XGrpAcpt $ memberId userMember
+    agentConnId <- withAgent $ \a -> joinConnection a connRequest . directMessage . XGrpAcpt $ memberId (userMember :: GroupMember)
     withStore $ \st -> do
       createMemberConnection st userId fromMember agentConnId
       updateGroupMemberStatus st userId fromMember GSMemAccepted
@@ -286,13 +285,13 @@ processChatCommand user@User {userId, profile} = \case
     Group {membership, members} <- withStore $ \st -> getGroup st user gName
     case find ((== cName) . (localDisplayName :: GroupMember -> ContactName)) members of
       Nothing -> chatError $ CEGroupMemberNotFound cName
-      Just member -> do
-        let userRole = memberRole membership
-        when (userRole < GRAdmin || userRole < memberRole member) $ chatError CEGroupUserRole
-        when (memberStatus member /= GSMemInvited) . sendGroupMessage members $ XGrpMemDel (memberId member)
-        deleteMemberConnection member
-        withStore $ \st -> updateGroupMemberStatus st userId member GSMemRemoved
-        showDeletedMember gName Nothing (Just member)
+      Just m@GroupMember {memberId = mId, memberRole = mRole, memberStatus = mStatus} -> do
+        let userRole = memberRole (membership :: GroupMember)
+        when (userRole < GRAdmin || userRole < mRole) $ chatError CEGroupUserRole
+        when (mStatus /= GSMemInvited) . sendGroupMessage members $ XGrpMemDel mId
+        deleteMemberConnection m
+        withStore $ \st -> updateGroupMemberStatus st userId m GSMemRemoved
+        showDeletedMember gName Nothing (Just m)
   LeaveGroup gName -> do
     Group {membership, members} <- withStore $ \st -> getGroup st user gName
     sendGroupMessage members XGrpLeave
@@ -303,7 +302,7 @@ processChatCommand user@User {userId, profile} = \case
     g@Group {membership, members} <- withStore $ \st -> getGroup st user gName
     let s = memberStatus membership
         canDelete =
-          memberRole membership == GROwner
+          memberRole (membership :: GroupMember) == GROwner
             || (s == GSMemRemoved || s == GSMemLeft || s == GSMemGroupDeleted || s == GSMemInvited)
     unless canDelete $ chatError CEGroupUserRole
     when (memberActive membership) $ sendGroupMessage members XGrpDel
@@ -318,7 +317,7 @@ processChatCommand user@User {userId, profile} = \case
     -- TODO save pending message delivery for members without connections
     Group {members, membership} <- withStore $ \st -> getGroup st user gName
     unless (memberActive membership) $ chatError CEGroupMemberUserRemoved
-    let msgEvent = XMsgNew $ MsgContent MTText [] [MsgContentBody {contentType = SimplexContentType XCText, contentData = msg}]
+    let msgEvent = XMsgNew . MCText $ safeDecodeUtf8 msg
     sendGroupMessage members msgEvent
     setActive $ ActiveG gName
   SendFile cName f -> do
@@ -376,14 +375,14 @@ processChatCommand user@User {userId, profile} = \case
   QuitChat -> liftIO exitSuccess
   ShowVersion -> printToView clientVersionInfo
   where
-    connect :: ConnectionRequest c -> ChatMsgEvent -> m ()
+    connect :: ConnectionRequestUri c -> ChatMsgEvent -> m ()
     connect cReq msg = do
       connId <- withAgent $ \a -> joinConnection a cReq $ directMessage msg
       withStore $ \st -> createDirectConnection st userId connId
     sendMessageCmd :: ContactName -> ByteString -> m ()
     sendMessageCmd cName msg = do
       contact <- withStore $ \st -> getContact st userId cName
-      let msgEvent = XMsgNew $ MsgContent MTText [] [MsgContentBody {contentType = SimplexContentType XCText, contentData = msg}]
+      let msgEvent = XMsgNew . MCText $ safeDecodeUtf8 msg
       sendDirectMessage (contactConn contact) msgEvent
       setActive $ ActiveC cName
     contactMember :: Contact -> [GroupMember] -> Maybe GroupMember
@@ -426,7 +425,7 @@ processChatCommand user@User {userId, profile} = \case
               f = filePath `combine` (name <> suffix <> ext)
            in ifM (doesFileExist f) (tryCombine $ n + 1) (pure f)
 
-agentSubscriber :: (MonadUnliftIO m, MonadReader ChatController m) => m ()
+agentSubscriber :: (MonadUnliftIO m, MonadReader ChatController m, MonadFail m) => m ()
 agentSubscriber = do
   q <- asks $ subQ . smpAgent
   l <- asks chatLock
@@ -437,7 +436,7 @@ agentSubscriber = do
     withLock l . void . runExceptT $
       processAgentMessage user connId msg `catchError` showChatError
 
-subscribeUserConnections :: (MonadUnliftIO m, MonadReader ChatController m) => m ()
+subscribeUserConnections :: (MonadUnliftIO m, MonadReader ChatController m, MonadFail m) => m ()
 subscribeUserConnections = void . runExceptT $ do
   user <- readTVarIO =<< asks currentUser
   subscribeContacts user
@@ -519,7 +518,7 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
   where
     isMember :: MemberId -> Group -> Bool
     isMember memId Group {membership, members} =
-      memberId membership == memId || isJust (find ((== memId) . memberId) members)
+      sameMemberId memId membership || isJust (find (sameMemberId memId) members)
 
     contactIsReady :: Contact -> Bool
     contactIsReady Contact {activeConn} = connStatus activeConn == ConnReady
@@ -548,13 +547,17 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
           ackMsgDeliveryEvent conn meta
         SENT msgId ->
           sentMsgDeliveryEvent conn msgId
+        -- TODO print errors
+        MERR _ _ -> pure ()
+        ERR _ -> pure ()
+        -- TODO add debugging output
         _ -> pure ()
       Just ct@Contact {localDisplayName = c} -> case agentMsg of
         MSG meta msgBody -> do
           chatMsgEvent <- saveRcvMSG conn meta msgBody
           withAckMessage agentConnId meta $
             case chatMsgEvent of
-              XMsgNew (MsgContent MTText [] body) -> newTextMessage c meta $ find (isSimplexContentType XCText) body
+              XMsgNew (MCText text) -> newTextMessage c meta text
               XFile fInv -> processFileInvitation ct meta fInv
               XInfo p -> xInfo ct p
               XGrpInv gInv -> processGroupInvitation ct gInv
@@ -607,6 +610,10 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
           showContactSubscribed c
           showToast (c <> "> ") "is active"
           setActive $ ActiveC c
+        -- TODO print errors
+        MERR _ _ -> pure ()
+        ERR _ -> pure ()
+        -- TODO add debugging output
         _ -> pure ()
 
     processGroupMessage :: ACommand 'Agent -> Connection -> GroupName -> GroupMember -> m ()
@@ -617,7 +624,7 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
           GCInviteeMember ->
             case chatMsgEvent of
               XGrpAcpt memId
-                | memId == memberId m -> do
+                | sameMemberId memId m -> do
                   withStore $ \st -> updateGroupMemberStatus st userId m GSMemAccepted
                   allowAgentConnection conn confId XOk
                 | otherwise -> messageError "x.grp.acpt: memberId is different from expected"
@@ -625,17 +632,17 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
           _ ->
             case chatMsgEvent of
               XGrpMemInfo memId _memProfile
-                | memId == memberId m -> do
+                | sameMemberId memId m -> do
                   -- TODO update member profile
                   Group {membership} <- withStore $ \st -> getGroup st user gName
-                  allowAgentConnection conn confId $ XGrpMemInfo (memberId membership) profile
+                  allowAgentConnection conn confId $ XGrpMemInfo (memberId (membership :: GroupMember)) profile
                 | otherwise -> messageError "x.grp.mem.info: memberId is different from expected"
               _ -> messageError "CONF from member must have x.grp.mem.info"
       INFO connInfo -> do
         ChatMessage {chatMsgEvent} <- liftEither $ parseChatMessage connInfo
         case chatMsgEvent of
           XGrpMemInfo memId _memProfile
-            | memId == memberId m -> do
+            | sameMemberId memId m -> do
               -- TODO update member profile
               pure ()
             | otherwise -> messageError "x.grp.mem.info: memberId is different from expected"
@@ -678,8 +685,7 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
         chatMsgEvent <- saveRcvMSG conn meta msgBody
         withAckMessage agentConnId meta $
           case chatMsgEvent of
-            XMsgNew (MsgContent MTText [] body) ->
-              newGroupTextMessage gName m meta $ find (isSimplexContentType XCText) body
+            XMsgNew (MCText text) -> newGroupTextMessage gName m meta text
             XFile fInv -> processGroupFileInvitation gName m meta fInv
             XGrpMemNew memInfo -> xGrpMemNew gName m memInfo
             XGrpMemIntro memInfo -> xGrpMemIntro conn gName m memInfo
@@ -692,6 +698,10 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
         ackMsgDeliveryEvent conn meta
       SENT msgId ->
         sentMsgDeliveryEvent conn msgId
+      -- TODO print errors
+      MERR _ _ -> pure ()
+      ERR _ -> pure ()
+      -- TODO add debugging output
       _ -> pure ()
 
     processSndFileConn :: ACommand 'Agent -> Connection -> SndFileTransfer -> m ()
@@ -721,6 +731,9 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
             _ -> chatError $ CEFileSend fileId err
         MSG meta _ ->
           withAckMessage agentConnId meta $ pure ()
+        -- TODO print errors
+        ERR _ -> pure ()
+        -- TODO add debugging output
         _ -> pure ()
 
     processRcvFileConn :: ACommand 'Agent -> Connection -> RcvFileTransfer -> m ()
@@ -731,10 +744,10 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
           showRcvFileStart ft
         MSG meta@MsgMeta {recipient = (msgId, _), integrity} msgBody -> withAckMessage agentConnId meta $ do
           parseFileChunk msgBody >>= \case
-            (0, _) -> do
+            FileChunkCancel -> do
               cancelRcvFileTransfer ft
               showRcvFileSndCancelled ft
-            (chunkNo, chunk) -> do
+            FileChunk {chunkNo, chunkBytes = chunk} -> do
               case integrity of
                 MsgOk -> pure ()
                 MsgError MsgDuplicate -> pure () -- TODO remove once agent removes duplicates
@@ -758,6 +771,10 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
                       withAgent (`deleteConnection` agentConnId)
                 RcvChunkDuplicate -> pure ()
                 RcvChunkError -> badRcvFileChunk ft $ "incorrect chunk number " <> show chunkNo
+        -- TODO print errors
+        MERR _ _ -> pure ()
+        ERR _ -> pure ()
+        -- TODO add debugging output
         _ -> pure ()
 
     processUserContactRequest :: ACommand 'Agent -> Connection -> UserContact -> m ()
@@ -769,12 +786,17 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
           XInfo p -> profileContactRequest invId p
           -- TODO show/log error, other events in contact request
           _ -> pure ()
+      -- TODO print errors
+      MERR _ _ -> pure ()
+      ERR _ -> pure ()
+      -- TODO add debugging output
       _ -> pure ()
       where
         profileContactRequest :: InvitationId -> Profile -> m ()
         profileContactRequest invId p = do
           cName <- withStore $ \st -> createContactRequest st userId userContactLinkId invId p
           showReceivedContactRequest cName p
+          showToast (cName <> "> ") "wants to connect to you"
 
     withAckMessage :: ConnId -> MsgMeta -> m () -> m ()
     withAckMessage cId MsgMeta {recipient = (msgId, _)} action =
@@ -808,7 +830,7 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
       (probe, probeId) <- withStore $ \st -> createSentProbe st gVar userId ct
       sendDirectMessage (contactConn ct) $ XInfoProbe probe
       cs <- withStore (\st -> getMatchingContacts st userId ct)
-      let probeHash = C.sha256Hash probe
+      let probeHash = ProbeHash $ C.sha256Hash (unProbe probe)
       forM_ cs $ \c -> sendProbeHash c probeHash probeId `catchError` const (pure ())
       where
         sendProbeHash c probeHash probeId = do
@@ -821,23 +843,17 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
     messageError :: Text -> m ()
     messageError = showMessageError "error"
 
-    newTextMessage :: ContactName -> MsgMeta -> Maybe MsgContentBody -> m ()
-    newTextMessage c meta = \case
-      Just MsgContentBody {contentData = bs} -> do
-        let text = safeDecodeUtf8 bs
-        showReceivedMessage c (snd $ broker meta) (msgPlain text) (integrity (meta :: MsgMeta))
-        showToast (c <> "> ") text
-        setActive $ ActiveC c
-      _ -> messageError "x.msg.new: no expected message body"
+    newTextMessage :: ContactName -> MsgMeta -> Text -> m ()
+    newTextMessage c meta text = do
+      showReceivedMessage c (snd $ broker meta) (msgPlain text) (integrity (meta :: MsgMeta))
+      showToast (c <> "> ") text
+      setActive $ ActiveC c
 
-    newGroupTextMessage :: GroupName -> GroupMember -> MsgMeta -> Maybe MsgContentBody -> m ()
-    newGroupTextMessage gName GroupMember {localDisplayName = c} meta = \case
-      Just MsgContentBody {contentData = bs} -> do
-        let text = safeDecodeUtf8 bs
-        showReceivedGroupMessage gName c (snd $ broker meta) (msgPlain text) (integrity (meta :: MsgMeta))
-        showToast ("#" <> gName <> " " <> c <> "> ") text
-        setActive $ ActiveG gName
-      _ -> messageError "x.msg.new: no expected message body"
+    newGroupTextMessage :: GroupName -> GroupMember -> MsgMeta -> Text -> m ()
+    newGroupTextMessage gName GroupMember {localDisplayName = c} meta text = do
+      showReceivedGroupMessage gName c (snd $ broker meta) (msgPlain text) (integrity (meta :: MsgMeta))
+      showToast ("#" <> gName <> " " <> c <> "> ") text
+      setActive $ ActiveG gName
 
     processFileInvitation :: Contact -> MsgMeta -> FileInvitation -> m ()
     processFileInvitation contact@Contact {localDisplayName = c} meta fInv = do
@@ -845,6 +861,7 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
       chSize <- asks $ fileChunkSize . config
       ft <- withStore $ \st -> createRcvFileTransfer st userId contact fInv chSize
       showReceivedMessage c (snd $ broker meta) (receivedFileInvitation ft) (integrity (meta :: MsgMeta))
+      showToast (c <> "> ") "wants to send a file"
       setActive $ ActiveC c
 
     processGroupFileInvitation :: GroupName -> GroupMember -> MsgMeta -> FileInvitation -> m ()
@@ -852,37 +869,39 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
       chSize <- asks $ fileChunkSize . config
       ft <- withStore $ \st -> createRcvGroupFileTransfer st userId m fInv chSize
       showReceivedGroupMessage gName c (snd $ broker meta) (receivedFileInvitation ft) (integrity (meta :: MsgMeta))
+      showToast ("#" <> gName <> " " <> c <> "> ") "wants to send a file"
       setActive $ ActiveG gName
 
     processGroupInvitation :: Contact -> GroupInvitation -> m ()
-    processGroupInvitation ct@Contact {localDisplayName} inv@(GroupInvitation (fromMemId, fromRole) (memId, memRole) _ _) = do
-      when (fromRole < GRAdmin || fromRole < memRole) $ chatError (CEGroupContactRole localDisplayName)
+    processGroupInvitation ct@Contact {localDisplayName = c} inv@(GroupInvitation (MemberIdRole fromMemId fromRole) (MemberIdRole memId memRole) _ _) = do
+      when (fromRole < GRAdmin || fromRole < memRole) $ chatError (CEGroupContactRole c)
       when (fromMemId == memId) $ chatError CEGroupDuplicateMemberId
-      group <- withStore $ \st -> createGroupInvitation st user ct inv
-      showReceivedGroupInvitation group localDisplayName memRole
+      group@Group {localDisplayName = gName} <- withStore $ \st -> createGroupInvitation st user ct inv
+      showReceivedGroupInvitation group c memRole
+      showToast ("#" <> gName <> " " <> c <> "> ") $ "invited you to join the group"
 
     xInfo :: Contact -> Profile -> m ()
     xInfo c@Contact {profile = p} p' = unless (p == p') $ do
       c' <- withStore $ \st -> updateContactProfile st userId c p'
       showContactUpdated c c'
 
-    xInfoProbe :: Contact -> ByteString -> m ()
+    xInfoProbe :: Contact -> Probe -> m ()
     xInfoProbe c2 probe = do
       r <- withStore $ \st -> matchReceivedProbe st userId c2 probe
       forM_ r $ \c1 -> probeMatch c1 c2 probe
 
-    xInfoProbeCheck :: Contact -> ByteString -> m ()
+    xInfoProbeCheck :: Contact -> ProbeHash -> m ()
     xInfoProbeCheck c1 probeHash = do
       r <- withStore $ \st -> matchReceivedProbeHash st userId c1 probeHash
       forM_ r . uncurry $ probeMatch c1
 
-    probeMatch :: Contact -> Contact -> ByteString -> m ()
+    probeMatch :: Contact -> Contact -> Probe -> m ()
     probeMatch c1@Contact {profile = p1} c2@Contact {profile = p2} probe =
       when (p1 == p2) $ do
         sendDirectMessage (contactConn c1) $ XInfoProbeOk probe
         mergeContacts c1 c2
 
-    xInfoProbeOk :: Contact -> ByteString -> m ()
+    xInfoProbeOk :: Contact -> Probe -> m ()
     xInfoProbeOk c1 probe = do
       r <- withStore $ \st -> matchSentProbe st userId c1 probe
       forM_ r $ \c2 -> mergeContacts c1 c2
@@ -904,7 +923,7 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
     xGrpMemNew :: GroupName -> GroupMember -> MemberInfo -> m ()
     xGrpMemNew gName m memInfo@(MemberInfo memId _ _) = do
       group@Group {membership} <- withStore $ \st -> getGroup st user gName
-      when (memberId membership /= memId) $
+      unless (sameMemberId memId membership) $
         if isMember memId group
           then messageError "x.grp.mem.new error: member already exists"
           else do
@@ -932,7 +951,7 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
       case memberCategory m of
         GCInviteeMember -> do
           group <- withStore $ \st -> getGroup st user gName
-          case find ((== memId) . memberId) $ members group of
+          case find (sameMemberId memId) $ members group of
             Nothing -> messageError "x.grp.mem.inv error: referenced member does not exists"
             Just reMember -> do
               intro <- withStore $ \st -> saveIntroInvitation st reMember m introInv
@@ -946,7 +965,7 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
     xGrpMemFwd :: GroupName -> GroupMember -> MemberInfo -> IntroInvitation -> m ()
     xGrpMemFwd gName m memInfo@(MemberInfo memId _ _) introInv@IntroInvitation {groupConnReq, directConnReq} = do
       group@Group {membership} <- withStore $ \st -> getGroup st user gName
-      toMember <- case find ((== memId) . memberId) $ members group of
+      toMember <- case find (sameMemberId memId) $ members group of
         -- TODO if the missed messages are correctly sent as soon as there is connection before anything else is sent
         -- the situation when member does not exist is an error
         -- member receiving x.grp.mem.fwd should have also received x.grp.mem.new prior to that.
@@ -954,7 +973,7 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
         Nothing -> withStore $ \st -> createNewGroupMember st user group memInfo GCPostMember GSMemAnnounced
         Just m' -> pure m'
       withStore $ \st -> saveMemberInvitation st toMember introInv
-      let msg = XGrpMemInfo (memberId membership) profile
+      let msg = XGrpMemInfo (memberId (membership :: GroupMember)) profile
       groupConnId <- withAgent $ \a -> joinConnection a groupConnReq $ directMessage msg
       directConnId <- withAgent $ \a -> joinConnection a directConnReq $ directMessage msg
       withStore $ \st -> createIntroToMemberContact st userId m toMember groupConnId directConnId
@@ -962,22 +981,25 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
     xGrpMemDel :: GroupName -> GroupMember -> MemberId -> m ()
     xGrpMemDel gName m memId = do
       Group {membership, members} <- withStore $ \st -> getGroup st user gName
-      if memberId membership == memId
+      if memberId (membership :: GroupMember) == memId
         then do
           mapM_ deleteMemberConnection members
           withStore $ \st -> updateGroupMemberStatus st userId membership GSMemRemoved
           showDeletedMemberUser gName m
-        else case find ((== memId) . memberId) members of
+        else case find (sameMemberId memId) members of
           Nothing -> messageError "x.grp.mem.del with unknown member ID"
           Just member -> do
-            let mRole = memberRole m
-            if mRole < GRAdmin || mRole < memberRole member
+            let mRole = memberRole (m :: GroupMember)
+            if mRole < GRAdmin || mRole < memberRole (member :: GroupMember)
               then messageError "x.grp.mem.del with insufficient member permissions"
               else do
                 deleteMemberConnection member
                 withStore $ \st -> updateGroupMemberStatus st userId member GSMemRemoved
                 showDeletedMember gName (Just m) (Just member)
 
+    sameMemberId :: MemberId -> GroupMember -> Bool
+    sameMemberId memId GroupMember {memberId} = memId == memberId
+
     xGrpLeave :: GroupName -> GroupMember -> m ()
     xGrpLeave gName m = do
       deleteMemberConnection m
@@ -985,8 +1007,8 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
       showLeftMember gName m
 
     xGrpDel :: GroupName -> GroupMember -> m ()
-    xGrpDel gName m = do
-      when (memberRole m /= GROwner) $ chatError CEGroupUserRole
+    xGrpDel gName m@GroupMember {memberRole} = do
+      when (memberRole /= GROwner) $ chatError CEGroupUserRole
       ms <- withStore $ \st -> do
         Group {members, membership} <- getGroup st user gName
         updateGroupMemberStatus st userId membership GSMemGroupDeleted
@@ -995,7 +1017,7 @@ processAgentMessage user@User {userId, profile} agentConnId agentMessage = do
       showGroupDeleted gName m
 
 parseChatMessage :: ByteString -> Either ChatError ChatMessage
-parseChatMessage msgBody = first ChatErrorMessage (parseAll rawChatMessageP msgBody >>= toChatMessage)
+parseChatMessage = first ChatErrorMessage . strDecode
 
 sendFileChunk :: ChatMonad m => SndFileTransfer -> m ()
 sendFileChunk ft@SndFileTransfer {fileId, fileStatus, agentConnId} =
@@ -1012,8 +1034,8 @@ sendFileChunk ft@SndFileTransfer {fileId, fileStatus, agentConnId} =
 
 sendFileChunkNo :: ChatMonad m => SndFileTransfer -> Integer -> m ()
 sendFileChunkNo ft@SndFileTransfer {agentConnId} chunkNo = do
-  bytes <- readFileChunk ft chunkNo
-  msgId <- withAgent $ \a -> sendMessage a agentConnId $ serializeFileChunk chunkNo bytes
+  chunkBytes <- readFileChunk ft chunkNo
+  msgId <- withAgent $ \a -> sendMessage a agentConnId $ smpEncode FileChunk {chunkNo, chunkBytes}
   withStore $ \st -> updateSndFileChunkMsg st ft chunkNo msgId
 
 readFileChunk :: ChatMonad m => SndFileTransfer -> Integer -> m ByteString
@@ -1027,13 +1049,24 @@ readFileChunk SndFileTransfer {fileId, filePath, chunkSize} chunkNo =
       when (pos /= pos') $ hSeek h AbsoluteSeek pos'
       liftIO . B.hGet h $ fromInteger chunkSize
 
-parseFileChunk :: ChatMonad m => ByteString -> m (Integer, ByteString)
-parseFileChunk msg =
-  liftEither . first (ChatError . CEFileRcvChunk) $
-    parseAll ((,) <$> A.decimal <* A.space <*> A.takeByteString) msg
+data FileChunk = FileChunk {chunkNo :: Integer, chunkBytes :: ByteString} | FileChunkCancel
 
-serializeFileChunk :: Integer -> ByteString -> ByteString
-serializeFileChunk chunkNo bytes = bshow chunkNo <> " " <> bytes
+instance Encoding FileChunk where
+  smpEncode = \case
+    FileChunk {chunkNo, chunkBytes} -> smpEncode ('F', fromIntegral chunkNo :: Word32, Tail chunkBytes)
+    FileChunkCancel -> smpEncode 'C'
+  smpP =
+    smpP >>= \case
+      'F' -> do
+        chunkNo <- fromIntegral <$> smpP @Word32
+        Tail chunkBytes <- smpP
+        pure FileChunk {chunkNo, chunkBytes}
+      'C' -> pure FileChunkCancel
+      _ -> fail "bad FileChunk"
+
+parseFileChunk :: ChatMonad m => ByteString -> m FileChunk
+parseFileChunk msg =
+  liftEither . first (ChatError . CEFileRcvChunk) $ parseAll smpP msg
 
 appendFileChunk :: ChatMonad m => RcvFileTransfer -> Integer -> ByteString -> m ()
 appendFileChunk ft@RcvFileTransfer {fileId, fileStatus} chunkNo chunk =
@@ -1083,7 +1116,7 @@ cancelSndFileTransfer ft@SndFileTransfer {agentConnId, fileStatus} =
       updateSndFileStatus st ft FSCancelled
       deleteSndFileChunks st ft
     withAgent $ \a -> do
-      void (sendMessage a agentConnId "0 ") `catchError` \_ -> pure ()
+      void (sendMessage a agentConnId $ smpEncode FileChunkCancel) `catchError` \_ -> pure ()
       suspendConnection a agentConnId
 
 closeFileHandle :: ChatMonad m => Int64 -> (ChatController -> TVar (Map Int64 Handle)) -> m ()
@@ -1105,15 +1138,13 @@ deleteMemberConnection m@GroupMember {activeConn} = do
 sendDirectMessage :: ChatMonad m => Connection -> ChatMsgEvent -> m ()
 sendDirectMessage conn chatMsgEvent = do
   let msgBody = directMessage chatMsgEvent
-      newMsg = NewMessage {direction = MDSnd, chatMsgEventType = toChatEventType chatMsgEvent, msgBody}
+      newMsg = NewMessage {direction = MDSnd, chatMsgEventType = toChatEventTag chatMsgEvent, msgBody}
   -- can be done in transaction after sendMessage, probably shouldn't
   msgId <- withStore $ \st -> createNewMessage st newMsg
   deliverMessage conn msgBody msgId
 
 directMessage :: ChatMsgEvent -> ByteString
-directMessage chatMsgEvent =
-  serializeRawChatMessage $
-    rawChatMessage ChatMessage {chatMsgId = Nothing, chatMsgEvent, chatDAG = Nothing}
+directMessage chatMsgEvent = strEncode ChatMessage {chatMsgEvent}
 
 deliverMessage :: ChatMonad m => Connection -> MsgBody -> MessageId -> m ()
 deliverMessage Connection {connId, agentConnId} msgBody msgId = do
@@ -1124,7 +1155,7 @@ deliverMessage Connection {connId, agentConnId} msgBody msgId = do
 sendGroupMessage :: ChatMonad m => [GroupMember] -> ChatMsgEvent -> m ()
 sendGroupMessage members chatMsgEvent = do
   let msgBody = directMessage chatMsgEvent
-      newMsg = NewMessage {direction = MDSnd, chatMsgEventType = toChatEventType chatMsgEvent, msgBody}
+      newMsg = NewMessage {direction = MDSnd, chatMsgEventType = toChatEventTag chatMsgEvent, msgBody}
   msgId <- withStore $ \st -> createNewMessage st newMsg
   -- TODO once scheduled delivery is implemented memberActive should be changed to memberCurrent
   forM_ (map memberConn $ filter memberActive members) $
@@ -1133,7 +1164,7 @@ sendGroupMessage members chatMsgEvent = do
 saveRcvMSG :: ChatMonad m => Connection -> MsgMeta -> MsgBody -> m ChatMsgEvent
 saveRcvMSG Connection {connId} agentMsgMeta msgBody = do
   ChatMessage {chatMsgEvent} <- liftEither $ parseChatMessage msgBody
-  let newMsg = NewMessage {direction = MDRcv, chatMsgEventType = toChatEventType chatMsgEvent, msgBody}
+  let newMsg = NewMessage {direction = MDRcv, chatMsgEventType = toChatEventTag chatMsgEvent, msgBody}
       agentMsgId = fst $ recipient agentMsgMeta
       rcvMsgDelivery = RcvMsgDelivery {connId, agentMsgId, agentMsgMeta}
   withStore $ \st -> createNewMessageAndRcvMsgDelivery st newMsg rcvMsgDelivery
@@ -1241,7 +1272,7 @@ chatCommandP =
     <|> ("/groups" <|> "/gs") $> ListGroups
     <|> A.char '#' *> (SendGroupMessage <$> displayName <* A.space <*> A.takeByteString)
     <|> ("/contacts" <|> "/cs") $> ListContacts
-    <|> ("/connect " <|> "/c ") *> (Connect <$> ((Just <$> connReqP) <|> A.takeByteString $> Nothing))
+    <|> ("/connect " <|> "/c ") *> (Connect <$> ((Just <$> strP) <|> A.takeByteString $> Nothing))
     <|> ("/connect" <|> "/c") $> AddContact
     <|> ("/delete @" <|> "/delete " <|> "/d @" <|> "/d ") *> (DeleteContact <$> displayName)
     <|> A.char '@' *> (SendMessage <$> displayName <*> (A.space *> A.takeByteString))
@@ -1250,8 +1281,7 @@ chatCommandP =
     <|> ("/freceive " <|> "/fr ") *> (ReceiveFile <$> A.decimal <*> optional (A.space *> filePath))
     <|> ("/fcancel " <|> "/fc ") *> (CancelFile <$> A.decimal)
     <|> ("/fstatus " <|> "/fs ") *> (FileStatus <$> A.decimal)
-    <|> "/admin_welcome " *> (SendAdminWelcome <$> displayName)
-    <|> "/admin" $> ConnectAdmin
+    <|> "/simplex" $> ConnectAdmin
     <|> ("/address" <|> "/ad") $> CreateMyAddress
     <|> ("/delete_address" <|> "/da") $> DeleteMyAddress
     <|> ("/show_address" <|> "/sa") $> ShowMyAddress
@@ -1284,6 +1314,6 @@ chatCommandP =
         <|> (" member" $> GRMember)
         <|> pure GRAdmin
 
-adminContactReq :: ConnectionRequest 'CMContact
+adminContactReq :: ConnReqContact
 adminContactReq =
-  either error id $ parseAll connReqP' "https://simplex.chat/contact#/?smp=smp%3A%2F%2Fnxc7HnrnM8dOKgkMp008ub_9o9LXJlxlMrMpR-mfMQw%3D%40smp3.simplex.im%2F-TXnePw5eH5-4L7B%23&e2e=rsa%3AMIIBoDANBgkqhkiG9w0BAQEFAAOCAY0AMIIBiAKCAQEA6vpcsZggnYL38Qa2G5YU0W5uqnV8WAq_S3flIFU2kx4qW-aokVT8fo0CLJXv9aagdHObFfhc9SXcZPcm4T2NLnafKTgQa_HYFfj764l6cHkbSI-4JBE1gyhtaapsvrDGIdoiGDLgsF3AJVjqs8gavkuTsmw035aWMH-pkpc4qGlEWpNWp1Nn-7O4sdIIQ7yN48jsdCfeIY-BIk3kFR6s4oQOgiOcnir8e3x5tTuRMX1KWSiuzuqLHqgmcI1IqcPJPrBoTQLbXXEMGG1RsvIudxR03jejXXbQvlxXlNNrxwkniEe-P0rApGuCyv2NRMb4n0Wd3ZwewH7X-xtr16XNbQKBgDouGUHD1C55jB-w8W8VJRhFZS2xIYka9gJH1jjCFxHFzgjo69A_sObIamND1pF_JOzj_XCoA1fDICF95XbfS0rq9iS6xvX6M8Muq8QiJsfD5bRt5nh-Y3GK5rAFXS0ZtyOeh07iMLAMJ_EFxBQuKKDRu9_9KAvLL_plU0PuaMH3"
+  either error id $ strDecode "https://simplex.chat/contact#/?v=1&smp=smp%3A%2F%2FPQUV2eL0t7OStZOoAsPEV2QYWt4-xilbakvGUGOItUo%3D%40smp6.simplex.im%2FK1rslx-m5bpXVIdMZg9NLUZ_8JBm8xTt%23MCowBQYDK2VuAyEALDeVe-sG8mRY22LsXlPgiwTNs9dbiLrNuA7f3ZMAJ2w%3D"

src/Simplex/Chat/Controller.hs

@@ -26,7 +26,13 @@ import System.IO (Handle)
 import UnliftIO.STM
 
 versionNumber :: String
-versionNumber = "0.5.5"
+versionNumber = "1.0.0"
+
+versionStr :: String
+versionStr = "SimpleX Chat v" <> versionNumber
+
+updateStr :: String
+updateStr = "To update run: curl -o- https://raw.githubusercontent.com/simplex-chat/simplex-chat/master/install.sh | bash"
 
 data ChatConfig = ChatConfig
   { agentConfig :: AgentConfig,
@@ -78,9 +84,10 @@ data ChatErrorType
   | CEFileSend Int64 AgentErrorType
   | CEFileRcvChunk String
   | CEFileInternal String
+  | CEAgentVersion
   deriving (Show, Exception)
 
-type ChatMonad m = (MonadUnliftIO m, MonadReader ChatController m, MonadError ChatError m)
+type ChatMonad m = (MonadUnliftIO m, MonadReader ChatController m, MonadError ChatError m, MonadFail m)
 
 setActive :: (MonadUnliftIO m, MonadReader ChatController m) => ActiveTo -> m ()
 setActive to = asks (activeTo . chatTerminal) >>= atomically . (`writeTVar` to)

src/Simplex/Chat/Help.hs

@@ -3,7 +3,6 @@
 
 module Simplex.Chat.Help
   ( chatWelcome,
-    adminWelcomeMessages,
     chatHelpInfo,
     filesHelpInfo,
     groupsHelpInfo,
@@ -12,13 +11,12 @@ module Simplex.Chat.Help
   )
 where
 
-import Data.ByteString (ByteString)
 import Data.List (intersperse)
 import Data.Text (Text)
 import qualified Data.Text as T
 import Simplex.Chat.Markdown
 import Simplex.Chat.Styled
-import Simplex.Chat.Types (Onboarding (..), Profile (..), User (..))
+import Simplex.Chat.Types (Profile (..), User (..))
 import System.Console.ANSI.Types
 
 highlight :: Text -> Markdown
@@ -33,8 +31,8 @@ indent = "        "
 listHighlight :: [Text] -> Markdown
 listHighlight = mconcat . intersperse ", " . map highlight
 
-chatWelcome :: User -> Onboarding -> [StyledString]
-chatWelcome user Onboarding {contactsCount, createdGroups, membersCount, filesSentCount, addressCount} =
+chatWelcome :: User -> [StyledString]
+chatWelcome user =
   map
     styleMarkdown
     [ "                             __   __",
@@ -46,37 +44,21 @@ chatWelcome user Onboarding {contactsCount, createdGroups, membersCount, filesSe
       "Welcome " <> green userName <> "!",
       "Thank you for installing SimpleX Chat!",
       "",
-      "We have created several groups that you can join to play with SimpleX Chat:",
-      highlight "#simplex" <> " (SimpleX Engineers 💻) - technical questions about running or contributing to SimpleX Chat",
-      highlight "#hacks" <> " (Ethical Hacking 🔓) - chatting about privacy, security, announced vulnerabilities etc.",
+      "We have a couple of groups that you can join to play with SimpleX Chat:",
+      highlight "#termux" <> " (Android Termux 📱) - chatting about using SimpleX Chat on Android devices",
       highlight "#music" <> " (Music 🎸) - favorite music of our team and users",
-      highlight "#rand" <> " (Random 😇) - anything interesting, just keep it decent and friendly please :)",
       "",
-      "Connect to our groups admin to be added to these groups - " <> highlight "/admin",
+      "Connect to SimpleX Chat team to be added to these groups - type " <> highlight "/simplex",
       "",
-      "To continue:",
-      "[" <> check (contactsCount >= 2) <> "] " <> highlight "/connect" <> " with 2 friends - " <> highlight "/help" <> " for instructions",
-      "[" <> check (createdGroups >= 1 && membersCount >= 2) <> "] create a " <> highlight "/group" <> " with them - " <> highlight "/g #friends",
-      "[" <> check (filesSentCount >= 1) <> "] send " <> highlight "/file" <> ", e.g. your photo, to the group - " <> highlight "/f #friends ./photo.jpg",
-      "[" <> check (addressCount >= 1) <> "] create your optional chat " <> highlight "/address" <> " and share it with your friends - " <> highlight "/ad",
+      "Follow our updates:",
+      "> Reddit: https://www.reddit.com/r/SimpleXChat/",
+      "> Twitter: https://twitter.com/SimpleXChat",
       "",
-      "To help us build SimpleX Chat:",
-      "> star GitHub repo: https://github.com/simplex-chat/simplex-chat",
-      "> join Reddit group: https://www.reddit.com/r/SimpleXChat/",
-      "",
-      "To show this message again - " <> highlight "/welcome" <> " (or " <> highlight "/w" <> ")"
+      "Type " <> highlight "/help" <> " for usage info, " <> highlight "/welcome" <> " to show this message"
     ]
   where
     User {profile = Profile {displayName, fullName}} = user
     userName = if T.null fullName then displayName else fullName
-    check c = if c then green "*" else " "
-
-adminWelcomeMessages :: [ByteString]
-adminWelcomeMessages =
-  [ "Hello - and welcome to SimpleX Chat!",
-    "Which community groups you'd like to join:",
-    "!5 #simplex!, !5 #hacks!, !5 #music! or !5 #rand!"
-  ]
 
 chatHelpInfo :: [StyledString]
 chatHelpInfo =

src/Simplex/Chat/Options.hs

@@ -7,7 +7,9 @@ import qualified Data.ByteString.Char8 as B
 import Data.List.NonEmpty (NonEmpty)
 import qualified Data.List.NonEmpty as L
 import Options.Applicative
-import Simplex.Messaging.Agent.Protocol (SMPServer (..), smpServerP)
+import Simplex.Chat.Controller (updateStr, versionStr)
+import Simplex.Messaging.Agent.Protocol (SMPServer (..))
+import Simplex.Messaging.Encoding.String
 import Simplex.Messaging.Parsers (parseAll)
 import System.FilePath (combine)
 
@@ -23,8 +25,9 @@ chatOpts appDir =
       ( long "database"
           <> short 'd'
           <> metavar "DB_FILE"
-          <> help ("sqlite database file path (" <> defaultDbFilePath <> ")")
+          <> help "Path prefix to chat and agent database files"
           <> value defaultDbFilePath
+          <> showDefault
       )
     <*> option
       parseSMPServer
@@ -32,31 +35,30 @@ chatOpts appDir =
           <> short 's'
           <> metavar "SERVER"
           <> help
-            ( "SMP server(s) to use"
-                <> "\n(smp2.simplex.im,smp3.simplex.im)"
-            )
+            "Comma separated list of SMP server(s) to use \
+            \(default: smp4.simplex.im,smp5.simplex.im,smp6.simplex.im)"
           <> value
             ( L.fromList
-                [ "smp2.simplex.im#z5W2QLQ1Br3Yd6CoWg7bIq1bHdwK7Y8bEiEXBs/WfAg=", -- London, UK
-                  "smp3.simplex.im#nxc7HnrnM8dOKgkMp008ub/9o9LXJlxlMrMpR+mfMQw=" -- Fremont, CA
+                [ "smp://u2dS9sG8nMNURyZwqASV4yROM28Er0luVTx5X1CsMrU=@smp4.simplex.im",
+                  "smp://hpq7_4gGJiilmz5Rf-CswuU5kZGkm_zOIooSw6yALRg=@smp5.simplex.im",
+                  "smp://PQUV2eL0t7OStZOoAsPEV2QYWt4-xilbakvGUGOItUo=@smp6.simplex.im"
                 ]
             )
       )
   where
-    defaultDbFilePath = combine appDir "simplex"
+    defaultDbFilePath = combine appDir "simplex_v1"
 
 parseSMPServer :: ReadM (NonEmpty SMPServer)
 parseSMPServer = eitherReader $ parseAll servers . B.pack
   where
-    servers = L.fromList <$> smpServerP `A.sepBy1` A.char ','
+    servers = L.fromList <$> strP `A.sepBy1` A.char ','
 
 getChatOpts :: FilePath -> IO ChatOpts
-getChatOpts appDir = execParser opts
+getChatOpts appDir =
+  execParser $
+    info
+      (helper <*> versionOption <*> chatOpts appDir)
+      (header versionStr <> fullDesc <> progDesc "Start chat with DB_FILE file and use SERVER as SMP server")
   where
-    opts =
-      info
-        (chatOpts appDir <**> helper)
-        ( fullDesc
-            <> header "Chat prototype using Simplex Messaging Protocol (SMP)"
-            <> progDesc "Start chat with DB_FILE file and use SERVER as SMP server"
-        )
+    versionOption = infoOption versionAndUpdate (long "version" <> short 'v' <> help "Show version")
+    versionAndUpdate = versionStr <> "\n" <> updateStr

src/Simplex/Chat/Protocol.hs

@@ -1,35 +1,32 @@
 {-# LANGUAGE DataKinds #-}
+{-# LANGUAGE DeriveAnyClass #-}
+{-# LANGUAGE DeriveGeneric #-}
 {-# LANGUAGE DuplicateRecordFields #-}
 {-# LANGUAGE GADTs #-}
 {-# LANGUAGE KindSignatures #-}
 {-# LANGUAGE LambdaCase #-}
 {-# LANGUAGE NamedFieldPuns #-}
 {-# LANGUAGE OverloadedStrings #-}
+{-# LANGUAGE RankNTypes #-}
+{-# LANGUAGE ScopedTypeVariables #-}
 {-# LANGUAGE StandaloneDeriving #-}
-{-# LANGUAGE TupleSections #-}
 
 module Simplex.Chat.Protocol where
 
-import Control.Applicative (optional)
-import Control.Monad ((<=<), (>=>))
-import Data.Aeson (FromJSON, ToJSON)
+import Control.Monad ((<=<))
+import Data.Aeson (FromJSON, ToJSON, (.:), (.:?), (.=))
 import qualified Data.Aeson as J
-import Data.Attoparsec.ByteString.Char8 (Parser)
+import qualified Data.Aeson.Types as JT
 import qualified Data.Attoparsec.ByteString.Char8 as A
-import qualified Data.ByteString.Base64 as B64
-import Data.ByteString.Char8 (ByteString)
-import qualified Data.ByteString.Char8 as B
 import qualified Data.ByteString.Lazy.Char8 as LB
-import Data.Int (Int64)
-import Data.List (find, findIndex)
+import qualified Data.HashMap.Strict as H
 import Data.Text (Text)
-import qualified Data.Text as T
-import Data.Text.Encoding (encodeUtf8)
+import Data.Text.Encoding (decodeLatin1, encodeUtf8)
+import GHC.Generics
 import Simplex.Chat.Types
-import Simplex.Chat.Util (safeDecodeUtf8)
 import Simplex.Messaging.Agent.Protocol
-import Simplex.Messaging.Parsers (parseAll)
-import Simplex.Messaging.Util (bshow)
+import Simplex.Messaging.Encoding.String
+import Simplex.Messaging.Util ((<$?>))
 
 data ChatDirection (p :: AParty) where
   ReceivedDirectMessage :: Connection -> Maybe Contact -> ChatDirection 'Agent
@@ -52,6 +49,23 @@ fromConnection = \case
   RcvFileConnection conn _ -> conn
   UserContactConnection conn _ -> conn
 
+-- chat message is sent as JSON with these properties
+data AppMessage = AppMessage
+  { event :: Text,
+    params :: J.Object
+  }
+  deriving (Generic, FromJSON)
+
+instance ToJSON AppMessage where toEncoding = J.genericToEncoding J.defaultOptions {J.omitNothingFields = True}
+
+newtype ChatMessage = ChatMessage {chatMsgEvent :: ChatMsgEvent}
+  deriving (Eq, Show)
+
+instance StrEncoding ChatMessage where
+  strEncode = LB.toStrict . J.encode . chatToAppMessage
+  strDecode = appToChatMessage <=< J.eitherDecodeStrict'
+  strP = strDecode <$?> A.takeByteString
+
 data ChatMsgEvent
   = XMsgNew MsgContent
   | XFile FileInvitation
@@ -70,372 +84,215 @@ data ChatMsgEvent
   | XGrpMemDel MemberId
   | XGrpLeave
   | XGrpDel
-  | XInfoProbe ByteString
-  | XInfoProbeCheck ByteString
-  | XInfoProbeOk ByteString
+  | XInfoProbe Probe
+  | XInfoProbeCheck ProbeHash
+  | XInfoProbeOk Probe
   | XOk
   deriving (Eq, Show)
 
-data MessageType = MTText | MTImage deriving (Eq, Show)
+data MsgContentType = MCText_ | MCUnknown_
 
-data MsgContent = MsgContent
-  { messageType :: MessageType,
-    files :: [(ContentType, Int)],
-    content :: [MsgContentBody]
-  }
+instance StrEncoding MsgContentType where
+  strEncode = \case
+    MCText_ -> "text"
+    MCUnknown_ -> "text"
+  strDecode = \case
+    "text" -> Right MCText_
+    _ -> Right MCUnknown_
+  strP = strDecode <$?> A.takeTill (== ' ')
+
+instance FromJSON MsgContentType where
+  parseJSON = strParseJSON "MsgContentType"
+
+instance ToJSON MsgContentType where
+  toJSON = strToJSON
+  toEncoding = strToJEncoding
+
+data MsgContent = MCText Text | MCUnknown
   deriving (Eq, Show)
 
-toMsgType :: ByteString -> Either String MessageType
-toMsgType = \case
-  "c.text" -> Right MTText
-  "c.image" -> Right MTImage
-  t -> Left $ "invalid message type " <> B.unpack t
+toMsgContentType :: MsgContent -> MsgContentType
+toMsgContentType = \case
+  MCText _ -> MCText_
+  MCUnknown -> MCUnknown_
 
-rawMsgType :: MessageType -> ByteString
-rawMsgType = \case
-  MTText -> "c.text"
-  MTImage -> "c.image"
+instance FromJSON MsgContent where
+  parseJSON (J.Object v) = do
+    v .: "type" >>= \case
+      MCText_ -> MCText <$> v .: "text"
+      MCUnknown_ -> pure MCUnknown
+  parseJSON invalid =
+    JT.prependFailure "bad MsgContent, " (JT.typeMismatch "Object" invalid)
 
-data ChatMessage = ChatMessage
-  { chatMsgId :: Maybe Int64,
-    chatMsgEvent :: ChatMsgEvent,
-    chatDAG :: Maybe ByteString
-  }
-  deriving (Eq, Show)
+unknownMsgType :: Text
+unknownMsgType = "unknown message type"
 
-toChatEventType :: ChatMsgEvent -> Text
-toChatEventType = \case
-  XMsgNew _ -> "x.msg.new"
-  XFile _ -> "x.file"
-  XFileAcpt _ -> "x.file.acpt"
-  XInfo _ -> "x.info"
-  XContact _ _ -> "x.con"
-  XGrpInv _ -> "x.grp.inv"
-  XGrpAcpt _ -> "x.grp.acpt"
-  XGrpMemNew _ -> "x.grp.mem.new"
-  XGrpMemIntro _ -> "x.grp.mem.intro"
-  XGrpMemInv _ _ -> "x.grp.mem.inv"
-  XGrpMemFwd _ _ -> "x.grp.mem.fwd"
-  XGrpMemInfo _ _ -> "x.grp.mem.info"
-  XGrpMemCon _ -> "x.grp.mem.con"
-  XGrpMemConAll _ -> "x.grp.mem.con.all"
-  XGrpMemDel _ -> "x.grp.mem.del"
-  XGrpLeave -> "x.grp.leave"
-  XGrpDel -> "x.grp.del"
-  XInfoProbe _ -> "x.info.probe"
-  XInfoProbeCheck _ -> "x.info.probe.check"
-  XInfoProbeOk _ -> "x.info.probe.ok"
-  XOk -> "x.ok"
+instance ToJSON MsgContent where
+  toJSON mc =
+    J.object $
+      ("type" .= toMsgContentType mc) : case mc of
+        MCText t -> ["text" .= t]
+        MCUnknown -> ["text" .= unknownMsgType]
+  toEncoding mc =
+    J.pairs $
+      ("type" .= toMsgContentType mc) <> case mc of
+        MCText t -> "text" .= t
+        MCUnknown -> "text" .= unknownMsgType
 
-toChatMessage :: RawChatMessage -> Either String ChatMessage
-toChatMessage RawChatMessage {chatMsgId, chatMsgEvent, chatMsgParams, chatMsgBody} = do
-  (chatDAG, body) <- getDAG <$> mapM toMsgBodyContent chatMsgBody
-  let chatMsg msg = pure ChatMessage {chatMsgId, chatMsgEvent = msg, chatDAG}
-  case (chatMsgEvent, chatMsgParams) of
-    ("x.msg.new", mt : rawFiles) -> do
-      t <- toMsgType mt
-      files <- toFiles rawFiles
-      chatMsg . XMsgNew $ MsgContent {messageType = t, files, content = body}
-    ("x.file", [name, size, cReq]) -> do
-      let fileName = T.unpack $ safeDecodeUtf8 name
-      fileSize <- parseAll A.decimal size
-      fileConnReq <- parseAll connReqP' cReq
-      chatMsg . XFile $ FileInvitation {fileName, fileSize, fileConnReq}
-    ("x.file.acpt", [name]) ->
-      chatMsg . XFileAcpt . T.unpack $ safeDecodeUtf8 name
-    ("x.info", []) -> do
-      profile <- getJSON body
-      chatMsg $ XInfo profile
-    ("x.con", []) -> do
-      profile <- getJSON body
-      chatMsg $ XContact profile Nothing
-    ("x.con", mt : rawFiles) -> do
-      (profile, body') <- extractJSON body
-      t <- toMsgType mt
-      files <- toFiles rawFiles
-      chatMsg . XContact profile $ Just MsgContent {messageType = t, files, content = body'}
-    ("x.grp.inv", [fromMemId, fromRole, memId, role, cReq]) -> do
-      fromMem <- (,) <$> B64.decode fromMemId <*> toMemberRole fromRole
-      invitedMem <- (,) <$> B64.decode memId <*> toMemberRole role
-      groupConnReq <- parseAll connReqP' cReq
-      profile <- getJSON body
-      chatMsg . XGrpInv $ GroupInvitation fromMem invitedMem groupConnReq profile
-    ("x.grp.acpt", [memId]) ->
-      chatMsg . XGrpAcpt =<< B64.decode memId
-    ("x.grp.mem.new", [memId, role]) -> do
-      chatMsg . XGrpMemNew =<< toMemberInfo memId role body
-    ("x.grp.mem.intro", [memId, role]) ->
-      chatMsg . XGrpMemIntro =<< toMemberInfo memId role body
-    ("x.grp.mem.inv", [memId, groupConnReq, directConnReq]) ->
-      chatMsg =<< (XGrpMemInv <$> B64.decode memId <*> toIntroInv groupConnReq directConnReq)
-    ("x.grp.mem.fwd", [memId, role, groupConnReq, directConnReq]) -> do
-      chatMsg =<< (XGrpMemFwd <$> toMemberInfo memId role body <*> toIntroInv groupConnReq directConnReq)
-    ("x.grp.mem.info", [memId]) ->
-      chatMsg =<< (XGrpMemInfo <$> B64.decode memId <*> getJSON body)
-    ("x.grp.mem.con", [memId]) ->
-      chatMsg . XGrpMemCon =<< B64.decode memId
-    ("x.grp.mem.con.all", [memId]) ->
-      chatMsg . XGrpMemConAll =<< B64.decode memId
-    ("x.grp.mem.del", [memId]) ->
-      chatMsg . XGrpMemDel =<< B64.decode memId
-    ("x.grp.leave", []) ->
-      chatMsg XGrpLeave
-    ("x.grp.del", []) ->
-      chatMsg XGrpDel
-    ("x.info.probe", [probe]) -> do
-      chatMsg . XInfoProbe =<< B64.decode probe
-    ("x.info.probe.check", [probeHash]) -> do
-      chatMsg . XInfoProbeCheck =<< B64.decode probeHash
-    ("x.info.probe.ok", [probe]) -> do
-      chatMsg . XInfoProbeOk =<< B64.decode probe
-    ("x.ok", []) ->
-      chatMsg XOk
-    _ -> Left $ "bad syntax or unsupported event " <> B.unpack chatMsgEvent
+data CMEventTag
+  = XMsgNew_
+  | XFile_
+  | XFileAcpt_
+  | XInfo_
+  | XContact_
+  | XGrpInv_
+  | XGrpAcpt_
+  | XGrpMemNew_
+  | XGrpMemIntro_
+  | XGrpMemInv_
+  | XGrpMemFwd_
+  | XGrpMemInfo_
+  | XGrpMemCon_
+  | XGrpMemConAll_
+  | XGrpMemDel_
+  | XGrpLeave_
+  | XGrpDel_
+  | XInfoProbe_
+  | XInfoProbeCheck_
+  | XInfoProbeOk_
+  | XOk_
+
+instance StrEncoding CMEventTag where
+  strEncode = \case
+    XMsgNew_ -> "x.msg.new"
+    XFile_ -> "x.file"
+    XFileAcpt_ -> "x.file.acpt"
+    XInfo_ -> "x.info"
+    XContact_ -> "x.contact"
+    XGrpInv_ -> "x.grp.inv"
+    XGrpAcpt_ -> "x.grp.acpt"
+    XGrpMemNew_ -> "x.grp.mem.new"
+    XGrpMemIntro_ -> "x.grp.mem.intro"
+    XGrpMemInv_ -> "x.grp.mem.inv"
+    XGrpMemFwd_ -> "x.grp.mem.fwd"
+    XGrpMemInfo_ -> "x.grp.mem.info"
+    XGrpMemCon_ -> "x.grp.mem.con"
+    XGrpMemConAll_ -> "x.grp.mem.con.all"
+    XGrpMemDel_ -> "x.grp.mem.del"
+    XGrpLeave_ -> "x.grp.leave"
+    XGrpDel_ -> "x.grp.del"
+    XInfoProbe_ -> "x.info.probe"
+    XInfoProbeCheck_ -> "x.info.probe.check"
+    XInfoProbeOk_ -> "x.info.probe.ok"
+    XOk_ -> "x.ok"
+  strDecode = \case
+    "x.msg.new" -> Right XMsgNew_
+    "x.file" -> Right XFile_
+    "x.file.acpt" -> Right XFileAcpt_
+    "x.info" -> Right XInfo_
+    "x.contact" -> Right XContact_
+    "x.grp.inv" -> Right XGrpInv_
+    "x.grp.acpt" -> Right XGrpAcpt_
+    "x.grp.mem.new" -> Right XGrpMemNew_
+    "x.grp.mem.intro" -> Right XGrpMemIntro_
+    "x.grp.mem.inv" -> Right XGrpMemInv_
+    "x.grp.mem.fwd" -> Right XGrpMemFwd_
+    "x.grp.mem.info" -> Right XGrpMemInfo_
+    "x.grp.mem.con" -> Right XGrpMemCon_
+    "x.grp.mem.con.all" -> Right XGrpMemConAll_
+    "x.grp.mem.del" -> Right XGrpMemDel_
+    "x.grp.leave" -> Right XGrpLeave_
+    "x.grp.del" -> Right XGrpDel_
+    "x.info.probe" -> Right XInfoProbe_
+    "x.info.probe.check" -> Right XInfoProbeCheck_
+    "x.info.probe.ok" -> Right XInfoProbeOk_
+    "x.ok" -> Right XOk_
+    _ -> Left "bad CMEventTag"
+  strP = strDecode <$?> A.takeTill (== ' ')
+
+toCMEventTag :: ChatMsgEvent -> CMEventTag
+toCMEventTag = \case
+  XMsgNew _ -> XMsgNew_
+  XFile _ -> XFile_
+  XFileAcpt _ -> XFileAcpt_
+  XInfo _ -> XInfo_
+  XContact _ _ -> XContact_
+  XGrpInv _ -> XGrpInv_
+  XGrpAcpt _ -> XGrpAcpt_
+  XGrpMemNew _ -> XGrpMemNew_
+  XGrpMemIntro _ -> XGrpMemIntro_
+  XGrpMemInv _ _ -> XGrpMemInv_
+  XGrpMemFwd _ _ -> XGrpMemFwd_
+  XGrpMemInfo _ _ -> XGrpMemInfo_
+  XGrpMemCon _ -> XGrpMemCon_
+  XGrpMemConAll _ -> XGrpMemConAll_
+  XGrpMemDel _ -> XGrpMemDel_
+  XGrpLeave -> XGrpLeave_
+  XGrpDel -> XGrpDel_
+  XInfoProbe _ -> XInfoProbe_
+  XInfoProbeCheck _ -> XInfoProbeCheck_
+  XInfoProbeOk _ -> XInfoProbeOk_
+  XOk -> XOk_
+
+toChatEventTag :: ChatMsgEvent -> Text
+toChatEventTag = decodeLatin1 . strEncode . toCMEventTag
+
+appToChatMessage :: AppMessage -> Either String ChatMessage
+appToChatMessage AppMessage {event, params} = do
+  eventTag <- strDecode $ encodeUtf8 event
+  chatMsgEvent <- msg eventTag
+  pure ChatMessage {chatMsgEvent}
   where
-    getDAG :: [MsgContentBody] -> (Maybe ByteString, [MsgContentBody])
-    getDAG body = case break (isContentType SimplexDAG) body of
-      (b, MsgContentBody SimplexDAG dag : a) -> (Just dag, b <> a)
-      _ -> (Nothing, body)
-    toMemberInfo :: ByteString -> ByteString -> [MsgContentBody] -> Either String MemberInfo
-    toMemberInfo memId role body = MemberInfo <$> B64.decode memId <*> toMemberRole role <*> getJSON body
-    toIntroInv :: ByteString -> ByteString -> Either String IntroInvitation
-    toIntroInv groupConnReq directConnReq = IntroInvitation <$> parseAll connReqP' groupConnReq <*> parseAll connReqP' directConnReq
-    toContentInfo :: (RawContentType, Int) -> Either String (ContentType, Int)
-    toContentInfo (rawType, size) = (,size) <$> toContentType rawType
-    toFiles :: [ByteString] -> Either String [(ContentType, Int)]
-    toFiles = mapM $ toContentInfo <=< parseAll contentInfoP
-    getJSON :: FromJSON a => [MsgContentBody] -> Either String a
-    getJSON = J.eitherDecodeStrict' <=< getSimplexContentType XCJson
-    extractJSON :: FromJSON a => [MsgContentBody] -> Either String (a, [MsgContentBody])
-    extractJSON =
-      extractSimplexContentType XCJson >=> \(a, bs) -> do
-        j <- J.eitherDecodeStrict' a
-        pure (j, bs)
+    p :: FromJSON a => Text -> Either String a
+    p key = JT.parseEither (.: key) params
+    msg = \case
+      XMsgNew_ -> XMsgNew <$> p "content"
+      XFile_ -> XFile <$> p "file"
+      XFileAcpt_ -> XFileAcpt <$> p "fileName"
+      XInfo_ -> XInfo <$> p "profile"
+      XContact_ -> XContact <$> p "profile" <*> JT.parseEither (.:? "content") params
+      XGrpInv_ -> XGrpInv <$> p "groupInvitation"
+      XGrpAcpt_ -> XGrpAcpt <$> p "memberId"
+      XGrpMemNew_ -> XGrpMemNew <$> p "memberInfo"
+      XGrpMemIntro_ -> XGrpMemIntro <$> p "memberInfo"
+      XGrpMemInv_ -> XGrpMemInv <$> p "memberId" <*> p "memberIntro"
+      XGrpMemFwd_ -> XGrpMemFwd <$> p "memberInfo" <*> p "memberIntro"
+      XGrpMemInfo_ -> XGrpMemInfo <$> p "memberId" <*> p "profile"
+      XGrpMemCon_ -> XGrpMemCon <$> p "memberId"
+      XGrpMemConAll_ -> XGrpMemConAll <$> p "memberId"
+      XGrpMemDel_ -> XGrpMemDel <$> p "memberId"
+      XGrpLeave_ -> pure XGrpLeave
+      XGrpDel_ -> pure XGrpDel
+      XInfoProbe_ -> XInfoProbe <$> p "probe"
+      XInfoProbeCheck_ -> XInfoProbeCheck <$> p "probeHash"
+      XInfoProbeOk_ -> XInfoProbeOk <$> p "probe"
+      XOk_ -> pure XOk
 
-isContentType :: ContentType -> MsgContentBody -> Bool
-isContentType t MsgContentBody {contentType = t'} = t == t'
-
-isSimplexContentType :: XContentType -> MsgContentBody -> Bool
-isSimplexContentType = isContentType . SimplexContentType
-
-getContentType :: ContentType -> [MsgContentBody] -> Either String ByteString
-getContentType t body = case find (isContentType t) body of
-  Just MsgContentBody {contentData} -> Right contentData
-  Nothing -> Left "no required content type"
-
-extractContentType :: ContentType -> [MsgContentBody] -> Either String (ByteString, [MsgContentBody])
-extractContentType t body = case findIndex (isContentType t) body of
-  Just i -> case splitAt i body of
-    (b, el : a) -> Right (contentData (el :: MsgContentBody), b ++ a)
-    (_, []) -> Left "no required content type" -- this can only happen if findIndex returns incorrect result
-  Nothing -> Left "no required content type"
-
-getSimplexContentType :: XContentType -> [MsgContentBody] -> Either String ByteString
-getSimplexContentType = getContentType . SimplexContentType
-
-extractSimplexContentType :: XContentType -> [MsgContentBody] -> Either String (ByteString, [MsgContentBody])
-extractSimplexContentType = extractContentType . SimplexContentType
-
-rawChatMessage :: ChatMessage -> RawChatMessage
-rawChatMessage ChatMessage {chatMsgId, chatMsgEvent, chatDAG} =
-  case chatMsgEvent of
-    XMsgNew MsgContent {messageType = t, files, content} ->
-      rawMsg (rawMsgType t : toRawFiles files) content
-    XFile FileInvitation {fileName, fileSize, fileConnReq} ->
-      rawMsg [encodeUtf8 $ T.pack fileName, bshow fileSize, serializeConnReq' fileConnReq] []
-    XFileAcpt fileName ->
-      rawMsg [encodeUtf8 $ T.pack fileName] []
-    XInfo profile ->
-      rawMsg [] [jsonBody profile]
-    XContact profile Nothing ->
-      rawMsg [] [jsonBody profile]
-    XContact profile (Just MsgContent {messageType = t, files, content}) ->
-      rawMsg (rawMsgType t : toRawFiles files) (jsonBody profile : content)
-    XGrpInv (GroupInvitation (fromMemId, fromRole) (memId, role) cReq groupProfile) ->
-      let params =
-            [ B64.encode fromMemId,
-              serializeMemberRole fromRole,
-              B64.encode memId,
-              serializeMemberRole role,
-              serializeConnReq' cReq
-            ]
-       in rawMsg params [jsonBody groupProfile]
-    XGrpAcpt memId ->
-      rawMsg [B64.encode memId] []
-    XGrpMemNew (MemberInfo memId role profile) ->
-      let params = [B64.encode memId, serializeMemberRole role]
-       in rawMsg params [jsonBody profile]
-    XGrpMemIntro (MemberInfo memId role profile) ->
-      rawMsg [B64.encode memId, serializeMemberRole role] [jsonBody profile]
-    XGrpMemInv memId IntroInvitation {groupConnReq, directConnReq} ->
-      let params = [B64.encode memId, serializeConnReq' groupConnReq, serializeConnReq' directConnReq]
-       in rawMsg params []
-    XGrpMemFwd (MemberInfo memId role profile) IntroInvitation {groupConnReq, directConnReq} ->
-      let params =
-            [ B64.encode memId,
-              serializeMemberRole role,
-              serializeConnReq' groupConnReq,
-              serializeConnReq' directConnReq
-            ]
-       in rawMsg params [jsonBody profile]
-    XGrpMemInfo memId profile ->
-      rawMsg [B64.encode memId] [jsonBody profile]
-    XGrpMemCon memId ->
-      rawMsg [B64.encode memId] []
-    XGrpMemConAll memId ->
-      rawMsg [B64.encode memId] []
-    XGrpMemDel memId ->
-      rawMsg [B64.encode memId] []
-    XGrpLeave ->
-      rawMsg [] []
-    XGrpDel ->
-      rawMsg [] []
-    XInfoProbe probe ->
-      rawMsg [B64.encode probe] []
-    XInfoProbeCheck probeHash ->
-      rawMsg [B64.encode probeHash] []
-    XInfoProbeOk probe ->
-      rawMsg [B64.encode probe] []
-    XOk ->
-      rawMsg [] []
+chatToAppMessage :: ChatMessage -> AppMessage
+chatToAppMessage ChatMessage {chatMsgEvent} = AppMessage {event, params}
   where
-    rawMsg :: [ByteString] -> [MsgContentBody] -> RawChatMessage
-    rawMsg chatMsgParams body = do
-      let event = encodeUtf8 $ toChatEventType chatMsgEvent
-      RawChatMessage {chatMsgId, chatMsgEvent = event, chatMsgParams, chatMsgBody = rawWithDAG body}
-    rawContentInfo :: (ContentType, Int) -> (RawContentType, Int)
-    rawContentInfo (t, size) = (rawContentType t, size)
-    jsonBody :: ToJSON a => a -> MsgContentBody
-    jsonBody x =
-      let json = LB.toStrict $ J.encode x
-       in MsgContentBody {contentType = SimplexContentType XCJson, contentData = json}
-    rawWithDAG :: [MsgContentBody] -> [RawMsgBodyContent]
-    rawWithDAG body = map rawMsgBodyContent $ case chatDAG of
-      Nothing -> body
-      Just dag -> MsgContentBody {contentType = SimplexDAG, contentData = dag} : body
-    toRawFiles :: [(ContentType, Int)] -> [ByteString]
-    toRawFiles = map $ serializeContentInfo . rawContentInfo
-
-toMsgBodyContent :: RawMsgBodyContent -> Either String MsgContentBody
-toMsgBodyContent RawMsgBodyContent {contentType, contentData} = do
-  cType <- toContentType contentType
-  pure MsgContentBody {contentType = cType, contentData}
-
-rawMsgBodyContent :: MsgContentBody -> RawMsgBodyContent
-rawMsgBodyContent MsgContentBody {contentType = t, contentData} =
-  RawMsgBodyContent {contentType = rawContentType t, contentData}
-
-data MsgContentBody = MsgContentBody
-  { contentType :: ContentType,
-    contentData :: ByteString
-  }
-  deriving (Eq, Show)
-
-data ContentType
-  = SimplexContentType XContentType
-  | MimeContentType MContentType
-  | SimplexDAG
-  deriving (Eq, Show)
-
-data XContentType = XCText | XCImage | XCJson deriving (Eq, Show)
-
-data MContentType = MCImageJPG | MCImagePNG deriving (Eq, Show)
-
-toContentType :: RawContentType -> Either String ContentType
-toContentType (RawContentType ns cType) = case ns of
-  "x" -> case cType of
-    "text" -> Right $ SimplexContentType XCText
-    "image" -> Right $ SimplexContentType XCImage
-    "json" -> Right $ SimplexContentType XCJson
-    "dag" -> Right SimplexDAG
-    _ -> err
-  "m" -> case cType of
-    "image/jpg" -> Right $ MimeContentType MCImageJPG
-    "image/png" -> Right $ MimeContentType MCImagePNG
-    _ -> err
-  _ -> err
-  where
-    err = Left . B.unpack $ "invalid content type " <> ns <> "." <> cType
-
-rawContentType :: ContentType -> RawContentType
-rawContentType t = case t of
-  SimplexContentType t' -> RawContentType "x" $ case t' of
-    XCText -> "text"
-    XCImage -> "image"
-    XCJson -> "json"
-  MimeContentType t' -> RawContentType "m" $ case t' of
-    MCImageJPG -> "image/jpg"
-    MCImagePNG -> "image/png"
-  SimplexDAG -> RawContentType "x" "dag"
-
-newtype ContentMsg = NewContentMsg ContentData
-
-newtype ContentData = ContentText Text
-
-data RawChatMessage = RawChatMessage
-  { chatMsgId :: Maybe Int64,
-    chatMsgEvent :: ByteString,
-    chatMsgParams :: [ByteString],
-    chatMsgBody :: [RawMsgBodyContent]
-  }
-  deriving (Eq, Show)
-
-data RawMsgBodyContent = RawMsgBodyContent
-  { contentType :: RawContentType,
-    contentData :: ByteString
-  }
-  deriving (Eq, Show)
-
-data RawContentType = RawContentType NameSpace ByteString
-  deriving (Eq, Show)
-
-type NameSpace = ByteString
-
-newtype MsgData = MsgData ByteString
-  deriving (Eq, Show)
-
-class DataLength a where
-  dataLength :: a -> Int
-
-rawChatMessageP :: Parser RawChatMessage
-rawChatMessageP = do
-  chatMsgId <- optional A.decimal <* A.space
-  chatMsgEvent <- B.intercalate "." <$> identifierP `A.sepBy1'` A.char '.' <* A.space
-  chatMsgParams <- A.takeWhile1 (not . A.inClass ", ") `A.sepBy'` A.char ',' <* A.space
-  chatMsgBody <- msgBodyContent =<< contentInfoP `A.sepBy'` A.char ',' <* A.space
-  pure RawChatMessage {chatMsgId, chatMsgEvent, chatMsgParams, chatMsgBody}
-  where
-    msgBodyContent :: [(RawContentType, Int)] -> Parser [RawMsgBodyContent]
-    msgBodyContent [] = pure []
-    msgBodyContent ((contentType, size) : ps) = do
-      contentData <- A.take size <* A.space
-      ((RawMsgBodyContent {contentType, contentData}) :) <$> msgBodyContent ps
-
-contentInfoP :: Parser (RawContentType, Int)
-contentInfoP = do
-  contentType <- RawContentType <$> identifierP <* A.char '.' <*> A.takeTill (A.inClass ":, ")
-  size <- A.char ':' *> A.decimal
-  pure (contentType, size)
-
-identifierP :: Parser ByteString
-identifierP = B.cons <$> A.letter_ascii <*> A.takeWhile (\c -> A.isAlpha_ascii c || A.isDigit c)
-
-serializeRawChatMessage :: RawChatMessage -> ByteString
-serializeRawChatMessage RawChatMessage {chatMsgId, chatMsgEvent, chatMsgParams, chatMsgBody} =
-  B.unwords
-    [ maybe "" bshow chatMsgId,
-      chatMsgEvent,
-      B.intercalate "," chatMsgParams,
-      B.unwords $ map serializeBodyContentInfo chatMsgBody,
-      B.unwords $ map msgContentData chatMsgBody
-    ]
-
-serializeBodyContentInfo :: RawMsgBodyContent -> ByteString
-serializeBodyContentInfo RawMsgBodyContent {contentType = t, contentData} =
-  serializeContentInfo (t, B.length contentData)
-
-serializeContentInfo :: (RawContentType, Int) -> ByteString
-serializeContentInfo (RawContentType ns cType, size) = ns <> "." <> cType <> ":" <> bshow size
-
-msgContentData :: RawMsgBodyContent -> ByteString
-msgContentData RawMsgBodyContent {contentData} = contentData <> " "
+    event = toChatEventTag chatMsgEvent
+    o :: [(Text, J.Value)] -> J.Object
+    o = H.fromList
+    params = case chatMsgEvent of
+      XMsgNew content -> o ["content" .= content]
+      XFile fileInv -> o ["file" .= fileInv]
+      XFileAcpt fileName -> o ["fileName" .= fileName]
+      XInfo profile -> o ["profile" .= profile]
+      XContact profile content -> o $ maybe id ((:) . ("content" .=)) content ["profile" .= profile]
+      XGrpInv groupInv -> o ["groupInvitation" .= groupInv]
+      XGrpAcpt memId -> o ["memberId" .= memId]
+      XGrpMemNew memInfo -> o ["memberInfo" .= memInfo]
+      XGrpMemIntro memInfo -> o ["memberInfo" .= memInfo]
+      XGrpMemInv memId memIntro -> o ["memberId" .= memId, "memberIntro" .= memIntro]
+      XGrpMemFwd memInfo memIntro -> o ["memberInfo" .= memInfo, "memberIntro" .= memIntro]
+      XGrpMemInfo memId profile -> o ["memberId" .= memId, "profile" .= profile]
+      XGrpMemCon memId -> o ["memberId" .= memId]
+      XGrpMemConAll memId -> o ["memberId" .= memId]
+      XGrpMemDel memId -> o ["memberId" .= memId]
+      XGrpLeave -> H.empty
+      XGrpDel -> H.empty
+      XInfoProbe probe -> o ["probe" .= probe]
+      XInfoProbeCheck probeHash -> o ["probeHash" .= probeHash]
+      XInfoProbeOk probe -> o ["probe" .= probe]
+      XOk -> H.empty

src/Simplex/Chat/Store.hs

@@ -90,7 +90,6 @@ module Simplex.Chat.Store
     deleteRcvFileChunks,
     getFileTransfer,
     getFileTransferProgress,
-    getOnboarding,
     createNewMessage,
     createSndMsgDelivery,
     createNewMessageAndRcvMsgDelivery,
@@ -145,7 +144,7 @@ createStore :: FilePath -> Int -> IO SQLiteStore
 createStore dbFilePath poolSize = createSQLiteStore dbFilePath poolSize migrations
 
 chatStoreFile :: FilePath -> FilePath
-chatStoreFile = (<> ".chat.db")
+chatStoreFile = (<> "_chat.db")
 
 checkConstraint :: StoreError -> IO (Either StoreError a) -> IO (Either StoreError a)
 checkConstraint err action = action `E.catch` (pure . Left . handleSQLError err)
@@ -667,20 +666,20 @@ getMatchingContacts st userId Contact {contactId, profile = Profile {displayName
           ]
     rights <$> mapM (runExceptT . getContact_ db userId) contactNames
 
-createSentProbe :: StoreMonad m => SQLiteStore -> TVar ChaChaDRG -> UserId -> Contact -> m (ByteString, Int64)
+createSentProbe :: StoreMonad m => SQLiteStore -> TVar ChaChaDRG -> UserId -> Contact -> m (Probe, Int64)
 createSentProbe st gVar userId _to@Contact {contactId} =
   liftIOEither . withTransaction st $ \db ->
     createWithRandomBytes 32 gVar $ \probe -> do
       DB.execute db "INSERT INTO sent_probes (contact_id, probe, user_id) VALUES (?,?,?)" (contactId, probe, userId)
-      (probe,) <$> insertedRowId db
+      (Probe probe,) <$> insertedRowId db
 
 createSentProbeHash :: MonadUnliftIO m => SQLiteStore -> UserId -> Int64 -> Contact -> m ()
 createSentProbeHash st userId probeId _to@Contact {contactId} =
   liftIO . withTransaction st $ \db ->
     DB.execute db "INSERT INTO sent_probe_hashes (sent_probe_id, contact_id, user_id) VALUES (?,?,?)" (probeId, contactId, userId)
 
-matchReceivedProbe :: MonadUnliftIO m => SQLiteStore -> UserId -> Contact -> ByteString -> m (Maybe Contact)
-matchReceivedProbe st userId _from@Contact {contactId} probe =
+matchReceivedProbe :: MonadUnliftIO m => SQLiteStore -> UserId -> Contact -> Probe -> m (Maybe Contact)
+matchReceivedProbe st userId _from@Contact {contactId} (Probe probe) =
   liftIO . withTransaction st $ \db -> do
     let probeHash = C.sha256Hash probe
     contactNames <-
@@ -701,8 +700,8 @@ matchReceivedProbe st userId _from@Contact {contactId} probe =
         either (const Nothing) Just
           <$> runExceptT (getContact_ db userId cName)
 
-matchReceivedProbeHash :: MonadUnliftIO m => SQLiteStore -> UserId -> Contact -> ByteString -> m (Maybe (Contact, ByteString))
-matchReceivedProbeHash st userId _from@Contact {contactId} probeHash =
+matchReceivedProbeHash :: MonadUnliftIO m => SQLiteStore -> UserId -> Contact -> ProbeHash -> m (Maybe (Contact, Probe))
+matchReceivedProbeHash st userId _from@Contact {contactId} (ProbeHash probeHash) =
   liftIO . withTransaction st $ \db -> do
     namesAndProbes <-
       DB.query
@@ -718,11 +717,11 @@ matchReceivedProbeHash st userId _from@Contact {contactId} probeHash =
     case namesAndProbes of
       [] -> pure Nothing
       (cName, probe) : _ ->
-        either (const Nothing) (Just . (,probe))
+        either (const Nothing) (Just . (,Probe probe))
           <$> runExceptT (getContact_ db userId cName)
 
-matchSentProbe :: MonadUnliftIO m => SQLiteStore -> UserId -> Contact -> ByteString -> m (Maybe Contact)
-matchSentProbe st userId _from@Contact {contactId} probe =
+matchSentProbe :: MonadUnliftIO m => SQLiteStore -> UserId -> Contact -> Probe -> m (Maybe Contact)
+matchSentProbe st userId _from@Contact {contactId} (Probe probe) =
   liftIO . withTransaction st $ \db -> do
     contactNames <-
       map fromOnly
@@ -889,7 +888,7 @@ createNewGroup st gVar user groupProfile =
     DB.execute db "INSERT INTO groups (local_display_name, user_id, group_profile_id) VALUES (?, ?, ?)" (displayName, uId, profileId)
     groupId <- insertedRowId db
     memberId <- randomBytes gVar 12
-    membership <- createContactMember_ db user groupId user (memberId, GROwner) GCUserMember GSMemCreator IBUser
+    membership <- createContactMember_ db user groupId user (MemberIdRole (MemberId memberId) GROwner) GCUserMember GSMemCreator IBUser
     pure $ Right Group {groupId, localDisplayName = displayName, groupProfile, members = [], membership}
 
 -- | creates a new group record for the group the current user was invited to, or returns an existing one
@@ -1022,7 +1021,7 @@ getGroupInvitation st user localDisplayName =
     findFromContact (IBContact contactId) = find ((== Just contactId) . memberContactId)
     findFromContact _ = const Nothing
 
-type GroupMemberRow = (Int64, Int64, ByteString, GroupMemberRole, GroupMemberCategory, GroupMemberStatus, Maybe Int64, ContactName, Maybe Int64, ContactName, Text)
+type GroupMemberRow = (Int64, Int64, MemberId, GroupMemberRole, GroupMemberCategory, GroupMemberStatus, Maybe Int64, ContactName, Maybe Int64, ContactName, Text)
 
 toGroupMember :: Int64 -> GroupMemberRow -> GroupMember
 toGroupMember userContactId (groupMemberId, groupId, memberId, memberRole, memberCategory, memberStatus, invitedById, localDisplayName, memberContactId, displayName, fullName) =
@@ -1035,7 +1034,7 @@ createContactMember :: StoreMonad m => SQLiteStore -> TVar ChaChaDRG -> User ->
 createContactMember st gVar user groupId contact memberRole agentConnId connRequest =
   liftIOEither . withTransaction st $ \db ->
     createWithRandomId gVar $ \memId -> do
-      member@GroupMember {groupMemberId} <- createContactMemberInv_ db user groupId contact (memId, memberRole) GCInviteeMember GSMemInvited IBUser (Just connRequest)
+      member@GroupMember {groupMemberId} <- createContactMemberInv_ db user groupId contact (MemberIdRole (MemberId memId) memberRole) GCInviteeMember GSMemInvited IBUser (Just connRequest)
       void $ createMemberConnection_ db (userId user) groupMemberId agentConnId Nothing 0
       pure member
 
@@ -1043,7 +1042,7 @@ getMemberInvitation :: StoreMonad m => SQLiteStore -> User -> Int64 -> m (Maybe
 getMemberInvitation st User {userId} groupMemberId =
   liftIO . withTransaction st $ \db ->
     join . listToMaybe . map fromOnly
-      <$> DB.query db "SELECT inv_queue_info FROM group_members WHERE group_member_id = ? AND user_id = ?;" (groupMemberId, userId)
+      <$> DB.query db "SELECT sent_inv_queue_info FROM group_members WHERE group_member_id = ? AND user_id = ?;" (groupMemberId, userId)
 
 createMemberConnection :: MonadUnliftIO m => SQLiteStore -> UserId -> GroupMember -> ConnId -> m ()
 createMemberConnection st userId GroupMember {groupMemberId} agentConnId =
@@ -1269,12 +1268,12 @@ createIntroToMemberContact st userId GroupMember {memberContactId = viaContactId
 createMemberConnection_ :: DB.Connection -> UserId -> Int64 -> ConnId -> Maybe Int64 -> Int -> IO Connection
 createMemberConnection_ db userId groupMemberId = createConnection_ db userId ConnMember (Just groupMemberId)
 
-createContactMember_ :: IsContact a => DB.Connection -> User -> Int64 -> a -> (MemberId, GroupMemberRole) -> GroupMemberCategory -> GroupMemberStatus -> InvitedBy -> IO GroupMember
-createContactMember_ db user groupId userOrContact (memberId, memberRole) memberCategory memberStatus invitedBy =
-  createContactMemberInv_ db user groupId userOrContact (memberId, memberRole) memberCategory memberStatus invitedBy Nothing
+createContactMember_ :: IsContact a => DB.Connection -> User -> Int64 -> a -> MemberIdRole -> GroupMemberCategory -> GroupMemberStatus -> InvitedBy -> IO GroupMember
+createContactMember_ db user groupId userOrContact MemberIdRole {memberId, memberRole} memberCategory memberStatus invitedBy =
+  createContactMemberInv_ db user groupId userOrContact MemberIdRole {memberId, memberRole} memberCategory memberStatus invitedBy Nothing
 
-createContactMemberInv_ :: IsContact a => DB.Connection -> User -> Int64 -> a -> (MemberId, GroupMemberRole) -> GroupMemberCategory -> GroupMemberStatus -> InvitedBy -> Maybe ConnReqInvitation -> IO GroupMember
-createContactMemberInv_ db User {userId, userContactId} groupId userOrContact (memberId, memberRole) memberCategory memberStatus invitedBy connRequest = do
+createContactMemberInv_ :: IsContact a => DB.Connection -> User -> Int64 -> a -> MemberIdRole -> GroupMemberCategory -> GroupMemberStatus -> InvitedBy -> Maybe ConnReqInvitation -> IO GroupMember
+createContactMemberInv_ db User {userId, userContactId} groupId userOrContact MemberIdRole {memberId, memberRole} memberCategory memberStatus invitedBy connRequest = do
   insertMember_
   groupMemberId <- insertedRowId db
   let memberProfile = profile' userOrContact
@@ -1289,12 +1288,12 @@ createContactMemberInv_ db User {userId, userContactId} groupId userOrContact (m
         [sql|
           INSERT INTO group_members
             ( group_id, member_id, member_role, member_category, member_status, invited_by,
-              user_id, local_display_name, contact_profile_id, contact_id, inv_queue_info)
+              user_id, local_display_name, contact_profile_id, contact_id, sent_inv_queue_info)
           VALUES
             (:group_id,:member_id,:member_role,:member_category,:member_status,:invited_by,
              :user_id,:local_display_name,
               (SELECT contact_profile_id FROM contacts WHERE contact_id = :contact_id),
-              :contact_id, :inv_queue_info)
+              :contact_id, :sent_inv_queue_info)
         |]
         [ ":group_id" := groupId,
           ":member_id" := memberId,
@@ -1305,7 +1304,7 @@ createContactMemberInv_ db User {userId, userContactId} groupId userOrContact (m
           ":user_id" := userId,
           ":local_display_name" := localDisplayName' userOrContact,
           ":contact_id" := contactId' userOrContact,
-          ":inv_queue_info" := connRequest
+          ":sent_inv_queue_info" := connRequest
         ]
 
 getViaGroupMember :: MonadUnliftIO m => SQLiteStore -> User -> Contact -> m (Maybe (GroupName, GroupMember))
@@ -1625,21 +1624,6 @@ getSndFileTransfers_ db userId fileId =
         Just recipientDisplayName -> Right SndFileTransfer {..}
         Nothing -> Left $ SESndFileInvalid fileId
 
-getOnboarding :: MonadUnliftIO m => SQLiteStore -> UserId -> m Onboarding
-getOnboarding st userId =
-  liftIO . withTransaction st $ \db -> do
-    contactsCount <- intQuery db "SELECT COUNT(contact_id) FROM contacts WHERE user_id = ? AND is_user = 0"
-    createdGroups <- headOrZero <$> DB.query db "SELECT COUNT(g.group_id) FROM groups g JOIN group_members m WHERE g.user_id = ? AND m.member_status = ?" (userId, GSMemCreator)
-    membersCount <- headOrZero <$> DB.query db "SELECT COUNT(group_member_id) FROM group_members WHERE user_id = ? AND (member_status = ? OR member_status = ?)" (userId, GSMemConnected, GSMemComplete)
-    filesSentCount <- intQuery db "SELECT COUNT(s.file_id) FROM snd_files s JOIN files f USING (file_id) WHERE f.user_id = ?"
-    addressCount <- intQuery db "SELECT COUNT(user_contact_link_id) FROM user_contact_links WHERE user_id = ?"
-    pure $ Onboarding {..}
-  where
-    intQuery :: DB.Connection -> DB.Query -> IO Int
-    intQuery db q = headOrZero <$> DB.query db q (Only userId)
-    headOrZero [] = 0
-    headOrZero (n : _) = fromOnly n
-
 createNewMessage :: MonadUnliftIO m => SQLiteStore -> NewMessage -> m MessageId
 createNewMessage st newMsg =
   liftIO . withTransaction st $ \db ->

src/Simplex/Chat/Types.hs

@@ -1,4 +1,5 @@
 {-# LANGUAGE DataKinds #-}
+{-# LANGUAGE DeriveAnyClass #-}
 {-# LANGUAGE DeriveGeneric #-}
 {-# LANGUAGE DuplicateRecordFields #-}
 {-# LANGUAGE GADTs #-}
@@ -11,8 +12,10 @@
 
 module Simplex.Chat.Types where
 
-import Data.Aeson (FromJSON, ToJSON)
+import Data.Aeson (FromJSON, ToJSON, (.:), (.=))
 import qualified Data.Aeson as J
+import qualified Data.Aeson.Types as JT
+import qualified Data.Attoparsec.ByteString.Char8 as A
 import qualified Data.ByteString.Base64 as B64
 import Data.ByteString.Char8 (ByteString)
 import qualified Data.ByteString.Char8 as B
@@ -29,9 +32,11 @@ import Database.SQLite.Simple.Internal (Field (..))
 import Database.SQLite.Simple.Ok (Ok (Ok))
 import Database.SQLite.Simple.ToField (ToField (..))
 import GHC.Generics
-import Simplex.Messaging.Agent.Protocol (AgentMsgId, ConnId, ConnectionMode (..), ConnectionRequest, InvitationId, MsgMeta (..), serializeMsgIntegrity)
+import Simplex.Messaging.Agent.Protocol (AgentMsgId, ConnId, ConnectionMode (..), ConnectionRequestUri, InvitationId, MsgMeta (..), serializeMsgIntegrity)
 import Simplex.Messaging.Agent.Store.SQLite (fromTextField_)
+import Simplex.Messaging.Encoding.String
 import Simplex.Messaging.Protocol (MsgBody)
+import Simplex.Messaging.Util ((<$?>))
 
 class IsContact a where
   contactId' :: a -> Int64
@@ -106,41 +111,56 @@ data Profile = Profile
   { displayName :: ContactName,
     fullName :: Text
   }
-  deriving (Generic, Eq, Show)
+  deriving (Eq, Show, Generic, FromJSON)
 
 instance ToJSON Profile where toEncoding = J.genericToEncoding J.defaultOptions
 
-instance FromJSON Profile
-
 data GroupProfile = GroupProfile
   { displayName :: GroupName,
     fullName :: Text
   }
-  deriving (Generic, Eq, Show)
+  deriving (Eq, Show, Generic, FromJSON)
 
 instance ToJSON GroupProfile where toEncoding = J.genericToEncoding J.defaultOptions
 
-instance FromJSON GroupProfile
-
 data GroupInvitation = GroupInvitation
-  { fromMember :: (MemberId, GroupMemberRole),
-    invitedMember :: (MemberId, GroupMemberRole),
+  { fromMember :: MemberIdRole,
+    invitedMember :: MemberIdRole,
     connRequest :: ConnReqInvitation,
     groupProfile :: GroupProfile
   }
-  deriving (Eq, Show)
+  deriving (Eq, Show, Generic, FromJSON)
+
+instance ToJSON GroupInvitation where toEncoding = J.genericToEncoding J.defaultOptions
+
+data MemberIdRole = MemberIdRole
+  { memberId :: MemberId,
+    memberRole :: GroupMemberRole
+  }
+  deriving (Eq, Show, Generic, FromJSON)
+
+instance ToJSON MemberIdRole where toEncoding = J.genericToEncoding J.defaultOptions
 
 data IntroInvitation = IntroInvitation
   { groupConnReq :: ConnReqInvitation,
     directConnReq :: ConnReqInvitation
   }
-  deriving (Eq, Show)
+  deriving (Eq, Show, Generic, FromJSON)
 
-data MemberInfo = MemberInfo MemberId GroupMemberRole Profile
-  deriving (Eq, Show)
+instance ToJSON IntroInvitation where toEncoding = J.genericToEncoding J.defaultOptions
+
+data MemberInfo = MemberInfo
+  { memberId :: MemberId,
+    memberRole :: GroupMemberRole,
+    profile :: Profile
+  }
+  deriving (Eq, Show, Generic, FromJSON)
+
+instance ToJSON MemberInfo where toEncoding = J.genericToEncoding J.defaultOptions
 
 memberInfo :: GroupMember -> MemberInfo
-memberInfo m = MemberInfo (memberId m) (memberRole m) (memberProfile m)
+memberInfo GroupMember {memberId, memberRole, memberProfile} =
+  MemberInfo memberId memberRole memberProfile
 
 data ReceivedGroupInvitation = ReceivedGroupInvitation
   { fromMember :: GroupMember,
@@ -183,7 +203,24 @@ data NewGroupMember = NewGroupMember
     memContactId :: Maybe Int64
   }
 
-type MemberId = ByteString
+newtype MemberId = MemberId {unMemberId :: ByteString}
+  deriving (Eq, Show)
+
+instance FromField MemberId where fromField f = MemberId <$> fromField f
+
+instance ToField MemberId where toField (MemberId m) = toField m
+
+instance StrEncoding MemberId where
+  strEncode (MemberId m) = strEncode m
+  strDecode s = MemberId <$> strDecode s
+  strP = MemberId <$> strP
+
+instance FromJSON MemberId where
+  parseJSON = strParseJSON "MemberId"
+
+instance ToJSON MemberId where
+  toJSON = strToJSON
+  toEncoding = strToJEncoding
 
 data InvitedBy = IBContact Int64 | IBUser | IBUnknown
   deriving (Eq, Show)
@@ -203,22 +240,28 @@ fromInvitedBy userCtId = \case
 data GroupMemberRole = GRMember | GRAdmin | GROwner
   deriving (Eq, Show, Ord)
 
-instance FromField GroupMemberRole where fromField = fromBlobField_ toMemberRole
+instance FromField GroupMemberRole where fromField = fromBlobField_ strDecode
 
-instance ToField GroupMemberRole where toField = toField . serializeMemberRole
+instance ToField GroupMemberRole where toField = toField . strEncode
 
-toMemberRole :: ByteString -> Either String GroupMemberRole
-toMemberRole = \case
-  "owner" -> Right GROwner
-  "admin" -> Right GRAdmin
-  "member" -> Right GRMember
-  r -> Left $ "invalid group member role " <> B.unpack r
+instance StrEncoding GroupMemberRole where
+  strEncode = \case
+    GROwner -> "owner"
+    GRAdmin -> "admin"
+    GRMember -> "member"
+  strDecode = \case
+    "owner" -> Right GROwner
+    "admin" -> Right GRAdmin
+    "member" -> Right GRMember
+    r -> Left $ "bad GroupMemberRole " <> B.unpack r
+  strP = strDecode <$?> A.takeByteString
 
-serializeMemberRole :: GroupMemberRole -> ByteString
-serializeMemberRole = \case
-  GROwner -> "owner"
-  GRAdmin -> "admin"
-  GRMember -> "member"
+instance FromJSON GroupMemberRole where
+  parseJSON = strParseJSON "GroupMemberRole"
+
+instance ToJSON GroupMemberRole where
+  toJSON = strToJSON
+  toEncoding = strToJEncoding
 
 fromBlobField_ :: Typeable k => (ByteString -> Either String k) -> FieldParser k
 fromBlobField_ p = \case
@@ -228,6 +271,36 @@ fromBlobField_ p = \case
       Left e -> returnError ConversionFailed f ("could not parse field: " ++ e)
   f -> returnError ConversionFailed f "expecting SQLBlob column type"
 
+newtype Probe = Probe {unProbe :: ByteString}
+  deriving (Eq, Show)
+
+instance StrEncoding Probe where
+  strEncode (Probe p) = strEncode p
+  strDecode s = Probe <$> strDecode s
+  strP = Probe <$> strP
+
+instance FromJSON Probe where
+  parseJSON = strParseJSON "Probe"
+
+instance ToJSON Probe where
+  toJSON = strToJSON
+  toEncoding = strToJEncoding
+
+newtype ProbeHash = ProbeHash {unProbeHash :: ByteString}
+  deriving (Eq, Show)
+
+instance StrEncoding ProbeHash where
+  strEncode (ProbeHash p) = strEncode p
+  strDecode s = ProbeHash <$> strDecode s
+  strP = ProbeHash <$> strP
+
+instance FromJSON ProbeHash where
+  parseJSON = strParseJSON "ProbeHash"
+
+instance ToJSON ProbeHash where
+  toJSON = strToJSON
+  toEncoding = strToJEncoding
+
 data GroupMemberCategory
   = GCUserMember
   | GCInviteeMember -- member invited by the user
@@ -350,7 +423,24 @@ data FileInvitation = FileInvitation
     fileSize :: Integer,
     fileConnReq :: ConnReqInvitation
   }
-  deriving (Eq, Show)
+  deriving (Eq, Show, Generic)
+
+instance FromJSON FileInvitation where
+  parseJSON (J.Object v) = FileInvitation <$> v .: "fileName" <*> v .: "fileSize" <*> v .: "fileConnReq"
+  parseJSON invalid = JT.prependFailure "bad FileInvitation, " (JT.typeMismatch "Object" invalid)
+
+instance ToJSON FileInvitation where
+  toJSON (FileInvitation fileName fileSize fileConnReq) =
+    J.object
+      [ "fileName" .= fileName,
+        "fileSize" .= fileSize,
+        "fileConnReq" .= fileConnReq
+      ]
+  toEncoding (FileInvitation fileName fileSize fileConnReq) =
+    J.pairs $
+      "fileName" .= fileName
+        <> "fileSize" .= fileSize
+        <> "fileConnReq" .= fileConnReq
 
 data RcvFileTransfer = RcvFileTransfer
   { fileId :: Int64,
@@ -404,9 +494,9 @@ serializeFileStatus = \case
 data RcvChunkStatus = RcvChunkOk | RcvChunkFinal | RcvChunkDuplicate | RcvChunkError
   deriving (Eq, Show)
 
-type ConnReqInvitation = ConnectionRequest 'CMInvitation
+type ConnReqInvitation = ConnectionRequestUri 'CMInvitation
 
-type ConnReqContact = ConnectionRequest 'CMContact
+type ConnReqContact = ConnectionRequestUri 'CMContact
 
 data Connection = Connection
   { connId :: Int64,
@@ -534,14 +624,6 @@ serializeIntroStatus = \case
   GMIntroToConnected -> "to-con"
   GMIntroConnected -> "con"
 
-data Onboarding = Onboarding
-  { contactsCount :: Int,
-    createdGroups :: Int,
-    membersCount :: Int,
-    filesSentCount :: Int,
-    addressCount :: Int
-  }
-
 data NewMessage = NewMessage
   { direction :: MsgDirection,
     chatMsgEventType :: Text,
@@ -592,7 +674,7 @@ data RcvMsgDelivery = RcvMsgDelivery
     agentMsgMeta :: MsgMeta
   }
 
-data MsgMetaJ = MsgMetaJ
+data MsgMetaJSON = MsgMetaJSON
   { integrity :: Text,
     rcvId :: Int64,
     rcvTs :: UTCTime,
@@ -600,15 +682,13 @@ data MsgMetaJ = MsgMetaJ
     serverTs :: UTCTime,
     sndId :: Int64
   }
-  deriving (Generic, Eq, Show)
+  deriving (Eq, Show, FromJSON, Generic)
 
-instance ToJSON MsgMetaJ where toEncoding = J.genericToEncoding J.defaultOptions
+instance ToJSON MsgMetaJSON where toEncoding = J.genericToEncoding J.defaultOptions
 
-instance FromJSON MsgMetaJ
-
-msgMetaToJson :: MsgMeta -> MsgMetaJ
-msgMetaToJson MsgMeta {integrity, recipient = (rcvId, rcvTs), broker = (serverId, serverTs), sender = (sndId, _)} =
-  MsgMetaJ
+msgMetaToJson :: MsgMeta -> MsgMetaJSON
+msgMetaToJson MsgMeta {integrity, recipient = (rcvId, rcvTs), broker = (serverId, serverTs), sndMsgId = sndId} =
+  MsgMetaJSON
     { integrity = (decodeLatin1 . serializeMsgIntegrity) integrity,
       rcvId,
       rcvTs,

src/Simplex/Chat/View.hs

@@ -1,4 +1,5 @@
 {-# LANGUAGE ConstraintKinds #-}
+{-# LANGUAGE DataKinds #-}
 {-# LANGUAGE DuplicateRecordFields #-}
 {-# LANGUAGE FlexibleContexts #-}
 {-# LANGUAGE LambdaCase #-}
@@ -102,6 +103,7 @@ import Simplex.Chat.Terminal (printToTerminal)
 import Simplex.Chat.Types
 import Simplex.Chat.Util (safeDecodeUtf8)
 import Simplex.Messaging.Agent.Protocol
+import Simplex.Messaging.Encoding.String
 import qualified Simplex.Messaging.Protocol as SMP
 import System.Console.ANSI.Types
 
@@ -328,7 +330,7 @@ connReqInvitation_ :: ConnReqInvitation -> [StyledString]
 connReqInvitation_ cReq =
   [ "pass this invitation link to your contact (via another channel): ",
     "",
-    (plain . serializeConnReq') cReq,
+    (plain . strEncode) cReq,
     "",
     "and ask them to connect: " <> highlight' "/c <invitation_link_above>"
   ]
@@ -380,7 +382,7 @@ connReqContact_ :: StyledString -> ConnReqContact -> [StyledString]
 connReqContact_ intro cReq =
   [ intro,
     "",
-    (plain . serializeConnReq') cReq,
+    (plain . strEncode) cReq,
     "",
     "Anybody can send you contact requests with: " <> highlight' "/c <contact_link_above>",
     "to show it again: " <> highlight' "/sa",
@@ -444,7 +446,7 @@ cannotResendInvitation g c =
 
 receivedGroupInvitation :: Group -> ContactName -> GroupMemberRole -> [StyledString]
 receivedGroupInvitation g@Group {localDisplayName} c role =
-  [ ttyFullGroup g <> ": " <> ttyContact c <> " invites you to join the group as " <> plain (serializeMemberRole role),
+  [ ttyFullGroup g <> ": " <> ttyContact c <> " invites you to join the group as " <> plain (strEncode role),
     "use " <> highlight ("/j " <> localDisplayName) <> " to accept"
   ]
 
@@ -492,7 +494,7 @@ groupMembers Group {membership, members} = map groupMember . filter (not . remov
   where
     removedOrLeft m = let s = memberStatus m in s == GSMemRemoved || s == GSMemLeft
     groupMember m = ttyFullMember m <> ": " <> role m <> ", " <> category m <> status m
-    role = plain . serializeMemberRole . memberRole
+    role m = plain . strEncode $ memberRole (m :: GroupMember)
     category m = case memberCategory m of
       GCUserMember -> "you, "
       GCInviteeMember -> "invited, "
@@ -753,6 +755,7 @@ chatError = \case
     CEFileSend fileId e -> ["error sending file " <> sShow fileId <> ": " <> sShow e]
     CEFileRcvChunk e -> ["error receiving file: " <> plain e]
     CEFileInternal e -> ["file error: " <> plain e]
+    CEAgentVersion -> ["unsupported agent version"]
   -- e -> ["chat error: " <> sShow e]
   ChatErrorStore err -> case err of
     SEDuplicateName -> ["this display name is already used by user, contact or group"]
@@ -831,4 +834,4 @@ styleTime :: String -> StyledString
 styleTime = Styled [SetColor Foreground Vivid Black]
 
 clientVersionInfo :: [StyledString]
-clientVersionInfo = [plain $ "SimpleX Chat v" <> versionNumber]
+clientVersionInfo = [plain versionStr, plain updateStr]

src/Simplex/Chat/protocol.md

@@ -22,9 +22,6 @@ The syntax of the message inside agent MSG:
 ```abnf
 agentMessageBody = [chatMsgId] SP msgEvent SP [parameters] SP [contentParts [SP msgBodyParts]]
 chatMsgId = 1*DIGIT ; used to refer to previous message;
-                    ; in the group should only be used in messages sent to all members,
-                    ; which is the main reason not to use external agent ID -
-                    ; some messages are sent only to one member
 msgEvent = protocolNamespace 1*("." msgTypeName)
 protocolNamespace = 1*ALPHA ; "x" for all events defined in the protocol
 msgTypeName = 1*ALPHA
@@ -68,6 +65,219 @@ refMsgHash = 16*16(OCTET) ; SHA256 of agent message body
 ' x.file name,size x.text:NNN <invitation> '
 ```
 
+Chat message JTD:
+
+```jsonc
+{
+  "properties": {
+    "msgId": {"type": "string"},
+    "minVersion": {"type": "uint16"}, // Word16
+    "maxVersion": {"type": "uint16"}, // Word16
+    "event": {"type": "string"}, // Text e.g. s.ok
+    "params": {"values": {}}, // Map Text Value
+  },
+  "optionalProperties": {
+    "dag": {"type": "string"}
+  }
+}
+```
+
+Events:
+
+```jsonc
+"event": "x.msg.new" // XMsgNew
+"params":            // MsgContent
+{
+  "content": {
+    "msgType": "text",
+    // field "files" can be represented in content as contentType "file" with length prepended or as complex contentData
+    "text": "<msg text>"
+  }
+  // "content": [
+    // free form contentType for extensibility and/or complex content types? e.g. MIME
+    // could it be useful if contentData was free form as well? currently it is ByteString
+  //  {"contentType": <content type>, "contentData": "<content data>"},
+  //  ...
+  //  {"contentType": <content type N>, "contentData": "<content data N>"}
+  // ]
+}
+
+"event": "x.file" // XFile; TODO rename into x.file.inv?
+"params":         // FileInvitation
+{
+  "file": {
+    "fileName": "<file name>",
+    "fileSize": <file size>, // integer
+    "fileConnReq": "<file conn req>"
+  }
+}
+
+"event": "x.file.acpt" // XFileAcpt
+"params":              // String
+{
+  "fileName": "<file name>"
+}
+
+"event": "x.info" // XInfo
+"params":         // Profile
+{
+  "profile": {
+    "displayName": "<display name>",
+    "fullName": "<full name>"
+  }
+}
+
+"event": "x.contact" // XContact
+"params":        // Profile (Maybe MsgContent)
+{
+  "profile": {
+    "displayName": "<display name>",
+    "fullName": "<full name>"
+  },
+  "content": {
+    "msgType": "text",
+    "text": "<msg text>"
+  } // optional
+}
+
+"event": "x.grp.inv" // XGrpInv
+"params":            // GroupInvitation
+{
+  "groupInvitation": {
+    "fromMember": {
+      "memberId": "<from_member ID>",
+      "memberRole": "<from_member role>"
+    },
+    "invitedMember": {
+      "memberId": "<invited_member ID>",
+      "memberRole": "<invited_member role>"
+    },
+    "connRequest": "<conn request>",
+    "groupProfile": {
+      "displayName": "<display name>",
+      "fullName": "<full name>"
+    }
+  }
+}
+
+"event": "x.grp.acpt" // XGrpAcpt
+"params":             // MemberId
+{
+  "memberId": "<member ID>"
+}
+
+"event": "x.grp.mem.new" // XGrpMemNew
+"params":                // MemberInfo
+{
+  "memberInfo": {
+    "memberId": "<member ID>",
+    "memberRole": "<member role>",
+    "profile": {
+      "displayName": "<display name>",
+      "fullName": "<full name>"
+    }
+  }
+}
+
+"event": "x.grp.mem.intro" // XGrpMemIntro
+"params":                  // MemberInfo
+{
+  "memberInfo": {
+    "memberId": "<member ID>",
+    "memberRole": "<member role>",
+    "profile": {
+      "displayName": "<display name>",
+      "fullName": "<full name>"
+    }
+  }
+}
+
+"event": "x.grp.mem.inv" // XGrpMemInv
+"params":                // MemberId IntroInvitation
+{
+  "memberId": "<member ID>",
+  "memberIntro": {
+    "groupConnReq": "<group conn req>",
+    "directConnReq": "<direct conn req>"
+  }
+}
+
+"event": "x.grp.mem.fwd" // XGrpMemFwd
+"params":                // MemberInfo IntroInvitation
+{
+  "memberInfo": {
+    "memberId": "<member ID>",
+    "memberRole": "<member role>",
+    "profile": {
+      "displayName": "<display name>",
+      "fullName": "<full name>"
+    },
+  },
+  "memberIntro": {
+    "groupConnReq": "<group conn req>",
+    "directConnReq": "<direct conn req>"
+  }
+}
+
+"event": "x.grp.mem.info" // XGrpMemInfo
+"params":                 // MemberId Profile
+{
+  "memberId": "<member ID>",
+  "profile": {
+    "displayName": "<display name>",
+    "fullName": "<full name>"
+  }
+}
+
+"event": "x.grp.mem.con" // XGrpMemCon
+"params":                // MemberId
+{
+  "memberId": "<member ID>"
+}
+
+"event": "x.grp.mem.con.all" // XGrpMemConAll
+"params":                    // MemberId
+{
+  "memberId": "<member ID>"
+}
+
+"event": "x.grp.mem.del" // XGrpMemDel
+"params":                // MemberId
+{
+  "memberId": "<member ID>"
+}
+
+"event": "x.grp.leave" // XGrpLeave
+"params":
+{}
+
+"event": "x.grp.del" // XGrpDel
+"params":
+{}
+
+"event": "x.info.probe" // XInfoProbe
+"params":               // ByteString
+{
+  "probe": "<probe>"
+}
+
+"event": "x.info.probe.check" // XInfoProbeCheck
+"params":                     // ByteString
+{
+  "probeHash": "<probe hash>"
+}
+
+"event": "x.info.probe.ok" // XInfoProbeOk
+"params":                  // ByteString
+{
+  "probe": "<probe>"
+}
+
+"event": "x.ok" // XOk
+"params":
+{}
+```
+
 ### Group protocol
 
 #### Add group member

stack.yaml

@@ -38,10 +38,15 @@ extra-deps:
   - cryptostore-0.2.1.0@sha256:9896e2984f36a1c8790f057fd5ce3da4cbcaf8aa73eb2d9277916886978c5b19,3881
   - simple-logger-0.1.0@sha256:be8ede4bd251a9cac776533bae7fb643369ebd826eb948a9a18df1a8dd252ff8,1079
   - terminal-0.2.0.0@sha256:de6770ecaae3197c66ac1f0db5a80cf5a5b1d3b64a66a05b50f442de5ad39570,2977
-  - simplexmq-0.5.2@sha256:3544e479f353c1bbc6aa9405ef6976b78364f437d8af9cc45b9e0b228429e240,7884
+  - simplexmq-1.0.0@sha256:34b2004728ae396e3ae449cd090ba7410781e2b3cefc59259915f4ca5daa9ea8,8561
   # - ../simplexmq
   # - github: simplex-chat/simplexmq
-  #   commit: f15067cf6891bda3216c6cf6d2e3ecdba9b7269e
+  #   commit: bfa4911217b71527a6fbaf73b242b5684aaf9fce
+  - github: simplex-chat/hs-tls
+    commit: cea6d52c512716ff09adcac86ebc95bb0b3bb797
+    subdirs:
+      - core
+
 #
 # extra-deps: []
 

tests/ChatClient.hs

@@ -39,7 +39,7 @@ opts :: ChatOpts
 opts =
   ChatOpts
     { dbFile = undefined,
-      smpServers = ["localhost:5001"]
+      smpServers = ["smp://LcJUMfVhwD8yxjAiSaDzzGF3-kLG4Uh0Fl_ZIjrRwjI=@localhost:5001"]
     }
 
 termSettings :: VirtualTerminalSettings
@@ -66,12 +66,12 @@ cfg :: ChatConfig
 cfg =
   defaultChatConfig
     { agentConfig =
-        aCfg {retryInterval = (retryInterval aCfg) {initialInterval = 50000}}
+        aCfg {reconnectInterval = (reconnectInterval aCfg) {initialInterval = 50000}}
     }
 
 virtualSimplexChat :: FilePath -> Profile -> IO TestCC
 virtualSimplexChat dbFile profile = do
-  st <- createStore (dbFile <> ".chat.db") 1
+  st <- createStore (dbFile <> "_chat.db") 1
   void . runExceptT $ createUser st profile True
   t <- withVirtualTerminal termSettings pure
   cc <- newChatController cfg opts {dbFile} t . const $ pure () -- no notifications
@@ -143,44 +143,16 @@ concurrentlyN_ = mapConcurrently_ id
 serverCfg :: ServerConfig
 serverCfg =
   ServerConfig
-    { transports = [(serverPort, transport @TCP)],
+    { transports = [(serverPort, transport @TLS)],
       tbqSize = 1,
       serverTbqSize = 1,
       msgQueueQuota = 4,
       queueIdBytes = 12,
       msgIdBytes = 6,
       storeLog = Nothing,
-      blockSize = 4096,
-      serverPrivateKey =
-        -- full RSA private key (only for tests)
-        "MIIFIwIBAAKCAQEArZyrri/NAwt5buvYjwu+B/MQeJUszDBpRgVqNddlI9kNwDXu\
-        \kaJ8chEhrtaUgXeSWGooWwqjXEUQE6RVbCC6QVo9VEBSP4xFwVVd9Fj7OsgfcXXh\
-        \AqWxfctDcBZQ5jTUiJpdBc+Vz2ZkumVNl0W+j9kWm9nfkMLQj8c0cVSDxz4OKpZb\
-        \qFuj0uzHkis7e7wsrKSKWLPg3M5ZXPZM1m9qn7SfJzDRDfJifamxWI7uz9XK2+Dp\
-        \NkUQlGQgFJEv1cKN88JAwIqZ1s+TAQMQiB+4QZ2aNfSqGEzRJN7FMCKRK7pM0A9A\
-        \PCnijyuImvKFxTdk8Bx1q+XNJzsY6fBrLWJZ+QKBgQCySG4tzlcEm+tOVWRcwrWh\
-        \6zsczGZp9mbf9c8itRx6dlldSYuDG1qnddL70wuAZF2AgS1JZgvcRZECoZRoWP5q\
-        \Kq2wvpTIYjFPpC39lxgUoA/DXKVKZZdan+gwaVPAPT54my1CS32VrOiAY4gVJ3LJ\
-        \Mn1/FqZXUFQA326pau3loQKCAQEAoljmJMp88EZoy3HlHUbOjl5UEhzzVsU1TnQi\
-        \QmPm+aWRe2qelhjW4aTvSVE5mAUJsN6UWTeMf4uvM69Z9I5pfw2pEm8x4+GxRibY\
-        \iiwF2QNaLxxmzEHm1zQQPTgb39o8mgklhzFPill0JsnL3f6IkVwjFJofWSmpqEGs\
-        \dFSMRSXUTVXh1p/o7QZrhpwO/475iWKVS7o48N/0Xp513re3aXw+DRNuVnFEaBIe\
-        \TLvWM9Czn16ndAu1HYiTBuMvtRbAWnGZxU8ewzF4wlWK5tdIL5PTJDd1VhZJAKtB\
-        \npDvJpwxzKmjAhcTmjx0ckMIWtdVaOVm/2gWCXDty2FEdg7koQKBgQDOUUguJ/i7\
-        \q0jldWYRnVkotKnpInPdcEaodrehfOqYEHnvro9xlS6OeAS4Vz5AdH45zQ/4J3bV\
-        \2cH66tNr18ebM9nL//t5G69i89R9W7szyUxCI3LmAIdi3oSEbmz5GQBaw4l6h9Wi\
-        \n4FmFQaAXZrjQfO2qJcAHvWRsMp2pmqAGwKBgQDXaza0DRsKWywWznsHcmHa0cx8\
-        \I4jxqGaQmLO7wBJRP1NSFrywy1QfYrVX9CTLBK4V3F0PCgZ01Qv94751CzN43TgF\
-        \ebd/O9r5NjNTnOXzdWqETbCffLGd6kLgCMwPQWpM9ySVjXHWCGZsRAnF2F6M1O32\
-        \43StIifvwJQFqSM3ewKBgCaW6y7sRY90Ua7283RErezd9EyT22BWlDlACrPu3FNC\
-        \LtBf1j43uxBWBQrMLsHe2GtTV0xt9m0MfwZsm2gSsXcm4Xi4DJgfN+Z7rIlyy9UY\
-        \PCDSdZiU1qSr+NrffDrXlfiAM1cUmCdUX7eKjp/ltkUHNaOGfSn5Pdr3MkAiD/Hf\
-        \AoGBAKIdKCuOwuYlwjS9J+IRGuSSM4o+OxQdwGmcJDTCpyWb5dEk68e7xKIna3zf\
-        \jc+H+QdMXv1nkRK9bZgYheXczsXaNZUSTwpxaEldzVD3hNvsXSgJRy9fqHwA4PBq\
-        \vqiBHoO3RNbqg+2rmTMfDuXreME3S955ZiPZm4Z+T8Hj52mPAoGAQm5QH/gLFtY5\
-        \+znqU/0G8V6BKISCQMxbbmTQVcTgGySrP2gVd+e4MWvUttaZykhWqs8rpr7mgpIY\
-        \hul7Swx0SHFN3WpXu8uj+B6MLpRcCbDHO65qU4kQLs+IaXXsuuTjMvJ5LwjkZVrQ\
-        \TmKzSAw7iVWwEUZR/PeiEKazqrpp9VU="
+      caCertificateFile = "tests/fixtures/tls/ca.crt",
+      privateKeyFile = "tests/fixtures/tls/server.key",
+      certificateFile = "tests/fixtures/tls/server.crt"
     }
 
 withSmpServer :: IO a -> IO a

tests/ChatTests.hs

@@ -295,6 +295,8 @@ testGroup2 =
         ]
       dan #> "#club how is it going?"
       dan <## "you are no longer a member of the group"
+      dan ##> "/d #club"
+      dan <## "#club: you deleted the group"
       dan <##> cath
       dan <##> alice
       -- member leaves
@@ -316,6 +318,8 @@ testGroup2 =
         (bob </)
       bob #> "#club how is it going?"
       bob <## "you are no longer a member of the group"
+      bob ##> "/d #club"
+      bob <## "#club: you deleted the group"
       bob <##> cath
       bob <##> alice
 
@@ -338,6 +342,8 @@ testGroupDelete =
       bob <## "#team: you deleted the group"
       cath #> "#team hi"
       cath <## "you are no longer a member of the group"
+      cath ##> "/d #team"
+      cath <## "#team: you deleted the group"
 
 testGroupDeleteWhenInvited :: IO ()
 testGroupDeleteWhenInvited =

tests/ProtocolTests.hs

@@ -1,3 +1,5 @@
+{-# LANGUAGE DataKinds #-}
+{-# LANGUAGE DuplicateRecordFields #-}
 {-# LANGUAGE OverloadedLists #-}
 {-# LANGUAGE OverloadedStrings #-}
 
@@ -5,43 +7,115 @@ module ProtocolTests where
 
 import Data.ByteString.Char8 (ByteString)
 import Simplex.Chat.Protocol
+import Simplex.Chat.Types
+import Simplex.Messaging.Agent.Protocol
+import qualified Simplex.Messaging.Crypto as C
+import Simplex.Messaging.Crypto.Ratchet
+import Simplex.Messaging.Encoding.String
 import Simplex.Messaging.Parsers (parseAll)
+import Simplex.Messaging.Protocol (smpClientVRange)
 import Test.Hspec
 
 protocolTests :: Spec
-protocolTests = do
-  parseChatMessageTest
+protocolTests = decodeChatMessageTest
 
-(#==) :: ByteString -> RawChatMessage -> Expectation
-s #== msg = parseAll rawChatMessageP s `shouldBe` Right msg
+srv :: SMPServer
+srv =
+  SMPServer
+    { host = "smp.simplex.im",
+      port = "5223",
+      keyHash = C.KeyHash "\215m\248\251"
+    }
 
-parseChatMessageTest :: Spec
-parseChatMessageTest = describe "Raw chat message format" $ do
-  it "no parameters and content" $
-    "5 x.grp.mem.leave   " #== RawChatMessage (Just 5) "x.grp.mem.leave" [] []
-  it "one parameter, no content" $
-    "6 x.msg.del 3  " #== RawChatMessage (Just 6) "x.msg.del" ["3"] []
-  it "with content that fits the message" $
-    "7 x.msg.new c.text x.text:11 hello there "
-      #== RawChatMessage
-        (Just 7)
-        "x.msg.new"
-        ["c.text"]
-        [RawMsgBodyContent (RawContentType "x" "text") "hello there"]
-  it "with DAG reference and partial content" $
-    "8 x.msg.new c.image x.dag:16,x.text:7,m.image/jpg:6 0123456789012345 picture abcdef "
-      #== RawChatMessage
-        (Just 8)
-        "x.msg.new"
-        ["c.image"]
-        [ RawMsgBodyContent (RawContentType "x" "dag") "0123456789012345",
-          RawMsgBodyContent (RawContentType "x" "text") "picture",
-          RawMsgBodyContent (RawContentType "m" "image/jpg") "abcdef"
-        ]
-  it "without message id" $
-    " x.grp.mem.inv 23456,123 x.json:46 {\"contactRef\":\"john\",\"displayName\":\"John Doe\"} "
-      #== RawChatMessage
-        Nothing
-        "x.grp.mem.inv"
-        ["23456", "123"]
-        [RawMsgBodyContent (RawContentType "x" "json") "{\"contactRef\":\"john\",\"displayName\":\"John Doe\"}"]
+queue :: SMPQueueUri
+queue =
+  SMPQueueUri
+    { smpServer = srv,
+      senderId = "\223\142z\251",
+      clientVRange = smpClientVRange,
+      dhPublicKey = "MCowBQYDK2VuAyEAjiswwI3O/NlS8Fk3HJUW870EY2bAwmttMBsvRB9eV3o="
+    }
+
+connReqData :: ConnReqUriData
+connReqData =
+  ConnReqUriData
+    { crScheme = simplexChat,
+      crAgentVRange = smpAgentVRange,
+      crSmpQueues = [queue]
+    }
+
+testDhPubKey :: C.PublicKeyX448
+testDhPubKey = "MEIwBQYDK2VvAzkAmKuSYeQ/m0SixPDS8Wq8VBaTS1cW+Lp0n0h4Diu+kUpR+qXx4SDJ32YGEFoGFGSbGPry5Ychr6U="
+
+testE2ERatchetParams :: E2ERatchetParamsUri 'C.X448
+testE2ERatchetParams = E2ERatchetParamsUri e2eEncryptVRange testDhPubKey testDhPubKey
+
+testConnReq :: ConnectionRequestUri 'CMInvitation
+testConnReq = CRInvitationUri connReqData testE2ERatchetParams
+
+(==#) :: ByteString -> ChatMsgEvent -> Expectation
+s ==# msg = do
+  strDecode s `shouldBe` Right (ChatMessage msg)
+  parseAll strP s `shouldBe` Right (ChatMessage msg)
+
+(#==) :: ByteString -> ChatMsgEvent -> Expectation
+s #== msg = strEncode (ChatMessage msg) `shouldBe` s
+
+(#==#) :: ByteString -> ChatMsgEvent -> Expectation
+s #==# msg = do
+  s #== msg
+  s ==# msg
+
+testProfile :: Profile
+testProfile = Profile {displayName = "alice", fullName = "Alice"}
+
+testGroupProfile :: GroupProfile
+testGroupProfile = GroupProfile {displayName = "team", fullName = "Team"}
+
+decodeChatMessageTest :: Spec
+decodeChatMessageTest = describe "Chat message encoding/decoding" $ do
+  it "x.msg.new" $ "{\"event\":\"x.msg.new\",\"params\":{\"content\":{\"text\":\"hello\",\"type\":\"text\"}}}" #==# XMsgNew (MCText "hello")
+  it "x.file" $
+    "{\"event\":\"x.file\",\"params\":{\"file\":{\"fileConnReq\":\"https://simplex.chat/invitation#/?v=1&smp=smp%3A%2F%2F1234-w%3D%3D%40smp.simplex.im%3A5223%2F3456-w%3D%3D%23MCowBQYDK2VuAyEAjiswwI3O_NlS8Fk3HJUW870EY2bAwmttMBsvRB9eV3o%3D&e2e=v%3D1%26x3dh%3DMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D%2CMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D\",\"fileSize\":12345,\"fileName\":\"photo.jpg\"}}}"
+      #==# XFile FileInvitation {fileName = "photo.jpg", fileSize = 12345, fileConnReq = testConnReq}
+  it "x.file.acpt" $ "{\"event\":\"x.file.acpt\",\"params\":{\"fileName\":\"photo.jpg\"}}" #==# XFileAcpt "photo.jpg"
+  it "x.info" $ "{\"event\":\"x.info\",\"params\":{\"profile\":{\"fullName\":\"Alice\",\"displayName\":\"alice\"}}}" #==# XInfo testProfile
+  it "x.info" $ "{\"event\":\"x.info\",\"params\":{\"profile\":{\"fullName\":\"\",\"displayName\":\"alice\"}}}" #==# XInfo Profile {displayName = "alice", fullName = ""}
+  it "x.contact without content field" $
+    "{\"event\":\"x.contact\",\"params\":{\"profile\":{\"fullName\":\"Alice\",\"displayName\":\"alice\"}}}"
+      #==# XContact testProfile Nothing
+  it "x.contact with content null" $
+    "{\"event\":\"x.contact\",\"params\":{\"content\":null,\"profile\":{\"fullName\":\"Alice\",\"displayName\":\"alice\"}}}"
+      ==# XContact testProfile Nothing
+  it "x.contact with content" $
+    "{\"event\":\"x.contact\",\"params\":{\"content\":{\"text\":\"hello\",\"type\":\"text\"},\"profile\":{\"fullName\":\"Alice\",\"displayName\":\"alice\"}}}"
+      #==# XContact testProfile (Just $ MCText "hello")
+  it "x.grp.inv" $
+    "{\"event\":\"x.grp.inv\",\"params\":{\"groupInvitation\":{\"connRequest\":\"https://simplex.chat/invitation#/?v=1&smp=smp%3A%2F%2F1234-w%3D%3D%40smp.simplex.im%3A5223%2F3456-w%3D%3D%23MCowBQYDK2VuAyEAjiswwI3O_NlS8Fk3HJUW870EY2bAwmttMBsvRB9eV3o%3D&e2e=v%3D1%26x3dh%3DMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D%2CMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D\",\"invitedMember\":{\"memberRole\":\"member\",\"memberId\":\"BQYHCA==\"},\"groupProfile\":{\"fullName\":\"Team\",\"displayName\":\"team\"},\"fromMember\":{\"memberRole\":\"admin\",\"memberId\":\"AQIDBA==\"}}}}"
+      #==# XGrpInv GroupInvitation {fromMember = MemberIdRole (MemberId "\1\2\3\4") GRAdmin, invitedMember = MemberIdRole (MemberId "\5\6\7\8") GRMember, connRequest = testConnReq, groupProfile = testGroupProfile}
+  it "x.grp.acpt" $ "{\"event\":\"x.grp.acpt\",\"params\":{\"memberId\":\"AQIDBA==\"}}" #==# XGrpAcpt (MemberId "\1\2\3\4")
+  it "x.grp.acpt" $ "{\"event\":\"x.grp.acpt\",\"params\":{\"memberId\":\"AQIDBA==\"}}" #==# XGrpAcpt (MemberId "\1\2\3\4")
+  it "x.grp.mem.new" $
+    "{\"event\":\"x.grp.mem.new\",\"params\":{\"memberInfo\":{\"memberRole\":\"admin\",\"memberId\":\"AQIDBA==\",\"profile\":{\"fullName\":\"Alice\",\"displayName\":\"alice\"}}}}"
+      #==# XGrpMemNew MemberInfo {memberId = MemberId "\1\2\3\4", memberRole = GRAdmin, profile = testProfile}
+  it "x.grp.mem.intro" $
+    "{\"event\":\"x.grp.mem.intro\",\"params\":{\"memberInfo\":{\"memberRole\":\"admin\",\"memberId\":\"AQIDBA==\",\"profile\":{\"fullName\":\"Alice\",\"displayName\":\"alice\"}}}}"
+      #==# XGrpMemIntro MemberInfo {memberId = MemberId "\1\2\3\4", memberRole = GRAdmin, profile = testProfile}
+  it "x.grp.mem.inv" $
+    "{\"event\":\"x.grp.mem.inv\",\"params\":{\"memberId\":\"AQIDBA==\",\"memberIntro\":{\"directConnReq\":\"https://simplex.chat/invitation#/?v=1&smp=smp%3A%2F%2F1234-w%3D%3D%40smp.simplex.im%3A5223%2F3456-w%3D%3D%23MCowBQYDK2VuAyEAjiswwI3O_NlS8Fk3HJUW870EY2bAwmttMBsvRB9eV3o%3D&e2e=v%3D1%26x3dh%3DMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D%2CMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D\",\"groupConnReq\":\"https://simplex.chat/invitation#/?v=1&smp=smp%3A%2F%2F1234-w%3D%3D%40smp.simplex.im%3A5223%2F3456-w%3D%3D%23MCowBQYDK2VuAyEAjiswwI3O_NlS8Fk3HJUW870EY2bAwmttMBsvRB9eV3o%3D&e2e=v%3D1%26x3dh%3DMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D%2CMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D\"}}}"
+      #==# XGrpMemInv (MemberId "\1\2\3\4") IntroInvitation {groupConnReq = testConnReq, directConnReq = testConnReq}
+  it "x.grp.mem.fwd" $
+    "{\"event\":\"x.grp.mem.fwd\",\"params\":{\"memberIntro\":{\"directConnReq\":\"https://simplex.chat/invitation#/?v=1&smp=smp%3A%2F%2F1234-w%3D%3D%40smp.simplex.im%3A5223%2F3456-w%3D%3D%23MCowBQYDK2VuAyEAjiswwI3O_NlS8Fk3HJUW870EY2bAwmttMBsvRB9eV3o%3D&e2e=v%3D1%26x3dh%3DMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D%2CMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D\",\"groupConnReq\":\"https://simplex.chat/invitation#/?v=1&smp=smp%3A%2F%2F1234-w%3D%3D%40smp.simplex.im%3A5223%2F3456-w%3D%3D%23MCowBQYDK2VuAyEAjiswwI3O_NlS8Fk3HJUW870EY2bAwmttMBsvRB9eV3o%3D&e2e=v%3D1%26x3dh%3DMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D%2CMEIwBQYDK2VvAzkAmKuSYeQ_m0SixPDS8Wq8VBaTS1cW-Lp0n0h4Diu-kUpR-qXx4SDJ32YGEFoGFGSbGPry5Ychr6U%3D\"},\"memberInfo\":{\"memberRole\":\"admin\",\"memberId\":\"AQIDBA==\",\"profile\":{\"fullName\":\"Alice\",\"displayName\":\"alice\"}}}}"
+      #==# XGrpMemFwd MemberInfo {memberId = MemberId "\1\2\3\4", memberRole = GRAdmin, profile = testProfile} IntroInvitation {groupConnReq = testConnReq, directConnReq = testConnReq}
+  it "x.grp.mem.info" $
+    "{\"event\":\"x.grp.mem.info\",\"params\":{\"memberId\":\"AQIDBA==\",\"profile\":{\"fullName\":\"Alice\",\"displayName\":\"alice\"}}}"
+      #== XGrpMemInfo (MemberId "\1\2\3\4") testProfile
+  it "x.grp.mem.con" $ "{\"event\":\"x.grp.mem.con\",\"params\":{\"memberId\":\"AQIDBA==\"}}" #==# XGrpMemCon (MemberId "\1\2\3\4")
+  it "x.grp.mem.con.all" $ "{\"event\":\"x.grp.mem.con.all\",\"params\":{\"memberId\":\"AQIDBA==\"}}" #==# XGrpMemConAll (MemberId "\1\2\3\4")
+  it "x.grp.mem.del" $ "{\"event\":\"x.grp.mem.del\",\"params\":{\"memberId\":\"AQIDBA==\"}}" #==# XGrpMemDel (MemberId "\1\2\3\4")
+  it "x.grp.leave" $ "{\"event\":\"x.grp.leave\",\"params\":{}}" ==# XGrpLeave
+  it "x.grp.del" $ "{\"event\":\"x.grp.del\",\"params\":{}}" ==# XGrpDel
+  it "x.info.probe" $ "{\"event\":\"x.info.probe\",\"params\":{\"probe\":\"AQIDBA==\"}}" #==# XInfoProbe (Probe "\1\2\3\4")
+  it "x.info.probe.check" $ "{\"event\":\"x.info.probe.check\",\"params\":{\"probeHash\":\"AQIDBA==\"}}" #==# XInfoProbeCheck (ProbeHash "\1\2\3\4")
+  it "x.info.probe.ok" $ "{\"event\":\"x.info.probe.ok\",\"params\":{\"probe\":\"AQIDBA==\"}}" #==# XInfoProbeOk (Probe "\1\2\3\4")
+  it "x.ok" $ "{\"event\":\"x.ok\",\"params\":{}}" ==# XOk

tests/fixtures/tls/ca.crt

@@ -0,0 +1,12 @@
+-----BEGIN CERTIFICATE-----
+MIIBtjCCATagAwIBAgIUe2PryrWo0xXX9vcA3WfbCzcdmgAwBQYDK2VxMCoxFjAU
+BgNVBAMMDVNNUCBzZXJ2ZXIgQ0ExEDAOBgNVBAoMB1NpbXBsZVgwIBcNMjIwMTEx
+MTExNjM5WhgPNDc1OTEyMDgxMTE2MzlaMCoxFjAUBgNVBAMMDVNNUCBzZXJ2ZXIg
+Q0ExEDAOBgNVBAoMB1NpbXBsZVgwQzAFBgMrZXEDOgCAcvFwVicR+RLZpiEWPFNR
+XYTbf+mFcX1NHIyPQDugFwOCgqJAW1fsjYgFhtQJSMH/lc1N7clfm4CjUzBRMB0G
+A1UdDgQWBBQcUJvR7mm26yxMQfCsWgbnwMmJVDAfBgNVHSMEGDAWgBQcUJvR7mm2
+6yxMQfCsWgbnwMmJVDAPBgNVHRMBAf8EBTADAQH/MAUGAytlcQNzAAAP/hMPNxyW
+fyJi+iJViodU+C/aklnvHtjh5P3AbiVCSUfY6+PEdvkC8Ov0pBAYpYi5ukSNNVXl
+ABVRlipB+vOcLQStNyaZ7kXzQ2IO/0btmIidh+G6SP8I4aytYIYYcV5pEUZpG1L1
+57g8P29SDv81AA==
+-----END CERTIFICATE-----

tests/fixtures/tls/server.crt

@@ -0,0 +1,12 @@
+-----BEGIN CERTIFICATE-----
+MIIBvDCCATygAwIBAgIUUgqht3ibEJ9jp5JuroML5Baxa/IwBQYDK2VxMCoxFjAU
+BgNVBAMMDVNNUCBzZXJ2ZXIgQ0ExEDAOBgNVBAoMB1NpbXBsZVgwIBcNMjIwMTEx
+MTYzOTQ2WhgPNDc1OTEyMDgxNjM5NDZaMBQxEjAQBgNVBAMMCWxvY2FsaG9zdDBD
+MAUGAytlcQM6AFj+fmjmgOMBFgQ7lXfICSlSaB5sYp1wzhM2IaqgmVhWbDs3Tw7W
+oUXFJsWpKku1kFjLH2yhqIInAKNvMG0wCQYDVR0TBAIwADALBgNVHQ8EBAMCA8gw
+EwYDVR0lBAwwCgYIKwYBBQUHAwEwHQYDVR0OBBYEFMjbHbxESDI8YDPjJ8kXWjkg
+jHxmMB8GA1UdIwQYMBaAFBxQm9HuabbrLExB8KxaBufAyYlUMAUGAytlcQNzAP98
+RpFacmjxsc07GwC07uAxUxNgxX4R2nA+0St866uFbdyxApiyn44giG+m+YXCXLjh
+1ffKEhhqAPdlpzByLarx8EWgmOsisBCRWrmraksOV15FKxjPqm7OYKp4f9GvieAQ
+o/SKOyZVVdcZX7NcuDUxAA==
+-----END CERTIFICATE-----

tests/fixtures/tls/server.key

@@ -0,0 +1,4 @@
+-----BEGIN PRIVATE KEY-----
+MEcCAQAwBQYDK2VxBDsEOZcXSjKQ3FQ9EBfqHmFIidOu4LecpUgi5NVPMhx7mS2d
+QNQ0lFasp+EfvLErbs1Sd9xLVxA1hnZJng==
+-----END PRIVATE KEY-----