From 942e5eb8c49d8d28b51941f4f32f8b2cd610e54c Mon Sep 17 00:00:00 2001 From: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com> Date: Wed, 27 Sep 2023 22:19:20 +0100 Subject: [PATCH 1/8] docs: update branches --- docs/CONTRIBUTING.md | 34 +++++++++++++++++++++++----------- 1 file changed, 23 insertions(+), 11 deletions(-) diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md index dc18bebb0..0aa09c516 100644 --- a/docs/CONTRIBUTING.md +++ b/docs/CONTRIBUTING.md @@ -38,9 +38,15 @@ You will have to add `/opt/homebrew/opt/openssl@1.1/bin` to your PATH in order t - `master` - branch for beta version releases (GHC 9.6.2). -- `master-android` - used to build beta Android core library with Nix (GHC 8.10.7). +- `master-ghc8107` - branch for beta version releases (GHC 8.10.7). -- `master-ios` - used to build beta iOS core library with Nix (GHC 8.10.7) – this branch should be the same as `master-android` except Nix configuration files. +- `master-android` - used to build beta Android core library with Nix (GHC 8.10.7), same as `master-ghc8107` + +- `master-ios` - used to build beta iOS core library with Nix (GHC 8.10.7). + +- `windows-ghc8107` - branch for windows core library build (GHC 8.10.7). + +`master-ios` and `windows-ghc8107` branches should be the same as `master-ghc8107` except Nix configuration files. **In simplexmq repo** @@ -54,24 +60,30 @@ You will have to add `/opt/homebrew/opt/openssl@1.1/bin` to your PATH in order t 2. If simplexmq repo was changed, to build mobile core libraries you need to merge its `master` branch into `master-ghc8107` branch. -3. To build Android core library: -- merge `master` branch to `master-android` branch. +3. To build core libraries for Android, iOS and windows: +- merge `master` branch to `master-ghc8107` branch. +- update `simplexmq` commit in `master-ghc8107` branch to the commit in `master-ghc8107` branch (probably, when resolving merge conflicts). - update code to be compatible with GHC 8.10.7 (see below). -- update `simplexmq` commit in `master-android` branch to the commit in `master-ghc8107` branch. - push to GitHub. -4. To build iOS core library, merge `master-android` branch to `master-ios` branch, and push to GitHub. +4. To build Android core library, merge `master-ghc8107` branch to `master-android` branch, and push to GitHub. -5. To build Desktop and CLI apps, make tag in `master` branch, APK files should be attached to the release. +5. To build iOS core library, merge `master-ghc8107` branch to `master-ios` branch, and push to GitHub. -6. After the public release to App Store and Play Store, merge: +6. To build windows core library, merge `master-ghc8107` branch to `windows-ghc8107` branch, and push to GitHub. + +7. To build Desktop and CLI apps, make tag in `master` branch, APK files should be attached to the release. + +8. After the public release to App Store and Play Store, merge: - `master` to `stable` -- `master` to `master-android` (and compile/update code) -- `master-android` to `master-ios` +- `master` to `master-ghc8107` (and compile/update code) +- `master-ghc8107` to `master-android` +- `master-ghc8107` to `master-ios` +- `master-ghc8107` to `windows-ghc8107` - `master-android` to `stable-android` - `master-ios` to `stable-ios` -7. Independently, `master` branch of simplexmq repo should be merged to `stable` branch on stable releases. +9. Independently, `master` branch of simplexmq repo should be merged to `stable` branch on stable releases. ## Differences between GHC 8.10.7 and GHC 9.6.2 From dea96df27bf7002b1d52942c59bf8c1981dd67e7 Mon Sep 17 00:00:00 2001 From: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com> Date: Thu, 28 Sep 2023 09:26:54 +0100 Subject: [PATCH 2/8] docs: update join team --- docs/JOIN_TEAM.md | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/docs/JOIN_TEAM.md b/docs/JOIN_TEAM.md index 5a31b3e05..cf33df1ee 100644 --- a/docs/JOIN_TEAM.md +++ b/docs/JOIN_TEAM.md @@ -15,31 +15,30 @@ We want to add up to 3 people to the team. ## Who we are looking for -### Systems Haskell engineer +### Application Haskell engineer -You are a servers/network/Haskell expert: -- network libraries. +You are an expert in language models, databases and Haskell: +- expert knowledge of SQL. - exception handling, concurrency, STM. - type systems - we use ad hoc dependent types a lot. -- strictness. -- has some expertise in network protocols, cryptography and general information security principles and approaches. +- experience integrating open-source language models. +- experience developing community-centric applications. - interested to build the next generation of messaging network. -You will be focussed mostly on our servers code, and will also contribute to the core client code written in Haskell. +You will be focussed mostly on our client applications, and will also contribute to the servers also written in Haskell. +### iOS / Mac engineer -### Product engineer (iOS) - -You are a product UX expert who designs great user experiences directly in iOS code: -- iOS and Mac platforms, including: - - SwiftUI and UIKit. - - extensions, including notification service extension and sharing extension. - - low level inter-process communication primitives for concurrency. +You are an expert in Apple platforms, including: +- iOS and Mac platform architecture. +- Swift and Objective-C. +- SwiftUI and UIKit. +- extensions, including notification service extension and sharing extension. +- low level inter-process communication primitives for concurrency. - interested about creating the next generation of UX for a communication/social network. Knowledge of Android and Kotlin Multiplatform would be a bonus - we use Kotlin Jetpack Compose for our Android and desktop apps. - ## About you - **Passionate about joining SimpleX Chat team**: From 957f3b3eb0421096c59058f7870e7c16507956ae Mon Sep 17 00:00:00 2001 From: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com> Date: Thu, 28 Sep 2023 13:16:03 +0400 Subject: [PATCH 3/8] core: delete unused contact silently (#3140) --- src/Simplex/Chat.hs | 26 ++++++++++++++++---------- tests/ChatTests/Direct.hs | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+), 10 deletions(-) diff --git a/src/Simplex/Chat.hs b/src/Simplex/Chat.hs index a16cb98ae..822d6cd4b 100644 --- a/src/Simplex/Chat.hs +++ b/src/Simplex/Chat.hs @@ -4252,16 +4252,22 @@ processAgentMessageConn user@User {userId} corrId agentConnId agentMessage = do xInfo c p' = void $ processContactProfileUpdate c p' True xDirectDel :: Contact -> RcvMessage -> MsgMeta -> m () - xDirectDel c msg msgMeta = do - checkIntegrityCreateItem (CDDirectRcv c) msgMeta - ct' <- withStore' $ \db -> updateContactStatus db user c CSDeleted - contactConns <- withStore $ \db -> getContactConnections db userId ct' - deleteAgentConnectionsAsync user $ map aConnId contactConns - forM_ contactConns $ \conn -> withStore' $ \db -> updateConnectionStatus db conn ConnDeleted - let ct'' = ct' {activeConn = (contactConn ct') {connStatus = ConnDeleted}} :: Contact - ci <- saveRcvChatItem user (CDDirectRcv ct'') msg msgMeta (CIRcvDirectEvent RDEContactDeleted) - toView $ CRNewChatItem user (AChatItem SCTDirect SMDRcv (DirectChat ct'') ci) - toView $ CRContactDeletedByContact user ct'' + xDirectDel c msg msgMeta = + if directOrUsed c + then do + checkIntegrityCreateItem (CDDirectRcv c) msgMeta + ct' <- withStore' $ \db -> updateContactStatus db user c CSDeleted + contactConns <- withStore $ \db -> getContactConnections db userId ct' + deleteAgentConnectionsAsync user $ map aConnId contactConns + forM_ contactConns $ \conn -> withStore' $ \db -> updateConnectionStatus db conn ConnDeleted + let ct'' = ct' {activeConn = (contactConn ct') {connStatus = ConnDeleted}} :: Contact + ci <- saveRcvChatItem user (CDDirectRcv ct'') msg msgMeta (CIRcvDirectEvent RDEContactDeleted) + toView $ CRNewChatItem user (AChatItem SCTDirect SMDRcv (DirectChat ct'') ci) + toView $ CRContactDeletedByContact user ct'' + else do + contactConns <- withStore $ \db -> getContactConnections db userId c + deleteAgentConnectionsAsync user $ map aConnId contactConns + withStore' $ \db -> deleteContact db user c processContactProfileUpdate :: Contact -> Profile -> Bool -> m Contact processContactProfileUpdate c@Contact {profile = p} p' createItems diff --git a/tests/ChatTests/Direct.hs b/tests/ChatTests/Direct.hs index d9e8bac2f..445a5ab99 100644 --- a/tests/ChatTests/Direct.hs +++ b/tests/ChatTests/Direct.hs @@ -31,6 +31,7 @@ chatDirectTests = do describe "direct messages" $ do describe "add contact and send/receive message" testAddContact it "deleting contact deletes profile" testDeleteContactDeletesProfile + it "unused contact is deleted silently" testDeleteUnusedContactSilent it "direct message quoted replies" testDirectMessageQuotedReply it "direct message update" testDirectMessageUpdate it "direct message edit history" testDirectMessageEditHistory @@ -214,6 +215,42 @@ testDeleteContactDeletesProfile = (bob FilePath -> IO () +testDeleteUnusedContactSilent = + testChatCfg3 testCfgCreateGroupDirect aliceProfile bobProfile cathProfile $ + \alice bob cath -> do + createGroup3 "team" alice bob cath + bob ##> "/contacts" + bob <### ["alice (Alice)", "cath (Catherine)"] + bob `hasContactProfiles` ["bob", "alice", "cath"] + cath ##> "/contacts" + cath <### ["alice (Alice)", "bob (Bob)"] + cath `hasContactProfiles` ["cath", "alice", "bob"] + -- bob deletes cath, cath's bob contact is deleted silently + bob ##> "/d cath" + bob <## "cath: contact is deleted" + bob ##> "/contacts" + bob <## "alice (Alice)" + threadDelay 50000 + cath ##> "/contacts" + cath <## "alice (Alice)" + -- group messages work + alice #> "#team hello" + concurrentlyN_ + [ bob <# "#team alice> hello", + cath <# "#team alice> hello" + ] + bob #> "#team hi there" + concurrentlyN_ + [ alice <# "#team bob> hi there", + cath <# "#team bob> hi there" + ] + cath #> "#team hey" + concurrentlyN_ + [ alice <# "#team cath> hey", + bob <# "#team cath> hey" + ] + testDirectMessageQuotedReply :: HasCallStack => FilePath -> IO () testDirectMessageQuotedReply = testChat2 aliceProfile bobProfile $ From 682dfe503cdcaea6cb16d6f9eabe6fdcdb6afd3d Mon Sep 17 00:00:00 2001 From: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com> Date: Thu, 28 Sep 2023 13:52:43 +0400 Subject: [PATCH 4/8] android, desktop: notify contact about contact deletion (#3139) Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com> --- .../chat/simplex/common/model/ChatModel.kt | 23 ++++++- .../chat/simplex/common/model/SimpleXAPI.kt | 10 +++ .../simplex/common/views/chat/ChatInfoView.kt | 5 +- .../simplex/common/views/chat/ChatView.kt | 15 ++-- .../common/views/chat/item/ChatItemView.kt | 1 + .../common/views/chatlist/ChatPreviewView.kt | 68 +++++++++++-------- .../commonMain/resources/MR/base/strings.xml | 3 + 7 files changed, 87 insertions(+), 38 deletions(-) diff --git a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/model/ChatModel.kt b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/model/ChatModel.kt index 887abe756..0eb02515b 100644 --- a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/model/ChatModel.kt +++ b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/model/ChatModel.kt @@ -797,6 +797,7 @@ data class Contact( val activeConn: Connection, val viaGroup: Long? = null, val contactUsed: Boolean, + val contactStatus: ContactStatus, val chatSettings: ChatSettings, val userPreferences: ChatPreferences, val mergedPreferences: ContactUserPreferences, @@ -809,8 +810,9 @@ data class Contact( override val id get() = "@$contactId" override val apiId get() = contactId override val ready get() = activeConn.connStatus == ConnStatus.Ready + val active get() = contactStatus == ContactStatus.Active override val sendMsgEnabled get() = - (ready && !(activeConn.connectionStats?.ratchetSyncSendProhibited ?: false)) + (ready && active && !(activeConn.connectionStats?.ratchetSyncSendProhibited ?: false)) || nextSendGrpInv val nextSendGrpInv get() = contactGroupMemberId != null && !contactGrpInvSent override val ntfsEnabled get() = chatSettings.enableNtfs @@ -859,6 +861,7 @@ data class Contact( profile = LocalProfile.sampleData, activeConn = Connection.sampleData, contactUsed = true, + contactStatus = ContactStatus.Active, chatSettings = ChatSettings(enableNtfs = true, sendRcpts = null, favorite = false), userPreferences = ChatPreferences.sampleData, mergedPreferences = ContactUserPreferences.sampleData, @@ -869,6 +872,12 @@ data class Contact( } } +@Serializable +enum class ContactStatus { + @SerialName("active") Active, + @SerialName("deleted") Deleted; +} + @Serializable class ContactRef( val contactId: Long, @@ -1471,6 +1480,7 @@ data class ChatItem ( is CIContent.RcvDecryptionError -> showNtfDir is CIContent.RcvGroupInvitation -> showNtfDir is CIContent.SndGroupInvitation -> showNtfDir + is CIContent.RcvDirectEventContent -> false is CIContent.RcvGroupEventContent -> when (content.rcvGroupEvent) { is RcvGroupEvent.MemberAdded -> false is RcvGroupEvent.MemberConnected -> false @@ -1854,6 +1864,7 @@ sealed class CIContent: ItemContent { @Serializable @SerialName("rcvDecryptionError") class RcvDecryptionError(val msgDecryptError: MsgDecryptError, val msgCount: UInt): CIContent() { override val msgContent: MsgContent? get() = null } @Serializable @SerialName("rcvGroupInvitation") class RcvGroupInvitation(val groupInvitation: CIGroupInvitation, val memberRole: GroupMemberRole): CIContent() { override val msgContent: MsgContent? get() = null } @Serializable @SerialName("sndGroupInvitation") class SndGroupInvitation(val groupInvitation: CIGroupInvitation, val memberRole: GroupMemberRole): CIContent() { override val msgContent: MsgContent? get() = null } + @Serializable @SerialName("rcvDirectEvent") class RcvDirectEventContent(val rcvDirectEvent: RcvDirectEvent): CIContent() { override val msgContent: MsgContent? get() = null } @Serializable @SerialName("rcvGroupEvent") class RcvGroupEventContent(val rcvGroupEvent: RcvGroupEvent): CIContent() { override val msgContent: MsgContent? get() = null } @Serializable @SerialName("sndGroupEvent") class SndGroupEventContent(val sndGroupEvent: SndGroupEvent): CIContent() { override val msgContent: MsgContent? get() = null } @Serializable @SerialName("rcvConnEvent") class RcvConnEventContent(val rcvConnEvent: RcvConnEvent): CIContent() { override val msgContent: MsgContent? get() = null } @@ -1881,6 +1892,7 @@ sealed class CIContent: ItemContent { is RcvDecryptionError -> msgDecryptError.text is RcvGroupInvitation -> groupInvitation.text is SndGroupInvitation -> groupInvitation.text + is RcvDirectEventContent -> rcvDirectEvent.text is RcvGroupEventContent -> rcvGroupEvent.text is SndGroupEventContent -> sndGroupEvent.text is RcvConnEventContent -> rcvConnEvent.text @@ -2487,6 +2499,15 @@ sealed class MsgErrorType() { } } +@Serializable +sealed class RcvDirectEvent() { + @Serializable @SerialName("contactDeleted") class ContactDeleted(): RcvDirectEvent() + + val text: String get() = when (this) { + is ContactDeleted -> generalGetString(MR.strings.rcv_direct_event_contact_deleted) + } +} + @Serializable sealed class RcvGroupEvent() { @Serializable @SerialName("memberAdded") class MemberAdded(val groupMemberId: Long, val profile: Profile): RcvGroupEvent() diff --git a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/model/SimpleXAPI.kt b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/model/SimpleXAPI.kt index 87cac179f..619788f6e 100644 --- a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/model/SimpleXAPI.kt +++ b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/model/SimpleXAPI.kt @@ -1366,6 +1366,11 @@ object ChatController { chatModel.removeChat(r.connection.id) } } + is CR.ContactDeletedByContact -> { + if (active(r.user) && r.contact.directOrUsed) { + chatModel.updateContact(r.contact) + } + } is CR.ContactConnected -> { if (active(r.user) && r.contact.directOrUsed) { chatModel.updateContact(r.contact) @@ -3304,6 +3309,7 @@ sealed class CR { @Serializable @SerialName("contactAlreadyExists") class ContactAlreadyExists(val user: UserRef, val contact: Contact): CR() @Serializable @SerialName("contactRequestAlreadyAccepted") class ContactRequestAlreadyAccepted(val user: UserRef, val contact: Contact): CR() @Serializable @SerialName("contactDeleted") class ContactDeleted(val user: UserRef, val contact: Contact): CR() + @Serializable @SerialName("contactDeletedByContact") class ContactDeletedByContact(val user: UserRef, val contact: Contact): CR() @Serializable @SerialName("chatCleared") class ChatCleared(val user: UserRef, val chatInfo: ChatInfo): CR() @Serializable @SerialName("userProfileNoChange") class UserProfileNoChange(val user: User): CR() @Serializable @SerialName("userProfileUpdated") class UserProfileUpdated(val user: User, val fromProfile: Profile, val toProfile: Profile, val updateSummary: UserProfileUpdateSummary): CR() @@ -3435,6 +3441,7 @@ sealed class CR { is ContactAlreadyExists -> "contactAlreadyExists" is ContactRequestAlreadyAccepted -> "contactRequestAlreadyAccepted" is ContactDeleted -> "contactDeleted" + is ContactDeletedByContact -> "contactDeletedByContact" is ChatCleared -> "chatCleared" is UserProfileNoChange -> "userProfileNoChange" is UserProfileUpdated -> "userProfileUpdated" @@ -3563,6 +3570,7 @@ sealed class CR { is ContactAlreadyExists -> withUser(user, json.encodeToString(contact)) is ContactRequestAlreadyAccepted -> withUser(user, json.encodeToString(contact)) is ContactDeleted -> withUser(user, json.encodeToString(contact)) + is ContactDeletedByContact -> withUser(user, json.encodeToString(contact)) is ChatCleared -> withUser(user, json.encodeToString(chatInfo)) is UserProfileNoChange -> withUser(user, noDetails()) is UserProfileUpdated -> withUser(user, json.encodeToString(toProfile)) @@ -3831,6 +3839,7 @@ sealed class ChatErrorType { is InvalidConnReq -> "invalidConnReq" is InvalidChatMessage -> "invalidChatMessage" is ContactNotReady -> "contactNotReady" + is ContactNotActive -> "contactNotActive" is ContactDisabled -> "contactDisabled" is ConnectionDisabled -> "connectionDisabled" is GroupUserRole -> "groupUserRole" @@ -3906,6 +3915,7 @@ sealed class ChatErrorType { @Serializable @SerialName("invalidConnReq") object InvalidConnReq: ChatErrorType() @Serializable @SerialName("invalidChatMessage") class InvalidChatMessage(val connection: Connection, val message: String): ChatErrorType() @Serializable @SerialName("contactNotReady") class ContactNotReady(val contact: Contact): ChatErrorType() + @Serializable @SerialName("contactNotActive") class ContactNotActive(val contact: Contact): ChatErrorType() @Serializable @SerialName("contactDisabled") class ContactDisabled(val contact: Contact): ChatErrorType() @Serializable @SerialName("connectionDisabled") class ConnectionDisabled(val connection: Connection): ChatErrorType() @Serializable @SerialName("groupUserRole") class GroupUserRole(val groupInfo: GroupInfo, val requiredRole: GroupMemberRole): ChatErrorType() diff --git a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chat/ChatInfoView.kt b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chat/ChatInfoView.kt index 5fcb90c1c..1ba742b03 100644 --- a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chat/ChatInfoView.kt +++ b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chat/ChatInfoView.kt @@ -15,7 +15,6 @@ import androidx.compose.foundation.layout.* import androidx.compose.foundation.text.* import androidx.compose.material.* import androidx.compose.runtime.* -import androidx.compose.runtime.saveable.rememberSaveable import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier import androidx.compose.ui.graphics.Color @@ -291,7 +290,7 @@ fun ChatInfoLayout( SectionDividerSpaced() } - if (contact.ready) { + if (contact.ready && contact.active) { SectionView { if (connectionCode != null) { VerifyCodeButton(contact.verified, verifyClicked) @@ -318,7 +317,7 @@ fun ChatInfoLayout( SectionDividerSpaced() } - if (contact.ready) { + if (contact.ready && contact.active) { SectionView(title = stringResource(MR.strings.conn_stats_section_title_servers)) { SectionItemView({ AlertManager.shared.showAlertMsg( diff --git a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chat/ChatView.kt b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chat/ChatView.kt index 596a7426a..b22b2f91c 100644 --- a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chat/ChatView.kt +++ b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chat/ChatView.kt @@ -118,7 +118,12 @@ fun ChatView(chatId: String, chatModel: ChatModel, onComposed: suspend (chatId: Modifier.fillMaxWidth(), horizontalAlignment = Alignment.CenterHorizontally ) { - if (chat.chatInfo is ChatInfo.Direct && !chat.chatInfo.contact.ready && !chat.chatInfo.contact.nextSendGrpInv) { + if ( + chat.chatInfo is ChatInfo.Direct + && !chat.chatInfo.contact.ready + && chat.chatInfo.contact.active + && !chat.chatInfo.contact.nextSendGrpInv + ) { Text( generalGetString(MR.strings.contact_connection_pending), Modifier.padding(top = 4.dp), @@ -550,15 +555,15 @@ fun ChatInfoToolbar( showMenu.value = false startCall(CallMediaType.Audio) }, - enabled = chat.chatInfo.contact.ready) { + enabled = chat.chatInfo.contact.ready && chat.chatInfo.contact.active) { Icon( painterResource(MR.images.ic_call_500), stringResource(MR.strings.icon_descr_more_button), - tint = if (chat.chatInfo.contact.ready) MaterialTheme.colors.primary else MaterialTheme.colors.secondary + tint = if (chat.chatInfo.contact.ready && chat.chatInfo.contact.active) MaterialTheme.colors.primary else MaterialTheme.colors.secondary ) } } - if (chat.chatInfo.contact.ready) { + if (chat.chatInfo.contact.ready && chat.chatInfo.contact.active) { menuItems.add { ItemAction(stringResource(MR.strings.icon_descr_video_call).capitalize(Locale.current), painterResource(MR.images.ic_videocam), onClick = { showMenu.value = false @@ -576,7 +581,7 @@ fun ChatInfoToolbar( } } } - if ((chat.chatInfo is ChatInfo.Direct && chat.chatInfo.contact.ready) || chat.chatInfo is ChatInfo.Group) { + if ((chat.chatInfo is ChatInfo.Direct && chat.chatInfo.contact.ready && chat.chatInfo.contact.active) || chat.chatInfo is ChatInfo.Group) { val ntfsEnabled = remember { mutableStateOf(chat.chatInfo.ntfsEnabled) } menuItems.add { ItemAction( diff --git a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chat/item/ChatItemView.kt b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chat/item/ChatItemView.kt index e1d45ffb3..b5236e249 100644 --- a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chat/item/ChatItemView.kt +++ b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chat/item/ChatItemView.kt @@ -352,6 +352,7 @@ fun ChatItemView( is CIContent.RcvDecryptionError -> CIRcvDecryptionError(c.msgDecryptError, c.msgCount, cInfo, cItem, updateContactStats = updateContactStats, updateMemberStats = updateMemberStats, syncContactConnection = syncContactConnection, syncMemberConnection = syncMemberConnection, findModelChat = findModelChat, findModelMember = findModelMember) is CIContent.RcvGroupInvitation -> CIGroupInvitationView(cItem, c.groupInvitation, c.memberRole, joinGroup = joinGroup, chatIncognito = cInfo.incognito) is CIContent.SndGroupInvitation -> CIGroupInvitationView(cItem, c.groupInvitation, c.memberRole, joinGroup = joinGroup, chatIncognito = cInfo.incognito) + is CIContent.RcvDirectEventContent -> EventItemView() is CIContent.RcvGroupEventContent -> when (c.rcvGroupEvent) { is RcvGroupEvent.MemberConnected -> CIEventView(membersConnectedItemText()) is RcvGroupEvent.MemberCreatedContact -> CIMemberCreatedContactView(cItem, openDirectChat) diff --git a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chatlist/ChatPreviewView.kt b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chatlist/ChatPreviewView.kt index 780e3515d..a5775d369 100644 --- a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chatlist/ChatPreviewView.kt +++ b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/chatlist/ChatPreviewView.kt @@ -42,7 +42,7 @@ fun ChatPreviewView( val cInfo = chat.chatInfo @Composable - fun groupInactiveIcon() { + fun inactiveIcon() { Icon( painterResource(MR.images.ic_cancel_filled), stringResource(MR.strings.icon_descr_group_inactive), @@ -53,13 +53,19 @@ fun ChatPreviewView( @Composable fun chatPreviewImageOverlayIcon() { - if (cInfo is ChatInfo.Group) { - when (cInfo.groupInfo.membership.memberStatus) { - GroupMemberStatus.MemLeft -> groupInactiveIcon() - GroupMemberStatus.MemRemoved -> groupInactiveIcon() - GroupMemberStatus.MemGroupDeleted -> groupInactiveIcon() - else -> {} + when (cInfo) { + is ChatInfo.Direct -> + if (!cInfo.contact.active) { + inactiveIcon() + } + is ChatInfo.Group -> + when (cInfo.groupInfo.membership.memberStatus) { + GroupMemberStatus.MemLeft -> inactiveIcon() + GroupMemberStatus.MemRemoved -> inactiveIcon() + GroupMemberStatus.MemGroupDeleted -> inactiveIcon() + else -> {} } + else -> {} } } @@ -125,7 +131,7 @@ fun ChatPreviewView( if (cInfo.contact.verified) { VerifiedIcon() } - chatPreviewTitleText(if (cInfo.ready) Color.Unspecified else MaterialTheme.colors.secondary) + chatPreviewTitleText() } is ChatInfo.Group -> when (cInfo.groupInfo.membership.memberStatus) { @@ -174,7 +180,7 @@ fun ChatPreviewView( is ChatInfo.Direct -> if (cInfo.contact.nextSendGrpInv) { Text(stringResource(MR.strings.member_contact_send_direct_message), color = MaterialTheme.colors.secondary) - } else if (!cInfo.ready) { + } else if (!cInfo.ready && cInfo.contact.active) { Text(stringResource(MR.strings.contact_connection_pending), color = MaterialTheme.colors.secondary) } is ChatInfo.Group -> @@ -191,28 +197,32 @@ fun ChatPreviewView( @Composable fun chatStatusImage() { if (cInfo is ChatInfo.Direct) { - val descr = contactNetworkStatus?.statusString - when (contactNetworkStatus) { - is NetworkStatus.Connected -> - IncognitoIcon(chat.chatInfo.incognito) + if (cInfo.contact.active) { + val descr = contactNetworkStatus?.statusString + when (contactNetworkStatus) { + is NetworkStatus.Connected -> + IncognitoIcon(chat.chatInfo.incognito) - is NetworkStatus.Error -> - Icon( - painterResource(MR.images.ic_error), - contentDescription = descr, - tint = MaterialTheme.colors.secondary, - modifier = Modifier - .size(19.dp) - ) + is NetworkStatus.Error -> + Icon( + painterResource(MR.images.ic_error), + contentDescription = descr, + tint = MaterialTheme.colors.secondary, + modifier = Modifier + .size(19.dp) + ) - else -> - CircularProgressIndicator( - Modifier - .padding(horizontal = 2.dp) - .size(15.dp), - color = MaterialTheme.colors.secondary, - strokeWidth = 1.5.dp - ) + else -> + CircularProgressIndicator( + Modifier + .padding(horizontal = 2.dp) + .size(15.dp), + color = MaterialTheme.colors.secondary, + strokeWidth = 1.5.dp + ) + } + } else { + IncognitoIcon(chat.chatInfo.incognito) } } else { IncognitoIcon(chat.chatInfo.incognito) diff --git a/apps/multiplatform/common/src/commonMain/resources/MR/base/strings.xml b/apps/multiplatform/common/src/commonMain/resources/MR/base/strings.xml index ab0d943f3..26e9948d6 100644 --- a/apps/multiplatform/common/src/commonMain/resources/MR/base/strings.xml +++ b/apps/multiplatform/common/src/commonMain/resources/MR/base/strings.xml @@ -1105,6 +1105,9 @@ You rejected group invitation Group invitation expired + + deleted contact + invited %1$s connected From c1854b7d507708942397869d09b8828096e9d0d4 Mon Sep 17 00:00:00 2001 From: Stanislav Dmitrenko <7953703+avently@users.noreply.github.com> Date: Thu, 28 Sep 2023 18:39:43 +0800 Subject: [PATCH 5/8] desktop: fix script for building the lib (#3141) --- scripts/desktop/build-lib-mac.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/desktop/build-lib-mac.sh b/scripts/desktop/build-lib-mac.sh index 8b0386473..303e33154 100755 --- a/scripts/desktop/build-lib-mac.sh +++ b/scripts/desktop/build-lib-mac.sh @@ -117,7 +117,7 @@ for lib in $(find . -type f -name "*.$LIB_EXT"); do done done -LOCAL_DIRS=`for lib in $(find . -type f -name "*.$LIB_EXT"); do otool -l $lib | grep -E "/Users|/opt/|/usr/local" && echo $lib; done` +LOCAL_DIRS=`for lib in $(find . -type f -name "*.$LIB_EXT"); do otool -l $lib | grep -E "/Users|/opt/|/usr/local" && echo $lib || true; done` if [ -n "$LOCAL_DIRS" ]; then echo These libs still point to local directories: echo $LOCAL_DIRS From bc7baf560be0f6ec88bcba5b55f1b6dd5fa8a513 Mon Sep 17 00:00:00 2001 From: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com> Date: Fri, 29 Sep 2023 11:24:16 +0400 Subject: [PATCH 6/8] core: filter out connections of deleted contacts and group members on subscribe (#3144) --- src/Simplex/Chat.hs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/Simplex/Chat.hs b/src/Simplex/Chat.hs index 822d6cd4b..896edd26b 100644 --- a/src/Simplex/Chat.hs +++ b/src/Simplex/Chat.hs @@ -2570,7 +2570,7 @@ subscribeUserConnections onlyNeeded agentBatchSubscribe user@User {userId} = do getContactConns :: m ([ConnId], Map ConnId Contact) getContactConns = do cts <- withStore_ ("subscribeUserConnections " <> show userId <> ", getUserContacts") getUserContacts - let connIds = map contactConnId cts + let connIds = map contactConnId (filter contactActive cts) pure (connIds, M.fromList $ zip connIds cts) getUserContactLinkConns :: m ([ConnId], Map ConnId UserContact) getUserContactLinkConns = do @@ -2580,7 +2580,7 @@ subscribeUserConnections onlyNeeded agentBatchSubscribe user@User {userId} = do getGroupMemberConns :: m ([Group], [ConnId], Map ConnId GroupMember) getGroupMemberConns = do gs <- withStore_ ("subscribeUserConnections " <> show userId <> ", getUserGroups") getUserGroups - let mPairs = concatMap (\(Group _ ms) -> mapMaybe (\m -> (,m) <$> memberConnId m) ms) gs + let mPairs = concatMap (\(Group _ ms) -> mapMaybe (\m -> (,m) <$> memberConnId m) (filter (not . memberRemoved) ms)) gs pure (gs, map fst mPairs, M.fromList mPairs) getSndFileTransferConns :: m ([ConnId], Map ConnId SndFileTransfer) getSndFileTransferConns = do From 1d34500fba510a01be73dad81d6fb9f7447e7a41 Mon Sep 17 00:00:00 2001 From: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com> Date: Fri, 29 Sep 2023 11:14:10 +0100 Subject: [PATCH 7/8] core: revert stop/close changes made for Windows (#3145) * Revert "core: return error response when wrong passphrase is passed to start" This reverts commit ea319313f10c41127e604cee2391ce759aaab8b3. * Revert "core: support closing/re-opening store on chat stop/start (#3132)" This reverts commit 3c7fc6b0ee1949dbe731bc11ad4e4809474ae7fd. --- .../chat/simplex/common/model/SimpleXAPI.kt | 25 ++++------- .../common/views/database/DatabaseView.kt | 2 +- cabal.project | 2 +- scripts/nix/sha256map.nix | 2 +- src/Simplex/Chat.hs | 43 ++++++------------- src/Simplex/Chat/Archive.hs | 28 ++++++------ src/Simplex/Chat/Controller.hs | 12 +----- src/Simplex/Chat/View.hs | 2 +- stack.yaml | 2 +- tests/ChatClient.hs | 2 +- tests/ChatTests/Direct.hs | 9 +--- 11 files changed, 42 insertions(+), 87 deletions(-) diff --git a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/model/SimpleXAPI.kt b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/model/SimpleXAPI.kt index 619788f6e..4bb06afd8 100644 --- a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/model/SimpleXAPI.kt +++ b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/model/SimpleXAPI.kt @@ -516,8 +516,8 @@ object ChatController { throw Exception("failed to delete the user ${r.responseType} ${r.details}") } - suspend fun apiStartChat(openDBWithKey: String? = null): Boolean { - val r = sendCmd(CC.StartChat(ChatCtrlCfg(subConns = true, enableExpireCIs = true, startXFTPWorkers = true, openDBWithKey = openDBWithKey))) + suspend fun apiStartChat(): Boolean { + val r = sendCmd(CC.StartChat(expire = true)) when (r) { is CR.ChatStarted -> return true is CR.ChatRunning -> return false @@ -525,8 +525,8 @@ object ChatController { } } - suspend fun apiStopChat(closeStore: Boolean): Boolean { - val r = sendCmd(CC.ApiStopChat(closeStore)) + suspend fun apiStopChat(): Boolean { + val r = sendCmd(CC.ApiStopChat()) when (r) { is CR.ChatStopped -> return true else -> throw Error("failed stopping chat: ${r.responseType} ${r.details}") @@ -1834,8 +1834,8 @@ sealed class CC { class ApiMuteUser(val userId: Long): CC() class ApiUnmuteUser(val userId: Long): CC() class ApiDeleteUser(val userId: Long, val delSMPQueues: Boolean, val viewPwd: String?): CC() - class StartChat(val cfg: ChatCtrlCfg): CC() - class ApiStopChat(val closeStore: Boolean): CC() + class StartChat(val expire: Boolean): CC() + class ApiStopChat: CC() class SetTempFolder(val tempFolder: String): CC() class SetFilesFolder(val filesFolder: String): CC() class ApiSetXFTPConfig(val config: XFTPFileConfig?): CC() @@ -1938,9 +1938,8 @@ sealed class CC { is ApiMuteUser -> "/_mute user $userId" is ApiUnmuteUser -> "/_unmute user $userId" is ApiDeleteUser -> "/_delete user $userId del_smp=${onOff(delSMPQueues)}${maybePwd(viewPwd)}" -// is StartChat -> "/_start ${json.encodeToString(cfg)}" // this can be used with the new core - is StartChat -> "/_start subscribe=on expire=${onOff(cfg.enableExpireCIs)} xftp=on" - is ApiStopChat -> if (closeStore) "/_stop close" else "/_stop" + is StartChat -> "/_start subscribe=on expire=${onOff(expire)} xftp=on" + is ApiStopChat -> "/_stop" is SetTempFolder -> "/_temp_folder $tempFolder" is SetFilesFolder -> "/_files_folder $filesFolder" is ApiSetXFTPConfig -> if (config != null) "/_xftp on ${json.encodeToString(config)}" else "/_xftp off" @@ -2157,14 +2156,6 @@ sealed class CC { } } -@Serializable -data class ChatCtrlCfg ( - val subConns: Boolean, - val enableExpireCIs: Boolean, - val startXFTPWorkers: Boolean, - val openDBWithKey: String? -) - @Serializable data class NewUser( val profile: Profile?, diff --git a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/database/DatabaseView.kt b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/database/DatabaseView.kt index 3eb2e7d73..fa0f8f54d 100644 --- a/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/database/DatabaseView.kt +++ b/apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/views/database/DatabaseView.kt @@ -419,7 +419,7 @@ private fun stopChat(m: ChatModel) { } suspend fun stopChatAsync(m: ChatModel) { - m.controller.apiStopChat(false) + m.controller.apiStopChat() m.chatRunning.value = false } diff --git a/cabal.project b/cabal.project index b9753c9c0..b4024f088 100644 --- a/cabal.project +++ b/cabal.project @@ -9,7 +9,7 @@ constraints: zip +disable-bzip2 +disable-zstd source-repository-package type: git location: https://github.com/simplex-chat/simplexmq.git - tag: fda1284ae4b7c33cae2eb8ed0376a511aecc1d51 + tag: 8d47f690838371bc848e4b31a4b09ef6bf67ccc5 source-repository-package type: git diff --git a/scripts/nix/sha256map.nix b/scripts/nix/sha256map.nix index faa2401b1..26f4ea112 100644 --- a/scripts/nix/sha256map.nix +++ b/scripts/nix/sha256map.nix @@ -1,5 +1,5 @@ { - "https://github.com/simplex-chat/simplexmq.git"."fda1284ae4b7c33cae2eb8ed0376a511aecc1d51" = "1gq7scv9z8x3xhzl914xr46na0kkrqd1i743xbw69lyx33kj9xb5"; + "https://github.com/simplex-chat/simplexmq.git"."8d47f690838371bc848e4b31a4b09ef6bf67ccc5" = "1pwasv22ii3wy4xchaknlwczmy5ws7adx7gg2g58lxzrgdjm3650"; "https://github.com/simplex-chat/hs-socks.git"."a30cc7a79a08d8108316094f8f2f82a0c5e1ac51" = "0yasvnr7g91k76mjkamvzab2kvlb1g5pspjyjn2fr6v83swjhj38"; "https://github.com/kazu-yamamoto/http2.git"."b5a1b7200cf5bc7044af34ba325284271f6dff25" = "0dqb50j57an64nf4qcf5vcz4xkd1vzvghvf8bk529c1k30r9nfzb"; "https://github.com/simplex-chat/direct-sqlcipher.git"."f814ee68b16a9447fbb467ccc8f29bdd3546bfd9" = "0kiwhvml42g9anw4d2v0zd1fpc790pj9syg5x3ik4l97fnkbbwpp"; diff --git a/src/Simplex/Chat.hs b/src/Simplex/Chat.hs index 896edd26b..10f925471 100644 --- a/src/Simplex/Chat.hs +++ b/src/Simplex/Chat.hs @@ -34,7 +34,6 @@ import qualified Data.ByteString.Base64 as B64 import Data.ByteString.Char8 (ByteString) import qualified Data.ByteString.Char8 as B import Data.Char (isSpace, toLower) -import Data.Composition ((.:)) import Data.Constraint (Dict (..)) import Data.Either (fromRight, rights) import Data.Fixed (div') @@ -84,7 +83,7 @@ import Simplex.Messaging.Agent.Client (AgentStatsKey (..), SubInfo (..), agentCl import Simplex.Messaging.Agent.Env.SQLite (AgentConfig (..), InitialAgentServers (..), createAgentStore, defaultAgentConfig) import Simplex.Messaging.Agent.Lock import Simplex.Messaging.Agent.Protocol -import Simplex.Messaging.Agent.Store.SQLite (MigrationConfirmation (..), MigrationError, SQLiteStore (dbNew), execSQL, upMigration, withConnection, closeSQLiteStore, openSQLiteStore) +import Simplex.Messaging.Agent.Store.SQLite (MigrationConfirmation (..), MigrationError, SQLiteStore (dbNew), execSQL, upMigration, withConnection) import Simplex.Messaging.Agent.Store.SQLite.DB (SlowQueryStats (..)) import qualified Simplex.Messaging.Agent.Store.SQLite.DB as DB import qualified Simplex.Messaging.Agent.Store.SQLite.Migrations as Migrations @@ -218,8 +217,8 @@ newChatController ChatDatabase {chatStore, agentStore} user cfg@ChatConfig {agen where configServers :: DefaultAgentServers configServers = - let smp' = fromMaybe defaultServers.smp (nonEmpty smpServers) - xftp' = fromMaybe defaultServers.xftp (nonEmpty xftpServers) + let smp' = fromMaybe (defaultServers.smp) (nonEmpty smpServers) + xftp' = fromMaybe (defaultServers.xftp) (nonEmpty xftpServers) in defaultServers {smp = smp', xftp = xftp', netCfg = networkConfig} agentServers :: ChatConfig -> IO InitialAgentServers agentServers config@ChatConfig {defaultServers = defServers@DefaultAgentServers {ntf, netCfg}} = do @@ -252,7 +251,7 @@ cfgServers p s = case p of startChatController :: forall m. ChatMonad' m => Bool -> Bool -> Bool -> m (Async ()) startChatController subConns enableExpireCIs startXFTPWorkers = do - resumeAgentClient =<< asks smpAgent + asks smpAgent >>= resumeAgentClient unless subConns $ chatWriteVar subscriptionMode SMOnlyCreate users <- fromRight [] <$> runExceptT (withStoreCtx' (Just "startChatController, getUsers") getUsers) @@ -324,8 +323,8 @@ restoreCalls = do calls <- asks currentCalls atomically $ writeTVar calls callsMap -stopChatController :: forall m. MonadUnliftIO m => ChatController -> Bool -> m () -stopChatController ChatController {chatStore, smpAgent, agentAsync = s, sndFiles, rcvFiles, expireCIFlags} closeStore = do +stopChatController :: forall m. MonadUnliftIO m => ChatController -> m () +stopChatController ChatController {smpAgent, agentAsync = s, sndFiles, rcvFiles, expireCIFlags} = do disconnectAgentClient smpAgent readTVarIO s >>= mapM_ (\(a1, a2) -> uninterruptibleCancel a1 >> mapM_ uninterruptibleCancel a2) closeFiles sndFiles @@ -334,9 +333,6 @@ stopChatController ChatController {chatStore, smpAgent, agentAsync = s, sndFiles keys <- M.keys <$> readTVar expireCIFlags forM_ keys $ \k -> TM.insert k False expireCIFlags writeTVar s Nothing - when closeStore $ liftIO $ do - closeSQLiteStore chatStore - closeSQLiteStore $ agentClientStore smpAgent where closeFiles :: TVar (Map Int64 Handle) -> m () closeFiles files = do @@ -466,19 +462,12 @@ processChatCommand = \case checkDeleteChatUser user' withChatLock "deleteUser" . procCmd $ deleteChatUser user' delSMPQueues DeleteUser uName delSMPQueues viewPwd_ -> withUserName uName $ \userId -> APIDeleteUser userId delSMPQueues viewPwd_ - APIStartChat ChatCtrlCfg {subConns, enableExpireCIs, startXFTPWorkers, openDBWithKey} -> withUser' $ \_ -> + StartChat subConns enableExpireCIs startXFTPWorkers -> withUser' $ \_ -> asks agentAsync >>= readTVarIO >>= \case Just _ -> pure CRChatRunning - _ -> checkStoreNotChanged $ do - forM_ openDBWithKey $ \(DBEncryptionKey dbKey) -> do - ChatController {chatStore, smpAgent} <- ask - open chatStore dbKey - open (agentClientStore smpAgent) dbKey - startChatController subConns enableExpireCIs startXFTPWorkers $> CRChatStarted - where - open = handleDBError DBErrorOpen .: openSQLiteStore - APIStopChat closeStore -> do - ask >>= (`stopChatController` closeStore) + _ -> checkStoreNotChanged $ startChatController subConns enableExpireCIs startXFTPWorkers $> CRChatStarted + APIStopChat -> do + ask >>= stopChatController pure CRChatStopped APIActivateChat -> withUser $ \_ -> do restoreCalls @@ -5411,9 +5400,9 @@ chatCommandP = "/_delete user " *> (APIDeleteUser <$> A.decimal <* " del_smp=" <*> onOffP <*> optional (A.space *> jsonP)), "/delete user " *> (DeleteUser <$> displayName <*> pure True <*> optional (A.space *> pwdP)), ("/user" <|> "/u") $> ShowActiveUser, - "/_start" *> (APIStartChat <$> ((A.space *> jsonP) <|> chatCtrlCfgP)), - "/_stop close" $> APIStopChat {closeStore = True}, - "/_stop" $> APIStopChat False, + "/_start subscribe=" *> (StartChat <$> onOffP <* " expire=" <*> onOffP <* " xftp=" <*> onOffP), + "/_start" $> StartChat True True True, + "/_stop" $> APIStopChat, "/_app activate" $> APIActivateChat, "/_app suspend " *> (APISuspendChat <$> A.decimal), "/_resubscribe all" $> ResubscribeAllConnections, @@ -5641,12 +5630,6 @@ chatCommandP = ] where choice = A.choice . map (\p -> p <* A.takeWhile (== ' ') <* A.endOfInput) - chatCtrlCfgP = do - subConns <- (" subscribe=" *> onOffP) <|> pure True - enableExpireCIs <- (" expire=" *> onOffP) <|> pure True - startXFTPWorkers <- (" xftp=" *> onOffP) <|> pure True - openDBWithKey <- optional $ " key=" *> dbKeyP - pure ChatCtrlCfg {subConns, enableExpireCIs, startXFTPWorkers, openDBWithKey} incognitoP = (A.space *> ("incognito" <|> "i")) $> True <|> pure False incognitoOnOffP = (A.space *> "incognito=" *> onOffP) <|> pure False imagePrefix = (<>) <$> "data:" <*> ("image/png;base64," <|> "image/jpg;base64,") diff --git a/src/Simplex/Chat/Archive.hs b/src/Simplex/Chat/Archive.hs index 8d1b99328..f8fa0d152 100644 --- a/src/Simplex/Chat/Archive.hs +++ b/src/Simplex/Chat/Archive.hs @@ -9,7 +9,6 @@ module Simplex.Chat.Archive importArchive, deleteStorage, sqlCipherExport, - handleDBError, ) where @@ -125,7 +124,7 @@ sqlCipherExport DBEncryptionConfig {currentKey = DBEncryptionKey key, newKey = D checkFile `with` fs backup `with` fs (export chatDb chatEncrypted >> export agentDb agentEncrypted) - `catchChatError` \e -> tryChatError (restore `with` fs) >> throwError e + `catchChatError` \e -> (restore `with` fs) >> throwError e where action `with` StorageFiles {chatDb, agentDb} = action chatDb >> action agentDb backup f = copyFile f (f <> ".bak") @@ -140,7 +139,17 @@ sqlCipherExport DBEncryptionConfig {currentKey = DBEncryptionKey key, newKey = D withDB (`SQL.exec` testSQL) DBErrorOpen atomically $ writeTVar dbEnc $ not (null key') where - withDB a err = handleDBError err $ bracket (SQL.open $ T.pack f) SQL.close a + withDB a err = + liftIO (bracket (SQL.open $ T.pack f) SQL.close a $> Nothing) + `catch` checkSQLError + `catch` (\(e :: SomeException) -> sqliteError' e) + >>= mapM_ (throwDBError . err) + where + checkSQLError e = case SQL.sqlError e of + SQL.ErrorNotADatabase -> pure $ Just SQLiteErrorNotADatabase + _ -> sqliteError' e + sqliteError' :: Show e => e -> m (Maybe SQLiteError) + sqliteError' = pure . Just . SQLiteError . show exportSQL = T.unlines $ keySQL key @@ -157,16 +166,3 @@ sqlCipherExport DBEncryptionConfig {currentKey = DBEncryptionKey key, newKey = D "SELECT count(*) FROM sqlite_master;" ] keySQL k = ["PRAGMA key = " <> sqlString k <> ";" | not (null k)] - -handleDBError :: forall m. ChatMonad m => (SQLiteError -> DatabaseError) -> IO () -> m () -handleDBError err a = - (liftIO a $> Nothing) - `catch` checkSQLError - `catch` (\(e :: SomeException) -> sqliteError' e) - >>= mapM_ (throwDBError . err) - where - checkSQLError e = case SQL.sqlError e of - SQL.ErrorNotADatabase -> pure $ Just SQLiteErrorNotADatabase - _ -> sqliteError' e - sqliteError' :: Show e => e -> m (Maybe SQLiteError) - sqliteError' = pure . Just . SQLiteError . show diff --git a/src/Simplex/Chat/Controller.hs b/src/Simplex/Chat/Controller.hs index 3f3dae94f..122a4be3f 100644 --- a/src/Simplex/Chat/Controller.hs +++ b/src/Simplex/Chat/Controller.hs @@ -221,8 +221,8 @@ data ChatCommand | UnmuteUser | APIDeleteUser UserId Bool (Maybe UserPwd) | DeleteUser UserName Bool (Maybe UserPwd) - | APIStartChat ChatCtrlCfg - | APIStopChat {closeStore :: Bool} + | StartChat {subscribeConnections :: Bool, enableExpireChatItems :: Bool, startXFTPWorkers :: Bool} + | APIStopChat | APIActivateChat | APISuspendChat {suspendTimeout :: Int} | ResubscribeAllConnections @@ -621,14 +621,6 @@ instance ToJSON ChatResponse where toJSON = J.genericToJSON . sumTypeJSON $ dropPrefix "CR" toEncoding = J.genericToEncoding . sumTypeJSON $ dropPrefix "CR" -data ChatCtrlCfg = ChatCtrlCfg - { subConns :: Bool, - enableExpireCIs :: Bool, - startXFTPWorkers :: Bool, - openDBWithKey :: Maybe DBEncryptionKey - } - deriving (Show, Generic, FromJSON) - newtype UserPwd = UserPwd {unUserPwd :: Text} deriving (Eq, Show) diff --git a/src/Simplex/Chat/View.hs b/src/Simplex/Chat/View.hs index 19d729bf7..01bdfba95 100644 --- a/src/Simplex/Chat/View.hs +++ b/src/Simplex/Chat/View.hs @@ -1648,7 +1648,7 @@ viewChatError logLevel = \case DBErrorEncrypted -> ["error: chat database is already encrypted"] DBErrorPlaintext -> ["error: chat database is not encrypted"] DBErrorExport e -> ["error encrypting database: " <> sqliteError' e] - DBErrorOpen e -> ["error opening database: " <> sqliteError' e] + DBErrorOpen e -> ["error opening database after encryption: " <> sqliteError' e] e -> ["chat database error: " <> sShow e] ChatErrorAgent err entity_ -> case err of CMD PROHIBITED -> [withConnEntity <> "error: command is prohibited"] diff --git a/stack.yaml b/stack.yaml index a466178ce..0840970e4 100644 --- a/stack.yaml +++ b/stack.yaml @@ -49,7 +49,7 @@ extra-deps: # - simplexmq-1.0.0@sha256:34b2004728ae396e3ae449cd090ba7410781e2b3cefc59259915f4ca5daa9ea8,8561 # - ../simplexmq - github: simplex-chat/simplexmq - commit: fda1284ae4b7c33cae2eb8ed0376a511aecc1d51 + commit: 8d47f690838371bc848e4b31a4b09ef6bf67ccc5 - github: kazu-yamamoto/http2 commit: b5a1b7200cf5bc7044af34ba325284271f6dff25 # - ../direct-sqlcipher diff --git a/tests/ChatClient.hs b/tests/ChatClient.hs index d947fb63b..7da526325 100644 --- a/tests/ChatClient.hs +++ b/tests/ChatClient.hs @@ -171,7 +171,7 @@ startTestChat_ db cfg opts user = do stopTestChat :: TestCC -> IO () stopTestChat TestCC {chatController = cc, chatAsync, termAsync} = do - stopChatController cc False + stopChatController cc uninterruptibleCancel termAsync uninterruptibleCancel chatAsync threadDelay 200000 diff --git a/tests/ChatTests/Direct.hs b/tests/ChatTests/Direct.hs index 445a5ab99..5c4d96cc9 100644 --- a/tests/ChatTests/Direct.hs +++ b/tests/ChatTests/Direct.hs @@ -992,17 +992,10 @@ testDatabaseEncryption tmp = do alice ##> "/_start" alice <## "chat started" testChatWorking alice bob - alice ##> "/_stop close" + alice ##> "/_stop" alice <## "chat stopped" alice ##> "/db key wrongkey nextkey" alice <## "error encrypting database: wrong passphrase or invalid database file" - alice ##> "/_start key=wrongkey" - alice <## "error opening database: wrong passphrase or invalid database file" - alice ##> "/_start key=mykey" - alice <## "chat started" - testChatWorking alice bob - alice ##> "/_stop close" - alice <## "chat stopped" alice ##> "/db key mykey nextkey" alice <## "ok" alice ##> "/_db encryption {\"currentKey\":\"nextkey\",\"newKey\":\"anotherkey\"}" From 70a65e8969a67905a36116b04806aa7bd7d0c800 Mon Sep 17 00:00:00 2001 From: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com> Date: Fri, 29 Sep 2023 13:09:48 +0100 Subject: [PATCH 8/8] core: close stores before import/delete/encryption operations to make compatible with windows, make encryption more resilient (#3146) * core: close stores before import/delete/encryption operations to make compatible with windows, make encryption more resilient * remove file names * do not remove files if already removed --- cabal.project | 2 +- scripts/nix/sha256map.nix | 2 +- src/Simplex/Chat/Archive.hs | 86 +++++++++++++++++++++---------------- stack.yaml | 2 +- 4 files changed, 52 insertions(+), 40 deletions(-) diff --git a/cabal.project b/cabal.project index b4024f088..af664652d 100644 --- a/cabal.project +++ b/cabal.project @@ -9,7 +9,7 @@ constraints: zip +disable-bzip2 +disable-zstd source-repository-package type: git location: https://github.com/simplex-chat/simplexmq.git - tag: 8d47f690838371bc848e4b31a4b09ef6bf67ccc5 + tag: ec1b72cb8013a65a5d9783104a47ae44f5730089 source-repository-package type: git diff --git a/scripts/nix/sha256map.nix b/scripts/nix/sha256map.nix index 26f4ea112..b6ca36e31 100644 --- a/scripts/nix/sha256map.nix +++ b/scripts/nix/sha256map.nix @@ -1,5 +1,5 @@ { - "https://github.com/simplex-chat/simplexmq.git"."8d47f690838371bc848e4b31a4b09ef6bf67ccc5" = "1pwasv22ii3wy4xchaknlwczmy5ws7adx7gg2g58lxzrgdjm3650"; + "https://github.com/simplex-chat/simplexmq.git"."ec1b72cb8013a65a5d9783104a47ae44f5730089" = "1lz5rvgxp242zg95r9zd9j50y45314cf8nfpjg1qsa55nrk2w19b"; "https://github.com/simplex-chat/hs-socks.git"."a30cc7a79a08d8108316094f8f2f82a0c5e1ac51" = "0yasvnr7g91k76mjkamvzab2kvlb1g5pspjyjn2fr6v83swjhj38"; "https://github.com/kazu-yamamoto/http2.git"."b5a1b7200cf5bc7044af34ba325284271f6dff25" = "0dqb50j57an64nf4qcf5vcz4xkd1vzvghvf8bk529c1k30r9nfzb"; "https://github.com/simplex-chat/direct-sqlcipher.git"."f814ee68b16a9447fbb467ccc8f29bdd3546bfd9" = "0kiwhvml42g9anw4d2v0zd1fpc790pj9syg5x3ik4l97fnkbbwpp"; diff --git a/src/Simplex/Chat/Archive.hs b/src/Simplex/Chat/Archive.hs index f8fa0d152..e0de971bd 100644 --- a/src/Simplex/Chat/Archive.hs +++ b/src/Simplex/Chat/Archive.hs @@ -21,7 +21,7 @@ import qualified Data.Text as T import qualified Database.SQLite3 as SQL import Simplex.Chat.Controller import Simplex.Messaging.Agent.Client (agentClientStore) -import Simplex.Messaging.Agent.Store.SQLite (SQLiteStore (..), sqlString) +import Simplex.Messaging.Agent.Store.SQLite (SQLiteStore (..), sqlString, closeSQLiteStore) import Simplex.Messaging.Util import System.FilePath import UnliftIO.Directory @@ -42,9 +42,9 @@ archiveFilesFolder = "simplex_v1_files" exportArchive :: ChatMonad m => ArchiveConfig -> m () exportArchive cfg@ArchiveConfig {archivePath, disableCompression} = withTempDir cfg "simplex-chat." $ \dir -> do - StorageFiles {chatDb, agentDb, filesPath} <- storageFiles - copyFile chatDb $ dir archiveChatDbFile - copyFile agentDb $ dir archiveAgentDbFile + StorageFiles {chatStore, agentStore, filesPath} <- storageFiles + copyFile (dbFilePath chatStore) $ dir archiveChatDbFile + copyFile (dbFilePath agentStore) $ dir archiveAgentDbFile forM_ filesPath $ \fp -> copyDirectoryFiles fp $ dir archiveFilesFolder let method = if disableCompression == Just True then Z.Store else Z.Deflate @@ -54,11 +54,11 @@ importArchive :: ChatMonad m => ArchiveConfig -> m [ArchiveError] importArchive cfg@ArchiveConfig {archivePath} = withTempDir cfg "simplex-chat." $ \dir -> do Z.withArchive archivePath $ Z.unpackInto dir - StorageFiles {chatDb, agentDb, filesPath} <- storageFiles - backup chatDb - backup agentDb - copyFile (dir archiveChatDbFile) chatDb - copyFile (dir archiveAgentDbFile) agentDb + fs@StorageFiles {chatStore, agentStore, filesPath} <- storageFiles + liftIO $ closeSQLiteStore `withStores` fs + backup `withDBs` fs + copyFile (dir archiveChatDbFile) $ dbFilePath chatStore + copyFile (dir archiveAgentDbFile) $ dbFilePath agentStore copyFiles dir filesPath `E.catch` \(e :: E.SomeException) -> pure [AEImport . ChatError . CEException $ show e] where @@ -94,53 +94,60 @@ copyDirectoryFiles fromDir toDir = do deleteStorage :: ChatMonad m => m () deleteStorage = do - StorageFiles {chatDb, agentDb, filesPath} <- storageFiles - removeFile chatDb - removeFile agentDb - mapM_ removePathForcibly filesPath - tmpPath <- readTVarIO =<< asks tempDirectory - mapM_ removePathForcibly tmpPath + fs <- storageFiles + liftIO $ closeSQLiteStore `withStores` fs + remove `withDBs` fs + mapM_ removeDir $ filesPath fs + mapM_ removeDir =<< chatReadVar tempDirectory + where + remove f = whenM (doesFileExist f) $ removeFile f + removeDir d = whenM (doesDirectoryExist d) $ removePathForcibly d data StorageFiles = StorageFiles - { chatDb :: FilePath, - chatEncrypted :: TVar Bool, - agentDb :: FilePath, - agentEncrypted :: TVar Bool, + { chatStore :: SQLiteStore, + agentStore :: SQLiteStore, filesPath :: Maybe FilePath } storageFiles :: ChatMonad m => m StorageFiles storageFiles = do ChatController {chatStore, filesFolder, smpAgent} <- ask - let SQLiteStore {dbFilePath = chatDb, dbEncrypted = chatEncrypted} = chatStore - SQLiteStore {dbFilePath = agentDb, dbEncrypted = agentEncrypted} = agentClientStore smpAgent + let agentStore = agentClientStore smpAgent filesPath <- readTVarIO filesFolder - pure StorageFiles {chatDb, chatEncrypted, agentDb, agentEncrypted, filesPath} + pure StorageFiles {chatStore, agentStore, filesPath} sqlCipherExport :: forall m. ChatMonad m => DBEncryptionConfig -> m () sqlCipherExport DBEncryptionConfig {currentKey = DBEncryptionKey key, newKey = DBEncryptionKey key'} = when (key /= key') $ do - fs@StorageFiles {chatDb, chatEncrypted, agentDb, agentEncrypted} <- storageFiles - checkFile `with` fs - backup `with` fs - (export chatDb chatEncrypted >> export agentDb agentEncrypted) - `catchChatError` \e -> (restore `with` fs) >> throwError e + fs <- storageFiles + checkFile `withDBs` fs + backup `withDBs` fs + checkEncryption `withStores` fs + removeExported `withDBs` fs + export `withDBs` fs + -- closing after encryption prevents closing in case wrong encryption key was passed + liftIO $ closeSQLiteStore `withStores` fs + (moveExported `withStores` fs) + `catchChatError` \e -> (restore `withDBs` fs) >> throwError e where - action `with` StorageFiles {chatDb, agentDb} = action chatDb >> action agentDb backup f = copyFile f (f <> ".bak") restore f = copyFile (f <> ".bak") f checkFile f = unlessM (doesFileExist f) $ throwDBError $ DBErrorNoFile f - export f dbEnc = do - enc <- readTVarIO dbEnc + checkEncryption SQLiteStore {dbEncrypted} = do + enc <- readTVarIO dbEncrypted when (enc && null key) $ throwDBError DBErrorEncrypted when (not enc && not (null key)) $ throwDBError DBErrorPlaintext - withDB (`SQL.exec` exportSQL) DBErrorExport - renameFile (f <> ".exported") f - withDB (`SQL.exec` testSQL) DBErrorOpen - atomically $ writeTVar dbEnc $ not (null key') + exported = (<> ".exported") + removeExported f = whenM (doesFileExist $ exported f) $ removeFile (exported f) + moveExported SQLiteStore {dbFilePath = f, dbEncrypted} = do + renameFile (exported f) f + atomically $ writeTVar dbEncrypted $ not (null key') + export f = do + withDB f (`SQL.exec` exportSQL) DBErrorExport + withDB (exported f) (`SQL.exec` testSQL) DBErrorOpen where - withDB a err = - liftIO (bracket (SQL.open $ T.pack f) SQL.close a $> Nothing) + withDB f' a err = + liftIO (bracket (SQL.open $ T.pack f') SQL.close a $> Nothing) `catch` checkSQLError `catch` (\(e :: SomeException) -> sqliteError' e) >>= mapM_ (throwDBError . err) @@ -162,7 +169,12 @@ sqlCipherExport DBEncryptionConfig {currentKey = DBEncryptionKey key, newKey = D keySQL key' <> [ "PRAGMA foreign_keys = ON;", "PRAGMA secure_delete = ON;", - "PRAGMA auto_vacuum = FULL;", "SELECT count(*) FROM sqlite_master;" ] keySQL k = ["PRAGMA key = " <> sqlString k <> ";" | not (null k)] + +withDBs :: Monad m => (FilePath -> m b) -> StorageFiles -> m b +action `withDBs` StorageFiles {chatStore, agentStore} = action (dbFilePath chatStore) >> action (dbFilePath agentStore) + +withStores :: Monad m => (SQLiteStore -> m b) -> StorageFiles -> m b +action `withStores` StorageFiles {chatStore, agentStore} = action chatStore >> action agentStore diff --git a/stack.yaml b/stack.yaml index 0840970e4..bce5dd3a6 100644 --- a/stack.yaml +++ b/stack.yaml @@ -49,7 +49,7 @@ extra-deps: # - simplexmq-1.0.0@sha256:34b2004728ae396e3ae449cd090ba7410781e2b3cefc59259915f4ca5daa9ea8,8561 # - ../simplexmq - github: simplex-chat/simplexmq - commit: 8d47f690838371bc848e4b31a4b09ef6bf67ccc5 + commit: ec1b72cb8013a65a5d9783104a47ae44f5730089 - github: kazu-yamamoto/http2 commit: b5a1b7200cf5bc7044af34ba325284271f6dff25 # - ../direct-sqlcipher