check display name

filter on contact deletion
core: filter out user contact on merge
2024-02-19 12:28:04 +04:00 · 2024-02-19 12:05:40 +04:00 · 2024-02-19 11:37:36 +04:00
26 changed files with 322 additions and 768 deletions
--- a/cabal.project
+++ b/cabal.project
@@ -12,7 +12,7 @@ constraints: zip +disable-bzip2 +disable-zstd
 source-repository-package
    type: git
    location: https://github.com/simplex-chat/simplexmq.git
-    tag: e6c444f5d1e94f057ac776b8c6c6c8663236831f
+    tag: caeeb2df9ccca29a6bb504886736502d081fba0e

 source-repository-package
    type: git
--- a/docs/rfcs/2024-02-12-database-migration.md
+++ b/docs/rfcs/2024-02-12-database-migration.md
@@ -1,130 +0,0 @@
-# Database migration and other operations
-
-## Problem
-
-Migrating database to another device is very complex for most people - it is multi-step and error-prone.
-
-In addition to that, any database operation is confusing as it requires stopping chat.
-
-## Solution
-
-Let users migrate database to another device by scanning QR code.
-
-Simplify other database operations by removing the need to compose multiple actions, stop chat, etc.
-
-To support it, we already added the way to represent the file as link/QR code (by uploading file description to XFTP, and supporting "recursive" descriptions).
-
-There will be these actions in the Database settings (no stop/start chat toggle):
-
- Export database.
- Import database.
- Migrate from another device.
- Set passphrase (or Change passphrase if it was set).
- Remove passphrase from device / Store passphrase on the device.
-
-Stop chat toggle will be moved to dev tools.
-
-Migrate to another device will be available in the top part of the settings, 
-
-
-### Database export
-
-Currently, it requires these steps:
-
-1. Open Database settings.
-2. Stop chat (many users don't understand it).
-3. Tap "Export database" in settings.
-4. Look at the alert that says "set passphrase".
-5. Tap Ok.
-6. Tap Set passphrase.
-7. Enter passphrase and confirm.
-8. Exit back to Database settings.
-9. Tap "Export database" again.
-10. Choose file location and save.
-11. Tap "New archive".
-12. Remove exported archive.
-
-These steps are all very confusing, and if they were to stay as composable steps, they belong to dev tools.
-
-Instead we can offer these simple steps:
-
-1. Open Database settings.
-2. Tap "Export database".
-3. Alert will appear saying: "The chat will stop, and you will need to set (or verify) database passphrase. Continue?".
-4. Tap "Ok".
-5. Enter passphrase and confirm in the window that appears (or verify if it was already set, possibly allowing to skip this step).
-7. Choose whether to save file or upload to XFTP and generate link.
-8. File: choose file location and save.
-   Link: show upload progress and then show link to copy.
-9. Alert will appear saying: "Database exported!", exported archive will be automatically removed.
-
-So instead of asking users to understand the required sequence of steps, we will guide them through the required process.
-
-### Database import
-
-1. Open Database settings.
-2. Tap "Import database".
-3. Alert will appear saying: "The chat will stop, you will import?".
-4. File: choose file location and tap "Import".
-   Link: paste link (or scan QR code) and tap "Import".
-5. Confirm to replace database.
-6. Start chat automatically once imported.
-
-### Set or change passphrase
-
-1. Open Database settings.
-2. Tap "Set passphrase" or "Change passphrase" (if it was set).
-3. Choose - store passphrase on the device or enter it every time the app starts.
-
-### Remove / store passphrase from the device
-
-To remove:
-
-1. Open Database settings.
-2. Tap "Remove passphrase".
-3. Confirm to remove passphrase in alert.
-4. Button is replaced with Store.
-
-To store:
-
-1. Open Database settings.
-2. Tap "Store passphrase".
-3. Enter current passphrase - it is verified.
-4. Button is replaced with Remove.
-
-### Migrate database to / from another device
-
-#### User experience
-
-This function is the most important, and it should be available from the main section in settings, under "Use from desktop" (or under "Link from mobile" on desktop).
-
-On the receiving device it will be available via Database settings and also on the Onboarding screen, so users don't need to create a profile.
-
-The steps are:
-
-On the source device:
-1. Tap "Migrate to another device".
-2. The chat will stop showing "Stopping chat" to the user.
-3. If passphrase was:
-   - not set: make user set it in a separate screen.
-   - set: make user verify it.
-5. Show the screen to confirm the upload.
-6. Upload progress (full screen circular progress showing the share, with the %s and total/uploaded size) will be shown.
-7. Once upload is completed, show QR code (with option to copy link), instruct to tap "Migrate from another device" on the receiving device.
-
-On the receiving device:
-2. Tap "Migrate from another device".
-2. The chat will stop (if not from Onboarding) showing "Stopping chat" to the user.
-4. Scan QR code (with option to paste link on desktop only).
-5. Show similar download progress, but probably in reversed direction - design TBC.
-6. Once download is completed, show "Replace the current database" (if not from Onboarding).
-7. Once imported, start chat automatically, and once chat started show "Tap remove database on source device".
-
-On the source device:
-1. Tap "Remove database" on the showing screen (this should also remove uploaded file).
-
-#### Implementation considerations
-
-The latest updates allow uploading and downloading XFTP files without messages.
-
-So to perform the above, the second instance of the chat controller will be required, that probably requires supporting additional/optional chat controller parameter in the APIs that are required for that process.
--- a/scripts/nix/sha256map.nix
+++ b/scripts/nix/sha256map.nix
@@ -1,5 +1,5 @@
 {
-  "https://github.com/simplex-chat/simplexmq.git"."e6c444f5d1e94f057ac776b8c6c6c8663236831f" = "0r66s7q9l8ccpmg4gnk8z1yby9zp9p0c4gjsgx54cnc0rdl7nr4w";
+  "https://github.com/simplex-chat/simplexmq.git"."caeeb2df9ccca29a6bb504886736502d081fba0e" = "187avx8h014fhik76qv1l0nifv6db6nrg9kjk2azqia21n4s2m38";
  "https://github.com/simplex-chat/hs-socks.git"."a30cc7a79a08d8108316094f8f2f82a0c5e1ac51" = "0yasvnr7g91k76mjkamvzab2kvlb1g5pspjyjn2fr6v83swjhj38";
  "https://github.com/simplex-chat/direct-sqlcipher.git"."f814ee68b16a9447fbb467ccc8f29bdd3546bfd9" = "1ql13f4kfwkbaq7nygkxgw84213i0zm7c1a8hwvramayxl38dq5d";
  "https://github.com/simplex-chat/sqlcipher-simple.git"."a46bd361a19376c5211f1058908fc0ae6bf42446" = "1z0r78d8f0812kxbgsm735qf6xx8lvaz27k1a0b4a2m0sshpd5gl";
--- a/simplex-chat.cabal
+++ b/simplex-chat.cabal
@@ -133,7 +133,6 @@ library
      Simplex.Chat.Migrations.M20240104_members_profile_update
      Simplex.Chat.Migrations.M20240115_block_member_for_all
      Simplex.Chat.Migrations.M20240122_indexes
-      Simplex.Chat.Migrations.M20240214_redirect_file_id
      Simplex.Chat.Mobile
      Simplex.Chat.Mobile.File
      Simplex.Chat.Mobile.Shared
--- a/src/Simplex/Chat.hs
+++ b/src/Simplex/Chat.hs
@@ -6,7 +6,6 @@
 {-# LANGUAGE MultiWayIf #-}
 {-# LANGUAGE NamedFieldPuns #-}
 {-# LANGUAGE OverloadedStrings #-}
-{-# LANGUAGE PatternSynonyms #-}
 {-# LANGUAGE RankNTypes #-}
 {-# LANGUAGE ScopedTypeVariables #-}
 {-# LANGUAGE TupleSections #-}
@@ -26,6 +25,7 @@ import qualified Data.Aeson as J
 import Data.Attoparsec.ByteString.Char8 (Parser)
 import qualified Data.Attoparsec.ByteString.Char8 as A
 import Data.Bifunctor (bimap, first, second)
+import Data.ByteArray (ScrubbedBytes)
 import qualified Data.ByteArray as BA
 import qualified Data.ByteString.Base64 as B64
 import Data.ByteString.Char8 (ByteString)
@@ -81,8 +81,7 @@ import Simplex.Chat.Types.Util
 import Simplex.Chat.Util (encryptFile, shuffle)
 import Simplex.FileTransfer.Client.Main (maxFileSize)
 import Simplex.FileTransfer.Client.Presets (defaultXFTPServers)
-import Simplex.FileTransfer.Description (FileDescriptionURI (..), ValidFileDescription, gb, kb, mb)
-import qualified Simplex.FileTransfer.Description as FD
+import Simplex.FileTransfer.Description (ValidFileDescription, gb, kb, mb)
 import Simplex.FileTransfer.Protocol (FileParty (..), FilePartyI)
 import Simplex.Messaging.Agent as Agent
 import Simplex.Messaging.Agent.Client (AgentStatsKey (..), SubInfo (..), agentClientStore, getAgentWorkersDetails, getAgentWorkersSummary, temporaryAgentError)
@@ -98,7 +97,6 @@ import Simplex.Messaging.Client (defaultNetworkConfig)
 import qualified Simplex.Messaging.Crypto as C
 import Simplex.Messaging.Crypto.File (CryptoFile (..), CryptoFileArgs (..))
 import qualified Simplex.Messaging.Crypto.File as CF
-import Simplex.Messaging.Crypto.Memory (LockedBytes)
 import Simplex.Messaging.Encoding
 import Simplex.Messaging.Encoding.String
 import Simplex.Messaging.Parsers (base64P)
@@ -197,7 +195,7 @@ smallGroupsRcptsMemLimit = 20
 logCfg :: LogConfig
 logCfg = LogConfig {lc_file = Nothing, lc_stderr = True}

-createChatDatabase :: FilePath -> LockedBytes -> Bool -> MigrationConfirmation -> IO (Either MigrationError ChatDatabase)
+createChatDatabase :: FilePath -> ScrubbedBytes -> Bool -> MigrationConfirmation -> IO (Either MigrationError ChatDatabase)
 createChatDatabase filePrefix key keepKey confirmMigrations = runExceptT $ do
  chatStore <- ExceptT $ createChatStore (chatStoreFile filePrefix) key keepKey confirmMigrations
  agentStore <- ExceptT $ createAgentStore (agentStoreFile filePrefix) key keepKey confirmMigrations
@@ -772,18 +770,28 @@ processChatCommand' vr = \case
    CTContactConnection -> pure $ chatCmdError (Just user) "not supported"
    where
      xftpSndFileTransfer :: User -> CryptoFile -> Integer -> Int -> ContactOrGroup -> m (FileInvitation, CIFile 'MDSnd, FileTransferMeta)
-      xftpSndFileTransfer user file fileSize n contactOrGroup = do
-        (fInv, ciFile, ft) <- xftpSndFileTransfer_ user file fileSize n $ Just contactOrGroup
+      xftpSndFileTransfer user file@(CryptoFile filePath cfArgs) fileSize n contactOrGroup = do
+        let fileName = takeFileName filePath
+            fileDescr = FileDescr {fileDescrText = "", fileDescrPartNo = 0, fileDescrComplete = False}
+            fInv = xftpFileInvitation fileName fileSize fileDescr
+        fsFilePath <- toFSFilePath filePath
+        let srcFile = CryptoFile fsFilePath cfArgs
+        aFileId <- withAgent $ \a -> xftpSendFile a (aUserId user) srcFile (roundedFDCount n)
+        -- TODO CRSndFileStart event for XFTP
+        chSize <- asks $ fileChunkSize . config
+        ft@FileTransferMeta {fileId} <- withStore' $ \db -> createSndFileTransferXFTP db user contactOrGroup file fInv (AgentSndFileId aFileId) chSize
+        let fileSource = Just $ CryptoFile filePath cfArgs
+            ciFile = CIFile {fileId, fileName, fileSize, fileSource, fileStatus = CIFSSndStored, fileProtocol = FPXFTP}
        case contactOrGroup of
          CGContact Contact {activeConn} -> forM_ activeConn $ \conn ->
-            withStore' $ \db -> createSndFTDescrXFTP db user Nothing conn ft dummyFileDescr
+            withStore' $ \db -> createSndFTDescrXFTP db user Nothing conn ft fileDescr
          CGGroup (Group _ ms) -> forM_ ms $ \m -> saveMemberFD m `catchChatError` (toView . CRChatError (Just user))
            where
              -- we are not sending files to pending members, same as with inline files
              saveMemberFD m@GroupMember {activeConn = Just conn@Connection {connStatus}} =
                when ((connStatus == ConnReady || connStatus == ConnSndReady) && not (connDisabled conn)) $
                  withStore' $
-                    \db -> createSndFTDescrXFTP db user (Just m) conn ft dummyFileDescr
+                    \db -> createSndFTDescrXFTP db user (Just m) conn ft fileDescr
              saveMemberFD _ = pure ()
        pure (fInv, ciFile, ft)
      unzipMaybe3 :: Maybe (a, b, c) -> (Maybe a, Maybe b, Maybe c)
@@ -1951,16 +1959,16 @@ processChatCommand' vr = \case
          | otherwise -> do
              fileAgentConnIds <- cancelSndFile user ftm fts True
              deleteAgentConnectionsAsync user fileAgentConnIds
-              withStore (\db -> liftIO $ lookupChatRefByFileId db user fileId) >>= \case
-                Nothing -> pure ()
-                Just (ChatRef CTDirect contactId) -> do
-                  (contact, sharedMsgId) <- withStore $ \db -> (,) <$> getContact db user contactId <*> getSharedMsgIdByFileId db userId fileId
+              sharedMsgId <- withStore $ \db -> getSharedMsgIdByFileId db userId fileId
+              withStore (\db -> getChatRefByFileId db user fileId) >>= \case
+                ChatRef CTDirect contactId -> do
+                  contact <- withStore $ \db -> getContact db user contactId
                  void . sendDirectContactMessage contact $ XFileCancel sharedMsgId
-                Just (ChatRef CTGroup groupId) -> do
-                  (Group gInfo ms, sharedMsgId) <- withStore $ \db -> (,) <$> getGroup db vr user groupId <*> getSharedMsgIdByFileId db userId fileId
+                ChatRef CTGroup groupId -> do
+                  Group gInfo ms <- withStore $ \db -> getGroup db vr user groupId
                  void . sendGroupMessage user gInfo ms $ XFileCancel sharedMsgId
-                Just _ -> throwChatError $ CEFileInternal "invalid chat ref for file transfer"
-              ci <- withStore $ \db -> lookupChatItemByFileId db vr user fileId
+                _ -> throwChatError $ CEFileInternal "invalid chat ref for file transfer"
+              ci <- withStore $ \db -> getChatItemByFileId db vr user fileId
              pure $ CRSndFileCancelled user ci ftm fts
          where
            fileCancelledOrCompleteSMP SndFileTransfer {fileStatus = s} =
@@ -1971,7 +1979,7 @@ processChatCommand' vr = \case
          | otherwise -> case xftpRcvFile of
              Nothing -> do
                cancelRcvFileTransfer user ftr >>= mapM_ (deleteAgentConnectionAsync user)
-                ci <- withStore $ \db -> lookupChatItemByFileId db vr user fileId
+                ci <- withStore $ \db -> getChatItemByFileId db vr user fileId
                pure $ CRRcvFileCancelled user ci ftr
              Just XFTPRcvFile {agentRcvFileId} -> do
                forM_ (liveRcvFileTransferPath ftr) $ \filePath -> do
@@ -1984,21 +1992,18 @@ processChatCommand' vr = \case
                    updateCIFileStatus db user fileId CIFSRcvInvitation
                    updateRcvFileStatus db fileId FSNew
                    updateRcvFileAgentId db fileId Nothing
-                  lookupChatItemByFileId db vr user fileId
+                  getChatItemByFileId db vr user fileId
                pure $ CRRcvFileCancelled user ci ftr
  FileStatus fileId -> withUser $ \user -> do
-    withStore (\db -> lookupChatItemByFileId db vr user fileId) >>= \case
-      Nothing -> do
+    ci@(AChatItem _ _ _ ChatItem {file}) <- withStore $ \db -> getChatItemByFileId db vr user fileId
+    case file of
+      Just CIFile {fileProtocol = FPLocal} ->
+        throwChatError $ CECommandError "not supported for local files"
+      Just CIFile {fileProtocol = FPXFTP} ->
+        pure $ CRFileTransferStatusXFTP user ci
+      _ -> do
        fileStatus <- withStore $ \db -> getFileTransferProgress db user fileId
        pure $ CRFileTransferStatus user fileStatus
-      Just ci@(AChatItem _ _ _ ChatItem {file}) -> case file of
-        Just CIFile {fileProtocol = FPLocal} ->
-          throwChatError $ CECommandError "not supported for local files"
-        Just CIFile {fileProtocol = FPXFTP} ->
-          pure $ CRFileTransferStatusXFTP user ci
-        _ -> do
-          fileStatus <- withStore $ \db -> getFileTransferProgress db user fileId
-          pure $ CRFileTransferStatus user fileStatus
  ShowProfile -> withUser $ \user@User {profile} -> pure $ CRUserProfile user (fromLocalProfile profile)
  UpdateProfile displayName fullName -> withUser $ \user@User {profile} -> do
    let p = (fromLocalProfile profile :: Profile) {displayName = displayName, fullName = fullName}
@@ -2053,13 +2058,6 @@ processChatCommand' vr = \case
  StopRemoteCtrl -> withUser_ $ stopRemoteCtrl >> ok_
  ListRemoteCtrls -> withUser_ $ CRRemoteCtrlList <$> listRemoteCtrls
  DeleteRemoteCtrl rc -> withUser_ $ deleteRemoteCtrl rc >> ok_
-  APIUploadStandaloneFile userId file -> withUserId userId $ \user -> do
-    fileSize <- liftIO $ CF.getFileContentsSize file
-    (_, _, fileTransferMeta) <- xftpSndFileTransfer_ user file fileSize 1 Nothing
-    pure CRSndStandaloneFileCreated {user, fileTransferMeta}
-  APIDownloadStandaloneFile userId uri file -> withUserId userId $ \user -> do
-    ft <- receiveViaURI user uri file
-    pure $ CRRcvStandaloneFileCreated user ft
  QuitChat -> liftIO exitSuccess
  ShowVersion -> do
    -- simplexmqCommitQ makes iOS builds crash m(
@@ -2813,19 +2811,6 @@ receiveViaCompleteFD user fileId RcvFileDescr {fileDescrText, fileDescrComplete}
    startReceivingFile user fileId
    withStoreCtx' (Just "receiveViaCompleteFD, updateRcvFileAgentId") $ \db -> updateRcvFileAgentId db fileId (Just $ AgentRcvFileId aFileId)

-receiveViaURI :: ChatMonad m => User -> FileDescriptionURI -> CryptoFile -> m RcvFileTransfer
-receiveViaURI user@User {userId} FileDescriptionURI {description} cf@CryptoFile {cryptoArgs} = do
-  fileId <- withStore $ \db -> createRcvStandaloneFileTransfer db userId cf fileSize chunkSize
-  aFileId <- withAgent $ \a -> xftpReceiveFile a (aUserId user) description cryptoArgs
-  withStore $ \db -> do
-    liftIO $ do
-      updateRcvFileStatus db fileId FSConnected
-      updateCIFileStatus db user fileId $ CIFSRcvTransfer 0 1
-      updateRcvFileAgentId db fileId (Just $ AgentRcvFileId aFileId)
-    getRcvFileTransfer db user fileId
-  where
-    FD.ValidFileDescription FD.FileDescription {size = FD.FileSize fileSize, chunkSize = FD.FileSize chunkSize} = description
-
 startReceivingFile :: ChatMonad m => User -> FileTransferId -> m ()
 startReceivingFile user fileId = do
  vr <- chatVersionRange
@@ -3291,7 +3276,7 @@ processAgentMsgSndFile _corrId aFileId msg =
  where
    process :: User -> m ()
    process user = do
-      (ft@FileTransferMeta {fileId, xftpRedirectFor, cancelled}, sfts) <- withStore $ \db -> do
+      (ft@FileTransferMeta {fileId, cancelled}, sfts) <- withStore $ \db -> do
        fileId <- getXFTPSndFileDBId db user $ AgentSndFileId aFileId
        getSndFileTransfer db user fileId
      vr <- chatVersionRange
@@ -3300,76 +3285,61 @@ processAgentMsgSndFile _corrId aFileId msg =
          let status = CIFSSndTransfer {sndProgress, sndTotal}
          ci <- withStore $ \db -> do
            liftIO $ updateCIFileStatus db user fileId status
-            lookupChatItemByFileId db vr user fileId
+            getChatItemByFileId db vr user fileId
          toView $ CRSndFileProgressXFTP user ci ft sndProgress sndTotal
        SFDONE sndDescr rfds -> do
          withStore' $ \db -> setSndFTPrivateSndDescr db user fileId (fileDescrText sndDescr)
-          ci <- withStore $ \db -> lookupChatItemByFileId db vr user fileId
-          case ci of
-            Nothing -> do
-              withAgent (`xftpDeleteSndFileInternal` aFileId)
-              withStore' $ \db -> createExtraSndFTDescrs db user fileId (map fileDescrText rfds)
-              case mapMaybe fileDescrURI rfds of
-                [] -> case rfds of
-                  [] -> logError "File sent without receiver descriptions" -- should not happen
-                  (rfd : _) -> xftpSndFileRedirect user fileId rfd >>= toView . CRSndFileRedirectStartXFTP user ft
-                uris -> do
-                  ft' <- maybe (pure ft) (\fId -> withStore $ \db -> getFileTransferMeta db user fId) xftpRedirectFor
-                  toView $ CRSndStandaloneFileComplete user ft' uris
-            Just (AChatItem _ d cInfo _ci@ChatItem {meta = CIMeta {itemSharedMsgId = msgId_, itemDeleted}}) ->
-              case (msgId_, itemDeleted) of
-                (Just sharedMsgId, Nothing) -> do
-                  when (length rfds < length sfts) $ throwChatError $ CEInternalError "not enough XFTP file descriptions to send"
-                  -- TODO either update database status or move to SFPROG
-                  toView $ CRSndFileProgressXFTP user ci ft 1 1
-                  case (rfds, sfts, d, cInfo) of
-                    (rfd : extraRFDs, sft : _, SMDSnd, DirectChat ct) -> do
-                      withStore' $ \db -> createExtraSndFTDescrs db user fileId (map fileDescrText extraRFDs)
-                      msgDeliveryId <- sendFileDescription sft rfd sharedMsgId $ sendDirectContactMessage ct
-                      withStore' $ \db -> updateSndFTDeliveryXFTP db sft msgDeliveryId
-                      withAgent (`xftpDeleteSndFileInternal` aFileId)
-                    (_, _, SMDSnd, GroupChat g@GroupInfo {groupId}) -> do
-                      ms <- withStore' $ \db -> getGroupMembers db user g
-                      let rfdsMemberFTs = zip rfds $ memberFTs ms
-                          extraRFDs = drop (length rfdsMemberFTs) rfds
-                      withStore' $ \db -> createExtraSndFTDescrs db user fileId (map fileDescrText extraRFDs)
-                      forM_ rfdsMemberFTs $ \mt -> sendToMember mt `catchChatError` (toView . CRChatError (Just user))
-                      ci' <- withStore $ \db -> do
-                        liftIO $ updateCIFileStatus db user fileId CIFSSndComplete
-                        getChatItemByFileId db vr user fileId
-                      withAgent (`xftpDeleteSndFileInternal` aFileId)
-                      toView $ CRSndFileCompleteXFTP user ci' ft
+          ci@(AChatItem _ d cInfo _ci@ChatItem {meta = CIMeta {itemSharedMsgId = msgId_, itemDeleted}}) <-
+            withStore $ \db -> getChatItemByFileId db vr user fileId
+          case (msgId_, itemDeleted) of
+            (Just sharedMsgId, Nothing) -> do
+              when (length rfds < length sfts) $ throwChatError $ CEInternalError "not enough XFTP file descriptions to send"
+              -- TODO either update database status or move to SFPROG
+              toView $ CRSndFileProgressXFTP user ci ft 1 1
+              case (rfds, sfts, d, cInfo) of
+                (rfd : extraRFDs, sft : _, SMDSnd, DirectChat ct) -> do
+                  withStore' $ \db -> createExtraSndFTDescrs db user fileId (map fileDescrText extraRFDs)
+                  msgDeliveryId <- sendFileDescription sft rfd sharedMsgId $ sendDirectContactMessage ct
+                  withStore' $ \db -> updateSndFTDeliveryXFTP db sft msgDeliveryId
+                  withAgent (`xftpDeleteSndFileInternal` aFileId)
+                (_, _, SMDSnd, GroupChat g@GroupInfo {groupId}) -> do
+                  ms <- withStore' $ \db -> getGroupMembers db user g
+                  let rfdsMemberFTs = zip rfds $ memberFTs ms
+                      extraRFDs = drop (length rfdsMemberFTs) rfds
+                  withStore' $ \db -> createExtraSndFTDescrs db user fileId (map fileDescrText extraRFDs)
+                  forM_ rfdsMemberFTs $ \mt -> sendToMember mt `catchChatError` (toView . CRChatError (Just user))
+                  ci' <- withStore $ \db -> do
+                    liftIO $ updateCIFileStatus db user fileId CIFSSndComplete
+                    getChatItemByFileId db vr user fileId
+                  withAgent (`xftpDeleteSndFileInternal` aFileId)
+                  toView $ CRSndFileCompleteXFTP user ci' ft
+                  where
+                    memberFTs :: [GroupMember] -> [(Connection, SndFileTransfer)]
+                    memberFTs ms = M.elems $ M.intersectionWith (,) (M.fromList mConns') (M.fromList sfts')
                      where
-                        memberFTs :: [GroupMember] -> [(Connection, SndFileTransfer)]
-                        memberFTs ms = M.elems $ M.intersectionWith (,) (M.fromList mConns') (M.fromList sfts')
-                          where
-                            mConns' = mapMaybe useMember ms
-                            sfts' = mapMaybe (\sft@SndFileTransfer {groupMemberId} -> (,sft) <$> groupMemberId) sfts
-                            useMember GroupMember {groupMemberId, activeConn = Just conn@Connection {connStatus}}
-                              | (connStatus == ConnReady || connStatus == ConnSndReady) && not (connDisabled conn) = Just (groupMemberId, conn)
-                              | otherwise = Nothing
-                            useMember _ = Nothing
-                        sendToMember :: (ValidFileDescription 'FRecipient, (Connection, SndFileTransfer)) -> m ()
-                        sendToMember (rfd, (conn, sft)) =
-                          void $ sendFileDescription sft rfd sharedMsgId $ \msg' -> sendDirectMessage conn msg' $ GroupId groupId
-                    _ -> pure ()
-                _ -> pure () -- TODO error?
+                        mConns' = mapMaybe useMember ms
+                        sfts' = mapMaybe (\sft@SndFileTransfer {groupMemberId} -> (,sft) <$> groupMemberId) sfts
+                        useMember GroupMember {groupMemberId, activeConn = Just conn@Connection {connStatus}}
+                          | (connStatus == ConnReady || connStatus == ConnSndReady) && not (connDisabled conn) = Just (groupMemberId, conn)
+                          | otherwise = Nothing
+                        useMember _ = Nothing
+                    sendToMember :: (ValidFileDescription 'FRecipient, (Connection, SndFileTransfer)) -> m ()
+                    sendToMember (rfd, (conn, sft)) =
+                      void $ sendFileDescription sft rfd sharedMsgId $ \msg' -> sendDirectMessage conn msg' $ GroupId groupId
+                _ -> pure ()
+            _ -> pure () -- TODO error?
        SFERR e
          | temporaryAgentError e ->
              throwChatError $ CEXFTPSndFile fileId (AgentSndFileId aFileId) e
          | otherwise -> do
              ci <- withStore $ \db -> do
                liftIO $ updateFileCancelled db user fileId CIFSSndError
-                lookupChatItemByFileId db vr user fileId
+                getChatItemByFileId db vr user fileId
              withAgent (`xftpDeleteSndFileInternal` aFileId)
-              toView $ CRSndFileError user ci ft
+              toView $ CRSndFileError user ci
      where
        fileDescrText :: FilePartyI p => ValidFileDescription p -> T.Text
        fileDescrText = safeDecodeUtf8 . strEncode
-        fileDescrURI :: ValidFileDescription 'FRecipient -> Maybe T.Text
-        fileDescrURI vfd = if T.length uri < FD.qrSizeLimit then Just uri else Nothing
-          where
-            uri = decodeLatin1 . strEncode $ FD.fileDescriptionURI vfd
        sendFileDescription :: SndFileTransfer -> ValidFileDescription 'FRecipient -> SharedMsgId -> (ChatMsgEvent 'Json -> m (SndMessage, Int64)) -> m Int64
        sendFileDescription sft rfd msgId sendMsg = do
          let rfdText = fileDescrText rfd
@@ -3417,30 +3387,30 @@ processAgentMsgRcvFile _corrId aFileId msg =
          let status = CIFSRcvTransfer {rcvProgress, rcvTotal}
          ci <- withStore $ \db -> do
            liftIO $ updateCIFileStatus db user fileId status
-            lookupChatItemByFileId db vr user fileId
-          toView $ CRRcvFileProgressXFTP user ci rcvProgress rcvTotal ft
+            getChatItemByFileId db vr user fileId
+          toView $ CRRcvFileProgressXFTP user ci rcvProgress rcvTotal
        RFDONE xftpPath ->
          case liveRcvFileTransferPath ft of
            Nothing -> throwChatError $ CEInternalError "no target path for received XFTP file"
            Just targetPath -> do
              fsTargetPath <- toFSFilePath targetPath
              renameFile xftpPath fsTargetPath
-              ci_ <- withStore $ \db -> do
+              ci <- withStore $ \db -> do
                liftIO $ do
                  updateRcvFileStatus db fileId FSComplete
                  updateCIFileStatus db user fileId CIFSRcvComplete
-                lookupChatItemByFileId db vr user fileId
+                getChatItemByFileId db vr user fileId
              agentXFTPDeleteRcvFile aFileId fileId
-              toView $ maybe (CRRcvStandaloneFileComplete user fsTargetPath ft) (CRRcvFileComplete user) ci_
+              toView $ CRRcvFileComplete user ci
        RFERR e
          | temporaryAgentError e ->
              throwChatError $ CEXFTPRcvFile fileId (AgentRcvFileId aFileId) e
          | otherwise -> do
              ci <- withStore $ \db -> do
                liftIO $ updateFileCancelled db user fileId CIFSRcvError
-                lookupChatItemByFileId db vr user fileId
+                getChatItemByFileId db vr user fileId
              agentXFTPDeleteRcvFile aFileId fileId
-              toView $ CRRcvFileError user ci e ft
+              toView $ CRRcvFileError user ci e

 processAgentMessageConn :: forall m. ChatMonad m => VersionRange -> User -> ACorrId -> ConnId -> ACommand 'Agent 'AEConn -> m ()
 processAgentMessageConn vr user@User {userId} corrId agentConnId agentMessage = do
@@ -3648,15 +3618,18 @@ processAgentMessageConn vr user@User {userId} corrId agentConnId agentMessage =
            processErr cryptoErr = do
              let e@(mde, n) = agentMsgDecryptError cryptoErr
              ci_ <- withStore $ \db ->
-                getDirectChatItemLast db user contactId
+                getDirectChatItemsLast db user contactId 1 ""
                  >>= liftIO
                    . mapM (\(ci, content') -> updateDirectChatItem' db user contactId ci content' False Nothing)
-                    . mdeUpdatedCI e
+                    . (mdeUpdatedCI e <=< headMaybe)
              case ci_ of
                Just ci -> toView $ CRChatItemUpdated user (AChatItem SCTDirect SMDRcv (DirectChat ct) ci)
                _ -> do
                  toView $ CRContactRatchetSync user ct (RatchetSyncProgress rss cStats)
                  createInternalChatItem user (CDDirectRcv ct) (CIRcvDecryptionError mde n) Nothing
+            headMaybe = \case
+              x : _ -> Just x
+              _ -> Nothing
            ratchetSyncEventItem ct' = do
              toView $ CRContactRatchetSync user ct' (RatchetSyncProgress rss cStats)
              createInternalChatItem user (CDDirectRcv ct') (CIRcvConnEvent $ RCERatchetSync rss) Nothing
@@ -4116,10 +4089,10 @@ processAgentMessageConn vr user@User {userId} corrId agentConnId agentMessage =
          case err of
            SMP SMP.AUTH -> unless (fileStatus == FSCancelled) $ do
              ci <- withStore $ \db -> do
-                liftIO (lookupChatRefByFileId db user fileId) >>= \case
-                  Just (ChatRef CTDirect _) -> liftIO $ updateFileCancelled db user fileId CIFSSndCancelled
+                getChatRefByFileId db user fileId >>= \case
+                  ChatRef CTDirect _ -> liftIO $ updateFileCancelled db user fileId CIFSSndCancelled
                  _ -> pure ()
-                lookupChatItemByFileId db vr user fileId
+                getChatItemByFileId db vr user fileId
              toView $ CRSndFileRcvCancelled user ci ft
            _ -> throwChatError $ CEFileSend fileId err
        MSG meta _ _ -> withAckMessage' agentConnId conn meta $ pure ()
@@ -6301,19 +6274,12 @@ agentXFTPDeleteRcvFile aFileId fileId = do
  withStore' $ \db -> setRcvFTAgentDeleted db fileId

 agentXFTPDeleteSndFileRemote :: ChatMonad m => User -> XFTPSndFile -> FileTransferId -> m ()
-agentXFTPDeleteSndFileRemote user sndFile fileId = do
-    -- the agent doesn't know about redirect, delete explicitly
-  redirect_ <- withStore' $ \db -> lookupFileTransferRedirectMeta db user fileId
-  forM_ redirect_ $ \FileTransferMeta {fileId = fileIdRedirect, xftpSndFile = sndFileRedirect_} ->
-    mapM_ (handleError (const $ pure ()) . remove fileIdRedirect) sndFileRedirect_
-  remove fileId sndFile
-  where
-    remove fId XFTPSndFile {agentSndFileId = AgentSndFileId aFileId, privateSndFileDescr, agentSndFileDeleted} =
-      unless agentSndFileDeleted $ do
-        forM_ privateSndFileDescr $ \sfdText -> do
-          sd <- parseFileDescription sfdText
-          withAgent $ \a -> xftpDeleteSndFileRemote a (aUserId user) aFileId sd
-          withStore' $ \db -> setSndFTAgentDeleted db user fId
+agentXFTPDeleteSndFileRemote user XFTPSndFile {agentSndFileId = AgentSndFileId aFileId, privateSndFileDescr, agentSndFileDeleted} fileId =
+  unless agentSndFileDeleted $
+    forM_ privateSndFileDescr $ \sfdText -> do
+      sd <- parseFileDescription sfdText
+      withAgent $ \a -> xftpDeleteSndFileRemote a (aUserId user) aFileId sd
+      withStore' $ \db -> setSndFTAgentDeleted db user fileId

 userProfileToSend :: User -> Maybe Profile -> Maybe Contact -> Bool -> Profile
 userProfileToSend user@User {profile = p} incognitoProfile ct inGroup = do
@@ -6784,8 +6750,6 @@ chatCommandP =
      "/list remote ctrls" $> ListRemoteCtrls,
      "/stop remote ctrl" $> StopRemoteCtrl,
      "/delete remote ctrl " *> (DeleteRemoteCtrl <$> A.decimal),
-      "/_upload " *> (APIUploadStandaloneFile <$> A.decimal <* A.space <*> cryptoFileP),
-      "/_download " *> (APIDownloadStandaloneFile <$> A.decimal <* A.space <*> strP_ <*> cryptoFileP),
      ("/quit" <|> "/q" <|> "/exit") $> QuitChat,
      ("/version" <|> "/v") $> ShowVersion,
      "/debug locks" $> DebugLocks,
@@ -6973,29 +6937,3 @@ mkValidName = reverse . dropWhile isSpace . fst3 . foldl' addChar ("", '\NUL', 0
          | isPunctuation prev = validFirstChar || isSpace c || (punct < 3 && isPunctuation c)
          | otherwise = validFirstChar || isSpace c || isMark c || isPunctuation c
        validFirstChar = isLetter c || isNumber c || isSymbol c
-
-xftpSndFileTransfer_ :: ChatMonad m => User -> CryptoFile -> Integer -> Int -> Maybe ContactOrGroup -> m (FileInvitation, CIFile 'MDSnd, FileTransferMeta)
-xftpSndFileTransfer_ user file@(CryptoFile filePath cfArgs) fileSize n contactOrGroup_ = do
-  let fileName = takeFileName filePath
-      fInv = xftpFileInvitation fileName fileSize dummyFileDescr
-  fsFilePath <- toFSFilePath filePath
-  let srcFile = CryptoFile fsFilePath cfArgs
-  aFileId <- withAgent $ \a -> xftpSendFile a (aUserId user) srcFile (roundedFDCount n)
-  -- TODO CRSndFileStart event for XFTP
-  chSize <- asks $ fileChunkSize . config
-  ft@FileTransferMeta {fileId} <- withStore' $ \db -> createSndFileTransferXFTP db user contactOrGroup_ file fInv (AgentSndFileId aFileId) Nothing chSize
-  let fileSource = Just $ CryptoFile filePath cfArgs
-      ciFile = CIFile {fileId, fileName, fileSize, fileSource, fileStatus = CIFSSndStored, fileProtocol = FPXFTP}
-  pure (fInv, ciFile, ft)
-
-xftpSndFileRedirect :: ChatMonad m => User -> FileTransferId -> ValidFileDescription 'FRecipient -> m FileTransferMeta
-xftpSndFileRedirect user ftId vfd = do
-  let fileName = "redirect.yaml"
-      file = CryptoFile fileName Nothing
-      fInv = xftpFileInvitation fileName (fromIntegral $ B.length $ strEncode vfd) dummyFileDescr
-  aFileId <- withAgent $ \a -> xftpSendDescription a (aUserId user) vfd (roundedFDCount 1)
-  chSize <- asks $ fileChunkSize . config
-  withStore' $ \db -> createSndFileTransferXFTP db user Nothing file fInv (AgentSndFileId aFileId) (Just ftId) chSize
-
-dummyFileDescr :: FileDescr
-dummyFileDescr = FileDescr {fileDescrText = "", fileDescrPartNo = 0, fileDescrComplete = False}
--- a/src/Simplex/Chat/Archive.hs
+++ b/src/Simplex/Chat/Archive.hs
@@ -27,7 +27,6 @@ import qualified Database.SQLite3 as SQL
 import Simplex.Chat.Controller
 import Simplex.Messaging.Agent.Client (agentClientStore)
 import Simplex.Messaging.Agent.Store.SQLite (SQLiteStore (..), closeSQLiteStore, keyString, sqlString, storeKey)
-import Simplex.Messaging.Crypto.Memory (LockedBytes)
 import Simplex.Messaging.Util
 import System.FilePath
 import UnliftIO.Directory
@@ -173,7 +172,7 @@ withDB f' a err =
    sqliteError' :: Show e => e -> m (Maybe SQLiteError)
    sqliteError' = pure . Just . SQLiteError . show

-testSQL :: LockedBytes -> Text
+testSQL :: BA.ScrubbedBytes -> Text
 testSQL k =
  T.unlines $
    keySQL k
@@ -182,7 +181,7 @@ testSQL k =
            "SELECT count(*) FROM sqlite_master;"
          ]

-keySQL :: LockedBytes -> [Text]
+keySQL :: BA.ScrubbedBytes -> [Text]
 keySQL k = ["PRAGMA key = " <> keyString k <> ";" | not (BA.null k)]

 sqlCipherTestKey :: forall m. ChatMonad m => DBEncryptionKey -> m ()
--- a/src/Simplex/Chat/Controller.hs
+++ b/src/Simplex/Chat/Controller.hs
@@ -29,6 +29,7 @@ import qualified Data.Aeson.TH as JQ
 import qualified Data.Aeson.Types as JT
 import qualified Data.Attoparsec.ByteString.Char8 as A
 import Data.Bifunctor (first)
+import Data.ByteArray (ScrubbedBytes)
 import qualified Data.ByteArray as BA
 import Data.ByteString.Char8 (ByteString)
 import qualified Data.ByteString.Char8 as B
@@ -58,7 +59,6 @@ import Simplex.Chat.Remote.Types
 import Simplex.Chat.Store (AutoAccept, StoreError (..), UserContactLink, UserMsgReceiptSettings)
 import Simplex.Chat.Types
 import Simplex.Chat.Types.Preferences
-import Simplex.FileTransfer.Description (FileDescriptionURI)
 import Simplex.Messaging.Agent (AgentClient, SubscriptionsInfo)
 import Simplex.Messaging.Agent.Client (AgentLocks, AgentWorkersDetails (..), AgentWorkersSummary (..), ProtocolTestFailure)
 import Simplex.Messaging.Agent.Env.SQLite (AgentConfig, NetworkConfig)
@@ -70,7 +70,6 @@ import qualified Simplex.Messaging.Agent.Store.SQLite.DB as DB
 import qualified Simplex.Messaging.Crypto as C
 import Simplex.Messaging.Crypto.File (CryptoFile (..))
 import qualified Simplex.Messaging.Crypto.File as CF
-import Simplex.Messaging.Crypto.Memory (LockedBytes)
 import Simplex.Messaging.Encoding.String
 import Simplex.Messaging.Notifications.Protocol (DeviceToken (..), NtfTknStatus)
 import Simplex.Messaging.Parsers (defaultJSON, dropPrefix, enumJSON, parseAll, parseString, sumTypeJSON)
@@ -454,8 +453,6 @@ data ChatCommand
  | ListRemoteCtrls
  | StopRemoteCtrl -- Stop listening for announcements or terminate an active session
  | DeleteRemoteCtrl RemoteCtrlId -- Remove all local data associated with a remote controller session
-  | APIUploadStandaloneFile UserId CryptoFile
-  | APIDownloadStandaloneFile UserId FileDescriptionURI CryptoFile
  | QuitChat
  | ShowVersion
  | DebugLocks
@@ -596,26 +593,21 @@ data ChatResponse
  | CRRcvFileAccepted {user :: User, chatItem :: AChatItem}
  | CRRcvFileAcceptedSndCancelled {user :: User, rcvFileTransfer :: RcvFileTransfer}
  | CRRcvFileDescrNotReady {user :: User, chatItem :: AChatItem}
-  | CRRcvStandaloneFileCreated {user :: User, rcvFileTransfer :: RcvFileTransfer} -- returned by _download
-  | CRRcvFileStart {user :: User, chatItem :: AChatItem} -- sent by chats
-  | CRRcvFileProgressXFTP {user :: User, chatItem_ :: Maybe AChatItem, receivedSize :: Int64, totalSize :: Int64, rcvFileTransfer :: RcvFileTransfer}
+  | CRRcvFileStart {user :: User, chatItem :: AChatItem}
+  | CRRcvFileProgressXFTP {user :: User, chatItem :: AChatItem, receivedSize :: Int64, totalSize :: Int64}
  | CRRcvFileComplete {user :: User, chatItem :: AChatItem}
-  | CRRcvStandaloneFileComplete {user :: User, targetPath :: FilePath, rcvFileTransfer :: RcvFileTransfer}
-  | CRRcvFileCancelled {user :: User, chatItem_ :: Maybe AChatItem, rcvFileTransfer :: RcvFileTransfer}
+  | CRRcvFileCancelled {user :: User, chatItem :: AChatItem, rcvFileTransfer :: RcvFileTransfer}
  | CRRcvFileSndCancelled {user :: User, chatItem :: AChatItem, rcvFileTransfer :: RcvFileTransfer}
-  | CRRcvFileError {user :: User, chatItem_ :: Maybe AChatItem, agentError :: AgentErrorType, rcvFileTransfer :: RcvFileTransfer}
+  | CRRcvFileError {user :: User, chatItem :: AChatItem, agentError :: AgentErrorType}
  | CRSndFileStart {user :: User, chatItem :: AChatItem, sndFileTransfer :: SndFileTransfer}
  | CRSndFileComplete {user :: User, chatItem :: AChatItem, sndFileTransfer :: SndFileTransfer}
-  | CRSndFileRcvCancelled {user :: User, chatItem_ :: Maybe AChatItem, sndFileTransfer :: SndFileTransfer}
-  | CRSndFileCancelled {user :: User, chatItem_ :: Maybe AChatItem, fileTransferMeta :: FileTransferMeta, sndFileTransfers :: [SndFileTransfer]}
-  | CRSndStandaloneFileCreated {user :: User, fileTransferMeta :: FileTransferMeta} -- returned by _upload
-  | CRSndFileStartXFTP {user :: User, chatItem :: AChatItem, fileTransferMeta :: FileTransferMeta} -- not used
-  | CRSndFileProgressXFTP {user :: User, chatItem_ :: Maybe AChatItem, fileTransferMeta :: FileTransferMeta, sentSize :: Int64, totalSize :: Int64}
-  | CRSndFileRedirectStartXFTP {user :: User, fileTransferMeta :: FileTransferMeta, redirectMeta :: FileTransferMeta}
+  | CRSndFileRcvCancelled {user :: User, chatItem :: AChatItem, sndFileTransfer :: SndFileTransfer}
+  | CRSndFileCancelled {user :: User, chatItem :: AChatItem, fileTransferMeta :: FileTransferMeta, sndFileTransfers :: [SndFileTransfer]}
+  | CRSndFileStartXFTP {user :: User, chatItem :: AChatItem, fileTransferMeta :: FileTransferMeta}
+  | CRSndFileProgressXFTP {user :: User, chatItem :: AChatItem, fileTransferMeta :: FileTransferMeta, sentSize :: Int64, totalSize :: Int64}
  | CRSndFileCompleteXFTP {user :: User, chatItem :: AChatItem, fileTransferMeta :: FileTransferMeta}
-  | CRSndStandaloneFileComplete {user :: User, fileTransferMeta :: FileTransferMeta, rcvURIs :: [Text]}
-  | CRSndFileCancelledXFTP {user :: User, chatItem_ :: Maybe AChatItem, fileTransferMeta :: FileTransferMeta}
-  | CRSndFileError {user :: User, chatItem_ :: Maybe AChatItem, fileTransferMeta :: FileTransferMeta}
+  | CRSndFileCancelledXFTP {user :: User, chatItem :: AChatItem, fileTransferMeta :: FileTransferMeta}
+  | CRSndFileError {user :: User, chatItem :: AChatItem}
  | CRUserProfileUpdated {user :: User, fromProfile :: Profile, toProfile :: Profile, updateSummary :: UserProfileUpdateSummary}
  | CRUserProfileImage {user :: User, profile :: Profile}
  | CRContactAliasUpdated {user :: User, toContact :: Contact}
@@ -880,7 +872,7 @@ data ArchiveConfig = ArchiveConfig {archivePath :: FilePath, disableCompression
 data DBEncryptionConfig = DBEncryptionConfig {currentKey :: DBEncryptionKey, newKey :: DBEncryptionKey, keepKey :: Maybe Bool}
  deriving (Show)

-newtype DBEncryptionKey = DBEncryptionKey LockedBytes
+newtype DBEncryptionKey = DBEncryptionKey ScrubbedBytes
  deriving (Show)

 instance IsString DBEncryptionKey where fromString = parseString $ parseAll strP
--- a/src/Simplex/Chat/Messages.hs
+++ b/src/Simplex/Chat/Messages.hs
@@ -360,24 +360,6 @@ mkCIMeta itemId itemContent itemText itemStatus itemSharedMsgId itemDeleted item
        _ -> False
   in CIMeta {itemId, itemTs, itemText, itemStatus, itemSharedMsgId, itemDeleted, itemEdited, itemTimed, itemLive, editable, forwardedByMember, createdAt, updatedAt}

-dummyMeta :: ChatItemId -> UTCTime -> Text -> CIMeta c 'MDSnd
-dummyMeta itemId ts itemText =
-  CIMeta
-    { itemId,
-      itemTs = ts,
-      itemText,
-      itemStatus = CISSndNew,
-      itemSharedMsgId = Nothing,
-      itemDeleted = Nothing,
-      itemEdited = False,
-      itemTimed = Nothing,
-      itemLive = Nothing,
-      editable = False,
-      forwardedByMember = Nothing,
-      createdAt = ts,
-      updatedAt = ts
-    }
-
 data CITimed = CITimed
  { ttl :: Int, -- seconds
    deleteAt :: Maybe UTCTime -- this is initially Nothing for received items, the timer starts when they are read
--- a/src/Simplex/Chat/Messages/CIContent.hs
+++ b/src/Simplex/Chat/Messages/CIContent.hs
@@ -139,7 +139,7 @@ data CIContent (d :: MsgDirection) where
  CISndModerated :: CIContent 'MDSnd
  CIRcvModerated :: CIContent 'MDRcv
  CIRcvBlocked :: CIContent 'MDRcv
-  CIInvalidJSON :: Text -> CIContent d -- this is also used for logical database errors, e.g. SEBadChatItem
+  CIInvalidJSON :: Text -> CIContent d
 -- ^ This type is used both in API and in DB, so we use different JSON encodings for the database and for the API
 -- ! ^ Nested sum types also have to use different encodings for database and API
 -- ! ^ to avoid breaking cross-platform compatibility, see RcvGroupEvent and SndGroupEvent
--- a/src/Simplex/Chat/Migrations/M20240214_redirect_file_id.hs
+++ b/src/Simplex/Chat/Migrations/M20240214_redirect_file_id.hs
@@ -1,22 +0,0 @@
-{-# LANGUAGE QuasiQuotes #-}
-
-module Simplex.Chat.Migrations.M20240214_redirect_file_id where
-
-import Database.SQLite.Simple (Query)
-import Database.SQLite.Simple.QQ (sql)
-
-m20240214_redirect_file_id :: Query
-m20240214_redirect_file_id =
-  [sql|
-ALTER TABLE files ADD COLUMN redirect_file_id INTEGER REFERENCES files ON DELETE CASCADE;
-
-CREATE INDEX idx_files_redirect_file_id on files(redirect_file_id);
-|]
-
-down_m20240214_redirect_file_id :: Query
-down_m20240214_redirect_file_id =
-  [sql|
-DROP INDEX idx_files_redirect_file_id;
-
-ALTER TABLE files DROP COLUMN redirect_file_id;
-|]
--- a/src/Simplex/Chat/Migrations/chat_schema.sql
+++ b/src/Simplex/Chat/Migrations/chat_schema.sql
@@ -193,8 +193,7 @@ CREATE TABLE files(
  protocol TEXT NOT NULL DEFAULT 'smp',
  file_crypto_key BLOB,
  file_crypto_nonce BLOB,
-  note_folder_id INTEGER DEFAULT NULL REFERENCES note_folders ON DELETE CASCADE,
-  redirect_file_id INTEGER REFERENCES files ON DELETE CASCADE
+  note_folder_id INTEGER DEFAULT NULL REFERENCES note_folders ON DELETE CASCADE
 );
 CREATE TABLE snd_files(
  file_id INTEGER NOT NULL REFERENCES files ON DELETE CASCADE,
@@ -855,4 +854,3 @@ CREATE INDEX idx_chat_items_notes_item_status on chat_items(
  note_folder_id,
  item_status
 );
-CREATE INDEX idx_files_redirect_file_id on files(redirect_file_id);
--- a/src/Simplex/Chat/Mobile.hs
+++ b/src/Simplex/Chat/Mobile.hs
@@ -15,6 +15,7 @@ import Control.Monad.Reader
 import qualified Data.Aeson as J
 import qualified Data.Aeson.TH as JQ
 import Data.Bifunctor (first)
+import Data.ByteArray (ScrubbedBytes)
 import qualified Data.ByteArray as BA
 import qualified Data.ByteString.Base64.URL as U
 import Data.ByteString.Char8 (ByteString)
@@ -49,7 +50,6 @@ import Simplex.Messaging.Agent.Env.SQLite (createAgentStore)
 import Simplex.Messaging.Agent.Store.SQLite (MigrationConfirmation (..), MigrationError, closeSQLiteStore, reopenSQLiteStore)
 import Simplex.Messaging.Client (defaultNetworkConfig)
 import qualified Simplex.Messaging.Crypto as C
-import Simplex.Messaging.Crypto.Memory (LockedBytes)
 import Simplex.Messaging.Encoding.String
 import Simplex.Messaging.Parsers (defaultJSON, dropPrefix, sumTypeJSON)
 import Simplex.Messaging.Protocol (AProtoServerWithAuth (..), AProtocolType (..), BasicAuth (..), CorrId (..), ProtoServerWithAuth (..), ProtocolServer (..))
@@ -227,10 +227,10 @@ defaultMobileConfig =
 getActiveUser_ :: SQLiteStore -> IO (Maybe User)
 getActiveUser_ st = find activeUser <$> withTransaction st getUsers

-chatMigrateInit :: String -> LockedBytes -> String -> IO (Either DBMigrationResult ChatController)
+chatMigrateInit :: String -> ScrubbedBytes -> String -> IO (Either DBMigrationResult ChatController)
 chatMigrateInit dbFilePrefix dbKey confirm = chatMigrateInitKey dbFilePrefix dbKey False confirm False

-chatMigrateInitKey :: String -> LockedBytes -> Bool -> String -> Bool -> IO (Either DBMigrationResult ChatController)
+chatMigrateInitKey :: String -> ScrubbedBytes -> Bool -> String -> Bool -> IO (Either DBMigrationResult ChatController)
 chatMigrateInitKey dbFilePrefix dbKey keepKey confirm backgroundMode = runExceptT $ do
  confirmMigrations <- liftEitherWith (const DBMInvalidConfirmation) $ strDecode $ B.pack confirm
  chatStore <- migrate createChatStore (chatStoreFile dbFilePrefix) confirmMigrations
--- a/src/Simplex/Chat/Options.hs
+++ b/src/Simplex/Chat/Options.hs
@@ -19,6 +19,7 @@ where

 import Control.Logger.Simple (LogLevel (..))
 import qualified Data.Attoparsec.ByteString.Char8 as A
+import Data.ByteArray (ScrubbedBytes)
 import qualified Data.ByteString.Char8 as B
 import Data.Text (Text)
 import Numeric.Natural (Natural)
@@ -26,7 +27,6 @@ import Options.Applicative
 import Simplex.Chat.Controller (ChatLogLevel (..), updateStr, versionNumber, versionString)
 import Simplex.FileTransfer.Description (mb)
 import Simplex.Messaging.Client (NetworkConfig (..), defaultNetworkConfig)
-import Simplex.Messaging.Crypto.Memory (LockedBytes)
 import Simplex.Messaging.Encoding.String
 import Simplex.Messaging.Parsers (parseAll)
 import Simplex.Messaging.Protocol (ProtoServerWithAuth, ProtocolTypeI, SMPServerWithAuth, XFTPServerWithAuth)
@@ -51,7 +51,7 @@ data ChatOpts = ChatOpts

 data CoreChatOpts = CoreChatOpts
  { dbFilePrefix :: String,
-    dbKey :: LockedBytes,
+    dbKey :: ScrubbedBytes,
    smpServers :: [SMPServerWithAuth],
    xftpServers :: [XFTPServerWithAuth],
    networkConfig :: NetworkConfig,
--- a/src/Simplex/Chat/Store.hs
+++ b/src/Simplex/Chat/Store.hs
@@ -12,13 +12,13 @@ module Simplex.Chat.Store
  )
 where

+import Data.ByteArray (ScrubbedBytes)
 import Simplex.Chat.Store.Migrations
 import Simplex.Chat.Store.Profiles
 import Simplex.Chat.Store.Shared
 import Simplex.Messaging.Agent.Store.SQLite (MigrationConfirmation, MigrationError, SQLiteStore (..), createSQLiteStore, withTransaction)
-import Simplex.Messaging.Crypto.Memory (LockedBytes)

-createChatStore :: FilePath -> LockedBytes -> Bool -> MigrationConfirmation -> IO (Either MigrationError SQLiteStore)
+createChatStore :: FilePath -> ScrubbedBytes -> Bool -> MigrationConfirmation -> IO (Either MigrationError SQLiteStore)
 createChatStore dbPath key keepKey = createSQLiteStore dbPath key keepKey migrations

 chatStoreFile :: FilePath -> FilePath
--- a/src/Simplex/Chat/Store/Direct.hs
+++ b/src/Simplex/Chat/Store/Direct.hs
@@ -236,11 +236,18 @@ deleteContact db user@User {userId} Contact {contactId, localDisplayName, active
  if isNothing ctMember
    then do
      deleteContactProfile_ db userId contactId
-      DB.execute db "DELETE FROM display_names WHERE user_id = ? AND local_display_name = ?" (userId, localDisplayName)
+      DB.execute
+        db
+        [sql|
+          DELETE FROM display_names
+          WHERE user_id = ? AND local_display_name = ?
+            AND local_display_name NOT IN (SELECT local_display_name FROM users)
+        |]
+        (userId, localDisplayName)
    else do
      currentTs <- getCurrentTime
      DB.execute db "UPDATE group_members SET contact_id = NULL, updated_at = ? WHERE user_id = ? AND contact_id = ?" (currentTs, userId, contactId)
-  DB.execute db "DELETE FROM contacts WHERE user_id = ? AND contact_id = ?" (userId, contactId)
+  DB.execute db "DELETE FROM contacts WHERE user_id = ? AND contact_id = ? AND is_user = 0" (userId, contactId)
  forM_ activeConn $ \Connection {customUserProfileId} ->
    forM_ customUserProfileId $ \profileId ->
      deleteUnusedIncognitoProfileById_ db user profileId
@@ -250,8 +257,15 @@ deleteContactWithoutGroups :: DB.Connection -> User -> Contact -> IO ()
 deleteContactWithoutGroups db user@User {userId} Contact {contactId, localDisplayName, activeConn} = do
  DB.execute db "DELETE FROM chat_items WHERE user_id = ? AND contact_id = ?" (userId, contactId)
  deleteContactProfile_ db userId contactId
-  DB.execute db "DELETE FROM display_names WHERE user_id = ? AND local_display_name = ?" (userId, localDisplayName)
-  DB.execute db "DELETE FROM contacts WHERE user_id = ? AND contact_id = ?" (userId, contactId)
+  DB.execute
+    db
+    [sql|
+      DELETE FROM display_names
+      WHERE user_id = ? AND local_display_name = ?
+        AND local_display_name NOT IN (SELECT local_display_name FROM users)
+    |]
+    (userId, localDisplayName)
+  DB.execute db "DELETE FROM contacts WHERE user_id = ? AND contact_id = ? AND is_user = 0" (userId, contactId)
  forM_ activeConn $ \Connection {customUserProfileId} ->
    forM_ customUserProfileId $ \profileId ->
      deleteUnusedIncognitoProfileById_ db user profileId
@@ -259,7 +273,7 @@ deleteContactWithoutGroups db user@User {userId} Contact {contactId, localDispla
 setContactDeleted :: DB.Connection -> User -> Contact -> IO ()
 setContactDeleted db User {userId} Contact {contactId} = do
  currentTs <- getCurrentTime
-  DB.execute db "UPDATE contacts SET deleted = 1, updated_at = ? WHERE user_id = ? AND contact_id = ?" (currentTs, userId, contactId)
+  DB.execute db "UPDATE contacts SET deleted = 1, updated_at = ? WHERE user_id = ? AND contact_id = ? AND is_user = 0" (currentTs, userId, contactId)

 getDeletedContacts :: DB.Connection -> User -> IO [Contact]
 getDeletedContacts db user@User {userId} = do
@@ -501,7 +515,14 @@ updateContactLDN_ db userId contactId displayName newName updatedAt = do
    db
    "UPDATE group_members SET local_display_name = ?, updated_at = ? WHERE user_id = ? AND contact_id = ?"
    (newName, updatedAt, userId, contactId)
-  DB.execute db "DELETE FROM display_names WHERE local_display_name = ? AND user_id = ?" (displayName, userId)
+  DB.execute
+    db
+    [sql|
+      DELETE FROM display_names
+      WHERE local_display_name = ? AND user_id = ?
+        AND local_display_name NOT IN (SELECT local_display_name FROM users)
+    |]
+    (displayName, userId)

 getContactByName :: DB.Connection -> User -> ContactName -> ExceptT StoreError IO Contact
 getContactByName db user localDisplayName = do
@@ -614,7 +635,14 @@ createOrUpdateContactRequest db user@User {userId} userContactLinkId invId (Vers
                WHERE user_id = ? AND contact_request_id = ?
              |]
              (invId, minV, maxV, ldn, currentTs, userId, cReqId)
-            DB.execute db "DELETE FROM display_names WHERE local_display_name = ? AND user_id = ?" (oldLdn, userId)
+            DB.execute
+              db
+              [sql|
+                DELETE FROM display_names
+                WHERE local_display_name = ? AND user_id = ?
+                  AND local_display_name NOT IN (SELECT local_display_name FROM users)
+              |]
+              (oldLdn, userId)
      where
        updateProfile currentTs =
          DB.execute
@@ -684,6 +712,7 @@ deleteContactRequest db User {userId} contactRequestId = do
        SELECT local_display_name FROM contact_requests
        WHERE user_id = ? AND contact_request_id = ?
      )
+      AND local_display_name NOT IN (SELECT local_display_name FROM users)
    |]
    (userId, userId, contactRequestId)
  DB.execute db "DELETE FROM contact_requests WHERE user_id = ? AND contact_request_id = ?" (userId, contactRequestId)
--- a/src/Simplex/Chat/Store/Files.hs
+++ b/src/Simplex/Chat/Store/Files.hs
@@ -39,7 +39,6 @@ module Simplex.Chat.Store.Files
    getGroupFileIdBySharedMsgId,
    getDirectFileIdBySharedMsgId,
    getChatRefByFileId,
-    lookupChatRefByFileId,
    updateSndFileStatus,
    createSndFileChunk,
    updateSndFileChunkMsg,
@@ -47,7 +46,6 @@ module Simplex.Chat.Store.Files
    deleteSndFileChunks,
    createRcvFileTransfer,
    createRcvGroupFileTransfer,
-    createRcvStandaloneFileTransfer,
    appendRcvFD,
    getRcvFileDescrByRcvFileId,
    getRcvFileDescrBySndFileId,
@@ -72,7 +70,6 @@ module Simplex.Chat.Store.Files
    getFileTransfer,
    getFileTransferProgress,
    getFileTransferMeta,
-    lookupFileTransferRedirectMeta,
    getSndFileTransfer,
    getSndFileTransfers,
    getContactFileInfo,
@@ -89,14 +86,12 @@ import Control.Monad
 import Control.Monad.Except
 import Control.Monad.IO.Class
 import Data.Either (rights)
-import Data.Functor ((<&>))
 import Data.Int (Int64)
 import Data.Maybe (fromMaybe, isJust, listToMaybe)
 import Data.Text (Text)
 import Data.Time (addUTCTime)
 import Data.Time.Clock (UTCTime (..), getCurrentTime, nominalDay)
 import Data.Type.Equality
-import Data.Word (Word32)
 import Database.SQLite.Simple (Only (..), (:.) (..))
 import Database.SQLite.Simple.QQ (sql)
 import Database.SQLite.Simple.ToField (ToField)
@@ -189,7 +184,7 @@ createSndDirectFileTransfer db userId Contact {contactId} filePath FileInvitatio
      db
      "INSERT INTO snd_files (file_id, file_status, file_inline, connection_id, created_at, updated_at) VALUES (?,?,?,?,?,?)"
      (fileId, fileStatus, fileInline, connId, currentTs, currentTs)
-  pure FileTransferMeta {fileId, xftpSndFile = Nothing, xftpRedirectFor = Nothing, fileName, filePath, fileSize, fileInline, chunkSize, cancelled = False}
+  pure FileTransferMeta {fileId, xftpSndFile = Nothing, fileName, filePath, fileSize, fileInline, chunkSize, cancelled = False}

 createSndDirectFTConnection :: DB.Connection -> User -> Int64 -> (CommandId, ConnId) -> SubscriptionMode -> IO ()
 createSndDirectFTConnection db user@User {userId} fileId (cmdId, acId) subMode = do
@@ -209,7 +204,7 @@ createSndGroupFileTransfer db userId GroupInfo {groupId} filePath FileInvitation
    "INSERT INTO files (user_id, group_id, file_name, file_path, file_size, chunk_size, file_inline, ci_file_status, protocol, created_at, updated_at) VALUES (?,?,?,?,?,?,?,?,?,?,?)"
    ((userId, groupId, fileName, filePath, fileSize, chunkSize) :. (fileInline, CIFSSndStored, FPSMP, currentTs, currentTs))
  fileId <- insertedRowId db
-  pure FileTransferMeta {fileId, xftpSndFile = Nothing, xftpRedirectFor = Nothing, fileName, filePath, fileSize, fileInline, chunkSize, cancelled = False}
+  pure FileTransferMeta {fileId, xftpSndFile = Nothing, fileName, filePath, fileSize, fileInline, chunkSize, cancelled = False}

 createSndGroupFileTransferConnection :: DB.Connection -> User -> Int64 -> (CommandId, ConnId) -> GroupMember -> SubscriptionMode -> IO ()
 createSndGroupFileTransferConnection db user@User {userId} fileId (cmdId, acId) GroupMember {groupMemberId} subMode = do
@@ -282,16 +277,16 @@ getSndFTViaMsgDelivery db User {userId} Connection {connId, agentConnId} agentMs
      (\n -> SndFileTransfer {fileId, fileStatus, fileName, fileSize, chunkSize, filePath, fileDescrId, fileInline, groupMemberId, recipientDisplayName = n, connId, agentConnId})
        <$> (contactName_ <|> memberName_)

-createSndFileTransferXFTP :: DB.Connection -> User -> Maybe ContactOrGroup -> CryptoFile -> FileInvitation -> AgentSndFileId -> Maybe FileTransferId -> Integer -> IO FileTransferMeta
-createSndFileTransferXFTP db User {userId} contactOrGroup_ (CryptoFile filePath cryptoArgs) FileInvitation {fileName, fileSize} agentSndFileId xftpRedirectFor chunkSize = do
+createSndFileTransferXFTP :: DB.Connection -> User -> ContactOrGroup -> CryptoFile -> FileInvitation -> AgentSndFileId -> Integer -> IO FileTransferMeta
+createSndFileTransferXFTP db User {userId} contactOrGroup (CryptoFile filePath cryptoArgs) FileInvitation {fileName, fileSize} agentSndFileId chunkSize = do
  currentTs <- getCurrentTime
  let xftpSndFile = Just XFTPSndFile {agentSndFileId, privateSndFileDescr = Nothing, agentSndFileDeleted = False, cryptoArgs}
  DB.execute
    db
-    "INSERT INTO files (contact_id, group_id, user_id, file_name, file_path, file_crypto_key, file_crypto_nonce, file_size, chunk_size, redirect_file_id, agent_snd_file_id, ci_file_status, protocol, created_at, updated_at) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)"
-    (maybe (Nothing, Nothing) contactAndGroupIds contactOrGroup_ :. (userId, fileName, filePath, CF.fileKey <$> cryptoArgs, CF.fileNonce <$> cryptoArgs, fileSize, chunkSize) :. (xftpRedirectFor, agentSndFileId, CIFSSndStored, FPXFTP, currentTs, currentTs))
+    "INSERT INTO files (contact_id, group_id, user_id, file_name, file_path, file_crypto_key, file_crypto_nonce, file_size, chunk_size, agent_snd_file_id, ci_file_status, protocol, created_at, updated_at) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?)"
+    (contactAndGroupIds contactOrGroup :. (userId, fileName, filePath, CF.fileKey <$> cryptoArgs, CF.fileNonce <$> cryptoArgs, fileSize, chunkSize, agentSndFileId, CIFSSndStored, FPXFTP, currentTs, currentTs))
  fileId <- insertedRowId db
-  pure FileTransferMeta {fileId, xftpSndFile, xftpRedirectFor, fileName, filePath, fileSize, fileInline = Nothing, chunkSize, cancelled = False}
+  pure FileTransferMeta {fileId, xftpSndFile, fileName, filePath, fileSize, fileInline = Nothing, chunkSize, cancelled = False}

 createSndFTDescrXFTP :: DB.Connection -> User -> Maybe GroupMember -> Connection -> FileTransferMeta -> FileDescr -> IO ()
 createSndFTDescrXFTP db User {userId} m Connection {connId} FileTransferMeta {fileId} FileDescr {fileDescrText, fileDescrPartNo, fileDescrComplete} = do
@@ -426,14 +421,11 @@ getDirectFileIdBySharedMsgId db User {userId} Contact {contactId} sharedMsgId =
      (userId, contactId, sharedMsgId)

 getChatRefByFileId :: DB.Connection -> User -> Int64 -> ExceptT StoreError IO ChatRef
-getChatRefByFileId db user fileId = liftIO (lookupChatRefByFileId db user fileId) >>= maybe (throwError $ SEInternalError "could not retrieve chat ref by file id") pure
-
-lookupChatRefByFileId :: DB.Connection -> User -> Int64 -> IO (Maybe ChatRef)
-lookupChatRefByFileId db User {userId} fileId =
-  getChatRef <&> \case
-    [(Just contactId, Nothing)] -> Just $ ChatRef CTDirect contactId
-    [(Nothing, Just groupId)] -> Just $ ChatRef CTGroup groupId
-    _ -> Nothing
+getChatRefByFileId db User {userId} fileId =
+  liftIO getChatRef >>= \case
+    [(Just contactId, Nothing)] -> pure $ ChatRef CTDirect contactId
+    [(Nothing, Just groupId)] -> pure $ ChatRef CTGroup groupId
+    _ -> throwError $ SEInternalError "could not retrieve chat ref by file id"
  where
    getChatRef =
      DB.query
@@ -544,23 +536,6 @@ createRcvGroupFileTransfer db userId GroupMember {groupId, groupMemberId, localD
      (fileId, FSNew, fileConnReq, fileInline, rcvFileInline, groupMemberId, rfdId, currentTs, currentTs)
  pure RcvFileTransfer {fileId, xftpRcvFile, fileInvitation = f, fileStatus = RFSNew, rcvFileInline, senderDisplayName = c, chunkSize, cancelled = False, grpMemberId = Just groupMemberId, cryptoArgs = Nothing}

-createRcvStandaloneFileTransfer :: DB.Connection -> UserId -> CryptoFile -> Int64 -> Word32 -> ExceptT StoreError IO Int64
-createRcvStandaloneFileTransfer db userId (CryptoFile filePath cfArgs_) fileSize chunkSize = do
-  currentTs <- liftIO getCurrentTime
-  fileId <- liftIO $ do
-    DB.execute
-      db
-      "INSERT INTO files (user_id, file_name, file_path, file_size, chunk_size, ci_file_status, protocol, created_at, updated_at) VALUES (?,?,?,?,?,?,?,?,?)"
-      (userId, takeFileName filePath, filePath, fileSize, chunkSize, CIFSRcvInvitation, FPXFTP, currentTs, currentTs)
-    insertedRowId db
-  liftIO . forM_ cfArgs_ $ \cfArgs -> setFileCryptoArgs_ db fileId cfArgs currentTs
-  liftIO $
-    DB.execute
-      db
-      "INSERT INTO rcv_files (file_id, file_status, created_at, updated_at) VALUES (?,?,?,?)"
-      (fileId, FSNew, currentTs, currentTs)
-  pure fileId
-
 createRcvFD_ :: DB.Connection -> UserId -> UTCTime -> FileDescr -> ExceptT StoreError IO RcvFileDescr
 createRcvFD_ db userId currentTs FileDescr {fileDescrText, fileDescrPartNo, fileDescrComplete} = do
  when (fileDescrPartNo /= 0) $ throwError SERcvFileInvalidDescrPart
@@ -687,9 +662,9 @@ getRcvFileTransfer_ db userId fileId = do
      (FileStatus, Maybe ConnReqInvitation, Maybe Int64, String, Integer, Integer, Maybe Bool) :. (Maybe ContactName, Maybe ContactName, Maybe FilePath, Maybe C.SbKey, Maybe C.CbNonce, Maybe InlineFileMode, Maybe InlineFileMode, Maybe AgentRcvFileId, Bool) :. (Maybe Int64, Maybe AgentConnId) ->
      ExceptT StoreError IO RcvFileTransfer
    rcvFileTransfer rfd_ ((fileStatus', fileConnReq, grpMemberId, fileName, fileSize, chunkSize, cancelled_) :. (contactName_, memberName_, filePath_, fileKey, fileNonce, fileInline, rcvFileInline, agentRcvFileId, agentRcvFileDeleted) :. (connId_, agentConnId_)) =
-      case contactName_ <|> memberName_ <|> standaloneName_ of
+      case contactName_ <|> memberName_ of
        Nothing -> throwError $ SERcvFileInvalid fileId
-        Just name ->
+        Just name -> do
          case fileStatus' of
            FSNew -> pure $ ft name RFSNew
            FSAccepted -> ft name . RFSAccepted <$> rfi
@@ -697,9 +672,6 @@ getRcvFileTransfer_ db userId fileId = do
            FSComplete -> ft name . RFSComplete <$> rfi
            FSCancelled -> ft name . RFSCancelled <$> rfi_
      where
-        standaloneName_ = case (connId_, agentRcvFileId, filePath_) of
-          (Nothing, Just _, Just _) -> Just "" -- filePath marks files that are accepted from contact or, in this case, set by createRcvDirectFileTransfer
-          _ -> Nothing
        ft senderDisplayName fileStatus =
          let fileInvitation = FileInvitation {fileName, fileSize, fileDigest = Nothing, fileConnReq, fileInline, fileDescr = Nothing}
              cryptoArgs = CFArgs <$> fileKey <*> fileNonce
@@ -934,22 +906,17 @@ getFileTransferMeta_ db userId fileId =
    DB.query
      db
      [sql|
-        SELECT file_name, file_size, chunk_size, file_path, file_crypto_key, file_crypto_nonce, file_inline, agent_snd_file_id, agent_snd_file_deleted, private_snd_file_descr, cancelled, redirect_file_id
+        SELECT file_name, file_size, chunk_size, file_path, file_crypto_key, file_crypto_nonce, file_inline, agent_snd_file_id, agent_snd_file_deleted, private_snd_file_descr, cancelled
        FROM files
        WHERE user_id = ? AND file_id = ?
      |]
      (userId, fileId)
  where
-    fileTransferMeta :: (String, Integer, Integer, FilePath, Maybe C.SbKey, Maybe C.CbNonce, Maybe InlineFileMode, Maybe AgentSndFileId, Bool, Maybe Text, Maybe Bool, Maybe FileTransferId) -> FileTransferMeta
-    fileTransferMeta (fileName, fileSize, chunkSize, filePath, fileKey, fileNonce, fileInline, aSndFileId_, agentSndFileDeleted, privateSndFileDescr, cancelled_, xftpRedirectFor) =
+    fileTransferMeta :: (String, Integer, Integer, FilePath, Maybe C.SbKey, Maybe C.CbNonce, Maybe InlineFileMode, Maybe AgentSndFileId, Bool, Maybe Text, Maybe Bool) -> FileTransferMeta
+    fileTransferMeta (fileName, fileSize, chunkSize, filePath, fileKey, fileNonce, fileInline, aSndFileId_, agentSndFileDeleted, privateSndFileDescr, cancelled_) =
      let cryptoArgs = CFArgs <$> fileKey <*> fileNonce
          xftpSndFile = (\fId -> XFTPSndFile {agentSndFileId = fId, privateSndFileDescr, agentSndFileDeleted, cryptoArgs}) <$> aSndFileId_
-        in FileTransferMeta {fileId, xftpSndFile, xftpRedirectFor, fileName, fileSize, chunkSize, filePath, fileInline, cancelled = fromMaybe False cancelled_}
-
-lookupFileTransferRedirectMeta :: DB.Connection -> User -> Int64 -> IO [FileTransferMeta]
-lookupFileTransferRedirectMeta db User {userId} fileId = do
-  redirects <- DB.query db "SELECT file_id FROM files WHERE user_id = ? AND redirect_file_id = ?" (userId, fileId)
-  rights <$> mapM (runExceptT . getFileTransferMeta_ db userId . fromOnly) redirects
+       in FileTransferMeta {fileId, xftpSndFile, fileName, fileSize, chunkSize, filePath, fileInline, cancelled = fromMaybe False cancelled_}

 createLocalFile :: ToField (CIFileStatus d) => CIFileStatus d -> DB.Connection -> User -> NoteFolder -> ChatItemId -> UTCTime -> CryptoFile -> Integer -> Integer -> IO Int64
 createLocalFile fileStatus db User {userId} NoteFolder {noteFolderId} chatItemId itemTs CryptoFile {filePath, cryptoArgs} fileSize fileChunkSize = do
--- a/src/Simplex/Chat/Store/Groups.hs
+++ b/src/Simplex/Chat/Store/Groups.hs
@@ -225,6 +225,7 @@ deleteGroupLink db User {userId} GroupInfo {groupId} = do
          JOIN user_contact_links uc USING (user_contact_link_id)
          WHERE uc.user_id = ? AND uc.group_id = ?
        )
+        AND local_display_name NOT IN (SELECT local_display_name FROM users)
    |]
    (userId, userId, groupId)
  DB.execute
@@ -586,7 +587,14 @@ deleteGroup :: DB.Connection -> User -> GroupInfo -> IO ()
 deleteGroup db user@User {userId} g@GroupInfo {groupId, localDisplayName} = do
  deleteGroupProfile_ db userId groupId
  DB.execute db "DELETE FROM groups WHERE user_id = ? AND group_id = ?" (userId, groupId)
-  DB.execute db "DELETE FROM display_names WHERE user_id = ? AND local_display_name = ?" (userId, localDisplayName)
+  DB.execute
+    db
+    [sql|
+      DELETE FROM display_names
+      WHERE user_id = ? AND local_display_name = ?
+        AND local_display_name NOT IN (SELECT local_display_name FROM users)
+    |]
+    (userId, localDisplayName)
  forM_ (incognitoMembershipProfile g) $ deleteUnusedIncognitoProfileById_ db user . localProfileId

 deleteGroupProfile_ :: DB.Connection -> UserId -> GroupId -> IO ()
@@ -1051,7 +1059,14 @@ cleanupMemberProfileAndName_ db User {userId} GroupMember {groupMemberId, member
    sameProfileMember :: (Maybe GroupMemberId) <- maybeFirstRow fromOnly $ DB.query db "SELECT group_member_id FROM group_members WHERE user_id = ? AND contact_profile_id = ? AND group_member_id != ? LIMIT 1" (userId, memberContactProfileId, groupMemberId)
    when (isNothing sameProfileMember) $ do
      DB.execute db "DELETE FROM contact_profiles WHERE user_id = ? AND contact_profile_id = ?" (userId, memberContactProfileId)
-      DB.execute db "DELETE FROM display_names WHERE user_id = ? AND local_display_name = ?" (userId, localDisplayName)
+      DB.execute
+        db
+        [sql|
+          DELETE FROM display_names
+          WHERE user_id = ? AND local_display_name = ?
+            AND local_display_name NOT IN (SELECT local_display_name FROM users)
+        |]
+        (userId, localDisplayName)

 deleteGroupMemberConnection :: DB.Connection -> User -> GroupMember -> IO ()
 deleteGroupMemberConnection db User {userId} GroupMember {groupMemberId} =
@@ -1361,7 +1376,14 @@ updateGroupProfile db User {userId} g@GroupInfo {groupId, localDisplayName, grou
        db
        "UPDATE groups SET local_display_name = ?, updated_at = ? WHERE user_id = ? AND group_id = ?"
        (ldn, currentTs, userId, groupId)
-      DB.execute db "DELETE FROM display_names WHERE local_display_name = ? AND user_id = ?" (localDisplayName, userId)
+      DB.execute
+        db
+        [sql|
+          DELETE FROM display_names
+          WHERE local_display_name = ? AND user_id = ?
+            AND local_display_name NOT IN (SELECT local_display_name FROM users)
+        |]
+        (localDisplayName, userId)

 getGroupInfo :: DB.Connection -> VersionRange -> User -> Int64 -> ExceptT StoreError IO GroupInfo
 getGroupInfo db vr User {userId, userContactId} groupId =
@@ -1464,7 +1486,7 @@ getMatchingContacts db user@User {userId} Contact {contactId, profile = LocalPro
        FROM contacts ct
        JOIN contact_profiles p ON ct.contact_profile_id = p.contact_profile_id
        WHERE ct.user_id = ? AND ct.contact_id != ?
-          AND ct.contact_status = ? AND ct.deleted = 0
+          AND ct.contact_status = ? AND ct.deleted = 0 AND ct.is_user = 0
          AND p.display_name = ? AND p.full_name = ?
      |]

@@ -1502,7 +1524,7 @@ getMatchingMemberContacts db user@User {userId} GroupMember {memberProfile = Loc
        FROM contacts ct
        JOIN contact_profiles p ON ct.contact_profile_id = p.contact_profile_id
        WHERE ct.user_id = ?
-          AND ct.contact_status = ? AND ct.deleted = 0
+          AND ct.contact_status = ? AND ct.deleted = 0 AND ct.is_user = 0
          AND p.display_name = ? AND p.full_name = ?
      |]

@@ -1656,7 +1678,7 @@ mergeContactRecords db user@User {userId} to@Contact {localDisplayName = keepLDN
        ":updated_at" := currentTs
      ]
    deleteContactProfile_ db userId fromContactId
-    DB.execute db "DELETE FROM contacts WHERE contact_id = ? AND user_id = ?" (fromContactId, userId)
+    DB.execute db "DELETE FROM contacts WHERE contact_id = ? AND user_id = ? AND is_user = 0" (fromContactId, userId)
    deleteUnusedDisplayName_ db userId fromLDN
    when (keepLDN /= toLDN && keepLDN == fromLDN) $
      DB.execute
@@ -2030,7 +2052,14 @@ updateMemberProfile db User {userId} m p'
          db
          "UPDATE group_members SET local_display_name = ?, updated_at = ? WHERE user_id = ? AND group_member_id = ?"
          (ldn, currentTs, userId, groupMemberId)
-        DB.execute db "DELETE FROM display_names WHERE local_display_name = ? AND user_id = ?" (localDisplayName, userId)
+        DB.execute
+          db
+          [sql|
+            DELETE FROM display_names
+            WHERE local_display_name = ? AND user_id = ?
+              AND local_display_name NOT IN (SELECT local_display_name FROM users)
+          |]
+          (localDisplayName, userId)
        pure $ Right m {localDisplayName = ldn, memberProfile = profile}
  where
    GroupMember {groupMemberId, localDisplayName, memberProfile = LocalProfile {profileId, displayName, localAlias}} = m
--- a/src/Simplex/Chat/Store/Messages.hs
+++ b/src/Simplex/Chat/Store/Messages.hs
@@ -39,7 +39,7 @@ module Simplex.Chat.Store.Messages
    getDirectChat,
    getGroupChat,
    getLocalChat,
-    getDirectChatItemLast,
+    getDirectChatItemsLast,
    getAllChatItems,
    getAChatItem,
    updateDirectChatItem,
@@ -92,7 +92,6 @@ module Simplex.Chat.Store.Messages
    getLocalChatItemIdByText,
    getLocalChatItemIdByText',
    getChatItemByFileId,
-    lookupChatItemByFileId,
    getChatItemByGroupId,
    updateDirectChatItemStatus,
    getTimedItems,
@@ -126,7 +125,6 @@ import Data.List (sortBy)
 import Data.Maybe (fromMaybe, isJust, mapMaybe)
 import Data.Ord (Down (..), comparing)
 import Data.Text (Text)
-import qualified Data.Text as T
 import Data.Time (addUTCTime)
 import Data.Time.Clock (UTCTime (..), getCurrentTime)
 import Database.SQLite.Simple (NamedParam (..), Only (..), Query, (:.) (..))
@@ -830,7 +828,7 @@ toLocalChatItem currentTs ((itemId, itemTs, AMsgDirection msgDir, itemContentTex
    cItem :: MsgDirectionI d => SMsgDirection d -> CIDirection 'CTLocal d -> CIStatus d -> CIContent d -> Maybe (CIFile d) -> CChatItem 'CTLocal
    cItem d chatDir ciStatus content file =
      CChatItem d ChatItem {chatDir, meta = ciMeta content ciStatus, content, formattedText = parseMaybeMarkdownList itemText, quotedItem = Nothing, reactions = [], file}
-    badItem = Left $ SEBadChatItem itemId (Just itemTs)
+    badItem = Left $ SEBadChatItem itemId
    ciMeta :: CIContent d -> CIStatus d -> CIMeta 'CTLocal d
    ciMeta content status =
      let itemDeleted' = case itemDeleted of
@@ -924,118 +922,97 @@ getDirectChat :: DB.Connection -> User -> Int64 -> ChatPagination -> Maybe Strin
 getDirectChat db user contactId pagination search_ = do
  let search = fromMaybe "" search_
  ct <- getContact db user contactId
-  liftIO $ case pagination of
+  liftIO . getDirectChatReactions_ db ct =<< case pagination of
    CPLast count -> getDirectChatLast_ db user ct count search
    CPAfter afterId count -> getDirectChatAfter_ db user ct afterId count search
    CPBefore beforeId count -> getDirectChatBefore_ db user ct beforeId count search

-- the last items in reverse order (the last item in the conversation is the first in the returned list)
-getDirectChatLast_ :: DB.Connection -> User -> Contact -> Int -> String -> IO (Chat 'CTDirect)
-getDirectChatLast_ db user@User {userId} ct@Contact {contactId} count search = do
+getDirectChatLast_ :: DB.Connection -> User -> Contact -> Int -> String -> ExceptT StoreError IO (Chat 'CTDirect)
+getDirectChatLast_ db user ct@Contact {contactId} count search = do
  let stats = ChatStats {unreadCount = 0, minUnreadItemId = 0, unreadChat = False}
-  chatItemIds <- getDirectChatItemIdsLast_
-  currentTs <- getCurrentTime
-  chatItems <- mapM (safeGetDirectItem db user ct currentTs) chatItemIds
+  chatItems <- getDirectChatItemsLast db user contactId count search
  pure $ Chat (DirectChat ct) (reverse chatItems) stats
-  where
-    getDirectChatItemIdsLast_ :: IO [ChatItemId]
-    getDirectChatItemIdsLast_ =
-      map fromOnly
-        <$> DB.query
-          db
-          [sql|
-            SELECT chat_item_id
-            FROM chat_items
-            WHERE user_id = ? AND contact_id = ? AND item_text LIKE '%' || ? || '%'
-            ORDER BY created_at DESC, chat_item_id DESC
-            LIMIT ?
-          |]
-          (userId, contactId, search, count)

-safeGetDirectItem :: DB.Connection -> User -> Contact -> UTCTime -> ChatItemId -> IO (CChatItem 'CTDirect)
-safeGetDirectItem db user ct currentTs itemId =
-  runExceptT (getDirectCIWithReactions db user ct itemId)
-    >>= pure <$> safeToDirectItem currentTs itemId
-
-safeToDirectItem :: UTCTime -> ChatItemId -> Either StoreError (CChatItem 'CTDirect) -> CChatItem 'CTDirect
-safeToDirectItem currentTs itemId = \case
-  Right ci -> ci
-  Left e@(SEBadChatItem _ (Just itemTs)) -> badDirectItem itemTs e
-  Left e -> badDirectItem currentTs e
-  where
-    badDirectItem :: UTCTime -> StoreError -> CChatItem 'CTDirect
-    badDirectItem ts e =
-      let errorText = T.pack $ show e
-       in CChatItem
-            SMDSnd
-            ChatItem
-              { chatDir = CIDirectSnd,
-                meta = dummyMeta itemId ts errorText,
-                content = CIInvalidJSON errorText,
-                formattedText = Nothing,
-                quotedItem = Nothing,
-                reactions = [],
-                file = Nothing
-              }
-
-getDirectChatItemLast :: DB.Connection -> User -> ContactId -> ExceptT StoreError IO (CChatItem 'CTDirect)
-getDirectChatItemLast db user@User {userId} contactId = do
-  chatItemId <-
-    ExceptT . firstRow fromOnly (SEChatItemNotFoundByContactId contactId) $
-      DB.query
-        db
-        [sql|
-          SELECT chat_item_id
-          FROM chat_items
-          WHERE user_id = ? AND contact_id = ?
-          ORDER BY created_at DESC, chat_item_id DESC
-          LIMIT 1
-        |]
-        (userId, contactId)
-  getDirectChatItem db user contactId chatItemId
-
-getDirectChatAfter_ :: DB.Connection -> User -> Contact -> ChatItemId -> Int -> String -> IO (Chat 'CTDirect)
-getDirectChatAfter_ db user@User {userId} ct@Contact {contactId} afterChatItemId count search = do
-  let stats = ChatStats {unreadCount = 0, minUnreadItemId = 0, unreadChat = False}
-  chatItemIds <- getDirectChatItemIdsAfter_
+-- the last items in reverse order (the last item in the conversation is the first in the returned list)
+getDirectChatItemsLast :: DB.Connection -> User -> ContactId -> Int -> String -> ExceptT StoreError IO [CChatItem 'CTDirect]
+getDirectChatItemsLast db User {userId} contactId count search = ExceptT $ do
  currentTs <- getCurrentTime
-  chatItems <- mapM (safeGetDirectItem db user ct currentTs) chatItemIds
+  mapM (toDirectChatItem currentTs)
+    <$> DB.query
+      db
+      [sql|
+        SELECT
+          -- ChatItem
+          i.chat_item_id, i.item_ts, i.item_sent, i.item_content, i.item_text, i.item_status, i.shared_msg_id, i.item_deleted, i.item_deleted_ts, i.item_edited, i.created_at, i.updated_at, i.timed_ttl, i.timed_delete_at, i.item_live,
+          -- CIFile
+          f.file_id, f.file_name, f.file_size, f.file_path, f.file_crypto_key, f.file_crypto_nonce, f.ci_file_status, f.protocol,
+          -- DirectQuote
+          ri.chat_item_id, i.quoted_shared_msg_id, i.quoted_sent_at, i.quoted_content, i.quoted_sent
+        FROM chat_items i
+        LEFT JOIN files f ON f.chat_item_id = i.chat_item_id
+        LEFT JOIN chat_items ri ON ri.user_id = i.user_id AND ri.contact_id = i.contact_id AND ri.shared_msg_id = i.quoted_shared_msg_id
+        WHERE i.user_id = ? AND i.contact_id = ? AND i.item_text LIKE '%' || ? || '%'
+        ORDER BY i.created_at DESC, i.chat_item_id DESC
+        LIMIT ?
+      |]
+      (userId, contactId, search, count)
+
+getDirectChatAfter_ :: DB.Connection -> User -> Contact -> ChatItemId -> Int -> String -> ExceptT StoreError IO (Chat 'CTDirect)
+getDirectChatAfter_ db User {userId} ct@Contact {contactId} afterChatItemId count search = do
+  let stats = ChatStats {unreadCount = 0, minUnreadItemId = 0, unreadChat = False}
+  chatItems <- ExceptT getDirectChatItemsAfter_
  pure $ Chat (DirectChat ct) chatItems stats
  where
-    getDirectChatItemIdsAfter_ :: IO [ChatItemId]
-    getDirectChatItemIdsAfter_ =
-      map fromOnly
+    getDirectChatItemsAfter_ :: IO (Either StoreError [CChatItem 'CTDirect])
+    getDirectChatItemsAfter_ = do
+      currentTs <- getCurrentTime
+      mapM (toDirectChatItem currentTs)
        <$> DB.query
          db
          [sql|
-            SELECT chat_item_id
-            FROM chat_items
-            WHERE user_id = ? AND contact_id = ? AND item_text LIKE '%' || ? || '%'
-              AND chat_item_id > ?
-            ORDER BY created_at ASC, chat_item_id ASC
+            SELECT
+              -- ChatItem
+              i.chat_item_id, i.item_ts, i.item_sent, i.item_content, i.item_text, i.item_status, i.shared_msg_id, i.item_deleted, i.item_deleted_ts, i.item_edited, i.created_at, i.updated_at, i.timed_ttl, i.timed_delete_at, i.item_live,
+              -- CIFile
+              f.file_id, f.file_name, f.file_size, f.file_path, f.file_crypto_key, f.file_crypto_nonce, f.ci_file_status, f.protocol,
+              -- DirectQuote
+              ri.chat_item_id, i.quoted_shared_msg_id, i.quoted_sent_at, i.quoted_content, i.quoted_sent
+            FROM chat_items i
+            LEFT JOIN files f ON f.chat_item_id = i.chat_item_id
+            LEFT JOIN chat_items ri ON ri.user_id = i.user_id AND ri.contact_id = i.contact_id AND ri.shared_msg_id = i.quoted_shared_msg_id
+            WHERE i.user_id = ? AND i.contact_id = ? AND i.item_text LIKE '%' || ? || '%'
+              AND i.chat_item_id > ?
+            ORDER BY i.created_at ASC, i.chat_item_id ASC
            LIMIT ?
          |]
          (userId, contactId, search, afterChatItemId, count)

-getDirectChatBefore_ :: DB.Connection -> User -> Contact -> ChatItemId -> Int -> String -> IO (Chat 'CTDirect)
-getDirectChatBefore_ db user@User {userId} ct@Contact {contactId} beforeChatItemId count search = do
+getDirectChatBefore_ :: DB.Connection -> User -> Contact -> ChatItemId -> Int -> String -> ExceptT StoreError IO (Chat 'CTDirect)
+getDirectChatBefore_ db User {userId} ct@Contact {contactId} beforeChatItemId count search = do
  let stats = ChatStats {unreadCount = 0, minUnreadItemId = 0, unreadChat = False}
-  chatItemIds <- getDirectChatItemsIdsBefore_
-  currentTs <- getCurrentTime
-  chatItems <- mapM (safeGetDirectItem db user ct currentTs) chatItemIds
+  chatItems <- ExceptT getDirectChatItemsBefore_
  pure $ Chat (DirectChat ct) (reverse chatItems) stats
  where
-    getDirectChatItemsIdsBefore_ :: IO [ChatItemId]
-    getDirectChatItemsIdsBefore_ =
-      map fromOnly
+    getDirectChatItemsBefore_ :: IO (Either StoreError [CChatItem 'CTDirect])
+    getDirectChatItemsBefore_ = do
+      currentTs <- getCurrentTime
+      mapM (toDirectChatItem currentTs)
        <$> DB.query
          db
          [sql|
-            SELECT chat_item_id
-            FROM chat_items
-            WHERE user_id = ? AND contact_id = ? AND item_text LIKE '%' || ? || '%'
-              AND chat_item_id < ?
-            ORDER BY created_at DESC, chat_item_id DESC
+            SELECT
+              -- ChatItem
+              i.chat_item_id, i.item_ts, i.item_sent, i.item_content, i.item_text, i.item_status, i.shared_msg_id, i.item_deleted, i.item_deleted_ts, i.item_edited, i.created_at, i.updated_at, i.timed_ttl, i.timed_delete_at, i.item_live,
+              -- CIFile
+              f.file_id, f.file_name, f.file_size, f.file_path, f.file_crypto_key, f.file_crypto_nonce, f.ci_file_status, f.protocol,
+              -- DirectQuote
+              ri.chat_item_id, i.quoted_shared_msg_id, i.quoted_sent_at, i.quoted_content, i.quoted_sent
+            FROM chat_items i
+            LEFT JOIN files f ON f.chat_item_id = i.chat_item_id
+            LEFT JOIN chat_items ri ON ri.user_id = i.user_id AND ri.contact_id = i.contact_id AND ri.shared_msg_id = i.quoted_shared_msg_id
+            WHERE i.user_id = ? AND i.contact_id = ? AND i.item_text LIKE '%' || ? || '%'
+              AND i.chat_item_id < ?
+            ORDER BY i.created_at DESC, i.chat_item_id DESC
            LIMIT ?
          |]
          (userId, contactId, search, beforeChatItemId, count)
@@ -1045,16 +1022,15 @@ getGroupChat db vr user groupId pagination search_ = do
  let search = fromMaybe "" search_
  g <- getGroupInfo db vr user groupId
  case pagination of
-    CPLast count -> liftIO $ getGroupChatLast_ db user g count search
+    CPLast count -> getGroupChatLast_ db user g count search
    CPAfter afterId count -> getGroupChatAfter_ db user g afterId count search
    CPBefore beforeId count -> getGroupChatBefore_ db user g beforeId count search

-getGroupChatLast_ :: DB.Connection -> User -> GroupInfo -> Int -> String -> IO (Chat 'CTGroup)
+getGroupChatLast_ :: DB.Connection -> User -> GroupInfo -> Int -> String -> ExceptT StoreError IO (Chat 'CTGroup)
 getGroupChatLast_ db user@User {userId} g@GroupInfo {groupId} count search = do
  let stats = ChatStats {unreadCount = 0, minUnreadItemId = 0, unreadChat = False}
-  chatItemIds <- getGroupChatItemIdsLast_
-  currentTs <- getCurrentTime
-  chatItems <- mapM (safeGetGroupItem db user g currentTs) chatItemIds
+  chatItemIds <- liftIO getGroupChatItemIdsLast_
+  chatItems <- mapM (getGroupCIWithReactions db user g) chatItemIds
  pure $ Chat (GroupChat g) (reverse chatItems) stats
  where
    getGroupChatItemIdsLast_ :: IO [ChatItemId]
@@ -1071,32 +1047,6 @@ getGroupChatLast_ db user@User {userId} g@GroupInfo {groupId} count search = do
          |]
          (userId, groupId, search, count)

-safeGetGroupItem :: DB.Connection -> User -> GroupInfo -> UTCTime -> ChatItemId -> IO (CChatItem 'CTGroup)
-safeGetGroupItem db user g currentTs itemId =
-  runExceptT (getGroupCIWithReactions db user g itemId)
-    >>= pure <$> safeToGroupItem currentTs itemId
-
-safeToGroupItem :: UTCTime -> ChatItemId -> Either StoreError (CChatItem 'CTGroup) -> CChatItem 'CTGroup
-safeToGroupItem currentTs itemId = \case
-  Right ci -> ci
-  Left e@(SEBadChatItem _ (Just itemTs)) -> badGroupItem itemTs e
-  Left e -> badGroupItem currentTs e
-  where
-    badGroupItem :: UTCTime -> StoreError -> CChatItem 'CTGroup
-    badGroupItem ts e =
-      let errorText = T.pack $ show e
-       in CChatItem
-            SMDSnd
-            ChatItem
-              { chatDir = CIGroupSnd,
-                meta = dummyMeta itemId ts errorText,
-                content = CIInvalidJSON errorText,
-                formattedText = Nothing,
-                quotedItem = Nothing,
-                reactions = [],
-                file = Nothing
-              }
-
 getGroupMemberChatItemLast :: DB.Connection -> User -> GroupId -> GroupMemberId -> ExceptT StoreError IO (CChatItem 'CTGroup)
 getGroupMemberChatItemLast db user@User {userId} groupId groupMemberId = do
  chatItemId <-
@@ -1118,8 +1068,7 @@ getGroupChatAfter_ db user@User {userId} g@GroupInfo {groupId} afterChatItemId c
  let stats = ChatStats {unreadCount = 0, minUnreadItemId = 0, unreadChat = False}
  afterChatItem <- getGroupChatItem db user groupId afterChatItemId
  chatItemIds <- liftIO $ getGroupChatItemIdsAfter_ (chatItemTs afterChatItem)
-  currentTs <- liftIO getCurrentTime
-  chatItems <- liftIO $ mapM (safeGetGroupItem db user g currentTs) chatItemIds
+  chatItems <- mapM (getGroupCIWithReactions db user g) chatItemIds
  pure $ Chat (GroupChat g) chatItems stats
  where
    getGroupChatItemIdsAfter_ :: UTCTime -> IO [ChatItemId]
@@ -1142,8 +1091,7 @@ getGroupChatBefore_ db user@User {userId} g@GroupInfo {groupId} beforeChatItemId
  let stats = ChatStats {unreadCount = 0, minUnreadItemId = 0, unreadChat = False}
  beforeChatItem <- getGroupChatItem db user groupId beforeChatItemId
  chatItemIds <- liftIO $ getGroupChatItemIdsBefore_ (chatItemTs beforeChatItem)
-  currentTs <- liftIO getCurrentTime
-  chatItems <- liftIO $ mapM (safeGetGroupItem db user g currentTs) chatItemIds
+  chatItems <- mapM (getGroupCIWithReactions db user g) chatItemIds
  pure $ Chat (GroupChat g) (reverse chatItems) stats
  where
    getGroupChatItemIdsBefore_ :: UTCTime -> IO [ChatItemId]
@@ -1165,17 +1113,16 @@ getLocalChat :: DB.Connection -> User -> Int64 -> ChatPagination -> Maybe String
 getLocalChat db user folderId pagination search_ = do
  let search = fromMaybe "" search_
  nf <- getNoteFolder db user folderId
-  liftIO $ case pagination of
+  case pagination of
    CPLast count -> getLocalChatLast_ db user nf count search
    CPAfter afterId count -> getLocalChatAfter_ db user nf afterId count search
    CPBefore beforeId count -> getLocalChatBefore_ db user nf beforeId count search

-getLocalChatLast_ :: DB.Connection -> User -> NoteFolder -> Int -> String -> IO (Chat 'CTLocal)
+getLocalChatLast_ :: DB.Connection -> User -> NoteFolder -> Int -> String -> ExceptT StoreError IO (Chat 'CTLocal)
 getLocalChatLast_ db user@User {userId} nf@NoteFolder {noteFolderId} count search = do
  let stats = ChatStats {unreadCount = 0, minUnreadItemId = 0, unreadChat = False}
-  chatItemIds <- getLocalChatItemIdsLast_
-  currentTs <- getCurrentTime
-  chatItems <- mapM (safeGetLocalItem db user nf currentTs) chatItemIds
+  chatItemIds <- liftIO getLocalChatItemIdsLast_
+  chatItems <- mapM (getLocalChatItem db user noteFolderId) chatItemIds
  pure $ Chat (LocalChat nf) (reverse chatItems) stats
  where
    getLocalChatItemIdsLast_ :: IO [ChatItemId]
@@ -1192,38 +1139,11 @@ getLocalChatLast_ db user@User {userId} nf@NoteFolder {noteFolderId} count searc
          |]
          (userId, noteFolderId, search, count)

-safeGetLocalItem :: DB.Connection -> User -> NoteFolder -> UTCTime -> ChatItemId -> IO (CChatItem 'CTLocal)
-safeGetLocalItem db user NoteFolder {noteFolderId} currentTs itemId =
-  runExceptT (getLocalChatItem db user noteFolderId itemId)
-    >>= pure <$> safeToLocalItem currentTs itemId
-
-safeToLocalItem :: UTCTime -> ChatItemId -> Either StoreError (CChatItem 'CTLocal) -> CChatItem 'CTLocal
-safeToLocalItem currentTs itemId = \case
-  Right ci -> ci
-  Left e@(SEBadChatItem _ (Just itemTs)) -> badLocalItem itemTs e
-  Left e -> badLocalItem currentTs e
-  where
-    badLocalItem :: UTCTime -> StoreError -> CChatItem 'CTLocal
-    badLocalItem ts e =
-      let errorText = T.pack $ show e
-       in CChatItem
-            SMDSnd
-            ChatItem
-              { chatDir = CILocalSnd,
-                meta = dummyMeta itemId ts errorText,
-                content = CIInvalidJSON errorText,
-                formattedText = Nothing,
-                quotedItem = Nothing,
-                reactions = [],
-                file = Nothing
-              }
-
-getLocalChatAfter_ :: DB.Connection -> User -> NoteFolder -> ChatItemId -> Int -> String -> IO (Chat 'CTLocal)
+getLocalChatAfter_ :: DB.Connection -> User -> NoteFolder -> ChatItemId -> Int -> String -> ExceptT StoreError IO (Chat 'CTLocal)
 getLocalChatAfter_ db user@User {userId} nf@NoteFolder {noteFolderId} afterChatItemId count search = do
  let stats = ChatStats {unreadCount = 0, minUnreadItemId = 0, unreadChat = False}
-  chatItemIds <- getLocalChatItemIdsAfter_
-  currentTs <- getCurrentTime
-  chatItems <- mapM (safeGetLocalItem db user nf currentTs) chatItemIds
+  chatItemIds <- liftIO getLocalChatItemIdsAfter_
+  chatItems <- mapM (getLocalChatItem db user noteFolderId) chatItemIds
  pure $ Chat (LocalChat nf) chatItems stats
  where
    getLocalChatItemIdsAfter_ :: IO [ChatItemId]
@@ -1241,12 +1161,11 @@ getLocalChatAfter_ db user@User {userId} nf@NoteFolder {noteFolderId} afterChatI
          |]
          (userId, noteFolderId, search, afterChatItemId, count)

-getLocalChatBefore_ :: DB.Connection -> User -> NoteFolder -> ChatItemId -> Int -> String -> IO (Chat 'CTLocal)
+getLocalChatBefore_ :: DB.Connection -> User -> NoteFolder -> ChatItemId -> Int -> String -> ExceptT StoreError IO (Chat 'CTLocal)
 getLocalChatBefore_ db user@User {userId} nf@NoteFolder {noteFolderId} beforeChatItemId count search = do
  let stats = ChatStats {unreadCount = 0, minUnreadItemId = 0, unreadChat = False}
-  chatItemIds <- getLocalChatItemIdsBefore_
-  currentTs <- getCurrentTime
-  chatItems <- mapM (safeGetLocalItem db user nf currentTs) chatItemIds
+  chatItemIds <- liftIO getLocalChatItemIdsBefore_
+  chatItems <- mapM (getLocalChatItem db user noteFolderId) chatItemIds
  pure $ Chat (LocalChat nf) (reverse chatItems) stats
  where
    getLocalChatItemIdsBefore_ :: IO [ChatItemId]
@@ -1269,7 +1188,7 @@ toChatItemRef = \case
  (itemId, Just contactId, Nothing, Nothing) -> Right (ChatRef CTDirect contactId, itemId)
  (itemId, Nothing, Just groupId, Nothing) -> Right (ChatRef CTGroup groupId, itemId)
  (itemId, Nothing, Nothing, Just folderId) -> Right (ChatRef CTLocal folderId, itemId)
-  (itemId, _, _, _) -> Left $ SEBadChatItem itemId Nothing
+  (itemId, _, _, _) -> Left $ SEBadChatItem itemId

 updateDirectChatItemsRead :: DB.Connection -> User -> ContactId -> Maybe (ChatItemId, ChatItemId) -> IO ()
 updateDirectChatItemsRead db User {userId} contactId itemsRange_ = do
@@ -1442,7 +1361,7 @@ toDirectChatItem currentTs (((itemId, itemTs, AMsgDirection msgDir, itemContentT
    cItem :: MsgDirectionI d => SMsgDirection d -> CIDirection 'CTDirect d -> CIStatus d -> CIContent d -> Maybe (CIFile d) -> CChatItem 'CTDirect
    cItem d chatDir ciStatus content file =
      CChatItem d ChatItem {chatDir, meta = ciMeta content ciStatus, content, formattedText = parseMaybeMarkdownList itemText, quotedItem = toDirectQuote quoteRow, reactions = [], file}
-    badItem = Left $ SEBadChatItem itemId (Just itemTs)
+    badItem = Left $ SEBadChatItem itemId
    ciMeta :: CIContent d -> CIStatus d -> CIMeta 'CTDirect d
    ciMeta content status =
      let itemDeleted' = case itemDeleted of
@@ -1493,7 +1412,7 @@ toGroupChatItem currentTs userContactId (((itemId, itemTs, AMsgDirection msgDir,
    cItem :: MsgDirectionI d => SMsgDirection d -> CIDirection 'CTGroup d -> CIStatus d -> CIContent d -> Maybe (CIFile d) -> CChatItem 'CTGroup
    cItem d chatDir ciStatus content file =
      CChatItem d ChatItem {chatDir, meta = ciMeta content ciStatus, content, formattedText = parseMaybeMarkdownList itemText, quotedItem = toGroupQuote quoteRow quotedMember_, reactions = [], file}
-    badItem = Left $ SEBadChatItem itemId (Just itemTs)
+    badItem = Left $ SEBadChatItem itemId
    ciMeta :: CIContent d -> CIStatus d -> CIMeta 'CTGroup d
    ciMeta content status =
      let itemDeleted' = case itemDeleted of
@@ -2166,12 +2085,6 @@ getChatItemByFileId db vr user@User {userId} fileId = do
        (userId, fileId)
  getAChatItem db vr user chatRef itemId

-lookupChatItemByFileId :: DB.Connection -> VersionRange -> User -> Int64 -> ExceptT StoreError IO (Maybe AChatItem)
-lookupChatItemByFileId db vr user fileId = do
-  fmap Just (getChatItemByFileId db vr user fileId) `catchError` \case
-    SEChatItemNotFoundByFileId {} -> pure Nothing
-    e -> throwError e
-
 getChatItemByGroupId :: DB.Connection -> VersionRange -> User -> GroupId -> ExceptT StoreError IO AChatItem
 getChatItemByGroupId db vr user@User {userId} groupId = do
  (chatRef, itemId) <-
@@ -2196,7 +2109,7 @@ getChatRefViaItemId db User {userId} itemId = do
    toChatRef = \case
      (Just contactId, Nothing) -> Right $ ChatRef CTDirect contactId
      (Nothing, Just groupId) -> Right $ ChatRef CTGroup groupId
-      (_, _) -> Left $ SEBadChatItem itemId Nothing
+      (_, _) -> Left $ SEBadChatItem itemId

 getAChatItem :: DB.Connection -> VersionRange -> User -> ChatRef -> ChatItemId -> ExceptT StoreError IO AChatItem
 getAChatItem db vr user chatRef itemId = case chatRef of
@@ -2232,6 +2145,11 @@ getChatItemVersions db itemId = do
      let formattedText = parseMaybeMarkdownList $ msgContentText msgContent
       in ChatItemVersion {chatItemVersionId, msgContent, formattedText, itemVersionTs, createdAt}

+getDirectChatReactions_ :: DB.Connection -> Contact -> Chat 'CTDirect -> IO (Chat 'CTDirect)
+getDirectChatReactions_ db ct c@Chat {chatItems} = do
+  chatItems' <- mapM (directCIWithReactions db ct) chatItems
+  pure c {chatItems = chatItems'}
+
 directCIWithReactions :: DB.Connection -> Contact -> CChatItem 'CTDirect -> IO (CChatItem 'CTDirect)
 directCIWithReactions db ct cci@(CChatItem md ci@ChatItem {meta = CIMeta {itemSharedMsgId}}) = case itemSharedMsgId of
  Just sharedMsgId -> do
--- a/src/Simplex/Chat/Store/Migrations.hs
+++ b/src/Simplex/Chat/Store/Migrations.hs
@@ -98,7 +98,6 @@ import Simplex.Chat.Migrations.M20240102_note_folders
 import Simplex.Chat.Migrations.M20240104_members_profile_update
 import Simplex.Chat.Migrations.M20240115_block_member_for_all
 import Simplex.Chat.Migrations.M20240122_indexes
-import Simplex.Chat.Migrations.M20240214_redirect_file_id
 import Simplex.Messaging.Agent.Store.SQLite.Migrations (Migration (..))

 schemaMigrations :: [(String, Query, Maybe Query)]
@@ -196,8 +195,7 @@ schemaMigrations =
    ("20240102_note_folders", m20240102_note_folders, Just down_m20240102_note_folders),
    ("20240104_members_profile_update", m20240104_members_profile_update, Just down_m20240104_members_profile_update),
    ("20240115_block_member_for_all", m20240115_block_member_for_all, Just down_m20240115_block_member_for_all),
-    ("20240122_indexes", m20240122_indexes, Just down_m20240122_indexes),
-    ("20240214_redirect_file_id", m20240214_redirect_file_id, Just down_m20240214_redirect_file_id)
+    ("20240122_indexes", m20240122_indexes, Just down_m20240122_indexes)
  ]

 -- | The list of migrations in ascending order by date
--- a/src/Simplex/Chat/Store/Profiles.hs
+++ b/src/Simplex/Chat/Store/Profiles.hs
@@ -388,6 +388,7 @@ deleteUserAddress db user@User {userId} = do
          JOIN user_contact_links uc USING (user_contact_link_id)
          WHERE uc.user_id = :user_id AND uc.local_display_name = '' AND uc.group_id IS NULL
        )
+        AND local_display_name NOT IN (SELECT local_display_name FROM users)
    |]
    [":user_id" := userId]
  DB.executeNamed
--- a/src/Simplex/Chat/Store/Shared.hs
+++ b/src/Simplex/Chat/Store/Shared.hs
@@ -92,12 +92,11 @@ data StoreError
  | SEUniqueID
  | SELargeMsg
  | SEInternalError {message :: String}
-  | SEBadChatItem {itemId :: ChatItemId, itemTs :: Maybe ChatItemTs}
+  | SEBadChatItem {itemId :: ChatItemId}
  | SEChatItemNotFound {itemId :: ChatItemId}
  | SEChatItemNotFoundByText {text :: Text}
  | SEChatItemSharedMsgIdNotFound {sharedMsgId :: SharedMsgId}
  | SEChatItemNotFoundByFileId {fileId :: FileTransferId}
-  | SEChatItemNotFoundByContactId {contactId :: ContactId}
  | SEChatItemNotFoundByGroupId {groupId :: GroupId}
  | SEProfileNotFound {profileId :: Int64}
  | SEDuplicateGroupLink {groupInfo :: GroupInfo}
--- a/src/Simplex/Chat/Types.hs
+++ b/src/Simplex/Chat/Types.hs
@@ -1210,7 +1210,6 @@ data FileTransfer
 data FileTransferMeta = FileTransferMeta
  { fileId :: FileTransferId,
    xftpSndFile :: Maybe XFTPSndFile,
-    xftpRedirectFor :: Maybe FileTransferId,
    fileName :: String,
    filePath :: String,
    fileSize :: Integer,
--- a/src/Simplex/Chat/View.hs
+++ b/src/Simplex/Chat/View.hs
@@ -198,24 +198,17 @@ responseToView hu@(currentRH, user_) ChatConfig {logLevel, showReactions, showRe
  CRGroupMemberUpdated {} -> []
  CRContactsMerged u intoCt mergedCt ct' -> ttyUser u $ viewContactsMerged intoCt mergedCt ct'
  CRReceivedContactRequest u UserContactRequest {localDisplayName = c, profile} -> ttyUser u $ viewReceivedContactRequest c profile
-  CRRcvStandaloneFileCreated u ft -> ttyUser u $ receivingFileStandalone "started" ft
  CRRcvFileStart u ci -> ttyUser u $ receivingFile_' hu testView "started" ci
  CRRcvFileComplete u ci -> ttyUser u $ receivingFile_' hu testView "completed" ci
-  CRRcvStandaloneFileComplete u _ ft -> ttyUser u $ receivingFileStandalone "completed" ft
  CRRcvFileSndCancelled u _ ft -> ttyUser u $ viewRcvFileSndCancelled ft
-  CRRcvFileError u (Just ci) e _ -> ttyUser u $ receivingFile_' hu testView "error" ci <> [sShow e]
-  CRRcvFileError u Nothing e ft -> ttyUser u $ receivingFileStandalone "error" ft <> [sShow e]
+  CRRcvFileError u ci e -> ttyUser u $ receivingFile_' hu testView "error" ci <> [sShow e]
  CRSndFileStart u _ ft -> ttyUser u $ sendingFile_ "started" ft
  CRSndFileComplete u _ ft -> ttyUser u $ sendingFile_ "completed" ft
-  CRSndStandaloneFileCreated u ft -> ttyUser u $ uploadingFileStandalone "started" ft
  CRSndFileStartXFTP {} -> []
  CRSndFileProgressXFTP {} -> []
-  CRSndFileRedirectStartXFTP u ft ftRedirect -> ttyUser u $ standaloneUploadRedirect ft ftRedirect
-  CRSndStandaloneFileComplete u ft uris -> ttyUser u $ standaloneUploadComplete ft uris
  CRSndFileCompleteXFTP u ci _ -> ttyUser u $ uploadingFile "completed" ci
  CRSndFileCancelledXFTP {} -> []
-  CRSndFileError u Nothing ft -> ttyUser u $ uploadingFileStandalone "error" ft
-  CRSndFileError u (Just ci) _ -> ttyUser u $ uploadingFile "error" ci
+  CRSndFileError u ci -> ttyUser u $ uploadingFile "error" ci
  CRSndFileRcvCancelled u _ ft@SndFileTransfer {recipientDisplayName = c} ->
    ttyUser u [ttyContact c <> " cancelled receiving " <> sndFile ft]
  CRContactConnecting u _ -> ttyUser u []
@@ -1565,26 +1558,11 @@ sendingFile_ status ft@SndFileTransfer {recipientDisplayName = c} =
  [status <> " sending " <> sndFile ft <> " to " <> ttyContact c]

 uploadingFile :: StyledString -> AChatItem -> [StyledString]
-uploadingFile status = \case
-  AChatItem _ _ (DirectChat Contact {localDisplayName = c}) ChatItem {file = Just CIFile {fileId, fileName}, chatDir = CIDirectSnd} ->
-    [status <> " uploading " <> fileTransferStr fileId fileName <> " for " <> ttyContact c]
-  AChatItem _ _ (GroupChat g) ChatItem {file = Just CIFile {fileId, fileName}, chatDir = CIGroupSnd} ->
-    [status <> " uploading " <> fileTransferStr fileId fileName <> " for " <> ttyGroup' g]
-  _ -> [status <> " uploading file"]
-
-uploadingFileStandalone :: StyledString -> FileTransferMeta -> [StyledString]
-uploadingFileStandalone status FileTransferMeta {fileId, fileName} = [status <> " standalone uploading " <> fileTransferStr fileId fileName]
-
-standaloneUploadRedirect :: FileTransferMeta -> FileTransferMeta -> [StyledString]
-standaloneUploadRedirect FileTransferMeta {fileId, fileName} FileTransferMeta {fileId = redirectId} =
-  [fileTransferStr fileId fileName <> " uploaded, preparing redirect file " <> sShow redirectId]
-
-standaloneUploadComplete :: FileTransferMeta -> [Text] -> [StyledString]
-standaloneUploadComplete FileTransferMeta {fileId, fileName} = \case
-  [] -> [fileTransferStr fileId fileName <> " upload complete."]
-  uris ->
-    fileTransferStr fileId fileName <> " upload complete. download with:"
-    : map plain uris
+uploadingFile status (AChatItem _ _ (DirectChat Contact {localDisplayName = c}) ChatItem {file = Just CIFile {fileId, fileName}, chatDir = CIDirectSnd}) =
+  [status <> " uploading " <> fileTransferStr fileId fileName <> " for " <> ttyContact c]
+uploadingFile status (AChatItem _ _ (GroupChat g) ChatItem {file = Just CIFile {fileId, fileName}, chatDir = CIGroupSnd}) =
+  [status <> " uploading " <> fileTransferStr fileId fileName <> " for " <> ttyGroup' g]
+uploadingFile status _ = [status <> " uploading file"] -- shouldn't happen

 sndFile :: SndFileTransfer -> StyledString
 sndFile SndFileTransfer {fileId, fileName} = fileTransferStr fileId fileName
@@ -1630,11 +1608,7 @@ receivingFile_' hu testView status (AChatItem _ _ chat ChatItem {file = Just CIF
              highlight ("/get remote file " <> show rhId <> " " <> LB.unpack (J.encode RemoteFile {userId, fileId, sent = False, fileSource = f}))
            ]
      _ -> []
-receivingFile_' _ _ status _ = [plain status <> " receiving file"]
-
-receivingFileStandalone :: String -> RcvFileTransfer -> [StyledString]
-receivingFileStandalone status RcvFileTransfer {fileId, fileInvitation = FileInvitation {fileName}} =
-  [plain status <> " standalone receiving " <> fileTransferStr fileId fileName]
+receivingFile_' _ _ status _ = [plain status <> " receiving file"] -- shouldn't happen

 viewLocalFile :: StyledString -> CIFile d -> CurrentTime -> TimeZone -> CIMeta c d -> [StyledString]
 viewLocalFile to CIFile {fileId, fileSource} ts tz = case fileSource of
@@ -1653,7 +1627,7 @@ fileFrom _ _ = ""

 receivingFile_ :: StyledString -> RcvFileTransfer -> [StyledString]
 receivingFile_ status ft@RcvFileTransfer {senderDisplayName = c} =
-  [status <> " receiving " <> rcvFile ft <> if c == "" then "" else " from " <> ttyContact c]
+  [status <> " receiving " <> rcvFile ft <> " from " <> ttyContact c]

 rcvFile :: RcvFileTransfer -> StyledString
 rcvFile RcvFileTransfer {fileId, fileInvitation = FileInvitation {fileName}} = fileTransferStr fileId fileName
--- a/tests/ChatClient.hs
+++ b/tests/ChatClient.hs
@@ -15,6 +15,7 @@ import Control.Concurrent.STM
 import Control.Exception (bracket, bracket_)
 import Control.Monad
 import Control.Monad.Except
+import Data.ByteArray (ScrubbedBytes)
 import Data.Functor (($>))
 import Data.List (dropWhileEnd, find)
 import Data.Maybe (isNothing)
@@ -37,7 +38,6 @@ import Simplex.Messaging.Agent.RetryInterval
 import Simplex.Messaging.Agent.Store.SQLite (MigrationConfirmation (..))
 import qualified Simplex.Messaging.Agent.Store.SQLite.DB as DB
 import Simplex.Messaging.Client (ProtocolClientConfig (..), defaultNetworkConfig)
-import Simplex.Messaging.Crypto.Memory (LockedBytes)
 import Simplex.Messaging.Server (runSMPServerBlocking)
 import Simplex.Messaging.Server.Env.STM
 import Simplex.Messaging.Transport
@@ -92,7 +92,7 @@ testCoreOpts =
      highlyAvailable = False
    }

-getTestOpts :: Bool -> LockedBytes -> ChatOpts
+getTestOpts :: Bool -> ScrubbedBytes -> ChatOpts
 getTestOpts maintenance dbKey = testOpts {maintenance, coreOptions = testCoreOpts {dbKey}}

 termSettings :: VirtualTerminalSettings
--- a/tests/ChatTests/Files.hs
+++ b/tests/ChatTests/Files.hs
@@ -9,7 +9,6 @@ import ChatClient
 import ChatTests.Utils
 import Control.Concurrent (threadDelay)
 import Control.Concurrent.Async (concurrently_)
-import Control.Logger.Simple
 import qualified Data.Aeson as J
 import qualified Data.ByteString.Char8 as B
 import qualified Data.ByteString.Lazy.Char8 as LB
@@ -78,11 +77,6 @@ chatFileTests = do
    it "cancel receiving file, repeat receive" testXFTPCancelRcvRepeat
    it "should accept file automatically with CLI option" testAutoAcceptFile
    it "should prohibit file transfers in groups based on preference" testProhibitFiles
-  describe "file transfer over XFTP without chat items" $ do
-    it "send and receive small standalone file" testXFTPStandaloneSmall
-    it "send and receive large standalone file" testXFTPStandaloneLarge
-    xit "removes sent file from server" testXFTPStandaloneCancelSnd -- no error shown in tests
-    it "removes received temporary files" testXFTPStandaloneCancelRcv

 runTestFileTransfer :: HasCallStack => TestCC -> TestCC -> IO ()
 runTestFileTransfer alice bob = do
@@ -1551,116 +1545,6 @@ testProhibitFiles =
  where
    cfg = testCfg {xftpFileConfig = Just $ XFTPFileConfig {minFileSize = 0}, tempDir = Just "./tests/tmp"}

-testXFTPStandaloneSmall :: HasCallStack => FilePath -> IO ()
-testXFTPStandaloneSmall = testChat2 aliceProfile aliceDesktopProfile $ \src dst -> do
-  withXFTPServer $ do
-    logNote "sending"
-    src ##> "/_upload 1 ./tests/fixtures/test.jpg"
-    src <## "started standalone uploading file 1 (test.jpg)"
-    -- silent progress events
-    threadDelay 250000
-    src <## "file 1 (test.jpg) upload complete. download with:"
-    -- file description fits, enjoy the direct URIs
-    _uri1 <- getTermLine src
-    _uri2 <- getTermLine src
-    uri3 <- getTermLine src
-    _uri4 <- getTermLine src
-
-    logNote "receiving"
-    let dstFile = "./tests/tmp/test.jpg"
-    dst ##> ("/_download 1 " <> uri3 <> " " <> dstFile)
-    dst <## "started standalone receiving file 1 (test.jpg)"
-    -- silent progress events
-    threadDelay 250000
-    dst <## "completed standalone receiving file 1 (test.jpg)"
-    srcBody <- B.readFile "./tests/fixtures/test.jpg"
-    B.readFile dstFile `shouldReturn` srcBody
-
-testXFTPStandaloneLarge :: HasCallStack => FilePath -> IO ()
-testXFTPStandaloneLarge = testChat2 aliceProfile aliceDesktopProfile $ \src dst -> do
-  withXFTPServer $ do
-    xftpCLI ["rand", "./tests/tmp/testfile.in", "17mb"] `shouldReturn` ["File created: " <> "./tests/tmp/testfile.in"]
-
-    logNote "sending"
-    src ##> "/_upload 1 ./tests/tmp/testfile.in"
-    src <## "started standalone uploading file 1 (testfile.in)"
-    -- silent progress events
-    threadDelay 250000
-    src <## "file 1 (testfile.in) uploaded, preparing redirect file 2"
-    src <## "file 1 (testfile.in) upload complete. download with:"
-    uri <- getTermLine src
-    _uri2 <- getTermLine src
-    _uri3 <- getTermLine src
-    _uri4 <- getTermLine src
-
-    logNote "receiving"
-    let dstFile = "./tests/tmp/testfile.out"
-    dst ##> ("/_download 1 " <> uri <> " " <> dstFile)
-    dst <## "started standalone receiving file 1 (testfile.out)"
-    -- silent progress events
-    threadDelay 250000
-    dst <## "completed standalone receiving file 1 (testfile.out)"
-    srcBody <- B.readFile "./tests/tmp/testfile.in"
-    B.readFile dstFile `shouldReturn` srcBody
-
-testXFTPStandaloneCancelSnd :: HasCallStack => FilePath -> IO ()
-testXFTPStandaloneCancelSnd = testChat2 aliceProfile aliceDesktopProfile $ \src dst -> do
-  withXFTPServer $ do
-    xftpCLI ["rand", "./tests/tmp/testfile.in", "17mb"] `shouldReturn` ["File created: " <> "./tests/tmp/testfile.in"]
-
-    logNote "sending"
-    src ##> "/_upload 1 ./tests/tmp/testfile.in"
-    src <## "started standalone uploading file 1 (testfile.in)"
-    -- silent progress events
-    threadDelay 250000
-    src <## "file 1 (testfile.in) uploaded, preparing redirect file 2"
-    src <## "file 1 (testfile.in) upload complete. download with:"
-    uri <- getTermLine src
-    _uri2 <- getTermLine src
-    _uri3 <- getTermLine src
-    _uri4 <- getTermLine src
-
-    logNote "cancelling"
-    src ##> "/fc 1"
-    src <## "cancelled sending file 1 (testfile.in)"
-    threadDelay 1000000
-
-    logNote "trying to receive cancelled"
-    dst ##> ("/_download 1 " <> uri <> " " <> "./tests/tmp/should.not.extist")
-    dst <## "started standalone receiving file 1 (should.not.extist)"
-    threadDelay 100000
-    logWarn "no error?"
-    dst <## "error receiving file 1 (should.not.extist)"
-    dst <## "INTERNAL {internalErr = \"XFTP {xftpErr = AUTH}\"}"
-
-testXFTPStandaloneCancelRcv :: HasCallStack => FilePath -> IO ()
-testXFTPStandaloneCancelRcv = testChat2 aliceProfile aliceDesktopProfile $ \src dst -> do
-  withXFTPServer $ do
-    xftpCLI ["rand", "./tests/tmp/testfile.in", "17mb"] `shouldReturn` ["File created: " <> "./tests/tmp/testfile.in"]
-
-    logNote "sending"
-    src ##> "/_upload 1 ./tests/tmp/testfile.in"
-    src <## "started standalone uploading file 1 (testfile.in)"
-    -- silent progress events
-    threadDelay 250000
-    src <## "file 1 (testfile.in) uploaded, preparing redirect file 2"
-    src <## "file 1 (testfile.in) upload complete. download with:"
-    uri <- getTermLine src
-    _uri2 <- getTermLine src
-    _uri3 <- getTermLine src
-    _uri4 <- getTermLine src
-
-    logNote "receiving"
-    let dstFile = "./tests/tmp/testfile.out"
-    dst ##> ("/_download 1 " <> uri <> " " <> dstFile)
-    dst <## "started standalone receiving file 1 (testfile.out)"
-    threadDelay 25000 -- give workers some time to avoid internal errors from starting tasks
-    logNote "cancelling"
-    dst ##> "/fc 1"
-    dst <## "cancelled receiving file 1 (testfile.out)"
-    threadDelay 25000
-    doesFileExist dstFile `shouldReturn` False
-
 startFileTransfer :: HasCallStack => TestCC -> TestCC -> IO ()
 startFileTransfer alice bob =
  startFileTransfer' alice bob "test.jpg" "136.5 KiB / 139737 bytes"
--- a/tests/ChatTests/Local.hs
+++ b/tests/ChatTests/Local.hs
@@ -152,7 +152,7 @@ testFiles tmp = withNewTestChat tmp "alice" aliceProfile $ \alice -> do

  alice ##> "/clear *"
  alice ##> "/fs 1"
-  alice <## "file 1 not found"
+  alice <## "chat db error: SEChatItemNotFoundByFileId {fileId = 1}"
  alice ##> "/tail"
  doesFileExist stored `shouldReturn` False
Author	SHA1	Message	Date
spaced4ndy	1603309e60	check display name	2024-02-19 12:28:04 +04:00
spaced4ndy	203d793cf0	filter on contact deletion	2024-02-19 12:05:40 +04:00
spaced4ndy	acf6519e23	core: filter out user contact on merge	2024-02-19 11:37:36 +04:00