Skip to main content

matrix_sdk/send_queue/
mod.rs

1// Copyright 2024 The Matrix.org Foundation C.I.C.
2//
3// Licensed under the Apache License, Version 2.0 (the "License");
4// you may not use this file except in compliance with the License.
5// You may obtain a copy of the License at
6//
7//     http://www.apache.org/licenses/LICENSE-2.0
8//
9// Unless required by applicable law or agreed to in writing, software
10// distributed under the License is distributed on an "AS IS" BASIS,
11// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12// See the License for the specific language governing permissions and
13// limitations under the License.
14
15//! A send queue facility to serializing queuing and sending of messages.
16//!
17//! # [`Room`] send queue
18//!
19//! Each room gets its own [`RoomSendQueue`], that's available by calling
20//! [`Room::send_queue()`]. The first time this method is called, it will spawn
21//! a background task that's used to actually send events, in the order they
22//! were passed from calls to [`RoomSendQueue::send()`].
23//!
24//! This queue tries to simplify error management around sending events, using
25//! [`RoomSendQueue::send`] or [`RoomSendQueue::send_raw`]: by default, it will retry to send the
26//! same event a few times, before automatically disabling itself, and emitting
27//! a notification that can be listened to with the global send queue (see
28//! paragraph below) or using [`RoomSendQueue::subscribe()`].
29//!
30//! Requests are sent in the order they were queued. A request that failed with
31//! an unrecoverable error is marked as "wedged", and blocks all the requests
32//! queued after it (in the same room) from being sent, so events are never sent
33//! out of order; the queue resumes when the wedged request is retried (with
34//! [`SendHandle::unwedge`]) or removed (with [`SendHandle::abort`]).
35//!
36//! It is possible to control whether a single room is enabled using
37//! [`RoomSendQueue::set_enabled()`].
38//!
39//! # Global [`SendQueue`] object
40//!
41//! The [`Client::send_queue()`] method returns an API object allowing to
42//! control all the room send queues:
43//!
44//! - enable/disable them all at once with [`SendQueue::set_enabled()`].
45//! - get notifications about send errors with [`SendQueue::subscribe_errors`].
46//! - reload all unsent events that had been persisted in storage using
47//!   [`SendQueue::respawn_tasks_for_rooms_with_unsent_requests()`]. It is
48//!   recommended to call this method during initialization of a client,
49//!   otherwise persisted unsent events will only be re-sent after the send
50//!   queue for the given room has been reopened for the first time.
51//!
52//! # Send handle
53//!
54//! Just after queuing a request to send something, a [`SendHandle`] is
55//! returned, allowing manipulating the inflight request.
56//!
57//! For a send handle for an event, it's possible to edit the event / abort
58//! sending it. If it was still in the queue (i.e. not sent yet, or not being
59//! sent), then such an action would happen locally (i.e. in the database).
60//! Otherwise, it is "too late": the background task may be sending
61//! the event already, or has sent it; in that case, the edit/aborting must
62//! happen as an actual event materializing this, on the server. To accomplish
63//! this, the send queue may send such an event, using the dependency system
64//! described below.
65//!
66//! # Dependency system
67//!
68//! The send queue includes a simple dependency system, where a
69//! [`QueuedRequest`] can have zero or more dependents in the form of
70//! [`DependentQueuedRequest`]. A dependent queued request can have at most one
71//! depended-upon (parent) queued request.
72//!
73//! This allows implementing deferred edits/redacts, as hinted to in the
74//! previous section.
75//!
76//! ## Media upload
77//!
78//! This dependency system also allows uploading medias, since the media's
79//! *content* must be uploaded before we send the media *event* that describes
80//! it.
81//!
82//! In the simplest case, that is, a media file and its event must be sent (i.e.
83//! no thumbnails):
84//!
85//! - The file's content is immediately cached in the
86//!   [`matrix_sdk_base::event_cache::store::EventCacheStore`], using an MXC ID
87//!   that is temporary and designates a local URI without any possible doubt.
88//! - An initial media event is created and uses this temporary MXC ID, and
89//!   propagated as a local echo for an event.
90//! - A [`QueuedRequest`] is pushed to upload the file's media
91//!   ([`QueuedRequestKind::MediaUpload`]).
92//! - A [`DependentQueuedRequest`] is pushed to finish the upload
93//!   ([`DependentQueuedRequestKind::FinishUpload`]).
94//!
95//! What is expected to happen, if all goes well, is the following:
96//!
97//! - the media is uploaded to the media homeserver, which returns the final MXC
98//!   ID.
99//! - when marking the upload request as sent, the MXC ID is injected (as a
100//!   [`matrix_sdk_base::store::SentRequestKey`]) into the dependent request
101//!   [`DependentQueuedRequestKind::FinishUpload`] created in the last step
102//!   above.
103//! - next time the send queue handles dependent queries, it'll see this one is
104//!   ready to be sent, and it will transform it into an event queued request
105//!   ([`QueuedRequestKind::Event`]), with the event created in the local echo
106//!   before, updated with the MXC ID returned from the server.
107//! - this updated local echo is also propagated as an edit of the local echo to
108//!   observers, who get the final version with the final MXC IDs at this point
109//!   too.
110//! - then the event is sent normally, as any event sent with the send queue.
111//!
112//! When there is a thumbnail, things behave similarly, with some tweaks:
113//!
114//! - the thumbnail's content is also stored into the cache store immediately,
115//! - the thumbnail is sent first as an [`QueuedRequestKind::MediaUpload`]
116//!   request,
117//! - the file upload is pushed as a dependent request of kind
118//!   [`DependentQueuedRequestKind::UploadFileOrThumbnail`] (this variant keeps
119//!   the file's key used to look it up in the cache store).
120//! - the media event is then sent as a dependent request as described in the
121//!   previous section.
122//!
123//! What's expected to happen is thus the following:
124//!
125//! - After the thumbnail has been uploaded, the dependent query will retrieve
126//!   the final MXC ID returned by the homeserver for the thumbnail, and store
127//!   it into the [`QueuedRequestKind::MediaUpload`]'s `thumbnail_source` field,
128//!   allowing to remember the thumbnail MXC ID when it's time to finish the
129//!   upload later.
130//! - The dependent request is morphed into another
131//!   [`QueuedRequestKind::MediaUpload`], for the file itself.
132//!
133//! The rest of the process is then similar to that of uploading a file without
134//! a thumbnail. The only difference is that there's a thumbnail source (MXC ID)
135//! remembered and fixed up into the media event, just before sending it.
136
137use std::{
138    collections::{BTreeMap, HashMap},
139    future::IntoFuture,
140    ops::Not,
141    str::FromStr as _,
142    sync::{
143        Arc, OnceLock, RwLock,
144        atomic::{AtomicBool, Ordering},
145    },
146    time::Duration,
147};
148
149use eyeball::SharedObservable;
150#[cfg(feature = "e2e-encryption")]
151use matrix_sdk_base::crypto::{OlmError, SessionRecipientCollectionError};
152#[cfg(feature = "unstable-msc4274")]
153use matrix_sdk_base::store::FinishGalleryItemInfo;
154use matrix_sdk_base::{
155    RoomState, StoreError,
156    cross_process_lock::CrossProcessLockError,
157    deserialized_responses::{EncryptionInfo, TimelineEvent},
158    event_cache::store::EventCacheStoreError,
159    media::{MediaRequestParameters, store::MediaStoreError},
160    store::{
161        ChildTransactionId, DependentQueuedRequest, DependentQueuedRequestKind, DynStateStore,
162        FinishUploadThumbnailInfo, QueueWedgeError, QueuedRequest, QueuedRequestKind,
163        SentMediaInfo, SentRequestKey, SerializableEventContent,
164    },
165    task_monitor::BackgroundTaskHandle,
166};
167use matrix_sdk_common::{boxed_into_future, locks::Mutex as SyncMutex};
168use mime::Mime;
169#[cfg(feature = "unstable-msc4354")]
170use ruma::events::sticky::StickyDurationMs;
171use ruma::{
172    MilliSecondsSinceUnixEpoch, OwnedEventId, OwnedRoomId, OwnedTransactionId, RoomId,
173    TransactionId,
174    api::error::ErrorKind,
175    events::{
176        AnyMessageLikeEventContent, Mentions, MessageLikeEventContent as _, TimelineEventType,
177        reaction::ReactionEventContent,
178        relation::Annotation,
179        room::{
180            MediaSource,
181            message::{FormattedBody, RoomMessageEventContent},
182        },
183    },
184    serde::Raw,
185};
186use tokio::sync::{Mutex, Notify, OwnedMutexGuard, broadcast, oneshot};
187use tracing::{debug, error, info, instrument, trace, warn};
188
189#[cfg(feature = "unstable-msc4354")]
190use crate::utils::sticky_duration_ms;
191use crate::{
192    Client, Media, Room, SessionChange, TransmissionProgress,
193    client::WeakClient,
194    config::RequestConfig,
195    error::RetryKind,
196    room::{WeakRoom, edit::EditedContent},
197};
198
199mod progress;
200mod upload;
201
202pub use progress::AbstractProgress;
203
204/// How long to wait before retrying, when loading the next request to send
205/// failed against the state store.
206const STORE_ERROR_BACKOFF: Duration = Duration::from_millis(250);
207
208/// A client-wide send queue, for all the rooms known by a client.
209pub struct SendQueue {
210    client: Client,
211}
212
213#[cfg(not(tarpaulin_include))]
214impl std::fmt::Debug for SendQueue {
215    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
216        f.debug_struct("SendQueue").finish_non_exhaustive()
217    }
218}
219
220impl SendQueue {
221    pub(super) fn new(client: Client) -> Self {
222        Self { client }
223    }
224
225    /// Reload all the rooms which had unsent requests, and respawn tasks for
226    /// those rooms.
227    pub async fn respawn_tasks_for_rooms_with_unsent_requests(&self) {
228        if !self.is_enabled() {
229            return;
230        }
231
232        let room_ids =
233            self.client.state_store().load_rooms_with_unsent_requests().await.unwrap_or_else(
234                |err| {
235                    warn!("error when loading rooms with unsent requests: {err}");
236                    Vec::new()
237                },
238            );
239
240        // Getting the [`RoomSendQueue`] is sufficient to spawn the task if
241        // needs be.
242        for room_id in room_ids {
243            if let Some(room) = self.client.get_room(&room_id) {
244                let _ = self.for_room(room);
245            }
246        }
247    }
248
249    /// Watches for the session getting a working access token again and
250    /// unwedges the requests that were wedged because it didn't have one.
251    async fn session_change_task(client: WeakClient) {
252        let Some(strong_client) = client.get() else { return };
253        let mut session_changes = strong_client.subscribe_to_session_changes();
254        // Don't keep the client alive just by listening.
255        drop(strong_client);
256
257        loop {
258            match session_changes.recv().await {
259                Ok(SessionChange::TokensRefreshed) => {
260                    let Some(client) = client.get() else { break };
261                    client.send_queue().unwedge_requests_with_expired_token().await;
262                }
263
264                Ok(SessionChange::UnknownToken(_)) => {}
265
266                // A missed `TokensRefreshed` would leave requests wedged for good, while
267                // a spurious unwedge only costs a retry that gets wedged again.
268                Err(broadcast::error::RecvError::Lagged(num)) => {
269                    warn!(num, "missed some session changes, unwedging just in case");
270                    let Some(client) = client.get() else { break };
271                    client.send_queue().unwedge_requests_with_expired_token().await;
272                }
273
274                Err(broadcast::error::RecvError::Closed) => break,
275            }
276        }
277    }
278
279    /// Unwedge every request that was wedged because the access token had been
280    /// rejected, in every room that has unsent requests.
281    async fn unwedge_requests_with_expired_token(&self) {
282        let room_ids = match self.client.state_store().load_rooms_with_unsent_requests().await {
283            Ok(room_ids) => room_ids,
284            Err(err) => {
285                warn!("error when loading rooms with unsent requests: {err}");
286                return;
287            }
288        };
289
290        for room_id in room_ids {
291            let Some(room) = self.client.get_room(&room_id) else { continue };
292            self.for_room(room).unwedge_requests_with_expired_token().await;
293        }
294    }
295
296    /// Tiny helper to get the send queue's global context from the [`Client`].
297    #[inline(always)]
298    fn data(&self) -> &SendQueueData {
299        &self.client.inner.send_queue_data
300    }
301
302    /// Get or create a new send queue for a given room, and insert it into our
303    /// memoized rooms mapping.
304    pub(crate) fn for_room(&self, room: Room) -> RoomSendQueue {
305        let data = self.data();
306
307        data.session_change_task.get_or_init(|| {
308            let client = self.client.downgrade();
309            self.client
310                .task_monitor()
311                .spawn_infinite_task("send_queue_session_change", Self::session_change_task(client))
312                .abort_on_drop()
313        });
314
315        let mut map = data.rooms.write().unwrap();
316
317        let room_id = room.room_id();
318        if let Some(room_q) = map.get(room_id).cloned() {
319            return room_q;
320        }
321
322        let owned_room_id = room_id.to_owned();
323        let room_q = RoomSendQueue::new(
324            self.is_enabled(),
325            data.global_update_sender.clone(),
326            data.error_sender.clone(),
327            data.is_dropping.clone(),
328            &self.client,
329            owned_room_id.clone(),
330            data.report_media_upload_progress.clone(),
331        );
332
333        map.insert(owned_room_id, room_q.clone());
334
335        room_q
336    }
337
338    /// Enable or disable the send queue for the entire client, i.e. all rooms.
339    ///
340    /// If we're disabling the queue, and requests were being sent, they're not
341    /// aborted, and will continue until a status resolves (error responses will
342    /// keep the events in the buffer of events to send later). The disablement
343    /// will happen before the next request is sent.
344    ///
345    /// This may wake up background tasks and resume sending of requests in the
346    /// background.
347    pub async fn set_enabled(&self, enabled: bool) {
348        debug!(?enabled, "setting global send queue enablement");
349
350        self.data().globally_enabled.store(enabled, Ordering::SeqCst);
351
352        // Wake up individual rooms we already know about.
353        for room in self.data().rooms.read().unwrap().values() {
354            room.set_enabled(enabled);
355        }
356
357        // Reload some extra rooms that might not have been awaken yet, but
358        // could have requests from previous sessions.
359        self.respawn_tasks_for_rooms_with_unsent_requests().await;
360    }
361
362    /// Returns whether the send queue is enabled, at a client-wide granularity.
363    pub fn is_enabled(&self) -> bool {
364        self.data().globally_enabled.load(Ordering::SeqCst)
365    }
366
367    /// Enable or disable progress reporting for media uploads.
368    pub fn enable_upload_progress(&self, enabled: bool) {
369        self.data().report_media_upload_progress.store(enabled, Ordering::SeqCst);
370    }
371
372    /// Subscribe to all updates for all rooms.
373    ///
374    /// Use [`RoomSendQueue::subscribe`] to subscribe to update for a
375    /// _specific room_.
376    pub fn subscribe(&self) -> broadcast::Receiver<SendQueueUpdate> {
377        self.data().global_update_sender.subscribe()
378    }
379
380    /// Get local echoes from all room send queues.
381    pub async fn local_echoes(
382        &self,
383    ) -> Result<BTreeMap<OwnedRoomId, Vec<LocalEcho>>, RoomSendQueueError> {
384        let room_ids =
385            self.client.state_store().load_rooms_with_unsent_requests().await.unwrap_or_else(
386                |err| {
387                    warn!("error when loading rooms with unsent requests: {err}");
388                    Vec::new()
389                },
390            );
391
392        let mut local_echoes: BTreeMap<OwnedRoomId, Vec<LocalEcho>> = BTreeMap::new();
393
394        for room_id in room_ids {
395            if let Some(room) = self.client.get_room(&room_id) {
396                let queue = self.for_room(room);
397                local_echoes
398                    .insert(room_id.to_owned(), queue.inner.queue.local_echoes(&queue).await?);
399            }
400        }
401
402        Ok(local_echoes)
403    }
404
405    /// A subscriber to the enablement status (enabled or disabled) of the send
406    /// queue, along with useful errors.
407    pub fn subscribe_errors(&self) -> broadcast::Receiver<SendQueueRoomError> {
408        self.data().error_sender.subscribe()
409    }
410}
411
412/// Metadata about a thumbnail needed when pushing media uploads to the send
413/// queue.
414#[derive(Clone, Debug)]
415struct QueueThumbnailInfo {
416    /// Metadata about the thumbnail needed when finishing a media upload.
417    finish_upload_thumbnail_info: FinishUploadThumbnailInfo,
418
419    /// The parameters for the request to retrieve the thumbnail data.
420    media_request_parameters: MediaRequestParameters,
421
422    /// The thumbnail's mime type.
423    content_type: Mime,
424
425    /// The thumbnail's file size in bytes.
426    file_size: usize,
427}
428
429/// A specific room's send queue ran into an error.
430#[derive(Clone, Debug)]
431pub struct SendQueueRoomError {
432    /// For which room is the send queue failing?
433    pub room_id: OwnedRoomId,
434
435    /// The error the room has ran into, when trying to send a request.
436    pub error: Arc<crate::Error>,
437
438    /// Whether the error is considered recoverable or not.
439    ///
440    /// An error that's recoverable will disable the room's send queue, while an
441    /// unrecoverable error will be parked, until it's retried or aborted.
442    pub is_recoverable: bool,
443}
444
445impl Client {
446    /// Returns a [`SendQueue`] that handles sending, retrying and not
447    /// forgetting about requests that are to be sent.
448    pub fn send_queue(&self) -> SendQueue {
449        SendQueue::new(self.clone())
450    }
451}
452
453pub(super) struct SendQueueData {
454    /// Mapping of room to their unique send queue.
455    rooms: RwLock<BTreeMap<OwnedRoomId, RoomSendQueue>>,
456
457    /// Is the whole mechanism enabled or disabled?
458    ///
459    /// This is only kept in memory to initialize new room queues with an
460    /// initial enablement state.
461    globally_enabled: AtomicBool,
462
463    /// Global sender to send [`SendQueueUpdate`].
464    ///
465    /// See [`SendQueue::subscribe`].
466    global_update_sender: broadcast::Sender<SendQueueUpdate>,
467
468    /// Global error updates for the send queue.
469    error_sender: broadcast::Sender<SendQueueRoomError>,
470
471    /// Are we currently dropping the Client?
472    is_dropping: Arc<AtomicBool>,
473
474    /// Will media upload progress be reported via send queue updates?
475    report_media_upload_progress: Arc<AtomicBool>,
476
477    /// Task watching for the session getting a working access token again.
478    session_change_task: OnceLock<BackgroundTaskHandle>,
479}
480
481impl SendQueueData {
482    /// Create the data for a send queue, in the given enabled state.
483    pub fn new(globally_enabled: bool) -> Self {
484        let (global_update_sender, _) = broadcast::channel(32);
485        let (error_sender, _) = broadcast::channel(32);
486
487        Self {
488            rooms: Default::default(),
489            globally_enabled: AtomicBool::new(globally_enabled),
490            global_update_sender,
491            error_sender,
492            is_dropping: Arc::new(false.into()),
493            report_media_upload_progress: Arc::new(false.into()),
494            session_change_task: OnceLock::new(),
495        }
496    }
497}
498
499impl Drop for SendQueueData {
500    fn drop(&mut self) {
501        // Mark the whole send queue as shutting down, then wake up all the room
502        // queues so they're stopped too.
503        debug!("globally dropping the send queue");
504        self.is_dropping.store(true, Ordering::SeqCst);
505
506        let rooms = self.rooms.read().unwrap();
507        for room in rooms.values() {
508            room.inner.notifier.notify_one();
509        }
510    }
511}
512
513impl Room {
514    /// Returns the [`RoomSendQueue`] for this specific room.
515    pub fn send_queue(&self) -> RoomSendQueue {
516        self.client.send_queue().for_room(self.clone())
517    }
518}
519
520/// A per-room send queue.
521///
522/// This is cheap to clone.
523#[derive(Clone)]
524pub struct RoomSendQueue {
525    inner: Arc<RoomSendQueueInner>,
526}
527
528#[cfg(not(tarpaulin_include))]
529impl std::fmt::Debug for RoomSendQueue {
530    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
531        f.debug_struct("RoomSendQueue").finish_non_exhaustive()
532    }
533}
534
535impl RoomSendQueue {
536    fn new(
537        globally_enabled: bool,
538        global_update_sender: broadcast::Sender<SendQueueUpdate>,
539        global_error_sender: broadcast::Sender<SendQueueRoomError>,
540        is_dropping: Arc<AtomicBool>,
541        client: &Client,
542        room_id: OwnedRoomId,
543        report_media_upload_progress: Arc<AtomicBool>,
544    ) -> Self {
545        let (update_sender, _) = broadcast::channel(32);
546
547        let queue = QueueStorage::new(client.downgrade(), room_id.clone());
548        let notifier = Arc::new(Notify::new());
549
550        let weak_room = WeakRoom::new(client.downgrade(), room_id);
551        let locally_enabled = Arc::new(AtomicBool::new(globally_enabled));
552
553        let task = client.task_monitor().spawn_infinite_task(
554            "send_queue",
555            Self::sending_task(
556                weak_room.clone(),
557                queue.clone(),
558                notifier.clone(),
559                global_update_sender.clone(),
560                update_sender.clone(),
561                locally_enabled.clone(),
562                global_error_sender,
563                is_dropping,
564                report_media_upload_progress,
565            ),
566        );
567
568        Self {
569            inner: Arc::new(RoomSendQueueInner {
570                room: weak_room,
571                global_update_sender,
572                update_sender,
573                _task: task,
574                queue,
575                notifier,
576                locally_enabled,
577            }),
578        }
579    }
580
581    /// Queues a raw event for sending it to this room.
582    ///
583    /// This immediately returns, and will push the event to be sent into a
584    /// queue, handled in the background.
585    ///
586    /// Callers are expected to consume [`RoomSendQueueUpdate`] via calling the
587    /// [`Self::subscribe()`] method to get updates about the sending of that
588    /// event.
589    ///
590    /// By default, if sending failed on the first attempt, it will be retried a
591    /// few times. If sending failed after those retries, the entire client's
592    /// sending queue will be disabled, and it will need to be manually
593    /// re-enabled by the caller (e.g. after network is back, or when something
594    /// has been done about the faulty requests).
595    pub fn send_raw(
596        &self,
597        content: Raw<AnyMessageLikeEventContent>,
598        event_type: String,
599    ) -> SendRawEvent<'_> {
600        SendRawEvent {
601            queue: self,
602            content: SerializableEventContent::from_raw(content, event_type),
603            #[cfg(feature = "unstable-msc4354")]
604            sticky_duration: None,
605        }
606    }
607
608    /// Queues an already serialized event for sending it to this room.
609    async fn send_serialized(
610        &self,
611        content: SerializableEventContent,
612        #[cfg(feature = "unstable-msc4354")] sticky_duration: Option<StickyDurationMs>,
613    ) -> Result<SendHandle, RoomSendQueueError> {
614        let Some(room) = self.inner.room.get() else {
615            return Err(RoomSendQueueError::RoomDisappeared);
616        };
617        if room.state() != RoomState::Joined {
618            return Err(RoomSendQueueError::RoomNotJoined);
619        }
620
621        let request = QueuedRequestKind::Event {
622            content: content.clone(),
623            #[cfg(feature = "unstable-msc4354")]
624            sticky_duration,
625        };
626
627        let created_at = MilliSecondsSinceUnixEpoch::now();
628        let transaction_id = self.inner.queue.push(request, created_at).await?;
629        trace!(%transaction_id, "manager sends a raw event to the background task");
630
631        self.inner.notifier.notify_one();
632
633        let send_handle = SendHandle {
634            room: self.clone(),
635            transaction_id: transaction_id.clone(),
636            media_handles: vec![],
637            created_at,
638        };
639
640        self.send_update(RoomSendQueueUpdate::NewLocalEvent(LocalEcho {
641            transaction_id,
642            content: LocalEchoContent::Event {
643                serialized_event: content,
644                send_handle: send_handle.clone(),
645                send_error: None,
646            },
647        }));
648
649        Ok(send_handle)
650    }
651
652    /// Queues an event for sending it to this room.
653    ///
654    /// This immediately returns, and will push the event to be sent into a
655    /// queue, handled in the background.
656    ///
657    /// Callers are expected to consume [`RoomSendQueueUpdate`] via calling the
658    /// [`Self::subscribe()`] method to get updates about the sending of that
659    /// event.
660    ///
661    /// By default, if sending failed on the first attempt, it will be retried a
662    /// few times. If sending failed after those retries, the entire client's
663    /// sending queue will be disabled, and it will need to be manually
664    /// re-enabled by the caller (e.g. after network is back, or when something
665    /// has been done about the faulty requests).
666    pub fn send(&self, content: AnyMessageLikeEventContent) -> SendEvent<'_> {
667        SendEvent {
668            queue: self,
669            content,
670            extra_content: None,
671            #[cfg(feature = "unstable-msc4354")]
672            sticky_duration: None,
673        }
674    }
675
676    /// Queues a redaction of another event for sending it to this room.
677    ///
678    /// This immediately returns, and will push the redaction to be sent into a
679    /// queue, handled in the background.
680    ///
681    /// Callers are expected to consume [`RoomSendQueueUpdate`] via calling the
682    /// [`Self::subscribe()`] method to get updates about the sending of that
683    /// redaction.
684    ///
685    /// By default, if sending failed on the first attempt, it will be retried a
686    /// few times. If sending failed after those retries, the entire client's
687    /// sending queue will be disabled, and it will need to be manually
688    /// re-enabled by the caller (e.g. after network is back, or when something
689    /// has been done about the faulty requests).
690    pub async fn redact(
691        &self,
692        redacts: OwnedEventId,
693        reason: Option<&str>,
694    ) -> Result<SendHandle, RoomSendQueueError> {
695        let Some(room) = self.inner.room.get() else {
696            return Err(RoomSendQueueError::RoomDisappeared);
697        };
698        if room.state() != RoomState::Joined {
699            return Err(RoomSendQueueError::RoomNotJoined);
700        }
701
702        let request = QueuedRequestKind::Redaction {
703            redacts: redacts.clone(),
704            reason: reason.map(str::to_owned),
705        };
706
707        let created_at = MilliSecondsSinceUnixEpoch::now();
708        let transaction_id = self.inner.queue.push(request, created_at).await?;
709        trace!(%transaction_id, "manager sends a redaction event to the background task");
710
711        self.inner.notifier.notify_one();
712
713        let send_handle = SendHandle {
714            room: self.clone(),
715            transaction_id: transaction_id.clone(),
716            media_handles: vec![],
717            created_at,
718        };
719
720        self.send_update(RoomSendQueueUpdate::NewLocalEvent(LocalEcho {
721            transaction_id,
722            content: LocalEchoContent::Redaction {
723                redacts,
724                reason: reason.map(str::to_owned),
725                send_handle: send_handle.clone(),
726                send_error: None,
727            },
728        }));
729
730        Ok(send_handle)
731    }
732
733    /// Returns the current local requests as well as a receiver to listen to
734    /// the send queue updates, as defined in [`RoomSendQueueUpdate`].
735    ///
736    /// Use [`SendQueue::subscribe`] to subscribe to update for _all rooms_ with
737    /// a single receiver.
738    pub async fn subscribe(
739        &self,
740    ) -> Result<(Vec<LocalEcho>, broadcast::Receiver<RoomSendQueueUpdate>), RoomSendQueueError>
741    {
742        let local_echoes = self.inner.queue.local_echoes(self).await?;
743
744        Ok((local_echoes, self.inner.update_sender.subscribe()))
745    }
746
747    /// A task that must be spawned in the async runtime, running in the
748    /// background for each room that has a send queue.
749    ///
750    /// It only progresses forward: nothing can be cancelled at any point, which
751    /// makes the implementation not overly complicated to follow.
752    #[allow(clippy::too_many_arguments)]
753    #[instrument(skip_all, fields(room_id = %room.room_id()))]
754    async fn sending_task(
755        room: WeakRoom,
756        queue: QueueStorage,
757        notifier: Arc<Notify>,
758        global_update_sender: broadcast::Sender<SendQueueUpdate>,
759        update_sender: broadcast::Sender<RoomSendQueueUpdate>,
760        locally_enabled: Arc<AtomicBool>,
761        global_error_sender: broadcast::Sender<SendQueueRoomError>,
762        is_dropping: Arc<AtomicBool>,
763        report_media_upload_progress: Arc<AtomicBool>,
764    ) {
765        trace!("spawned the sending task");
766
767        let room_id = room.room_id();
768
769        loop {
770            // A request to shut down should be preferred above everything else.
771            if is_dropping.load(Ordering::SeqCst) {
772                trace!("shutting down!");
773                break;
774            }
775
776            // Try to apply dependent requests now; those applying to previously
777            // failed attempts (local echoes) would succeed now.
778            let mut new_updates = Vec::new();
779            if let Err(err) = queue.apply_dependent_requests(&mut new_updates).await {
780                warn!("errors when applying dependent requests: {err}");
781            }
782
783            for up in new_updates {
784                send_update(&global_update_sender, &update_sender, room_id, up);
785            }
786
787            if !locally_enabled.load(Ordering::SeqCst) {
788                trace!("not enabled, sleeping");
789                // Wait for an explicit wakeup.
790                notifier.notified().await;
791                continue;
792            }
793
794            let (queued_request, cancel_upload_rx) = match queue.peek_next_to_send().await {
795                Ok(Some(request)) => request,
796
797                Ok(None) => {
798                    trace!("queue is empty or blocked on a wedged request, sleeping");
799                    // Wait for an explicit wakeup.
800                    notifier.notified().await;
801                    continue;
802                }
803
804                Err(err) => {
805                    warn!("error when loading next request to send: {err}");
806                    // Don't hammer a failing store; back off a bit before
807                    // retrying.
808                    matrix_sdk_common::sleep::sleep(STORE_ERROR_BACKOFF).await;
809                    continue;
810                }
811            };
812
813            let txn_id = queued_request.transaction_id.clone();
814            trace!(txn_id = %txn_id, "received a request to send!");
815
816            let Some(room) = room.get() else {
817                if is_dropping.load(Ordering::SeqCst) {
818                    break;
819                }
820                error!("the weak room couldn't be upgraded but we're not shutting down?");
821                continue;
822            };
823
824            // If this is a media/gallery upload, prepare the following:
825            //
826            // - transaction id for the related media event request,
827            // - progress metadata to feed the final media upload progress
828            // - an observable to watch the media upload progress.
829            let (related_txn_id, media_upload_progress_info, http_progress) =
830                if let QueuedRequestKind::MediaUpload {
831                    cache_key,
832                    thumbnail_source,
833                    #[cfg(feature = "unstable-msc4274")]
834                    accumulated,
835                    related_to,
836                    ..
837                } = &queued_request.kind
838                {
839                    // Prepare to watch and communicate the request's progress
840                    // for media uploads, if it has been requested.
841                    let (media_upload_progress_info, http_progress) =
842                        if report_media_upload_progress.load(Ordering::SeqCst) {
843                            let media_upload_progress_info =
844                                RoomSendQueue::create_media_upload_progress_info(
845                                    &queued_request.transaction_id,
846                                    related_to,
847                                    cache_key,
848                                    thumbnail_source.as_ref(),
849                                    #[cfg(feature = "unstable-msc4274")]
850                                    accumulated,
851                                    &room,
852                                    &queue,
853                                )
854                                .await;
855
856                            let progress = RoomSendQueue::create_media_upload_progress_observable(
857                                &media_upload_progress_info,
858                                related_to,
859                                &update_sender,
860                            );
861
862                            (Some(media_upload_progress_info), Some(progress))
863                        } else {
864                            Default::default()
865                        };
866
867                    (Some(related_to.clone()), media_upload_progress_info, http_progress)
868                } else {
869                    Default::default()
870                };
871
872            match Self::handle_request(&room, queued_request, cancel_upload_rx, http_progress).await
873            {
874                Ok((Some(parent_key), encryption_info)) => match queue
875                    .mark_as_sent(&txn_id, parent_key.clone())
876                    .await
877                {
878                    Ok(()) => match parent_key {
879                        SentRequestKey::Event { event_id, event, event_type } => {
880                            send_update(
881                                &global_update_sender,
882                                &update_sender,
883                                room_id,
884                                RoomSendQueueUpdate::SentEvent {
885                                    transaction_id: txn_id,
886                                    event_id: event_id.clone(),
887                                },
888                            );
889
890                            // The event has been sent to the server and the
891                            // server has received it. Yepee! Now, we usually
892                            // wait on the server to give us back the event via
893                            // the sync.
894                            //
895                            // Problem: sometimes the network lags, can be down,
896                            // or the server may be slow; well, anything can
897                            // happen.
898                            //
899                            // It results in a weird situation where the user
900                            // sees its event being sent, then disappears before
901                            // it's received again from the server.
902                            //
903                            // To avoid this situation, we eagerly save the
904                            // event in the Event Cache. It's similar to what
905                            // would happen if the event was echoed back from
906                            // the server via the sync, but we avoid any network
907                            // issues. The Event Cache is smart enough to
908                            // deduplicate events based on the event ID, so it's
909                            // safe to do that.
910                            //
911                            // If this little feature fails, it MUST NOT stop
912                            // the Send Queue. Any errors are logged, but the
913                            // Send Queue will continue as if everything
914                            // happened successfully. This feature is not
915                            // considered “crucial”.
916                            if let Some(event_cache) = room.client().event_cache()
917                                && let Ok((room_event_cache, _drop_handles)) =
918                                    event_cache.room(room.room_id()).await
919                            {
920                                let timeline_event = match Raw::from_json_string(
921                                    // Create a compact string: remove all useless spaces.
922                                    format!(
923                                        "{{\
924                                            \"event_id\":\"{event_id}\",\
925                                            \"origin_server_ts\":{ts},\
926                                            \"sender\":\"{sender}\",\
927                                            \"type\":\"{type}\",\
928                                            \"content\":{content}\
929                                        }}",
930                                        event_id = event_id,
931                                        ts = MilliSecondsSinceUnixEpoch::now().get(),
932                                        sender = room.client().user_id().expect("Client must be logged-in"),
933                                        type = event_type,
934                                        content = event.into_json(),
935                                    ),
936                                ) {
937                                    Ok(event) => match encryption_info {
938                                        #[cfg(feature = "e2e-encryption")]
939                                        Some(encryption_info) => {
940                                            use matrix_sdk_base::deserialized_responses::DecryptedRoomEvent;
941                                            let decrypted_event = DecryptedRoomEvent {
942                                                event: event.cast_unchecked(),
943                                                encryption_info: Arc::new(encryption_info),
944                                                unsigned_encryption_info: None,
945                                            };
946                                            Some(TimelineEvent::from_decrypted(
947                                                decrypted_event,
948                                                None,
949                                            ))
950                                        }
951                                        _ => Some(TimelineEvent::from_plaintext(event)),
952                                    },
953                                    Err(err) => {
954                                        error!(
955                                            ?err,
956                                            "Failed to build the (sync) event before the saving in the Event Cache"
957                                        );
958                                        None
959                                    }
960                                };
961
962                                // In case of an error, just log the error but
963                                // don't stop the Send Queue. This feature is
964                                // not crucial.
965                                if let Some(timeline_event) = timeline_event
966                                    && let Err(err) = room_event_cache
967                                        .insert_sent_event_from_send_queue(timeline_event)
968                                        .await
969                                {
970                                    error!(
971                                        ?err,
972                                        "Failed to save the sent event in the Event Cache"
973                                    );
974                                }
975                            } else {
976                                info!(
977                                    "Cannot insert the sent event in the Event Cache because \
978                                    either the room no longer exists, or the Room Event Cache cannot be retrieved"
979                                );
980                            }
981                        }
982
983                        SentRequestKey::Media(sent_media_info) => {
984                            // Generate some final progress information, even if
985                            // incremental progress wasn't requested.
986                            let index =
987                                media_upload_progress_info.as_ref().map_or(0, |info| info.index);
988                            let progress = media_upload_progress_info
989                                .as_ref()
990                                .map(|info| {
991                                    AbstractProgress { current: info.bytes, total: info.bytes }
992                                        + info.offsets
993                                })
994                                .unwrap_or(AbstractProgress { current: 1, total: 1 });
995
996                            // Purposefully don't use `send_update` here,
997                            // because we don't want to notify the global
998                            // listeners about an upload progress update.
999                            let _ = update_sender.send(RoomSendQueueUpdate::MediaUpload {
1000                                related_to: related_txn_id.as_ref().unwrap_or(&txn_id).clone(),
1001                                file: Some(sent_media_info.file),
1002                                index,
1003                                progress,
1004                            });
1005                        }
1006
1007                        SentRequestKey::Redaction { event_id, redacts, reason } => {
1008                            send_update(
1009                                &global_update_sender,
1010                                &update_sender,
1011                                room_id,
1012                                RoomSendQueueUpdate::SentEvent {
1013                                    transaction_id: txn_id,
1014                                    event_id: event_id.clone(),
1015                                },
1016                            );
1017
1018                            // The redaction event has been sent to the server
1019                            // and the server has received it. It's safe to
1020                            // cache the event now to avoid any inconsistencies
1021                            // until the server sends down the remote echo via
1022                            // the sync.
1023                            if let Some(event_cache) = room.client().event_cache()
1024                                && let Ok((room_event_cache, _drop_handles)) =
1025                                    event_cache.room(room.room_id()).await
1026                            {
1027                                let content_field_redacts = room.version().is_some_and(|id| {
1028                                    id.rules()
1029                                        .is_some_and(|rules| rules.redaction.content_field_redacts)
1030                                });
1031                                let redacts = if content_field_redacts.not() {
1032                                    format!("\"redacts\":\"{redacts}\",")
1033                                } else {
1034                                    "".to_owned()
1035                                };
1036                                let reason = reason.map_or_else(
1037                                    || "".to_owned(),
1038                                    |r| format!("\"reason\": \"{r}\""),
1039                                );
1040                                let content = if content_field_redacts {
1041                                    format!("\"redacts\":\"{redacts}\",{reason}")
1042                                } else {
1043                                    reason
1044                                };
1045
1046                                let timeline_event = match Raw::from_json_string(
1047                                    // Create a compact string: remove all useless spaces.
1048                                    format!(
1049                                        "{{\
1050                                            {redacts}\
1051                                            \"event_id\":\"{event_id}\",\
1052                                            \"origin_server_ts\":{ts},\
1053                                            \"sender\":\"{sender}\",\
1054                                            \"type\":\"{type}\",\
1055                                            \"content\":{{{content}}}\
1056                                        }}",
1057                                        redacts = redacts,
1058                                        event_id = event_id,
1059                                        ts = MilliSecondsSinceUnixEpoch::now().get(),
1060                                        sender = room.client().user_id().expect("Client must be logged-in"),
1061                                        type = TimelineEventType::RoomRedaction,
1062                                        content = content
1063                                    ),
1064                                ) {
1065                                    Ok(event) => Some(TimelineEvent::from_plaintext(event)),
1066                                    Err(err) => {
1067                                        error!(
1068                                            ?err,
1069                                            "Failed to build the (sync) redaction event before the saving in the Event Cache"
1070                                        );
1071                                        None
1072                                    }
1073                                };
1074
1075                                // In case of an error, just log the error but
1076                                // don't stop the Send Queue. This feature is
1077                                // not crucial.
1078                                if let Some(timeline_event) = timeline_event
1079                                    && let Err(err) = room_event_cache
1080                                        .insert_sent_event_from_send_queue(timeline_event)
1081                                        .await
1082                                {
1083                                    error!(
1084                                        ?err,
1085                                        "Failed to save the sent redaction event in the Event Cache"
1086                                    );
1087                                }
1088                            } else {
1089                                info!(
1090                                    "Cannot insert the sent redaction event in the Event Cache because \
1091                                    either the room no longer exists, or the Room Event Cache cannot be retrieved"
1092                                );
1093                            }
1094                        }
1095                    },
1096
1097                    Err(err) => {
1098                        warn!("unable to mark queued request as sent: {err}");
1099                    }
1100                },
1101
1102                Ok((None, _)) => {
1103                    debug!("Request has been aborted while running, continuing.");
1104                }
1105
1106                // The server already has this exact reaction from us (sent from
1107                // another device, or by an earlier attempt it no longer
1108                // remembers), so there's nothing left to do. Wedging would
1109                // block the room for good: every retry gets the same answer.
1110                Err(err)
1111                    if err.client_api_error_kind() == Some(&ErrorKind::DuplicateAnnotation) =>
1112                {
1113                    debug!(txn_id = %txn_id, "the server already has this reaction, dropping it");
1114
1115                    queue.mark_as_not_being_sent(&txn_id).await;
1116
1117                    match queue.cancel_event(&txn_id, None).await {
1118                        Ok(true) => send_update(
1119                            &global_update_sender,
1120                            &update_sender,
1121                            room_id,
1122                            RoomSendQueueUpdate::CancelledLocalEvent { transaction_id: txn_id },
1123                        ),
1124
1125                        // Removed meanwhile (aborted before we got here);
1126                        // nothing to report.
1127                        Ok(false) => {}
1128
1129                        Err(storage_error) => {
1130                            // Still queued, so it would be sent again right
1131                            // away, over and over: stop the queue instead.
1132                            error!(
1133                                txn_id = %txn_id,
1134                                "couldn't drop the duplicate reaction, disabling the queue: {storage_error}"
1135                            );
1136                            locally_enabled.store(false, Ordering::SeqCst);
1137                        }
1138                    }
1139                }
1140
1141                Err(err) => {
1142                    let is_recoverable = match err {
1143                        crate::Error::Http(ref http_err) => {
1144                            // All transient errors are recoverable.
1145                            matches!(
1146                                http_err.retry_kind(),
1147                                RetryKind::Transient { .. } | RetryKind::NetworkFailure
1148                            )
1149                        }
1150
1151                        // `ConcurrentRequestFailed` typically happens because
1152                        // of an HTTP failure; since we don't get the underlying
1153                        // error, be lax and consider it recoverable, and let
1154                        // observers decide to retry it or not. At some point
1155                        // we'll get the actual underlying error.
1156                        crate::Error::ConcurrentRequestFailed => true,
1157
1158                        // As of 2024-06-27, all other error types are considered unrecoverable.
1159                        _ => false,
1160                    };
1161
1162                    if is_recoverable {
1163                        // Disable the queue for this room; there's nothing else
1164                        // blocking it, and whatever caused the failure is
1165                        // likely to affect the next requests too.
1166                        locally_enabled.store(false, Ordering::SeqCst);
1167
1168                        warn!(txn_id = %txn_id, error = ?err, "Recoverable error when sending request: {err}, disabling send queue");
1169
1170                        // In this case, we intentionally keep the request in
1171                        // the queue, but mark it as not being sent anymore.
1172                        queue.mark_as_not_being_sent(&txn_id).await;
1173
1174                        // Let observers know about a failure _after_ we've
1175                        // marked the item as not being sent anymore. Otherwise,
1176                        // there's a possible race where a caller might try to
1177                        // remove an item, while it's still marked as being
1178                        // sent, resulting in a cancellation failure.
1179                    } else {
1180                        warn!(txn_id = %txn_id, error = ?err, "Unrecoverable error when sending request: {err}");
1181
1182                        // Mark the request as wedged, so it's not picked at any
1183                        // future point; it will also block subsequent requests
1184                        // in the same room from being sent, until it's unwedged
1185                        // or removed, so as to preserve ordering.
1186                        if let Err(storage_error) =
1187                            queue.mark_as_wedged(&txn_id, QueueWedgeError::from(&err)).await
1188                        {
1189                            // Nothing recorded the wedge, so the request would
1190                            // be picked up and sent again right away, over and
1191                            // over; disabling the queue is the only brake left.
1192                            error!(
1193                                "unable to mark request as wedged, disabling the queue: {storage_error}"
1194                            );
1195                            locally_enabled.store(false, Ordering::SeqCst);
1196                        }
1197                    }
1198
1199                    let error = Arc::new(err);
1200
1201                    let _ = global_error_sender.send(SendQueueRoomError {
1202                        room_id: room_id.to_owned(),
1203                        error: error.clone(),
1204                        is_recoverable,
1205                    });
1206
1207                    send_update(
1208                        &global_update_sender,
1209                        &update_sender,
1210                        room_id,
1211                        RoomSendQueueUpdate::SendError {
1212                            transaction_id: related_txn_id.unwrap_or(txn_id),
1213                            error,
1214                            is_recoverable,
1215                        },
1216                    );
1217                }
1218            }
1219        }
1220
1221        info!("exited sending task");
1222    }
1223
1224    /// Handles a single request and returns the [`SentRequestKey`] on success
1225    /// (unless the request was cancelled, in which case it'll return `None`).
1226    async fn handle_request(
1227        room: &Room,
1228        request: QueuedRequest,
1229        cancel_upload_rx: Option<oneshot::Receiver<()>>,
1230        progress: Option<SharedObservable<TransmissionProgress>>,
1231    ) -> Result<(Option<SentRequestKey>, Option<EncryptionInfo>), crate::Error> {
1232        match request.kind {
1233            QueuedRequestKind::Event {
1234                content,
1235                #[cfg(feature = "unstable-msc4354")]
1236                sticky_duration,
1237            } => {
1238                let (event, event_type) = content.into_raw();
1239
1240                let future = room
1241                    .send_raw(&event_type, &event)
1242                    .with_transaction_id(&request.transaction_id)
1243                    .with_request_config(RequestConfig::short_retry());
1244
1245                #[cfg(feature = "unstable-msc4354")]
1246                let future = match sticky_duration {
1247                    Some(duration) => {
1248                        future.with_sticky_duration(Duration::from_millis(duration.get().into()))
1249                    }
1250                    None => future,
1251                };
1252
1253                let result = future.await?;
1254
1255                trace!(txn_id = %request.transaction_id, event_id = %result.response.event_id, "event successfully sent");
1256
1257                Ok((
1258                    Some(SentRequestKey::Event {
1259                        event_id: result.response.event_id,
1260                        event,
1261                        event_type,
1262                    }),
1263                    result.encryption_info,
1264                ))
1265            }
1266
1267            QueuedRequestKind::MediaUpload {
1268                content_type,
1269                cache_key,
1270                thumbnail_source,
1271                related_to: relates_to,
1272                #[cfg(feature = "unstable-msc4274")]
1273                accumulated,
1274            } => {
1275                trace!(%relates_to, "uploading media related to event");
1276
1277                let fut = async move {
1278                    let data = room
1279                        .client()
1280                        .media_store()
1281                        .lock()
1282                        .await?
1283                        .get_media_content(&cache_key)
1284                        .await?
1285                        .ok_or(crate::Error::SendQueueWedgeError(Box::new(
1286                            QueueWedgeError::MissingMediaContent,
1287                        )))?;
1288
1289                    let mime = Mime::from_str(&content_type).map_err(|_| {
1290                        crate::Error::SendQueueWedgeError(Box::new(
1291                            QueueWedgeError::InvalidMimeType { mime_type: content_type.clone() },
1292                        ))
1293                    })?;
1294
1295                    #[cfg(feature = "e2e-encryption")]
1296                    let media_source = if room.latest_encryption_state().await?.is_encrypted() {
1297                        trace!("upload will be encrypted (encrypted room)");
1298
1299                        let mut cursor = std::io::Cursor::new(data);
1300                        let mut req = room
1301                            .client
1302                            .upload_encrypted_file(&mut cursor)
1303                            .with_request_config(RequestConfig::short_retry());
1304                        if let Some(progress) = progress {
1305                            req = req.with_send_progress_observable(progress);
1306                        }
1307                        let encrypted_file = req.await?;
1308
1309                        MediaSource::Encrypted(Box::new(encrypted_file))
1310                    } else {
1311                        trace!("upload will be in clear text (room without encryption)");
1312
1313                        let request_config = RequestConfig::short_retry()
1314                            .timeout(Media::reasonable_upload_timeout(&data));
1315                        let mut req =
1316                            room.client().media().upload(&mime, data, Some(request_config));
1317                        if let Some(progress) = progress {
1318                            req = req.with_send_progress_observable(progress);
1319                        }
1320                        let res = req.await?;
1321
1322                        MediaSource::Plain(res.content_uri)
1323                    };
1324
1325                    #[cfg(not(feature = "e2e-encryption"))]
1326                    let media_source = {
1327                        let request_config = RequestConfig::short_retry()
1328                            .timeout(Media::reasonable_upload_timeout(&data));
1329                        let mut req =
1330                            room.client().media().upload(&mime, data, Some(request_config));
1331                        if let Some(progress) = progress {
1332                            req = req.with_send_progress_observable(progress);
1333                        }
1334                        let res = req.await?;
1335                        MediaSource::Plain(res.content_uri)
1336                    };
1337
1338                    let uri = match &media_source {
1339                        MediaSource::Plain(uri) => uri,
1340                        MediaSource::Encrypted(encrypted_file) => &encrypted_file.url,
1341                    };
1342                    trace!(%relates_to, mxc_uri = %uri, "media successfully uploaded");
1343
1344                    Ok((
1345                        Some(SentRequestKey::Media(SentMediaInfo {
1346                            file: media_source,
1347                            thumbnail: thumbnail_source,
1348                            #[cfg(feature = "unstable-msc4274")]
1349                            accumulated,
1350                        })),
1351                        None,
1352                    ))
1353                };
1354
1355                let wait_for_cancel = async move {
1356                    if let Some(rx) = cancel_upload_rx {
1357                        rx.await
1358                    } else {
1359                        std::future::pending().await
1360                    }
1361                };
1362
1363                tokio::select! {
1364                    biased;
1365
1366                    _ = wait_for_cancel => {
1367                        Ok((None, None))
1368                    }
1369
1370                    res = fut => {
1371                        res
1372                    }
1373                }
1374            }
1375
1376            QueuedRequestKind::Redaction { redacts, reason } => {
1377                let result = room
1378                    .redact(&redacts, reason.as_deref(), Some(request.transaction_id.clone()))
1379                    .await?;
1380
1381                trace!(txn_id = %request.transaction_id, event_id = %result.event_id, "redaction successfully sent");
1382
1383                Ok((
1384                    Some(SentRequestKey::Redaction { event_id: result.event_id, redacts, reason }),
1385                    None,
1386                ))
1387            }
1388        }
1389    }
1390
1391    /// Returns whether the room is enabled, at the room level.
1392    pub fn is_enabled(&self) -> bool {
1393        self.inner.locally_enabled.load(Ordering::SeqCst)
1394    }
1395
1396    /// Set the locally enabled flag for this room queue.
1397    pub fn set_enabled(&self, enabled: bool) {
1398        self.inner.locally_enabled.store(enabled, Ordering::SeqCst);
1399
1400        // No need to wake a task to tell it it's been disabled, so only notify
1401        // if we're re-enabling the queue.
1402        if enabled {
1403            self.inner.notifier.notify_one();
1404        }
1405    }
1406
1407    /// Send an update on the room send queue channel, and on the global send
1408    /// queue channel, i.e. it sends a [`RoomSendQueueUpdate`] and a
1409    /// [`SendQueueUpdate`].
1410    fn send_update(&self, update: RoomSendQueueUpdate) {
1411        let _ = self.inner.update_sender.send(update.clone());
1412        let _ = self
1413            .inner
1414            .global_update_sender
1415            .send(SendQueueUpdate { room_id: self.inner.room.room_id().to_owned(), update });
1416    }
1417
1418    /// Clear the wedged status of every request that was wedged because the
1419    /// access token had been rejected.
1420    async fn unwedge_requests_with_expired_token(&self) {
1421        let transaction_ids = match self.inner.queue.requests_with_expired_token().await {
1422            Ok(transaction_ids) => transaction_ids,
1423            Err(err) => {
1424                warn!("error when looking for requests wedged by an expired token: {err}");
1425                return;
1426            }
1427        };
1428
1429        for transaction_id in transaction_ids {
1430            if let Err(err) = self.unwedge_request(&transaction_id).await {
1431                warn!(%transaction_id, "error when unwedging a request: {err}");
1432            }
1433        }
1434    }
1435
1436    /// Clear a request's wedged status and wake the queue up so it's tried
1437    /// again.
1438    async fn unwedge_request(
1439        &self,
1440        transaction_id: &TransactionId,
1441    ) -> Result<(), RoomSendQueueError> {
1442        self.inner
1443            .queue
1444            .mark_as_unwedged(transaction_id)
1445            .await
1446            .map_err(RoomSendQueueError::StorageError)?;
1447
1448        // Wake up the queue, in case the room was asleep before unwedging the
1449        // request.
1450        self.inner.notifier.notify_one();
1451
1452        self.send_update(RoomSendQueueUpdate::RetryEvent {
1453            transaction_id: transaction_id.to_owned(),
1454        });
1455
1456        Ok(())
1457    }
1458}
1459
1460fn send_update(
1461    global_update_sender: &broadcast::Sender<SendQueueUpdate>,
1462    update_sender: &broadcast::Sender<RoomSendQueueUpdate>,
1463    room_id: &RoomId,
1464    update: RoomSendQueueUpdate,
1465) {
1466    let _ = update_sender.send(update.clone());
1467    let _ = global_update_sender.send(SendQueueUpdate { room_id: room_id.to_owned(), update });
1468}
1469
1470impl From<&crate::Error> for QueueWedgeError {
1471    fn from(value: &crate::Error) -> Self {
1472        match value {
1473            #[cfg(feature = "e2e-encryption")]
1474            crate::Error::OlmError(error) => match &**error {
1475                OlmError::SessionRecipientCollectionError(error) => match error {
1476                    SessionRecipientCollectionError::VerifiedUserHasUnsignedDevice(user_map) => {
1477                        QueueWedgeError::InsecureDevices { user_device_map: user_map.clone() }
1478                    }
1479
1480                    SessionRecipientCollectionError::VerifiedUserChangedIdentity(users) => {
1481                        QueueWedgeError::IdentityViolations { users: users.clone() }
1482                    }
1483
1484                    SessionRecipientCollectionError::CrossSigningNotSetup
1485                    | SessionRecipientCollectionError::SendingFromUnverifiedDevice => {
1486                        QueueWedgeError::CrossVerificationRequired
1487                    }
1488                },
1489                _ => QueueWedgeError::GenericApiError { msg: value.to_string() },
1490            },
1491
1492            // Flatten errors of `Self` type.
1493            crate::Error::SendQueueWedgeError(error) => *error.clone(),
1494
1495            crate::Error::Http(error) if error.is_rejected_token() => {
1496                QueueWedgeError::ExpiredAccessToken
1497            }
1498
1499            _ => QueueWedgeError::GenericApiError { msg: value.to_string() },
1500        }
1501    }
1502}
1503
1504struct RoomSendQueueInner {
1505    /// The room which this send queue relates to.
1506    room: WeakRoom,
1507
1508    /// Global sender to send [`SendQueueUpdate`].
1509    ///
1510    /// See [`SendQueue::subscribe`].
1511    global_update_sender: broadcast::Sender<SendQueueUpdate>,
1512
1513    /// Broadcaster for notifications about the statuses of requests to be sent.
1514    ///
1515    /// Can be subscribed to from the outside.
1516    ///
1517    /// See [`RoomSendQueue::subscribe`].
1518    update_sender: broadcast::Sender<RoomSendQueueUpdate>,
1519
1520    /// Queue of requests that are either to be sent, or being sent.
1521    ///
1522    /// When a request has been sent to the server, it is removed from that
1523    /// queue _after_ being sent. That way, we will retry sending upon failure,
1524    /// in the same order requests have been inserted in the first place.
1525    queue: QueueStorage,
1526
1527    /// A notifier that's updated any time common data is touched (stopped or
1528    /// enabled statuses), or the associated room [`QueueStorage`].
1529    notifier: Arc<Notify>,
1530
1531    /// Should the room process new requests or not (because e.g. it might be
1532    /// running off the network)?
1533    locally_enabled: Arc<AtomicBool>,
1534
1535    /// Handle to the actual sending task. Unused, but kept alive along this
1536    /// data structure.
1537    _task: BackgroundTaskHandle,
1538}
1539
1540/// Information about a request being sent right this moment.
1541struct BeingSentInfo {
1542    /// Transaction id of the thing being sent.
1543    transaction_id: OwnedTransactionId,
1544
1545    /// For an upload request, a trigger to cancel the upload before it
1546    /// completes.
1547    cancel_upload: Option<oneshot::Sender<()>>,
1548}
1549
1550impl BeingSentInfo {
1551    /// Aborts the upload, if a trigger is available.
1552    ///
1553    /// Consumes the object because the sender is a oneshot and will be consumed
1554    /// upon sending.
1555    fn cancel_upload(self) -> bool {
1556        if let Some(cancel_upload) = self.cancel_upload {
1557            let _ = cancel_upload.send(());
1558            true
1559        } else {
1560            false
1561        }
1562    }
1563}
1564
1565/// A specialized lock that guards both against the state store and the
1566/// [`Self::being_sent`] data.
1567#[derive(Clone)]
1568struct StoreLock {
1569    /// Reference to the client, to get access to the underlying store.
1570    client: WeakClient,
1571
1572    /// The one queued request that is being sent at the moment, along with
1573    /// associated data that can be useful to act upon it.
1574    ///
1575    /// Also used as the lock to access the state store.
1576    being_sent: Arc<Mutex<Option<BeingSentInfo>>>,
1577}
1578
1579impl StoreLock {
1580    /// Gets a hold of the locked store and [`Self::being_sent`] pair.
1581    async fn lock(&self) -> StoreLockGuard {
1582        StoreLockGuard {
1583            client: self.client.clone(),
1584            being_sent: self.being_sent.clone().lock_owned().await,
1585        }
1586    }
1587}
1588
1589/// A lock guard obtained through locking with [`StoreLock`]. `being_sent` data.
1590struct StoreLockGuard {
1591    /// Reference to the client, to get access to the underlying store.
1592    client: WeakClient,
1593
1594    /// The one queued request that is being sent at the moment, along with
1595    /// associated data that can be useful to act upon it.
1596    being_sent: OwnedMutexGuard<Option<BeingSentInfo>>,
1597}
1598
1599impl StoreLockGuard {
1600    /// Get a client from the locked state, useful to get a handle on a store.
1601    fn client(&self) -> Result<Client, RoomSendQueueStorageError> {
1602        self.client.get().ok_or(RoomSendQueueStorageError::ClientShuttingDown)
1603    }
1604}
1605
1606#[derive(Clone)]
1607struct QueueStorage {
1608    /// A lock to make sure the state store is only accessed once at a time, to
1609    /// make some store operations atomic.
1610    store: StoreLock,
1611
1612    /// To which room is this storage related.
1613    room_id: OwnedRoomId,
1614
1615    /// In-memory mapping of media transaction IDs to thumbnail sizes for the
1616    /// purpose of progress reporting.
1617    ///
1618    /// The keys are the transaction IDs for sending the media or gallery event
1619    /// after all uploads have finished. This allows us to easily clean up the
1620    /// cache after the event was sent.
1621    ///
1622    /// For media uploads, the value vector will always have a single element.
1623    ///
1624    /// For galleries, some gallery items might not have a thumbnail while
1625    /// others do. Since we access the thumbnails by their index within the
1626    /// gallery, the vector needs to hold optional usize's.
1627    thumbnail_file_sizes: Arc<SyncMutex<HashMap<OwnedTransactionId, Vec<Option<usize>>>>>,
1628}
1629
1630impl QueueStorage {
1631    /// Default priority for a queued request.
1632    const LOW_PRIORITY: usize = 0;
1633
1634    /// High priority for a queued request that must be handled before others.
1635    const HIGH_PRIORITY: usize = 10;
1636
1637    /// Create a new queue for queuing requests to be sent later.
1638    fn new(client: WeakClient, room: OwnedRoomId) -> Self {
1639        Self {
1640            room_id: room,
1641            store: StoreLock { client, being_sent: Default::default() },
1642            thumbnail_file_sizes: Default::default(),
1643        }
1644    }
1645
1646    /// Push a new event to be sent in the queue, with a default priority of 0.
1647    ///
1648    /// Returns the transaction id chosen to identify the request.
1649    async fn push(
1650        &self,
1651        request: QueuedRequestKind,
1652        created_at: MilliSecondsSinceUnixEpoch,
1653    ) -> Result<OwnedTransactionId, RoomSendQueueStorageError> {
1654        let transaction_id = TransactionId::new();
1655
1656        self.store
1657            .lock()
1658            .await
1659            .client()?
1660            .state_store()
1661            .save_send_queue_request(
1662                &self.room_id,
1663                transaction_id.clone(),
1664                created_at,
1665                request,
1666                Self::LOW_PRIORITY,
1667            )
1668            .await?;
1669
1670        Ok(transaction_id)
1671    }
1672
1673    /// Peeks the next request to be sent, marking it as being sent.
1674    ///
1675    /// It is required to call [`Self::mark_as_sent`] after it's been
1676    /// effectively sent.
1677    async fn peek_next_to_send(
1678        &self,
1679    ) -> Result<Option<(QueuedRequest, Option<oneshot::Receiver<()>>)>, RoomSendQueueStorageError>
1680    {
1681        let mut guard = self.store.lock().await;
1682        let queued_requests =
1683            guard.client()?.state_store().load_send_queue_requests(&self.room_id).await?;
1684
1685        // Only ever consider the head of the queue: requests must be sent in
1686        // the order they were queued, so a wedged request (which failed to be
1687        // sent with an unrecoverable error) blocks all the requests queued
1688        // after it. Otherwise, messages would be sent out of order, until the
1689        // wedged request is either manually unwedged or removed (both of which
1690        // will wake up the sending task).
1691        if let Some(request) = queued_requests.first().filter(|queued| !queued.is_wedged()) {
1692            let (cancel_upload_tx, cancel_upload_rx) =
1693                if matches!(request.kind, QueuedRequestKind::MediaUpload { .. }) {
1694                    let (tx, rx) = oneshot::channel();
1695                    (Some(tx), Some(rx))
1696                } else {
1697                    Default::default()
1698                };
1699
1700            let prev = guard.being_sent.replace(BeingSentInfo {
1701                transaction_id: request.transaction_id.clone(),
1702                cancel_upload: cancel_upload_tx,
1703            });
1704
1705            if let Some(prev) = prev {
1706                error!(
1707                    prev_txn = ?prev.transaction_id,
1708                    "a previous request was still active while picking a new one"
1709                );
1710            }
1711
1712            Ok(Some((request.clone(), cancel_upload_rx)))
1713        } else {
1714            Ok(None)
1715        }
1716    }
1717
1718    /// Marks a request popped with [`Self::peek_next_to_send`] and identified
1719    /// with the given transaction id as not being sent anymore, so it can be
1720    /// removed from the queue later.
1721    async fn mark_as_not_being_sent(&self, transaction_id: &TransactionId) {
1722        let was_being_sent = self.store.lock().await.being_sent.take();
1723
1724        let prev_txn = was_being_sent.as_ref().map(|info| info.transaction_id.as_ref());
1725        if prev_txn != Some(transaction_id) {
1726            error!(prev_txn = ?prev_txn, "previous active request didn't match that we expect (after transient error)");
1727        }
1728    }
1729
1730    /// Marks a request popped with [`Self::peek_next_to_send`] and identified
1731    /// with the given transaction id as being wedged (and not being sent
1732    /// anymore), so it can be removed from the queue later.
1733    async fn mark_as_wedged(
1734        &self,
1735        transaction_id: &TransactionId,
1736        reason: QueueWedgeError,
1737    ) -> Result<(), RoomSendQueueStorageError> {
1738        // Keep the lock until we're done touching the storage.
1739        let mut guard = self.store.lock().await;
1740        let was_being_sent = guard.being_sent.take();
1741
1742        let prev_txn = was_being_sent.as_ref().map(|info| info.transaction_id.as_ref());
1743        if prev_txn != Some(transaction_id) {
1744            error!(
1745                ?prev_txn,
1746                "previous active request didn't match that we expect (after permanent error)",
1747            );
1748        }
1749
1750        Ok(guard
1751            .client()?
1752            .state_store()
1753            .update_send_queue_request_status(&self.room_id, transaction_id, Some(reason))
1754            .await?)
1755    }
1756
1757    /// Marks a request identified with the given transaction id as being now
1758    /// unwedged and adds it back to the queue.
1759    async fn mark_as_unwedged(
1760        &self,
1761        transaction_id: &TransactionId,
1762    ) -> Result<(), RoomSendQueueStorageError> {
1763        Ok(self
1764            .store
1765            .lock()
1766            .await
1767            .client()?
1768            .state_store()
1769            .update_send_queue_request_status(&self.room_id, transaction_id, None)
1770            .await?)
1771    }
1772
1773    /// The transaction ids of the requests that were wedged because the
1774    /// access token had been rejected.
1775    async fn requests_with_expired_token(
1776        &self,
1777    ) -> Result<Vec<OwnedTransactionId>, RoomSendQueueStorageError> {
1778        Ok(self
1779            .store
1780            .lock()
1781            .await
1782            .client()?
1783            .state_store()
1784            .load_send_queue_requests(&self.room_id)
1785            .await?
1786            .into_iter()
1787            .filter(|request| matches!(request.error, Some(QueueWedgeError::ExpiredAccessToken)))
1788            .map(|request| request.transaction_id)
1789            .collect())
1790    }
1791
1792    /// Marks a request pushed with [`Self::push`] and identified with the given
1793    /// transaction id as sent, by removing it from the local queue.
1794    async fn mark_as_sent(
1795        &self,
1796        transaction_id: &TransactionId,
1797        parent_key: SentRequestKey,
1798    ) -> Result<(), RoomSendQueueStorageError> {
1799        // Keep the lock until we're done touching the storage.
1800        let mut guard = self.store.lock().await;
1801        let was_being_sent = guard.being_sent.take();
1802
1803        let prev_txn = was_being_sent.as_ref().map(|info| info.transaction_id.as_ref());
1804        if prev_txn != Some(transaction_id) {
1805            error!(
1806                ?prev_txn,
1807                "previous active request didn't match that we expect (after successful send)",
1808            );
1809        }
1810
1811        let client = guard.client()?;
1812        let store = client.state_store();
1813
1814        // Update all dependent requests.
1815        store
1816            .mark_dependent_queued_requests_as_ready(&self.room_id, transaction_id, parent_key)
1817            .await?;
1818
1819        let removed = store.remove_send_queue_request(&self.room_id, transaction_id).await?;
1820
1821        if !removed {
1822            warn!(txn_id = %transaction_id, "request marked as sent was missing from storage");
1823        }
1824
1825        self.thumbnail_file_sizes.lock().remove(transaction_id);
1826
1827        Ok(())
1828    }
1829
1830    /// Cancel a sending command for an event that has been sent with
1831    /// [`Self::push`] with the given transaction id.
1832    ///
1833    /// Returns whether the given transaction has been effectively removed. If
1834    /// false, this either means that the transaction id was unrelated to this
1835    /// queue, or that the request was sent before we cancelled it.
1836    async fn cancel_event(
1837        &self,
1838        transaction_id: &TransactionId,
1839        reason: Option<String>,
1840    ) -> Result<bool, RoomSendQueueStorageError> {
1841        let guard = self.store.lock().await;
1842
1843        if guard.being_sent.as_ref().map(|info| info.transaction_id.as_ref())
1844            == Some(transaction_id)
1845        {
1846            // Save the intent to redact the event.
1847            guard
1848                .client()?
1849                .state_store()
1850                .save_dependent_queued_request(
1851                    &self.room_id,
1852                    transaction_id,
1853                    ChildTransactionId::new(),
1854                    MilliSecondsSinceUnixEpoch::now(),
1855                    DependentQueuedRequestKind::RedactEventWithReason { reason },
1856                )
1857                .await?;
1858
1859            return Ok(true);
1860        }
1861
1862        let removed = guard
1863            .client()?
1864            .state_store()
1865            .remove_send_queue_request(&self.room_id, transaction_id)
1866            .await?;
1867
1868        self.thumbnail_file_sizes.lock().remove(transaction_id);
1869
1870        Ok(removed)
1871    }
1872
1873    /// Replace an event that has been sent with [`Self::push`] with the given
1874    /// transaction id, before it's been actually sent.
1875    ///
1876    /// Returns whether the given transaction has been effectively edited. If
1877    /// false, this either means that the transaction id was unrelated to this
1878    /// queue, or that the request was sent before we edited it.
1879    async fn replace_event(
1880        &self,
1881        transaction_id: &TransactionId,
1882        serializable: SerializableEventContent,
1883    ) -> Result<bool, RoomSendQueueStorageError> {
1884        let guard = self.store.lock().await;
1885        let client = guard.client()?;
1886        let store = client.state_store();
1887
1888        // Only an event the user composed has content to replace: a redaction
1889        // or a reaction has nothing to put the new content into.
1890        if !store.load_send_queue_requests(&self.room_id).await?.iter().any(|request| {
1891            request.transaction_id == transaction_id && is_own_event_request(request)
1892        }) {
1893            return Ok(false);
1894        }
1895
1896        if guard.being_sent.as_ref().map(|info| info.transaction_id.as_ref())
1897            == Some(transaction_id)
1898        {
1899            // Save the intent to edit the associated event.
1900            store
1901                .save_dependent_queued_request(
1902                    &self.room_id,
1903                    transaction_id,
1904                    ChildTransactionId::new(),
1905                    MilliSecondsSinceUnixEpoch::now(),
1906                    DependentQueuedRequestKind::EditEvent { new_content: serializable },
1907                )
1908                .await?;
1909
1910            return Ok(true);
1911        }
1912
1913        let request = QueuedRequestKind::Event {
1914            content: serializable,
1915            #[cfg(feature = "unstable-msc4354")]
1916            sticky_duration: self.sticky_duration_of(store, transaction_id).await?,
1917        };
1918
1919        let edited =
1920            store.update_send_queue_request(&self.room_id, transaction_id, request).await?;
1921
1922        Ok(edited)
1923    }
1924
1925    /// The sticky duration of the queued event `transaction_id`, if it is
1926    /// queued and sticky.
1927    #[cfg(feature = "unstable-msc4354")]
1928    async fn sticky_duration_of(
1929        &self,
1930        store: &DynStateStore,
1931        transaction_id: &TransactionId,
1932    ) -> Result<Option<StickyDurationMs>, RoomSendQueueStorageError> {
1933        let sticky_duration = store
1934            .load_send_queue_requests(&self.room_id)
1935            .await?
1936            .into_iter()
1937            .find(|request| request.transaction_id == *transaction_id)
1938            .and_then(|request| match request.kind {
1939                QueuedRequestKind::Event { sticky_duration, .. } => sticky_duration,
1940                _ => None,
1941            });
1942
1943        Ok(sticky_duration)
1944    }
1945
1946    /// Push requests (and dependents) to upload a media.
1947    ///
1948    /// See the module-level description for details of the whole processus.
1949    #[allow(clippy::too_many_arguments)]
1950    async fn push_media(
1951        &self,
1952        event: RoomMessageEventContent,
1953        content_type: Mime,
1954        send_event_txn: OwnedTransactionId,
1955        created_at: MilliSecondsSinceUnixEpoch,
1956        upload_file_txn: OwnedTransactionId,
1957        file_media_request: MediaRequestParameters,
1958        thumbnail: Option<QueueThumbnailInfo>,
1959        extra_content: Option<serde_json::Map<String, serde_json::Value>>,
1960    ) -> Result<(), RoomSendQueueStorageError> {
1961        let guard = self.store.lock().await;
1962        let client = guard.client()?;
1963        let store = client.state_store();
1964
1965        // There's only a single media to be sent, so it has at most one
1966        // thumbnail.
1967        let thumbnail_file_sizes = vec![thumbnail.as_ref().map(|t| t.file_size)];
1968
1969        let thumbnail_info = self
1970            .push_thumbnail_and_media_uploads(
1971                store,
1972                &content_type,
1973                send_event_txn.clone(),
1974                created_at,
1975                upload_file_txn.clone(),
1976                file_media_request,
1977                thumbnail,
1978            )
1979            .await?;
1980
1981        // Push the dependent request for the event itself.
1982        store
1983            .save_dependent_queued_request(
1984                &self.room_id,
1985                &upload_file_txn,
1986                send_event_txn.clone().into(),
1987                created_at,
1988                DependentQueuedRequestKind::FinishUpload {
1989                    local_echo: Box::new(event),
1990                    file_upload: upload_file_txn.clone(),
1991                    thumbnail_info,
1992                    extra_content,
1993                },
1994            )
1995            .await?;
1996
1997        self.thumbnail_file_sizes.lock().insert(send_event_txn, thumbnail_file_sizes);
1998
1999        Ok(())
2000    }
2001
2002    /// Push requests (and dependents) to upload a gallery.
2003    ///
2004    /// See the module-level description for details of the whole processus.
2005    #[cfg(feature = "unstable-msc4274")]
2006    #[allow(clippy::too_many_arguments)]
2007    async fn push_gallery(
2008        &self,
2009        event: RoomMessageEventContent,
2010        send_event_txn: OwnedTransactionId,
2011        created_at: MilliSecondsSinceUnixEpoch,
2012        item_queue_infos: Vec<GalleryItemQueueInfo>,
2013        extra_content: Option<serde_json::Map<String, serde_json::Value>>,
2014    ) -> Result<(), RoomSendQueueStorageError> {
2015        let guard = self.store.lock().await;
2016        let client = guard.client()?;
2017        let store = client.state_store();
2018
2019        let mut finish_item_infos = Vec::with_capacity(item_queue_infos.len());
2020        let mut thumbnail_file_sizes = Vec::with_capacity(item_queue_infos.len());
2021
2022        let Some((first, rest)) = item_queue_infos.split_first() else {
2023            return Ok(());
2024        };
2025
2026        let GalleryItemQueueInfo {
2027            content_type,
2028            upload_file_txn,
2029            file_media_request,
2030            thumbnail,
2031            extra_content: item_extra_content,
2032        } = first;
2033
2034        let thumbnail_info = self
2035            .push_thumbnail_and_media_uploads(
2036                store,
2037                content_type,
2038                send_event_txn.clone(),
2039                created_at,
2040                upload_file_txn.clone(),
2041                file_media_request.clone(),
2042                thumbnail.clone(),
2043            )
2044            .await?;
2045
2046        finish_item_infos.push(FinishGalleryItemInfo {
2047            file_upload: upload_file_txn.clone(),
2048            thumbnail_info,
2049            extra_content: item_extra_content.clone(),
2050        });
2051        thumbnail_file_sizes.push(thumbnail.as_ref().map(|t| t.file_size));
2052
2053        let mut last_upload_file_txn = upload_file_txn.clone();
2054
2055        for item_queue_info in rest {
2056            let GalleryItemQueueInfo {
2057                content_type,
2058                upload_file_txn,
2059                file_media_request,
2060                thumbnail,
2061                extra_content: item_extra_content,
2062            } = item_queue_info;
2063
2064            let thumbnail_info = if let Some(QueueThumbnailInfo {
2065                finish_upload_thumbnail_info: thumbnail_info,
2066                media_request_parameters: thumbnail_media_request,
2067                content_type: thumbnail_content_type,
2068                ..
2069            }) = thumbnail
2070            {
2071                let upload_thumbnail_txn = thumbnail_info.txn.clone();
2072
2073                // Save the thumbnail upload request as a dependent request of
2074                // the last file upload.
2075                store
2076                    .save_dependent_queued_request(
2077                        &self.room_id,
2078                        &last_upload_file_txn,
2079                        upload_thumbnail_txn.clone().into(),
2080                        created_at,
2081                        DependentQueuedRequestKind::UploadFileOrThumbnail {
2082                            content_type: thumbnail_content_type.to_string(),
2083                            cache_key: thumbnail_media_request.clone(),
2084                            related_to: send_event_txn.clone(),
2085                            parent_is_thumbnail_upload: false,
2086                        },
2087                    )
2088                    .await?;
2089
2090                last_upload_file_txn = upload_thumbnail_txn;
2091
2092                Some(thumbnail_info)
2093            } else {
2094                None
2095            };
2096
2097            // Save the file upload as a dependent request of the previous
2098            // upload.
2099            store
2100                .save_dependent_queued_request(
2101                    &self.room_id,
2102                    &last_upload_file_txn,
2103                    upload_file_txn.clone().into(),
2104                    created_at,
2105                    DependentQueuedRequestKind::UploadFileOrThumbnail {
2106                        content_type: content_type.to_string(),
2107                        cache_key: file_media_request.clone(),
2108                        related_to: send_event_txn.clone(),
2109                        parent_is_thumbnail_upload: thumbnail.is_some(),
2110                    },
2111                )
2112                .await?;
2113
2114            finish_item_infos.push(FinishGalleryItemInfo {
2115                file_upload: upload_file_txn.clone(),
2116                thumbnail_info: thumbnail_info.cloned(),
2117                extra_content: item_extra_content.clone(),
2118            });
2119            thumbnail_file_sizes.push(thumbnail.as_ref().map(|t| t.file_size));
2120
2121            last_upload_file_txn = upload_file_txn.clone();
2122        }
2123
2124        // Push the request for the event itself as a dependent request of the
2125        // last file upload.
2126        store
2127            .save_dependent_queued_request(
2128                &self.room_id,
2129                &last_upload_file_txn,
2130                send_event_txn.clone().into(),
2131                created_at,
2132                DependentQueuedRequestKind::FinishGallery {
2133                    local_echo: Box::new(event),
2134                    item_infos: finish_item_infos,
2135                    extra_content,
2136                },
2137            )
2138            .await?;
2139
2140        self.thumbnail_file_sizes.lock().insert(send_event_txn, thumbnail_file_sizes);
2141
2142        Ok(())
2143    }
2144
2145    /// If a thumbnail exists, pushes a [`QueuedRequestKind::MediaUpload`] to
2146    /// upload it and a [`DependentQueuedRequestKind::UploadFileOrThumbnail`] to
2147    /// upload the media itself. Otherwise, pushes a
2148    /// [`QueuedRequestKind::MediaUpload`] to upload the media directly.
2149    #[allow(clippy::too_many_arguments)]
2150    async fn push_thumbnail_and_media_uploads(
2151        &self,
2152        store: &DynStateStore,
2153        content_type: &Mime,
2154        send_event_txn: OwnedTransactionId,
2155        created_at: MilliSecondsSinceUnixEpoch,
2156        upload_file_txn: OwnedTransactionId,
2157        file_media_request: MediaRequestParameters,
2158        thumbnail: Option<QueueThumbnailInfo>,
2159    ) -> Result<Option<FinishUploadThumbnailInfo>, RoomSendQueueStorageError> {
2160        if let Some(QueueThumbnailInfo {
2161            finish_upload_thumbnail_info: thumbnail_info,
2162            media_request_parameters: thumbnail_media_request,
2163            content_type: thumbnail_content_type,
2164            ..
2165        }) = thumbnail
2166        {
2167            let upload_thumbnail_txn = thumbnail_info.txn.clone();
2168
2169            // Save the thumbnail upload request.
2170            store
2171                .save_send_queue_request(
2172                    &self.room_id,
2173                    upload_thumbnail_txn.clone(),
2174                    created_at,
2175                    QueuedRequestKind::MediaUpload {
2176                        content_type: thumbnail_content_type.to_string(),
2177                        cache_key: thumbnail_media_request,
2178                        thumbnail_source: None, // the thumbnail has no thumbnails :)
2179                        related_to: send_event_txn.clone(),
2180                        #[cfg(feature = "unstable-msc4274")]
2181                        accumulated: vec![],
2182                    },
2183                    Self::LOW_PRIORITY,
2184                )
2185                .await?;
2186
2187            // Save the file upload request as a dependent request of the
2188            // thumbnail upload.
2189            store
2190                .save_dependent_queued_request(
2191                    &self.room_id,
2192                    &upload_thumbnail_txn,
2193                    upload_file_txn.into(),
2194                    created_at,
2195                    DependentQueuedRequestKind::UploadFileOrThumbnail {
2196                        content_type: content_type.to_string(),
2197                        cache_key: file_media_request,
2198                        related_to: send_event_txn,
2199                        parent_is_thumbnail_upload: true,
2200                    },
2201                )
2202                .await?;
2203
2204            Ok(Some(thumbnail_info))
2205        } else {
2206            // Save the file upload as its own request, not a dependent one.
2207            store
2208                .save_send_queue_request(
2209                    &self.room_id,
2210                    upload_file_txn,
2211                    created_at,
2212                    QueuedRequestKind::MediaUpload {
2213                        content_type: content_type.to_string(),
2214                        cache_key: file_media_request,
2215                        thumbnail_source: None,
2216                        related_to: send_event_txn,
2217                        #[cfg(feature = "unstable-msc4274")]
2218                        accumulated: vec![],
2219                    },
2220                    Self::LOW_PRIORITY,
2221                )
2222                .await?;
2223
2224            Ok(None)
2225        }
2226    }
2227
2228    /// Reacts to the given local echo of an event.
2229    #[instrument(skip(self))]
2230    async fn react(
2231        &self,
2232        transaction_id: &TransactionId,
2233        key: String,
2234        created_at: MilliSecondsSinceUnixEpoch,
2235    ) -> Result<Option<ChildTransactionId>, RoomSendQueueStorageError> {
2236        let guard = self.store.lock().await;
2237        let client = guard.client()?;
2238        let store = client.state_store();
2239
2240        let requests = store.load_send_queue_requests(&self.room_id).await?;
2241
2242        // If the target event has been already sent, or isn't something that
2243        // can be reacted to in the first place, abort immediately.
2244        if !requests
2245            .iter()
2246            .any(|item| item.transaction_id == transaction_id && is_own_event_request(item))
2247        {
2248            // We didn't find it as a queued request; try to find it as a
2249            // dependent queued request.
2250            let dependent_requests = store.load_dependent_queued_requests(&self.room_id).await?;
2251            if !dependent_requests
2252                .into_iter()
2253                .filter_map(|item| item.is_own_event().then_some(item.own_transaction_id))
2254                .any(|child_txn| *child_txn == *transaction_id)
2255            {
2256                // We didn't find it as either a request or a dependent request,
2257                // abort.
2258                return Ok(None);
2259            }
2260        }
2261
2262        // Record the dependent request.
2263        let reaction_txn_id = ChildTransactionId::new();
2264        store
2265            .save_dependent_queued_request(
2266                &self.room_id,
2267                transaction_id,
2268                reaction_txn_id.clone(),
2269                created_at,
2270                DependentQueuedRequestKind::ReactEvent { key },
2271            )
2272            .await?;
2273
2274        Ok(Some(reaction_txn_id))
2275    }
2276
2277    /// Returns a list of the local echoes, that is, all the requests that we're
2278    /// about to send but that haven't been sent yet (or are being sent).
2279    async fn local_echoes(
2280        &self,
2281        room: &RoomSendQueue,
2282    ) -> Result<Vec<LocalEcho>, RoomSendQueueStorageError> {
2283        let guard = self.store.lock().await;
2284        let client = guard.client()?;
2285        let store = client.state_store();
2286
2287        let queued_requests = store.load_send_queue_requests(&self.room_id).await?;
2288
2289        // Media upload requests aren't returned as echoes themselves (the media
2290        // event, represented as a dependent request, is), so carry their send
2291        // errors over to the dependent request's echo: a wedged upload wedges
2292        // the media event.
2293        let mut media_upload_errors: HashMap<OwnedTransactionId, QueueWedgeError> = queued_requests
2294            .iter()
2295            .filter_map(|queued| match queued.kind {
2296                QueuedRequestKind::MediaUpload { .. } => {
2297                    queued.error.clone().map(|error| (queued.transaction_id.clone(), error))
2298                }
2299                _ => None,
2300            })
2301            .collect();
2302
2303        let local_requests = queued_requests.into_iter().filter_map(|queued| {
2304            Some(LocalEcho {
2305                transaction_id: queued.transaction_id.clone(),
2306                content: match queued.kind {
2307                    QueuedRequestKind::Event { content, .. } => LocalEchoContent::Event {
2308                        serialized_event: content,
2309                        send_handle: SendHandle {
2310                            room: room.clone(),
2311                            transaction_id: queued.transaction_id,
2312                            media_handles: vec![],
2313                            created_at: queued.created_at,
2314                        },
2315                        send_error: queued.error,
2316                    },
2317
2318                    QueuedRequestKind::MediaUpload { .. } => {
2319                        // Don't return uploaded medias as their own things; the
2320                        // accompanying event represented as a dependent request
2321                        // should be sufficient.
2322                        return None;
2323                    }
2324
2325                    QueuedRequestKind::Redaction { redacts, reason } => {
2326                        LocalEchoContent::Redaction {
2327                            redacts,
2328                            reason,
2329                            send_handle: SendHandle {
2330                                room: room.clone(),
2331                                transaction_id: queued.transaction_id,
2332                                media_handles: vec![],
2333                                created_at: queued.created_at,
2334                            },
2335                            send_error: queued.error,
2336                        }
2337                    }
2338                },
2339            })
2340        });
2341
2342        let dependent_requests = store.load_dependent_queued_requests(&self.room_id).await?;
2343
2344        let reactions_and_medias =
2345            dependent_requests.into_iter().filter_map(|dep| match dep.kind {
2346                DependentQueuedRequestKind::EditEvent { .. }
2347                | DependentQueuedRequestKind::RedactEvent
2348                | DependentQueuedRequestKind::RedactEventWithReason { .. } => {
2349                    // TODO: reflect local edits/redacts too?
2350                    None
2351                }
2352
2353                DependentQueuedRequestKind::ReactEvent { key } => Some(LocalEcho {
2354                    transaction_id: dep.own_transaction_id.clone().into(),
2355                    content: LocalEchoContent::React {
2356                        key,
2357                        send_handle: SendHandle {
2358                            room: room.clone(),
2359                            transaction_id: dep.own_transaction_id.into(),
2360                            media_handles: vec![],
2361                            created_at: dep.created_at,
2362                        },
2363                        applies_to: dep.parent_transaction_id,
2364                    },
2365                }),
2366
2367                DependentQueuedRequestKind::UploadFileOrThumbnail { .. } => {
2368                    // Don't reflect these: only the associated event is
2369                    // interesting to observers.
2370                    None
2371                }
2372
2373                DependentQueuedRequestKind::FinishUpload {
2374                    local_echo,
2375                    file_upload,
2376                    thumbnail_info,
2377                    extra_content,
2378                } => {
2379                    let upload_thumbnail_txn = thumbnail_info.map(|info| info.txn);
2380
2381                    // If one of the uploads wedged, the media event is wedged
2382                    // too.
2383                    let send_error = media_upload_errors.remove(&file_upload).or_else(|| {
2384                        upload_thumbnail_txn
2385                            .as_ref()
2386                            .and_then(|txn| media_upload_errors.remove(&**txn))
2387                    });
2388
2389                    // Materialize as an event local echo.
2390                    Some(LocalEcho {
2391                        transaction_id: dep.own_transaction_id.clone().into(),
2392                        content: LocalEchoContent::Event {
2393                            serialized_event: upload::merge_extra_content(
2394                                SerializableEventContent::new(&(*local_echo).into()).ok()?,
2395                                extra_content,
2396                            )
2397                            .ok()?,
2398                            send_handle: SendHandle {
2399                                room: room.clone(),
2400                                transaction_id: dep.own_transaction_id.into(),
2401                                media_handles: vec![MediaHandles {
2402                                    upload_thumbnail_txn,
2403                                    upload_file_txn: file_upload,
2404                                }],
2405                                created_at: dep.created_at,
2406                            },
2407                            send_error,
2408                        },
2409                    })
2410                }
2411
2412                #[cfg(feature = "unstable-msc4274")]
2413                DependentQueuedRequestKind::FinishGallery {
2414                    local_echo,
2415                    item_infos,
2416                    extra_content,
2417                } => {
2418                    // Materialize as an event local echo.
2419                    self.create_gallery_local_echo(
2420                        dep.own_transaction_id,
2421                        room,
2422                        dep.created_at,
2423                        local_echo,
2424                        item_infos,
2425                        extra_content,
2426                        &mut media_upload_errors,
2427                    )
2428                }
2429            });
2430
2431        Ok(local_requests.chain(reactions_and_medias).collect())
2432    }
2433
2434    /// Create a local echo for a gallery event.
2435    #[cfg(feature = "unstable-msc4274")]
2436    #[allow(clippy::too_many_arguments)]
2437    fn create_gallery_local_echo(
2438        &self,
2439        transaction_id: ChildTransactionId,
2440        room: &RoomSendQueue,
2441        created_at: MilliSecondsSinceUnixEpoch,
2442        local_echo: Box<RoomMessageEventContent>,
2443        item_infos: Vec<FinishGalleryItemInfo>,
2444        extra_content: Option<serde_json::Map<String, serde_json::Value>>,
2445        media_upload_errors: &mut HashMap<OwnedTransactionId, QueueWedgeError>,
2446    ) -> Option<LocalEcho> {
2447        // If any of the uploads wedged, the gallery event is wedged too.
2448        let send_error = item_infos.iter().find_map(|i| {
2449            media_upload_errors.remove(&i.file_upload).or_else(|| {
2450                i.thumbnail_info.as_ref().and_then(|info| media_upload_errors.remove(&*info.txn))
2451            })
2452        });
2453
2454        Some(LocalEcho {
2455            transaction_id: transaction_id.clone().into(),
2456            content: LocalEchoContent::Event {
2457                serialized_event: upload::merge_gallery_extra_content(
2458                    SerializableEventContent::new(&(*local_echo).into()).ok()?,
2459                    extra_content,
2460                    item_infos.iter().map(|item| item.extra_content.clone()),
2461                )
2462                .ok()?,
2463                send_handle: SendHandle {
2464                    room: room.clone(),
2465                    transaction_id: transaction_id.into(),
2466                    media_handles: item_infos
2467                        .into_iter()
2468                        .map(|i| MediaHandles {
2469                            upload_thumbnail_txn: i.thumbnail_info.map(|info| info.txn),
2470                            upload_file_txn: i.file_upload,
2471                        })
2472                        .collect(),
2473                    created_at,
2474                },
2475                send_error,
2476            },
2477        })
2478    }
2479
2480    /// Try to apply a single dependent request, whether it's local or remote.
2481    ///
2482    /// This swallows errors that would retrigger every time if we retried
2483    /// applying the dependent request: invalid edit content, etc.
2484    ///
2485    /// Returns true if the dependent request has been sent (or should not be
2486    /// retried later).
2487    #[instrument(skip_all)]
2488    async fn try_apply_single_dependent_request(
2489        &self,
2490        client: &Client,
2491        dependent_request: DependentQueuedRequest,
2492        new_updates: &mut Vec<RoomSendQueueUpdate>,
2493    ) -> Result<bool, RoomSendQueueError> {
2494        let store = client.state_store();
2495
2496        let parent_key = dependent_request.parent_key;
2497
2498        match dependent_request.kind {
2499            DependentQueuedRequestKind::EditEvent { new_content } => {
2500                if let Some(parent_key) = parent_key {
2501                    let Some(event_id) = parent_key.into_event_id() else {
2502                        return Err(RoomSendQueueError::StorageError(
2503                            RoomSendQueueStorageError::InvalidParentKey,
2504                        ));
2505                    };
2506
2507                    // The parent event has been sent, so send an edit event.
2508                    let room = client
2509                        .get_room(&self.room_id)
2510                        .ok_or(RoomSendQueueError::RoomDisappeared)?;
2511
2512                    // Check the event is one we know how to edit with an edit
2513                    // event.
2514
2515                    // Fields the typed content doesn't know about, like an
2516                    // attachment's extra content, to put back in the edit.
2517                    let mut unknown_fields = None;
2518
2519                    // It must be deserializable…
2520                    let edited_content = match new_content.deserialize() {
2521                        Ok(AnyMessageLikeEventContent::RoomMessage(c)) => {
2522                            unknown_fields = upload::unknown_fields(&new_content, &c);
2523
2524                            // Assume no relationships.
2525                            EditedContent::RoomMessage(c.into())
2526                        }
2527
2528                        Ok(AnyMessageLikeEventContent::UnstablePollStart(c)) => {
2529                            let poll_start = c.poll_start().clone();
2530                            EditedContent::PollStart {
2531                                fallback_text: poll_start.question.text.clone(),
2532                                new_content: poll_start,
2533                            }
2534                        }
2535
2536                        Ok(c) => {
2537                            warn!("Unsupported edit content type: {:?}", c.event_type());
2538                            return Ok(true);
2539                        }
2540
2541                        Err(err) => {
2542                            warn!("Unable to deserialize: {err}");
2543                            return Ok(true);
2544                        }
2545                    };
2546
2547                    let edit_event = match room.make_edit_event(&event_id, edited_content).await {
2548                        Ok(e) => e,
2549                        Err(err) => {
2550                            warn!("couldn't create edited event: {err}");
2551                            return Ok(true);
2552                        }
2553                    };
2554
2555                    let mut edit_json = serde_json::to_value(&edit_event)
2556                        .map_err(RoomSendQueueStorageError::JsonSerialization)?;
2557
2558                    // Only in the new content: the top level is what push rules
2559                    // see, and an edit mustn't notify again.
2560                    if let Some(serde_json::Value::Object(mut unknown)) = unknown_fields
2561                        && let Some(edited) = edit_json.get_mut("m.new_content")
2562                    {
2563                        // The edit has its own relation and new content.
2564                        unknown.remove("m.relates_to");
2565                        unknown.remove("m.new_content");
2566                        upload::restore_unknown_fields(edited, unknown.into());
2567                    }
2568
2569                    // Queue the edit event in the send queue 🧠.
2570                    let serializable = SerializableEventContent::from_raw(
2571                        Raw::from_json(
2572                            serde_json::value::to_raw_value(&edit_json)
2573                                .map_err(RoomSendQueueStorageError::JsonSerialization)?,
2574                        ),
2575                        edit_event.event_type().to_string(),
2576                    );
2577
2578                    store
2579                        .save_send_queue_request(
2580                            &self.room_id,
2581                            dependent_request.own_transaction_id.into(),
2582                            dependent_request.created_at,
2583                            serializable.into(),
2584                            Self::HIGH_PRIORITY,
2585                        )
2586                        .await
2587                        .map_err(RoomSendQueueStorageError::StateStoreError)?;
2588                } else {
2589                    // The parent event is still local; update the local echo.
2590                    let parent_transaction_id = &dependent_request.parent_transaction_id;
2591
2592                    let request = QueuedRequestKind::Event {
2593                        content: new_content,
2594                        #[cfg(feature = "unstable-msc4354")]
2595                        sticky_duration: self
2596                            .sticky_duration_of(store, parent_transaction_id)
2597                            .await?,
2598                    };
2599
2600                    let edited = store
2601                        .update_send_queue_request(&self.room_id, parent_transaction_id, request)
2602                        .await
2603                        .map_err(RoomSendQueueStorageError::StateStoreError)?;
2604
2605                    if !edited {
2606                        warn!("missing local echo upon dependent edit");
2607                    }
2608                }
2609            }
2610
2611            kind @ (DependentQueuedRequestKind::RedactEvent
2612            | DependentQueuedRequestKind::RedactEventWithReason { .. }) => {
2613                let reason = match kind {
2614                    DependentQueuedRequestKind::RedactEventWithReason { reason } => reason,
2615                    // The legacy variant carries no reason.
2616                    _ => None,
2617                };
2618
2619                if let Some(parent_key) = parent_key {
2620                    let Some(event_id) = parent_key.into_event_id() else {
2621                        return Err(RoomSendQueueError::StorageError(
2622                            RoomSendQueueStorageError::InvalidParentKey,
2623                        ));
2624                    };
2625
2626                    // The parent event has been sent; send a redaction.
2627                    let room = client
2628                        .get_room(&self.room_id)
2629                        .ok_or(RoomSendQueueError::RoomDisappeared)?;
2630
2631                    // Ideally we'd use the send queue to send the redaction,
2632                    // but the protocol has changed the shape of a
2633                    // room.redaction after v11, so keep it simple and try once
2634                    // here.
2635
2636                    if let Err(err) = room
2637                        .redact(
2638                            &event_id,
2639                            reason.as_deref(),
2640                            Some(dependent_request.own_transaction_id.into()),
2641                        )
2642                        .await
2643                    {
2644                        warn!("error when sending a redact for {event_id}: {err}");
2645                        return Ok(false);
2646                    }
2647                } else {
2648                    // The parent event is still local (sending must have
2649                    // failed); redact the local echo.
2650                    let removed = store
2651                        .remove_send_queue_request(
2652                            &self.room_id,
2653                            &dependent_request.parent_transaction_id,
2654                        )
2655                        .await
2656                        .map_err(RoomSendQueueStorageError::StateStoreError)?;
2657
2658                    if !removed {
2659                        warn!("missing local echo upon dependent redact");
2660                    }
2661                }
2662            }
2663
2664            DependentQueuedRequestKind::ReactEvent { key } => {
2665                if let Some(parent_key) = parent_key {
2666                    let Some(parent_event_id) = parent_key.into_event_id() else {
2667                        return Err(RoomSendQueueError::StorageError(
2668                            RoomSendQueueStorageError::InvalidParentKey,
2669                        ));
2670                    };
2671
2672                    // Queue the reaction event in the send queue 🧠.
2673                    let react_event =
2674                        ReactionEventContent::new(Annotation::new(parent_event_id, key)).into();
2675                    let serializable = SerializableEventContent::from_raw(
2676                        Raw::new(&react_event)
2677                            .map_err(RoomSendQueueStorageError::JsonSerialization)?,
2678                        react_event.event_type().to_string(),
2679                    );
2680
2681                    store
2682                        .save_send_queue_request(
2683                            &self.room_id,
2684                            dependent_request.own_transaction_id.into(),
2685                            dependent_request.created_at,
2686                            serializable.into(),
2687                            Self::HIGH_PRIORITY,
2688                        )
2689                        .await
2690                        .map_err(RoomSendQueueStorageError::StateStoreError)?;
2691                } else {
2692                    // Not applied yet, we should retry later => false.
2693                    return Ok(false);
2694                }
2695            }
2696
2697            DependentQueuedRequestKind::UploadFileOrThumbnail {
2698                content_type,
2699                cache_key,
2700                related_to,
2701                parent_is_thumbnail_upload,
2702            } => {
2703                let Some(parent_key) = parent_key else {
2704                    // Not finished yet, we should retry later => false.
2705                    return Ok(false);
2706                };
2707                self.handle_dependent_file_or_thumbnail_upload(
2708                    client,
2709                    dependent_request.own_transaction_id.into(),
2710                    parent_key,
2711                    content_type,
2712                    cache_key,
2713                    related_to,
2714                    parent_is_thumbnail_upload,
2715                )
2716                .await?;
2717            }
2718
2719            DependentQueuedRequestKind::FinishUpload {
2720                local_echo,
2721                file_upload,
2722                thumbnail_info,
2723                extra_content,
2724            } => {
2725                let Some(parent_key) = parent_key else {
2726                    // Not finished yet, we should retry later => false.
2727                    return Ok(false);
2728                };
2729                self.handle_dependent_finish_upload(
2730                    client,
2731                    dependent_request.own_transaction_id.into(),
2732                    parent_key,
2733                    *local_echo,
2734                    file_upload,
2735                    thumbnail_info,
2736                    extra_content,
2737                    new_updates,
2738                )
2739                .await?;
2740            }
2741
2742            #[cfg(feature = "unstable-msc4274")]
2743            DependentQueuedRequestKind::FinishGallery { local_echo, item_infos, extra_content } => {
2744                let Some(parent_key) = parent_key else {
2745                    // Not finished yet, we should retry later => false.
2746                    return Ok(false);
2747                };
2748                self.handle_dependent_finish_gallery_upload(
2749                    client,
2750                    dependent_request.own_transaction_id.into(),
2751                    parent_key,
2752                    *local_echo,
2753                    item_infos,
2754                    extra_content,
2755                    new_updates,
2756                )
2757                .await?;
2758            }
2759        }
2760
2761        Ok(true)
2762    }
2763
2764    #[instrument(skip(self))]
2765    async fn apply_dependent_requests(
2766        &self,
2767        new_updates: &mut Vec<RoomSendQueueUpdate>,
2768    ) -> Result<(), RoomSendQueueError> {
2769        let guard = self.store.lock().await;
2770
2771        let client = guard.client()?;
2772        let store = client.state_store();
2773
2774        let dependent_requests = store
2775            .load_dependent_queued_requests(&self.room_id)
2776            .await
2777            .map_err(RoomSendQueueStorageError::StateStoreError)?;
2778
2779        let num_initial_dependent_requests = dependent_requests.len();
2780        if num_initial_dependent_requests == 0 {
2781            // Returning early here avoids a bit of useless logging.
2782            return Ok(());
2783        }
2784
2785        let canonicalized_dependent_requests = canonicalize_dependent_requests(&dependent_requests);
2786
2787        // Get rid of the all non-canonical dependent events.
2788        for original in &dependent_requests {
2789            if !canonicalized_dependent_requests
2790                .iter()
2791                .any(|canonical| canonical.own_transaction_id == original.own_transaction_id)
2792            {
2793                store
2794                    .remove_dependent_queued_request(&self.room_id, &original.own_transaction_id)
2795                    .await
2796                    .map_err(RoomSendQueueStorageError::StateStoreError)?;
2797            }
2798        }
2799
2800        let mut num_dependent_requests = canonicalized_dependent_requests.len();
2801
2802        debug!(
2803            num_dependent_requests,
2804            num_initial_dependent_requests, "starting handling of dependent requests"
2805        );
2806
2807        for dependent in canonicalized_dependent_requests {
2808            let dependent_id = dependent.own_transaction_id.clone();
2809
2810            match self.try_apply_single_dependent_request(&client, dependent, new_updates).await {
2811                Ok(should_remove) => {
2812                    if should_remove {
2813                        // The dependent request has been successfully applied,
2814                        // forget about it.
2815                        store
2816                            .remove_dependent_queued_request(&self.room_id, &dependent_id)
2817                            .await
2818                            .map_err(RoomSendQueueStorageError::StateStoreError)?;
2819
2820                        num_dependent_requests -= 1;
2821                    }
2822                }
2823
2824                Err(err) => {
2825                    warn!("error when applying single dependent request: {err}");
2826                }
2827            }
2828        }
2829
2830        debug!(
2831            leftover_dependent_requests = num_dependent_requests,
2832            "stopped handling dependent request"
2833        );
2834
2835        Ok(())
2836    }
2837
2838    /// Remove a single dependent request from storage.
2839    async fn remove_dependent_send_queue_request(
2840        &self,
2841        dependent_event_id: &ChildTransactionId,
2842    ) -> Result<bool, RoomSendQueueStorageError> {
2843        Ok(self
2844            .store
2845            .lock()
2846            .await
2847            .client()?
2848            .state_store()
2849            .remove_dependent_queued_request(&self.room_id, dependent_event_id)
2850            .await?)
2851    }
2852}
2853
2854#[cfg(feature = "unstable-msc4274")]
2855/// Metadata needed for pushing gallery item uploads onto the send queue.
2856struct GalleryItemQueueInfo {
2857    content_type: Mime,
2858    upload_file_txn: OwnedTransactionId,
2859    file_media_request: MediaRequestParameters,
2860    thumbnail: Option<QueueThumbnailInfo>,
2861    extra_content: Option<serde_json::Map<String, serde_json::Value>>,
2862}
2863
2864/// The content of a local echo.
2865#[derive(Clone, Debug)]
2866pub enum LocalEchoContent {
2867    /// The local echo contains an actual event ready to display.
2868    Event {
2869        /// Content of the event itself (along with its type) that we are about
2870        /// to send.
2871        serialized_event: SerializableEventContent,
2872        /// A handle to manipulate the sending of the associated event.
2873        send_handle: SendHandle,
2874        /// Whether trying to send this local echo failed in the past with an
2875        /// unrecoverable error (see [`SendQueueRoomError::is_recoverable`]).
2876        send_error: Option<QueueWedgeError>,
2877    },
2878
2879    /// A local echo has been reacted to.
2880    React {
2881        /// The key with which the local echo has been reacted to.
2882        key: String,
2883        /// A handle to manipulate the sending of the reaction.
2884        send_handle: SendHandle,
2885        /// The local echo which has been reacted to.
2886        applies_to: OwnedTransactionId,
2887    },
2888
2889    /// A local echo of a redaction event.
2890    Redaction {
2891        /// The ID of the redacted event.
2892        redacts: OwnedEventId,
2893        /// The reason for the event being redacted.
2894        reason: Option<String>,
2895        /// A handle to manipulate the sending of the associated event.
2896        send_handle: SendHandle,
2897        /// Whether trying to send this local echo failed in the past with an
2898        /// unrecoverable error (see [`SendQueueRoomError::is_recoverable`]).
2899        send_error: Option<QueueWedgeError>,
2900    },
2901}
2902
2903/// A local representation for a request that hasn't been sent yet to the user's
2904/// homeserver.
2905#[derive(Clone, Debug)]
2906pub struct LocalEcho {
2907    /// Transaction id used to identify the associated request.
2908    pub transaction_id: OwnedTransactionId,
2909    /// The content for the local echo.
2910    pub content: LocalEchoContent,
2911}
2912
2913/// An update to a room send queue, observable with
2914/// [`RoomSendQueue::subscribe`].
2915#[derive(Clone, Debug)]
2916pub enum RoomSendQueueUpdate {
2917    /// A new local event is being sent.
2918    ///
2919    /// There's been a user query to create this event. It is being sent to the
2920    /// server.
2921    NewLocalEvent(LocalEcho),
2922
2923    /// A local event that hadn't been sent to the server yet has been cancelled
2924    /// before sending.
2925    CancelledLocalEvent {
2926        /// Transaction id used to identify this event.
2927        transaction_id: OwnedTransactionId,
2928    },
2929
2930    /// A local event's content has been replaced with something else.
2931    ReplacedLocalEvent {
2932        /// Transaction id used to identify this event.
2933        transaction_id: OwnedTransactionId,
2934
2935        /// The new content replacing the previous one.
2936        new_content: SerializableEventContent,
2937    },
2938
2939    /// An error happened when an event was being sent.
2940    ///
2941    /// The event has not been removed from the queue. A recoverable error
2942    /// disables the room's send queue, which must then be manually re-enabled;
2943    /// an unrecoverable one wedges the request, which blocks its room's queue
2944    /// until the request is unwedged or aborted.
2945    SendError {
2946        /// Transaction id used to identify this event.
2947        transaction_id: OwnedTransactionId,
2948        /// Error received while sending the event.
2949        error: Arc<crate::Error>,
2950        /// Whether the error is considered recoverable or not.
2951        ///
2952        /// An error that's recoverable will disable the room's send queue,
2953        /// while an unrecoverable error will be parked, until it's retried or
2954        /// aborted.
2955        is_recoverable: bool,
2956    },
2957
2958    /// The event has been unwedged and sending is now being retried.
2959    RetryEvent {
2960        /// Transaction id used to identify this event.
2961        transaction_id: OwnedTransactionId,
2962    },
2963
2964    /// The event has been sent to the server, and the query returned
2965    /// successfully.
2966    SentEvent {
2967        /// Transaction id used to identify this event.
2968        transaction_id: OwnedTransactionId,
2969        /// Received event id from the send response.
2970        event_id: OwnedEventId,
2971    },
2972
2973    /// A media upload (consisting of a file and possibly a thumbnail) has made
2974    /// progress.
2975    MediaUpload {
2976        /// The media event this uploaded media relates to.
2977        related_to: OwnedTransactionId,
2978
2979        /// The final media source for the file if it has finished uploading.
2980        file: Option<MediaSource>,
2981
2982        /// The index of the media within the transaction. A file and its
2983        /// thumbnail share the same index. Will always be 0 for non-gallery
2984        /// media uploads.
2985        index: u64,
2986
2987        /// The combined upload progress across the file and, if existing, its
2988        /// thumbnail. For gallery uploads, the progress is reported per indexed
2989        /// gallery item.
2990        progress: AbstractProgress,
2991    },
2992}
2993
2994/// A [`RoomSendQueueUpdate`] with an associated [`OwnedRoomId`].
2995///
2996/// This is used by [`SendQueue::subscribe`] to get a single channel to receive
2997/// updates for all [`RoomSendQueue`]s.
2998#[derive(Clone, Debug)]
2999pub struct SendQueueUpdate {
3000    /// The room where the update happened.
3001    pub room_id: OwnedRoomId,
3002
3003    /// The update for this room.
3004    pub update: RoomSendQueueUpdate,
3005}
3006
3007/// An error triggered by the send queue module.
3008#[derive(Debug, thiserror::Error)]
3009pub enum RoomSendQueueError {
3010    /// The room isn't in the joined state.
3011    #[error("the room isn't in the joined state")]
3012    RoomNotJoined,
3013
3014    /// The room is missing from the client.
3015    ///
3016    /// This happens only whenever the client is shutting down.
3017    #[error("the room is now missing from the client")]
3018    RoomDisappeared,
3019
3020    /// Error coming from storage.
3021    #[error(transparent)]
3022    StorageError(#[from] RoomSendQueueStorageError),
3023
3024    /// The attachment event failed to be created.
3025    #[error("the attachment event could not be created")]
3026    FailedToCreateAttachment,
3027
3028    /// The target of an [`RoomSendQueue::edit_with_attachment`] can't be
3029    /// edited with a new attachment.
3030    #[error(transparent)]
3031    Edit(#[from] crate::room::edit::EditError),
3032
3033    /// The gallery contains no items.
3034    #[cfg(feature = "unstable-msc4274")]
3035    #[error("the gallery contains no items")]
3036    EmptyGallery,
3037
3038    /// The gallery event failed to be created.
3039    #[cfg(feature = "unstable-msc4274")]
3040    #[error("the gallery event could not be created")]
3041    FailedToCreateGallery,
3042}
3043
3044/// An error triggered by the send queue storage.
3045#[derive(Debug, thiserror::Error)]
3046pub enum RoomSendQueueStorageError {
3047    /// Error caused by the state store.
3048    #[error(transparent)]
3049    StateStoreError(#[from] StoreError),
3050
3051    /// Error caused by the event cache store.
3052    #[error(transparent)]
3053    EventCacheStoreError(#[from] EventCacheStoreError),
3054
3055    /// Error caused by the event cache store.
3056    #[error(transparent)]
3057    MediaStoreError(#[from] MediaStoreError),
3058
3059    /// Error caused when attempting to get a handle on the event cache store.
3060    #[error(transparent)]
3061    LockError(#[from] CrossProcessLockError),
3062
3063    /// Error caused when (de)serializing into/from json.
3064    #[error(transparent)]
3065    JsonSerialization(#[from] serde_json::Error),
3066
3067    /// A parent key was expected to be of a certain type, and it was another
3068    /// type instead.
3069    #[error("a dependent event had an invalid parent key type")]
3070    InvalidParentKey,
3071
3072    /// The client is shutting down.
3073    #[error("The client is shutting down.")]
3074    ClientShuttingDown,
3075
3076    /// An operation not implemented on a send handle.
3077    #[error("This operation is not implemented for media uploads")]
3078    OperationNotImplementedYet,
3079
3080    /// Trying to edit a media caption for something that's not a media.
3081    #[error("Can't edit a media caption when the underlying event isn't a media")]
3082    InvalidMediaCaptionEdit,
3083}
3084
3085/// Extra transaction IDs useful during an upload.
3086#[derive(Clone, Debug)]
3087struct MediaHandles {
3088    /// Transaction id used when uploading the thumbnail.
3089    ///
3090    /// Optional because a media can be uploaded without a thumbnail.
3091    upload_thumbnail_txn: Option<OwnedTransactionId>,
3092
3093    /// Transaction id used when uploading the media itself.
3094    upload_file_txn: OwnedTransactionId,
3095}
3096
3097/// Future returned by [`RoomSendQueue::send`].
3098#[allow(missing_debug_implementations)]
3099pub struct SendEvent<'a> {
3100    queue: &'a RoomSendQueue,
3101    content: AnyMessageLikeEventContent,
3102    extra_content: Option<serde_json::Map<String, serde_json::Value>>,
3103    #[cfg(feature = "unstable-msc4354")]
3104    sticky_duration: Option<StickyDurationMs>,
3105}
3106
3107impl<'a> SendEvent<'a> {
3108    /// Merge additional top-level fields into the outgoing event's content.
3109    ///
3110    /// The event's own fields take precedence on conflicts.
3111    pub fn with_extra_content(
3112        mut self,
3113        extra_content: serde_json::Map<String, serde_json::Value>,
3114    ) -> Self {
3115        self.extra_content = Some(extra_content);
3116        self
3117    }
3118
3119    /// Send the event as a sticky event for `duration`, clamped to one hour.
3120    ///
3121    /// Note that if the homeserver doesn't support sticky events, it will
3122    /// ignore the duration and send the event unsticky. Server support can
3123    /// be checked with [`Client::supports_sticky_events`].
3124    #[cfg(feature = "unstable-msc4354")]
3125    pub fn with_sticky_duration(mut self, duration: Duration) -> Self {
3126        self.sticky_duration = Some(sticky_duration_ms(duration));
3127        self
3128    }
3129}
3130
3131impl<'a> IntoFuture for SendEvent<'a> {
3132    type Output = Result<SendHandle, RoomSendQueueError>;
3133    boxed_into_future!(extra_bounds: 'a);
3134
3135    fn into_future(self) -> Self::IntoFuture {
3136        Box::pin(async move {
3137            let serialized = upload::merge_extra_content(
3138                SerializableEventContent::new(&self.content)
3139                    .map_err(RoomSendQueueStorageError::JsonSerialization)?,
3140                self.extra_content,
3141            )?;
3142            self.queue
3143                .send_serialized(
3144                    serialized,
3145                    #[cfg(feature = "unstable-msc4354")]
3146                    self.sticky_duration,
3147                )
3148                .await
3149        })
3150    }
3151}
3152
3153/// Future returned by [`RoomSendQueue::send_raw`].
3154#[allow(missing_debug_implementations)]
3155pub struct SendRawEvent<'a> {
3156    queue: &'a RoomSendQueue,
3157    content: SerializableEventContent,
3158    #[cfg(feature = "unstable-msc4354")]
3159    sticky_duration: Option<StickyDurationMs>,
3160}
3161
3162impl SendRawEvent<'_> {
3163    /// Send the event as a sticky event for `duration`, clamped to one hour.
3164    ///
3165    /// Note that if the homeserver doesn't support sticky events, it will
3166    /// ignore the duration and send the event unsticky. Server support can
3167    /// be checked with [`Client::supports_sticky_events`].
3168    #[cfg(feature = "unstable-msc4354")]
3169    pub fn with_sticky_duration(mut self, duration: Duration) -> Self {
3170        self.sticky_duration = Some(sticky_duration_ms(duration));
3171        self
3172    }
3173}
3174
3175impl<'a> IntoFuture for SendRawEvent<'a> {
3176    type Output = Result<SendHandle, RoomSendQueueError>;
3177    boxed_into_future!(extra_bounds: 'a);
3178
3179    fn into_future(self) -> Self::IntoFuture {
3180        Box::pin(async move {
3181            self.queue
3182                .send_serialized(
3183                    self.content,
3184                    #[cfg(feature = "unstable-msc4354")]
3185                    self.sticky_duration,
3186                )
3187                .await
3188        })
3189    }
3190}
3191
3192/// A handle to manipulate an event that was scheduled to be sent to a room.
3193///
3194/// The event may be a room message, a media upload, a redaction or a reaction;
3195/// [`Self::edit`] and [`Self::react`] only apply to the first two, and return
3196/// `false`/`None` for the others.
3197#[derive(Clone, Debug)]
3198pub struct SendHandle {
3199    /// Link to the send queue used to send this request.
3200    room: RoomSendQueue,
3201
3202    /// Transaction id used for the sent request.
3203    ///
3204    /// If this is a media upload, this is the "main" transaction id, i.e. the
3205    /// one used to send the event, and that will be seen by observers.
3206    transaction_id: OwnedTransactionId,
3207
3208    /// Additional handles for a media upload.
3209    media_handles: Vec<MediaHandles>,
3210
3211    /// The time at which the event to be sent has been created.
3212    pub created_at: MilliSecondsSinceUnixEpoch,
3213}
3214
3215impl SendHandle {
3216    /// Creates a new [`SendHandle`].
3217    #[cfg(test)]
3218    pub(crate) fn new(
3219        room: RoomSendQueue,
3220        transaction_id: OwnedTransactionId,
3221        created_at: MilliSecondsSinceUnixEpoch,
3222    ) -> Self {
3223        Self { room, transaction_id, media_handles: vec![], created_at }
3224    }
3225
3226    /// Returns the [`TransactionId`] used for sending the associated event.
3227    pub fn transaction_id(&self) -> &TransactionId {
3228        &self.transaction_id
3229    }
3230
3231    fn nyi_for_uploads(&self) -> Result<(), RoomSendQueueStorageError> {
3232        if !self.media_handles.is_empty() {
3233            Err(RoomSendQueueStorageError::OperationNotImplementedYet)
3234        } else {
3235            Ok(())
3236        }
3237    }
3238
3239    /// Aborts the sending of the event, if it wasn't sent yet.
3240    ///
3241    /// Returns true if the sending could be aborted, false if not (i.e. the
3242    /// event had already been sent).
3243    pub async fn abort(&self) -> Result<bool, RoomSendQueueStorageError> {
3244        self.abort_with_reason(None).await
3245    }
3246
3247    /// Aborts the sending of the event, if it wasn't sent yet, with an optional
3248    /// reason.
3249    ///
3250    /// If the event was being sent when the abort was requested and the send
3251    /// succeeds, the event is redacted server-side; the given reason is applied
3252    /// to that redaction. It is unused in every other case (the local echo is
3253    /// simply dropped).
3254    ///
3255    /// Returns true if the sending could be aborted, false if not (i.e. the
3256    /// event had already been sent).
3257    #[instrument(skip(self), fields(room_id = %self.room.inner.room.room_id(), txn_id = %self.transaction_id))]
3258    pub async fn abort_with_reason(
3259        &self,
3260        reason: Option<String>,
3261    ) -> Result<bool, RoomSendQueueStorageError> {
3262        trace!("received an abort request");
3263
3264        let queue = &self.room.inner.queue;
3265
3266        for handles in &self.media_handles {
3267            if queue.abort_upload(&self.transaction_id, handles).await? {
3268                // Wake up the queue, in case it was blocked on this request
3269                // being wedged.
3270                self.room.inner.notifier.notify_one();
3271
3272                // Propagate a cancelled update.
3273                self.room.send_update(RoomSendQueueUpdate::CancelledLocalEvent {
3274                    transaction_id: self.transaction_id.clone(),
3275                });
3276
3277                return Ok(true);
3278            }
3279
3280            // If it failed, it means the sending of the event is not a
3281            // dependent request anymore. Fall back to the regular code path
3282            // below, that handles aborting sending of an event.
3283        }
3284
3285        // A reaction is queued as a dependent request of the event it applies
3286        // to, so it has no entry in the main queue as long as that
3287        // event hasn't been sent.
3288        let aborted =
3289            queue.remove_dependent_send_queue_request(&self.transaction_id.clone().into()).await?
3290                || queue.cancel_event(&self.transaction_id, reason).await?;
3291
3292        if aborted {
3293            trace!("successful abort");
3294
3295            // Wake up the queue, in case it was blocked on this request being
3296            // wedged.
3297            self.room.inner.notifier.notify_one();
3298
3299            // Propagate a cancelled update too.
3300            self.room.send_update(RoomSendQueueUpdate::CancelledLocalEvent {
3301                transaction_id: self.transaction_id.clone(),
3302            });
3303
3304            Ok(true)
3305        } else {
3306            debug!("local echo didn't exist anymore, can't abort");
3307            Ok(false)
3308        }
3309    }
3310
3311    /// Edits the content of a local echo with a raw event content.
3312    ///
3313    /// Returns true if the event to be sent was replaced, false if not (i.e.
3314    /// the event had already been sent).
3315    ///
3316    /// This method should not be used for editing sticky events. Sticky events
3317    /// are collected in an ephemeral map. Entries in that map need to be
3318    /// updated by sending a replacing sticky event with updated content,
3319    /// not by using an `m.replace` relation. Nevertheless, this method
3320    /// applies edits of an _unsent_ sticky event on the queued event
3321    /// directly because it is safe to do so. If, however, the event has
3322    /// already been sent, the edit will be sent as an unsticky event.
3323    #[instrument(skip(self, new_content), fields(room_id = %self.room.inner.room.room_id(), txn_id = %self.transaction_id))]
3324    pub async fn edit_raw(
3325        &self,
3326        new_content: Raw<AnyMessageLikeEventContent>,
3327        event_type: String,
3328    ) -> Result<bool, RoomSendQueueStorageError> {
3329        trace!("received an edit request");
3330        self.nyi_for_uploads()?;
3331
3332        let serializable = SerializableEventContent::from_raw(new_content, event_type);
3333
3334        if self.room.inner.queue.replace_event(&self.transaction_id, serializable.clone()).await? {
3335            trace!("successful edit");
3336
3337            // Wake up the queue, in case the room was asleep before the edit.
3338            self.room.inner.notifier.notify_one();
3339
3340            // Propagate a replaced update too.
3341            self.room.send_update(RoomSendQueueUpdate::ReplacedLocalEvent {
3342                transaction_id: self.transaction_id.clone(),
3343                new_content: serializable,
3344            });
3345
3346            Ok(true)
3347        } else {
3348            debug!("local echo doesn't exist anymore, can't edit");
3349            Ok(false)
3350        }
3351    }
3352
3353    /// Edits the content of a local echo with an event content.
3354    ///
3355    /// Returns true if the event to be sent was replaced, false if not (i.e.
3356    /// the event had already been sent).
3357    ///
3358    /// This method should not be used for editing sticky events. Sticky events
3359    /// are collected in an ephemeral map. Entries in that map need to be
3360    /// updated by sending a replacing sticky event with updated content,
3361    /// not by using an `m.replace` relation. Nevertheless, this method
3362    /// applies edits of an _unsent_ sticky event on the queued event
3363    /// directly because it is safe to do so. If, however, the event has
3364    /// already been sent, the edit will be sent as an unsticky event.
3365    pub async fn edit(
3366        &self,
3367        new_content: AnyMessageLikeEventContent,
3368    ) -> Result<bool, RoomSendQueueStorageError> {
3369        self.edit_raw(
3370            Raw::new(&new_content).map_err(RoomSendQueueStorageError::JsonSerialization)?,
3371            new_content.event_type().to_string(),
3372        )
3373        .await
3374    }
3375
3376    /// Edits the content of a local echo with a media caption.
3377    ///
3378    /// Will fail if the event to be sent, represented by this send handle,
3379    /// wasn't a media.
3380    pub async fn edit_media_caption(
3381        &self,
3382        caption: Option<String>,
3383        formatted_caption: Option<FormattedBody>,
3384        mentions: Option<Mentions>,
3385    ) -> Result<bool, RoomSendQueueStorageError> {
3386        if let Some(new_content) = self
3387            .room
3388            .inner
3389            .queue
3390            .edit_media_caption(&self.transaction_id, caption, formatted_caption, mentions)
3391            .await?
3392        {
3393            trace!("successful edit of media caption");
3394
3395            // Wake up the queue, in case the room was asleep before the edit.
3396            self.room.inner.notifier.notify_one();
3397
3398            // Propagate a replaced update too.
3399            self.room.send_update(RoomSendQueueUpdate::ReplacedLocalEvent {
3400                transaction_id: self.transaction_id.clone(),
3401                new_content,
3402            });
3403
3404            Ok(true)
3405        } else {
3406            debug!("local echo doesn't exist anymore, can't edit media caption");
3407            Ok(false)
3408        }
3409    }
3410
3411    /// Unwedge the local echo associated to this [`SendHandle`] and try to
3412    /// resend it.
3413    pub async fn unwedge(&self) -> Result<(), RoomSendQueueError> {
3414        let room = &self.room.inner;
3415
3416        // If we have media handles, try to unwedge them.
3417        //
3418        // It's fine to always do it to _all_ the transaction IDs at once,
3419        // because only one of the three requests will be active at the same
3420        // time, i.e. only one entry will be updated in the store. The other two
3421        // are either done, or dependent requests.
3422
3423        for handles in &self.media_handles {
3424            room.queue
3425                .mark_as_unwedged(&handles.upload_file_txn)
3426                .await
3427                .map_err(RoomSendQueueError::StorageError)?;
3428
3429            if let Some(txn) = &handles.upload_thumbnail_txn {
3430                room.queue.mark_as_unwedged(txn).await.map_err(RoomSendQueueError::StorageError)?;
3431            }
3432        }
3433
3434        self.room.unwedge_request(&self.transaction_id).await
3435    }
3436
3437    /// Send a reaction to the event as soon as it's sent.
3438    ///
3439    /// If returning `Ok(None)`; this means the reaction couldn't be sent
3440    /// because the event is already a remote one.
3441    #[instrument(skip(self), fields(room_id = %self.room.inner.room.room_id(), txn_id = %self.transaction_id))]
3442    pub async fn react(
3443        &self,
3444        key: String,
3445    ) -> Result<Option<SendHandle>, RoomSendQueueStorageError> {
3446        trace!("received an intent to react");
3447
3448        let created_at = MilliSecondsSinceUnixEpoch::now();
3449        if let Some(reaction_txn_id) =
3450            self.room.inner.queue.react(&self.transaction_id, key.clone(), created_at).await?
3451        {
3452            trace!("successfully queued react");
3453
3454            // Wake up the queue, in case the room was asleep before the
3455            // sending.
3456            self.room.inner.notifier.notify_one();
3457
3458            // Propagate a new local event.
3459            let send_handle = SendHandle {
3460                room: self.room.clone(),
3461                transaction_id: reaction_txn_id.clone().into(),
3462                media_handles: vec![],
3463                created_at,
3464            };
3465
3466            self.room.send_update(RoomSendQueueUpdate::NewLocalEvent(LocalEcho {
3467                // Note: we do want to use the `txn_id` we're going to use for
3468                // the reaction, not the one for the event we're reacting to.
3469                transaction_id: reaction_txn_id.into(),
3470                content: LocalEchoContent::React {
3471                    key,
3472                    send_handle: send_handle.clone(),
3473                    applies_to: self.transaction_id.clone(),
3474                },
3475            }));
3476
3477            Ok(Some(send_handle))
3478        } else {
3479            debug!("local echo doesn't exist anymore, can't react");
3480            Ok(None)
3481        }
3482    }
3483}
3484
3485/// Whether a queued request is an event the user composed, as opposed to a
3486/// redaction or a reaction.
3487///
3488/// A reaction is queued as a dependent request at first, but graduates into a
3489/// request of its own, under the same transaction id, once the event it
3490/// applies to has been sent; so being an event is not enough to tell the two
3491/// apart.
3492fn is_own_event_request(request: &QueuedRequest) -> bool {
3493    request.as_event().is_some_and(|content| {
3494        TimelineEventType::from(content.raw().1) != TimelineEventType::Reaction
3495    })
3496}
3497
3498/// From a given source of [`DependentQueuedRequest`], return only the most
3499/// meaningful, i.e. the ones that wouldn't be overridden after applying the
3500/// others.
3501fn canonicalize_dependent_requests(
3502    dependent: &[DependentQueuedRequest],
3503) -> Vec<DependentQueuedRequest> {
3504    let mut by_txn = HashMap::<OwnedTransactionId, Vec<&DependentQueuedRequest>>::new();
3505
3506    for d in dependent {
3507        let prevs = by_txn.entry(d.parent_transaction_id.clone()).or_default();
3508
3509        if prevs.iter().any(|prev| {
3510            matches!(
3511                prev.kind,
3512                DependentQueuedRequestKind::RedactEvent
3513                    | DependentQueuedRequestKind::RedactEventWithReason { .. }
3514            )
3515        }) {
3516            // The parent event has already been flagged for redaction, don't
3517            // consider the other dependent events.
3518            continue;
3519        }
3520
3521        match &d.kind {
3522            DependentQueuedRequestKind::EditEvent { .. } => {
3523                // Replace any previous edit with this one.
3524                if let Some(prev_edit) = prevs
3525                    .iter_mut()
3526                    .find(|prev| matches!(prev.kind, DependentQueuedRequestKind::EditEvent { .. }))
3527                {
3528                    *prev_edit = d;
3529                } else {
3530                    prevs.insert(0, d);
3531                }
3532            }
3533
3534            DependentQueuedRequestKind::UploadFileOrThumbnail { .. }
3535            | DependentQueuedRequestKind::FinishUpload { .. }
3536            | DependentQueuedRequestKind::ReactEvent { .. } => {
3537                // These requests can't be canonicalized, push them as is.
3538                prevs.push(d);
3539            }
3540
3541            #[cfg(feature = "unstable-msc4274")]
3542            DependentQueuedRequestKind::FinishGallery { .. } => {
3543                // This request can't be canonicalized, push it as is.
3544                prevs.push(d);
3545            }
3546
3547            DependentQueuedRequestKind::RedactEvent
3548            | DependentQueuedRequestKind::RedactEventWithReason { .. } => {
3549                // Remove every other dependent action.
3550                prevs.clear();
3551                prevs.push(d);
3552            }
3553        }
3554    }
3555
3556    by_txn.into_values().flat_map(|entries| entries.into_iter().cloned()).collect()
3557}
3558
3559#[cfg(all(test, not(target_family = "wasm")))]
3560mod tests {
3561    use std::{assert_matches, sync::Arc, time::Duration};
3562
3563    use matrix_sdk_base::store::{
3564        ChildTransactionId, DependentQueuedRequest, DependentQueuedRequestKind,
3565        SerializableEventContent,
3566    };
3567    use matrix_sdk_test::{JoinedRoomBuilder, SyncResponseBuilder, async_test};
3568    use ruma::{
3569        MilliSecondsSinceUnixEpoch, TransactionId,
3570        api::error::UnknownTokenErrorData,
3571        event_id,
3572        events::{AnyMessageLikeEventContent, room::message::RoomMessageEventContent},
3573        room_id,
3574    };
3575    use strass::assert_let;
3576
3577    use super::{RoomSendQueueUpdate, canonicalize_dependent_requests};
3578    use crate::{
3579        SessionChange,
3580        test_utils::{logged_in_client, mocks::MatrixMockServer},
3581    };
3582
3583    #[test]
3584    fn test_canonicalize_dependent_events_created_at() {
3585        // Test to ensure the created_at field is being serialized and retrieved
3586        // correctly.
3587        let txn = TransactionId::new();
3588        let created_at = MilliSecondsSinceUnixEpoch::now();
3589
3590        let edit = DependentQueuedRequest {
3591            own_transaction_id: ChildTransactionId::new(),
3592            parent_transaction_id: txn.clone(),
3593            kind: DependentQueuedRequestKind::EditEvent {
3594                new_content: SerializableEventContent::new(
3595                    &RoomMessageEventContent::text_plain("edit").into(),
3596                )
3597                .unwrap(),
3598            },
3599            parent_key: None,
3600            created_at,
3601        };
3602
3603        let res = canonicalize_dependent_requests(&[edit]);
3604
3605        assert_eq!(res.len(), 1);
3606        assert_let!(DependentQueuedRequestKind::EditEvent { new_content } = &res[0].kind);
3607        assert_let!(
3608            AnyMessageLikeEventContent::RoomMessage(msg) = new_content.deserialize().unwrap()
3609        );
3610        assert_eq!(msg.body(), "edit");
3611        assert_eq!(res[0].parent_transaction_id, txn);
3612        assert_eq!(res[0].created_at, created_at);
3613    }
3614
3615    #[async_test]
3616    async fn test_client_no_cycle_with_send_queue() {
3617        for enabled in [true, false] {
3618            let client = logged_in_client(None).await;
3619            let weak_client = client.downgrade();
3620
3621            {
3622                let mut sync_response_builder = SyncResponseBuilder::new();
3623
3624                let room_id = room_id!("!a:b.c");
3625
3626                // Make sure the client knows about the room.
3627                client
3628                    .base_client()
3629                    .receive_sync_response(
3630                        sync_response_builder
3631                            .add_joined_room(JoinedRoomBuilder::new(room_id))
3632                            .build_sync_response(),
3633                    )
3634                    .await
3635                    .unwrap();
3636
3637                let room = client.get_room(room_id).unwrap();
3638                let q = room.send_queue();
3639
3640                let _watcher = q.subscribe().await;
3641
3642                client.send_queue().set_enabled(enabled).await;
3643            }
3644
3645            drop(client);
3646
3647            // Give a bit of time for background tasks to die.
3648            tokio::time::sleep(Duration::from_millis(500)).await;
3649
3650            // The weak client must be the last reference to the client now.
3651            let client = weak_client.get();
3652            assert!(
3653                client.is_none(),
3654                "too many strong references to the client: {}",
3655                Arc::strong_count(&client.unwrap().inner)
3656            );
3657        }
3658    }
3659
3660    #[test]
3661    fn test_canonicalize_dependent_events_smoke_test() {
3662        // Smoke test: canonicalizing a single dependent event returns it.
3663        let txn = TransactionId::new();
3664
3665        let edit = DependentQueuedRequest {
3666            own_transaction_id: ChildTransactionId::new(),
3667            parent_transaction_id: txn.clone(),
3668            kind: DependentQueuedRequestKind::EditEvent {
3669                new_content: SerializableEventContent::new(
3670                    &RoomMessageEventContent::text_plain("edit").into(),
3671                )
3672                .unwrap(),
3673            },
3674            parent_key: None,
3675            created_at: MilliSecondsSinceUnixEpoch::now(),
3676        };
3677        let res = canonicalize_dependent_requests(&[edit]);
3678
3679        assert_eq!(res.len(), 1);
3680        assert_matches!(&res[0].kind, DependentQueuedRequestKind::EditEvent { .. });
3681        assert_eq!(res[0].parent_transaction_id, txn);
3682        assert!(res[0].parent_key.is_none());
3683    }
3684
3685    #[test]
3686    fn test_canonicalize_dependent_events_redaction_preferred() {
3687        // A redaction is preferred over any other kind of dependent event.
3688        let txn = TransactionId::new();
3689
3690        let mut inputs = Vec::with_capacity(100);
3691        let redact = DependentQueuedRequest {
3692            own_transaction_id: ChildTransactionId::new(),
3693            parent_transaction_id: txn.clone(),
3694            kind: DependentQueuedRequestKind::RedactEventWithReason { reason: None },
3695            parent_key: None,
3696            created_at: MilliSecondsSinceUnixEpoch::now(),
3697        };
3698
3699        let edit = DependentQueuedRequest {
3700            own_transaction_id: ChildTransactionId::new(),
3701            parent_transaction_id: txn.clone(),
3702            kind: DependentQueuedRequestKind::EditEvent {
3703                new_content: SerializableEventContent::new(
3704                    &RoomMessageEventContent::text_plain("edit").into(),
3705                )
3706                .unwrap(),
3707            },
3708            parent_key: None,
3709            created_at: MilliSecondsSinceUnixEpoch::now(),
3710        };
3711
3712        inputs.push({
3713            let mut edit = edit.clone();
3714            edit.own_transaction_id = ChildTransactionId::new();
3715            edit
3716        });
3717
3718        inputs.push(redact);
3719
3720        for _ in 0..98 {
3721            let mut edit = edit.clone();
3722            edit.own_transaction_id = ChildTransactionId::new();
3723            inputs.push(edit);
3724        }
3725
3726        let res = canonicalize_dependent_requests(&inputs);
3727
3728        assert_eq!(res.len(), 1);
3729        assert_matches!(&res[0].kind, DependentQueuedRequestKind::RedactEventWithReason { .. });
3730        assert_eq!(res[0].parent_transaction_id, txn);
3731    }
3732
3733    #[test]
3734    fn test_canonicalize_dependent_events_last_edit_preferred() {
3735        let parent_txn = TransactionId::new();
3736
3737        // The latest edit of a list is always preferred.
3738        let inputs = (0..10)
3739            .map(|i| DependentQueuedRequest {
3740                own_transaction_id: ChildTransactionId::new(),
3741                parent_transaction_id: parent_txn.clone(),
3742                kind: DependentQueuedRequestKind::EditEvent {
3743                    new_content: SerializableEventContent::new(
3744                        &RoomMessageEventContent::text_plain(format!("edit{i}")).into(),
3745                    )
3746                    .unwrap(),
3747                },
3748                parent_key: None,
3749                created_at: MilliSecondsSinceUnixEpoch::now(),
3750            })
3751            .collect::<Vec<_>>();
3752
3753        let txn = inputs[9].parent_transaction_id.clone();
3754
3755        let res = canonicalize_dependent_requests(&inputs);
3756
3757        assert_eq!(res.len(), 1);
3758        assert_let!(DependentQueuedRequestKind::EditEvent { new_content } = &res[0].kind);
3759        assert_let!(
3760            AnyMessageLikeEventContent::RoomMessage(msg) = new_content.deserialize().unwrap()
3761        );
3762        assert_eq!(msg.body(), "edit9");
3763        assert_eq!(res[0].parent_transaction_id, txn);
3764    }
3765
3766    #[test]
3767    fn test_canonicalize_multiple_local_echoes() {
3768        let txn1 = TransactionId::new();
3769        let txn2 = TransactionId::new();
3770
3771        let child1 = ChildTransactionId::new();
3772        let child2 = ChildTransactionId::new();
3773
3774        let inputs = vec![
3775            // This one pertains to txn1.
3776            DependentQueuedRequest {
3777                own_transaction_id: child1.clone(),
3778                kind: DependentQueuedRequestKind::RedactEventWithReason { reason: None },
3779                parent_transaction_id: txn1.clone(),
3780                parent_key: None,
3781                created_at: MilliSecondsSinceUnixEpoch::now(),
3782            },
3783            // This one pertains to txn2.
3784            DependentQueuedRequest {
3785                own_transaction_id: child2,
3786                kind: DependentQueuedRequestKind::EditEvent {
3787                    new_content: SerializableEventContent::new(
3788                        &RoomMessageEventContent::text_plain("edit").into(),
3789                    )
3790                    .unwrap(),
3791                },
3792                parent_transaction_id: txn2.clone(),
3793                parent_key: None,
3794                created_at: MilliSecondsSinceUnixEpoch::now(),
3795            },
3796        ];
3797
3798        let res = canonicalize_dependent_requests(&inputs);
3799
3800        // The canonicalization shouldn't depend per event id.
3801        assert_eq!(res.len(), 2);
3802
3803        for dependent in res {
3804            if dependent.own_transaction_id == child1 {
3805                assert_eq!(dependent.parent_transaction_id, txn1);
3806                assert_matches!(
3807                    dependent.kind,
3808                    DependentQueuedRequestKind::RedactEventWithReason { .. }
3809                );
3810            } else {
3811                assert_eq!(dependent.parent_transaction_id, txn2);
3812                assert_matches!(dependent.kind, DependentQueuedRequestKind::EditEvent { .. });
3813            }
3814        }
3815    }
3816
3817    #[test]
3818    fn test_canonicalize_reactions_after_edits() {
3819        // Sending reactions should happen after edits to a given event.
3820        let txn = TransactionId::new();
3821
3822        let react_id = ChildTransactionId::new();
3823        let react = DependentQueuedRequest {
3824            own_transaction_id: react_id.clone(),
3825            kind: DependentQueuedRequestKind::ReactEvent { key: "🧠".to_owned() },
3826            parent_transaction_id: txn.clone(),
3827            parent_key: None,
3828            created_at: MilliSecondsSinceUnixEpoch::now(),
3829        };
3830
3831        let edit_id = ChildTransactionId::new();
3832        let edit = DependentQueuedRequest {
3833            own_transaction_id: edit_id.clone(),
3834            kind: DependentQueuedRequestKind::EditEvent {
3835                new_content: SerializableEventContent::new(
3836                    &RoomMessageEventContent::text_plain("edit").into(),
3837                )
3838                .unwrap(),
3839            },
3840            parent_transaction_id: txn,
3841            parent_key: None,
3842            created_at: MilliSecondsSinceUnixEpoch::now(),
3843        };
3844
3845        let res = canonicalize_dependent_requests(&[react, edit]);
3846
3847        assert_eq!(res.len(), 2);
3848        assert_eq!(res[0].own_transaction_id, edit_id);
3849        assert_eq!(res[1].own_transaction_id, react_id);
3850    }
3851
3852    #[async_test]
3853    async fn test_request_wedged_by_a_rejected_token_is_unwedged_on_lag() {
3854        let mock = MatrixMockServer::new().await;
3855        let client = mock.client_builder().build().await;
3856        let room = mock.sync_joined_room(&client, room_id!("!a:b.c")).await;
3857
3858        let q = room.send_queue();
3859        let (_, mut watch) = q.subscribe().await.unwrap();
3860
3861        mock.mock_room_state_encryption().plain().mount().await;
3862        mock.mock_room_send().error_unknown_token(false).mock_once().mount().await;
3863        mock.mock_room_send().ok(event_id!("$42")).mock_once().mount().await;
3864
3865        q.send(RoomMessageEventContent::text_plain("hello").into()).await.unwrap();
3866
3867        assert_let!(Ok(RoomSendQueueUpdate::NewLocalEvent(_)) = watch.recv().await);
3868        assert_let!(
3869            Ok(RoomSendQueueUpdate::SendError { is_recoverable: false, .. }) = watch.recv().await
3870        );
3871
3872        // The token gets refreshed, then rejected again before the send queue
3873        // gets to see the refresh, so all it sees is a lag.
3874        let sender = &client.auth_ctx().session_change_sender;
3875        sender.send(SessionChange::TokensRefreshed).unwrap();
3876        sender.send(SessionChange::UnknownToken(UnknownTokenErrorData::new())).unwrap();
3877
3878        // Which is still enough for the request to be retried.
3879        assert_let!(Ok(RoomSendQueueUpdate::RetryEvent { .. }) = watch.recv().await);
3880        assert_let!(Ok(RoomSendQueueUpdate::SentEvent { .. }) = watch.recv().await);
3881    }
3882}