Module: store/webstorage

This is an internal module. Implementation details:
Room data is stored as follows:
  room_$ROOMID_timeline_$INDEX : [ Event, Event, Event ]
  room_$ROOMID_state : {
                         pagination_token: ,
                         events: {
                           : {  : {JSON} }
                         }
                       }
User data is stored as follows:
  user_$USERID : User
Sync token:
  sync_token : $TOKEN

Room Retrieval
--------------
Retrieving a room requires the $ROOMID which then pulls out the current state
from room_$ROOMID_state. A defined starting batch of timeline events are then
extracted from the highest numbered $INDEX for room_$ROOMID_timeline_$INDEX
(more indices as required). The $INDEX may be negative. These are
added to the timeline in the same way as /initialSync (old state will diverge).
If there exists a room_$ROOMID_timeline_live key, then a timeline sync should
be performed before retrieving.

Retrieval of earlier messages
-----------------------------
The earliest event the Room instance knows about is E. Retrieving earlier
messages requires a Room which has a storageToken defined.
This token maps to the index I where the Room is at. Events are then retrieved from
room_$ROOMID_timeline_{I} and elements before E are extracted. If the limit
demands more events, I-1 is retrieved, up until I=min $INDEX where it gives
less than the limit. Index may go negative if you have paginated in the past.

Full Insertion
--------------
Storing a room requires the timeline and state keys for $ROOMID to
be blown away and completely replaced, which is computationally expensive.
Room.timeline is batched according to the given batch size B. These batches
are then inserted into storage as room_$ROOMID_timeline_$INDEX. Finally,
the current room state is persisted to room_$ROOMID_state.

Incremental Insertion
---------------------
As events arrive, the store can quickly persist these new events. This
involves pushing the events to room_$ROOMID_timeline_live. If the
current room state has been modified by the new event, then
room_$ROOMID_state should be updated in addition to the timeline.

Timeline sync
-------------
Retrieval of events from the timeline depends on the proper batching of
events. This is computationally expensive to perform on every new event, so
is deferred by inserting live events to room_$ROOMID_timeline_live. A
timeline sync reconciles timeline_live and timeline_$INDEX. This involves
retrieving _live and the highest numbered $INDEX batch. If the batch is < B,
the earliest entries from _live are inserted into the $INDEX until the
batch == B. Then, the remaining entries in _live are batched to $INDEX+1,
$INDEX+2, and so on. The easiest way to visualise this is that the timeline
goes from old to new, left to right:
         -2         -1         0         1
<--OLD---------------------------------------NEW-->
       [a,b,c]    [d,e,f]   [g,h,i]   [j,k,l]

Purging
-------
Events from the timeline can be purged by removing the lowest
timeline_$INDEX in the store.

Example
-------
A room with room_id !foo:bar has 9 messages (M1->9 where 9=newest) with a
batch size of 4. The very first time, there is no entry for !foo:bar until
storeRoom() is called, which results in the keys: [Full Insert]
  room_!foo:bar_timeline_0 : [M1, M2, M3, M4]
  room_!foo:bar_timeline_1 : [M5, M6, M7, M8]
  room_!foo:bar_timeline_2 : [M9]
  room_!foo:bar_state: { ... }

5 new messages (N1-5, 5=newest) arrive and are then added: [Incremental Insert]
  room_!foo:bar_timeline_live: [N1]
  room_!foo:bar_timeline_live: [N1, N2]
  room_!foo:bar_timeline_live: [N1, N2, N3]
  room_!foo:bar_timeline_live: [N1, N2, N3, N4]
  room_!foo:bar_timeline_live: [N1, N2, N3, N4, N5]

App is shutdown. Restarts. The timeline is synced [Timeline Sync]
  room_!foo:bar_timeline_2 : [M9, N1, N2, N3]
  room_!foo:bar_timeline_3 : [N4, N5]
  room_!foo:bar_timeline_live: []

And the room is retrieved with 8 messages: [Room Retrieval]
  Room.timeline: [M7, M8, M9, N1, N2, N3, N4, N5]
  Room.storageToken: => early_index = 1 because that's where M7 is.

3 earlier messages are requested: [Earlier retrieval]
  Use storageToken to find batch index 1. Scan batch for earliest event ID.
  earliest event = M7
  events = room_!foo:bar_timeline_1 where event < M7 = [M5, M6]
Too few events, use next index (0) and get 1 more:
  events = room_!foo:bar_timeline_0 = [M1, M2, M3, M4] => [M4]
Return concatentation:
  [M4, M5, M6]

Purge oldest events: [Purge]
  del room_!foo:bar_timeline_0
Source:

module:store/webstorage

Web Storage Store class.
Source:

Classes

WebStorageStore