Persistence and Collaboration

Introduction

By default Variable state is purely client-side, meaning its source of truth is the browser memory. It also means that refreshing the browser page resets Variable state, and that each user has their own "copy" of the data. To customize the behavior, Variable accepts a store property. This allows changing Variable's source of truth, thereby enabling features like data persistence and collaborative interactions across users.

The rest of the page goes in-depth into available stores and how they can be used in your application.

BackendStore

This type of store changes the source of truth for a given Variable to live on the server, rather than on the client. This has two main benefits:

• the data can be persistent across page refreshes or even across server restarts
• since the data lives server-side, we can "sync" it across clients to enable collaboration

To use a BackendStore, simply create a store instance and attach it to a Variable:

from dara.core import Variable
from dara.core.persistence import BackendStore
from dara.components import Input

# Tag the variable to be "collaborative" and have a shared server state
collab_variable = Variable(default=1, store=BackendStore())

# Input value will automatically be kept in sync "live" for all users
Input(value=collab_variable)

Under the hood the BackendStore mechanism is very simple:

• the store data is initialized with the default Variable value
• on initial page load, the Variable value is fetched from the server
• whenever a client makes an update, it makes a request to the server to update the state, which then notifies all other clients about the new state

You can interact with the store programmatically to read, write or delete data from the store.

from dara.core import Variable
from dara.core.persistence import BackendStore
from dara.components import Input

store = BackendStore()
collab_variable = Variable(default='default value', store=store)

# get current value in the store
value = await store.read()
# store can also be accessed from the variable
value = await variable.store.read()

# write a new value to the store
await store.write('new value')

# delete the existing value from the store
await store.delete()

# get all value from the store - see `scope` section below for details
await store.get_all()

The behavior of the store can be customized by passing arguments to the BackendStore.

scope

Scope controls whether the data in a BackendStore is shared among all users (scope='global') or separate and specific to a given user (scope='user').

When using global scope, the state is shared and synced among the users so a change being made to an attached Variable in one client will immediately be propagated to other users. In this mode, the get_all API method will return a dictionary of the shape {'global': 'value'}.

When using user scope, each user has their own copy of the data. A change made to an attached Variable in one client will only be propagated to other sessions open for that particular user, e.g. among all currently open tabs for that user. In this mode, the get_all API returns a dictionary with user values keyed by the user identifiers, e.g. {'user1': 'value1', 'user2': 'value2'}.

danger

In user scope, the API methods read, write and delete can only be used in an authenticated context, i.e. within py_components, DerivedVariable resolvers or actions. This is because these APIs will look up the currently operating user and act on their behalf, e.g. write would update the value stored only for that particular user.

backend

You can select a particular backend implemention to be used for storage. By default BackendStore uses the InMemoryBackend implementation which simply stores the data in-memory.

Alternatively, you can use the FileBackend which uses a JSON file for storage. This enables persisting the data across server restarts and easy access to the data for external processes.

from dara.core import Variable
from dara.core.persistence import BackendStore, FileBackend
from dara.components import Input

collab_variable = Variable(
    default='default value',
    store=BackendStore(
        backend=FileBackend(path='path/to/file.json')
    )
)

Partial Updates

BackendStore supports efficient partial updates through the write_partial method, which allows you to update only specific parts of your data without sending the entire object. This is particularly useful for large objects where you only want to modify specific fields.

The write_partial method accepts two types of input:

1. JSON Patch Operations (RFC 6902)

You can provide a list of JSON Patch operations for precise, granular updates:

from dara.core import Variable
from dara.core.persistence import BackendStore

store = BackendStore()
# Initialize with some structured data
user_data = Variable(default={
    'user': {'name': 'John', 'age': 30},
    'items': ['apple', 'banana']
}, store=store)

# Apply specific patch operations
await store.write_partial([
    {'op': 'replace', 'path': '/user/age', 'value': 31},
    {'op': 'add', 'path': '/items/-', 'value': 'cherry'},
    {'op': 'add', 'path': '/user/city', 'value': 'New York'}
])

2. Full Object with Automatic Diffing

Alternatively, you can provide a complete updated object and write_partial will automatically calculate the differences:

# Get current data
current = await store.read()
# Modify the data
updated_data = {
    **current,
    'user': {**current['user'], 'age': 32},
    'items': current['items'] + ['orange']
}

# Apply the changes - only the differences will be sent to clients
await store.write_partial(updated_data)

Benefits of Partial Updates

• Performance: Only changed data is transmitted to clients via WebSocket
• Efficiency: Reduces network traffic and improves update speed for large objects
• Real-time sync: All connected clients receive the same patch operations, ensuring consistency
• Sequence validation: Updates include sequence numbers to prevent race conditions and ensure proper ordering

Supported JSON Patch Operations

• add: Add a new value to an object or array
• remove: Remove a value from an object or array
• replace: Replace a value in an object or array
• move: Move a value within the data structure
• copy: Copy a value to another location
• test: Test that a value is as expected

Note that JSON patches can only be applied to structured data (objects and arrays). For primitive values, use the regular write method instead.

You can also choose to provide a custom backend implementation as long as it extends the PersistenceBackend class. This is useful to e.g. use a database or other backend-data as a source of truth and expose it to the client. In particular, you might find the subscribe method on a PersistenceBackend helpful - it can be implemented to subscribe to changes in the backend data and update the variable accordingly. Make sure to use the readonly flag on the BackendStore if your backend does not support writing.

from dara.core import action
from dara.core.persistence import BackendStore, PersistenceBackend
from dara.core.variables import Variable

from dara.components import Button, Stack

DB: MyDatabase

class MyBackend(PersistenceBackend):
    def read(self, key: str) -> Any:
        # Read from the DB
        return DB.query("SELECT ...")

    def write(self, key: str, value: Any) -> None:
        raise NotImplementedError("Writing to the backend is not supported")

    def subscribe(self, callback: Callable[[Any], None]) -> None:
        # assume there is a way to subscribe to changes in the DB
        DB.subscribe("data_updated", callback)

    # ... other methods omitted for brevity

store = BackendStore(MyBackend(), readonly=True)
variable = Variable(store=store)

@action
async def update_data():
    # Update the data directly in the backend
    await DB.update("UPDATE ...")
    # Assuming this triggers a notification back from the DB into MyBackend, this will send a notification to the open clients
    await DB.notify("data_updated")

Stack(
    # displays the data from the backend
    display_data(variable),
    Button("Update data", onclick=update_data())
)