Data models and writers¶

Beamtime data models, catalog, schema, and writers for SQLite and Zarr storage.

class resonance.api.data.models.SampleMetadata(name: str, formula: str | None = None, serial: str | None = None, tags: list[str] = <factory>, beamline_pos: str | None = None, extra: dict[str, ~typing.Any]=<factory>, id: int | None = None)[source]¶

Bases: object

In-memory representation of a sample row from the samples table.

Parameters:

name (str) – Human-readable sample name, used as the primary display identifier.
formula (str | None) – Chemical formula string, e.g. “C8H8”. None if not recorded.
serial (str | None) – Physical identifier stamped on the sample or its holder, e.g. “S1234”.
tags (list[str]) – Arbitrary string tags for grouping or filtering, e.g. [“polymer”, “reference”].
beamline_pos (str | None) – Stage slot or carousel position where the sample was mounted.
extra (dict[str, Any]) – Arbitrary metadata that does not fit the fixed schema.
id (int | None) – Database primary key assigned on insertion. None before the row is written.

name¶

Human-readable sample name, used as the primary display identifier.

Type:: str

formula¶

Chemical formula string, e.g. “C8H8”. None if not recorded.

Type:: str | None

serial¶

Physical identifier stamped on the sample or its holder, e.g. “S1234”.

Type:: str | None

tags¶

Arbitrary string tags for grouping or filtering, e.g. [“polymer”, “reference”].

Type:: list[str]

beamline_pos¶

Stage slot or carousel position where the sample was mounted.

Type:: str | None

extra¶

Arbitrary metadata that does not fit the fixed schema.

Type:: dict[str, Any]

id¶

Database primary key assigned on insertion. None before the row is written.

Type:: int | None

class resonance.api.data.models.BeamtimeInfo(id: int, researcher_name: str, label: str, db_path: str, time_start: float | None = None, time_stop: float | None = None)[source]¶

Bases: object

Represents a beamtime entry as stored in the master index database.

Each beamtime corresponds to a single experimental session and owns its own SQLite database file. The master index holds one row per beamtime so that multiple sessions can be cataloged from a single entry point.

id¶

Primary key in the master index database.

Type:: int

researcher_name¶

Name of the researcher who led the beamtime session.

Type:: str

label¶

Human-readable label for the beamtime, typically an ISO date string such as “2026-03-05”.

Type:: str

db_path¶

Path to the beamtime SQLite file, relative to the DATA root directory.

Type:: str

time_start¶

Unix timestamp for the beginning of the beamtime. None if not recorded.

Type:: float | None

time_stop¶

Unix timestamp for the end of the beamtime. None if still in progress or not recorded.

Type:: float | None

class resonance.api.data.models.RunSummary(uid: str, plan_name: str, time_start: float, sample_name: str | None = None, time_stop: float | None = None, exit_status: str | None = None)[source]¶

Bases: object

Lightweight summary of a single run, used for catalog listing operations.

This is a denormalized projection built from joining the runs and samples tables. It is intended for display and filtering, not for full data retrieval.

uid¶

Run UUID in hexadecimal form, as stored in the database.

Type:: str

plan_name¶

Name of the scan plan that produced this run, e.g. “en_scan”.

Type:: str

time_start¶

Unix timestamp marking the start of the run.

Type:: float

sample_name¶

Sample name denormalized from the samples join. None if no sample was associated with the run or if the join produced no match.

Type:: str | None

time_stop¶

Unix timestamp marking the end of the run. None if the run did not finish cleanly or if the stop document was not recorded.

Type:: float | None

exit_status¶

Final status string from the RunStop document. Expected values are “success”, “aborted”, or “failed”. None if not recorded.

Type:: str | None

class resonance.api.data.writer.RunWriter(db_path: Path, sample: SampleMetadata)[source]¶

Bases: object

Manages a SQLite connection to a beamtime database and writes scan data.

Parameters:

db_path (Path) – Path to the beamtime SQLite database file. Created on first open if it does not exist.
sample (SampleMetadata) – Sample to associate with runs written through this writer. If sample.id is None, the sample is upserted on open.

_db_path¶

Resolved path to the SQLite file.

Type:: Path

_sample¶

Sample metadata, with id populated after open.

Type:: SampleMetadata

_conn¶

Active database connection, or None when closed.

Type:: sqlite3.Connection or None

_run_uid¶

Hex UID of the currently open run, or empty string.

Type:: str

_stream_uid¶

Hex UID of the currently open stream, or empty string.

Type:: str

_seq_num¶

Event sequence counter within the current stream.

Type:: int

_zarr_store¶

Open Zarr group for image storage, or None when closed.

Type:: zarr.Group or None

__enter__() → RunWriter[source]¶

Open the writer and return self.

Returns:: The opened writer instance.
Return type:: RunWriter

__exit__(exc_type: type[BaseException] | None, exc: BaseException | None, tb: object) → None[source]¶

Close the run and connection, propagating any exception.

Parameters:

exc_type (type[BaseException] or None) – Exception type if an exception occurred, otherwise None.
exc (BaseException or None) – Exception instance if an exception occurred, otherwise None.
tb (object) – Traceback object if an exception occurred, otherwise None.

close() → None[source]¶

Commit any remaining work and close the database connection.

Raises:: RuntimeError – If the writer is not open.

close_run(*, exit_status: str = 'success') → None[source]¶

Finalize the current run and commit all pending events.

Parameters:: exit_status (str, optional) – Final status string. Expected values are “success”, “aborted”, or “failed”. Defaults to “success”.
Raises:: RuntimeError – If the writer is not open or no run has been opened.

open() → None[source]¶

Open the database connection and upsert the sample.

Creates the beamtime schema if the database does not yet exist. If self._sample.id is None, the sample is looked up by name; if a matching row exists its id is loaded, otherwise a new row is inserted. A Zarr store is opened (or created) alongside the .db file at {db_path.stem}.zarr/.

Raises:

RuntimeError – If the writer is already open (self._conn is not None).
sqlite3.DatabaseError – If the database file is corrupt or unreadable.

open_run(plan_name: str, *, metadata: dict[str, Any] | None = None) → str[source]¶

Insert a new run row and return its UID.

Parameters:

plan_name (str) – Name of the scan plan, e.g. “en_scan”.
metadata (dict[str, Any] or None, optional) – Arbitrary run-level metadata serialized to JSON. Defaults to an empty dict.

Returns:

Hex UUID of the newly created run.

Return type:

str

Raises:

RuntimeError – If the writer is not open.

open_stream(name: str, data_keys: dict[str, Any]) → str[source]¶

Insert a new stream row and return its UID.

Parameters:

name (str) – Stream name, e.g. “primary”.
data_keys (dict[str, Any]) – Descriptor mapping field names to their metadata, serialized to JSON.

Returns:

Hex UUID of the newly created stream.

Return type:

str

Raises:

RuntimeError – If the writer is not open or no run has been opened.

write_event(data: dict[str, float | int | str | bool], timestamps: dict[str, float] | None = None) → str[source]¶

Insert an event row and return its UID.

Events are not committed individually; the commit is deferred to close_run for performance.

Parameters:

data (dict[str, float | int | str | bool]) – Measured values keyed by field name.
timestamps (dict[str, float] or None, optional) – Per-field acquisition timestamps. Defaults to an empty dict.

Returns:

Hex UUID of the newly inserted event.

Return type:

str

Raises:

RuntimeError – If the writer is not open, no run has been opened, or no stream has been opened.

write_image(event_uid: str, field_name: str, image: np.ndarray) → None[source]¶

Append a 2-D image frame to the Zarr store and record a reference in SQLite.

Parameters:

event_uid (str) – Hex UUID of the parent event row in the events table.
field_name (str) – Detector field name, e.g. "ccd".
image (np.ndarray) – 2-D array of shape (height, width) to append.

Raises:

RuntimeError – If the writer is not open (self._conn is None).
RuntimeError – If no run has been opened (self._run_uid is empty).
RuntimeError – If no stream has been opened (self._stream_uid is empty).

Notes

The Zarr store lives at {db_path.stem}.zarr/ alongside the .db file. Frames are appended to the array at runs/{run_uid}/{field_name} inside the store. Each call grows the array by one frame along axis 0. The image_refs row stores the zarr group path and the zero-based frame index so the frame can be retrieved without scanning the full array. compression_codec is recorded as "blosc" to document the Zarr default; actual compression is controlled by zarr’s compressor setting.

class resonance.api.data.writer.IndexWriter(index_db_path: Path)[source]¶

Bases: object

Writer for the master index database aggregating cross-beamtime metadata.

Parameters:: index_db_path (Path) – Path to the master index SQLite database file.

Notes

This class is a skeleton. All methods raise NotImplementedError until the master index feature is implemented.

ensure_schema() → None[source]¶

Create the index schema if it does not already exist.

Raises:: NotImplementedError – Always; not yet implemented.

index_run(uid: str, researcher_id: int, beamtime_id: int, plan_name: str, *, sample_name: str | None = None, time_start: float | None = None, tags: list[str] | None = None) → None[source]¶

Insert a run summary row into the master index.

Parameters:

uid (str) – Hex UUID of the run, matching the per-beamtime runs.uid.
researcher_id (int) – Foreign key into the researchers table.
beamtime_id (int) – Foreign key into the beamtimes table.
plan_name (str) – Name of the scan plan.
sample_name (str or None, optional) – Sample name denormalized from the per-beamtime database.
time_start (float or None, optional) – Unix timestamp for the start of the run.
tags (list[str] or None, optional) – Arbitrary string tags for filtering.

Raises:

NotImplementedError – Always; not yet implemented.

register_beamtime(researcher_id: int, label: str, db_path: str, *, time_start: float | None = None) → int[source]¶

Insert or retrieve a beamtime row and return its id.

Parameters:

researcher_id (int) – Foreign key into the researchers table.
label (str) – Human-readable beamtime label, e.g. “2026-03-05”.
db_path (str) – Path to the per-beamtime SQLite file.
time_start (float or None, optional) – Unix timestamp for the start of the beamtime session.

Returns:

Primary key of the beamtime row.

Return type:

int

Raises:

NotImplementedError – Always; not yet implemented.

register_researcher(name: str, root_path: str, *, email: str | None = None, orcid: str | None = None, affiliation: str | None = None) → int[source]¶

Insert or retrieve a researcher row and return its id.

Parameters:

name (str) – Full name of the researcher.
root_path (str) – Filesystem root under which this researcher’s data lives.
email (str or None, optional) – Contact email address.
orcid (str or None, optional) – ORCID identifier string.
affiliation (str or None, optional) – Institutional affiliation.

Returns:

Primary key of the researcher row.

Return type:

int

Raises:

NotImplementedError – Always; not yet implemented.

class resonance.api.data.catalog.Catalog(db_path: Path)[source]¶

Bases: object

Read-only catalog over a per-beamtime SQLite database.

The catalog reads from a .db SQLite file and an adjacent .zarr directory store that holds detector image arrays.

Parameters:: db_path (Path) – Path to the beamtime SQLite database file. The Zarr store is expected at db_path.with_suffix(".zarr").

_conn¶

Read-only connection to the database.

Type:: sqlite3.Connection

_db_path¶

Resolved path passed at construction time.

Type:: Path

_zarr_path¶

Path to the adjacent Zarr store directory.

Type:: Path

__getitem__(uid: str) → Run[source]¶

Return the full Run object for the given UID.

Parameters:: uid (str) – Hex UUID of the run.
Returns:: Fully-featured run accessor backed by the open connection.
Return type:: Run
Raises:: KeyError – If no run with the given UID exists in the database.

by_sample(name: str) → list[RunSummary][source]¶

Return all runs associated with a sample by name.

Parameters:: name (str) – Exact sample name as stored in the samples table.
Returns:: Run summaries for the given sample, ordered newest-first.
Return type:: list[RunSummary]

close() → None[source]¶: Close the underlying database connection.

recent(n: int = 10) → list[RunSummary][source]¶

Return the n most recently started runs.

Parameters:: n (int, optional) – Maximum number of runs to return. Defaults to 10.
Returns:: Run summaries ordered newest-first.
Return type:: list[RunSummary]

class resonance.api.data.catalog.Run(conn: Connection, row: dict[str, Any], zarr_path: Path)[source]¶

Bases: object

Full accessor for a single run, including scalar event data and sample metadata.

Parameters:

conn (sqlite3.Connection) – Active connection to the beamtime database.
row (dict[str, Any]) – Deserialized row from the runs table.
zarr_path (Path) – Path to the adjacent Zarr store directory.

_conn¶

Shared database connection from the parent Catalog.

Type:: sqlite3.Connection

_row¶

Raw column values for this run.

Type:: dict[str, Any]

_zarr_path¶

Path to the Zarr store for detector images.

Type:: Path

property exit_status: str | None¶

Final run status, e.g. ‘success’, ‘failed’.

Type:: str or None

images(field: str = 'detector_image') → LazyImageSequence[source]¶

Return a lazy accessor for detector images in this run.

Images are not loaded until explicitly indexed. Each frame is stored as a slice of a Zarr array on the filesystem.

Parameters:: field (str, optional) – The image field name to load (default: “detector_image”).
Returns:: Lazy accessor with length equal to the number of images in this run.
Return type:: LazyImageSequence

Notes

Returns an empty LazyImageSequence if no images are stored for the given field or if the Zarr store does not exist.

property num_events: int¶

Total number of events recorded in the primary stream.

Type:: int

property plan_name: str¶

Name of the scan plan.

Type:: str

property sample: SampleMetadata | None¶

Return the associated sample metadata, or None if not set.

Returns:: Populated from the samples table row, with tags and extra deserialized from JSON. Returns None if sample_id is null or the referenced row does not exist.
Return type:: SampleMetadata or None

table(stream: str = 'primary') → DataFrame[source]¶

Load scalar event data from a named stream as a DataFrame.

Parameters:: stream (str, optional) – Name of the stream to load. Defaults to “primary”.
Returns:: Columns are seq_num, time, followed by all scalar fields present in the event data JSON. Returns an empty DataFrame with columns ["seq_num", "time"] if the stream has no events.
Return type:: pd.DataFrame

property time_start: float¶

Unix timestamp of run start.

Type:: float

property time_stop: float | None¶

Unix timestamp of run stop.

Type:: float or None

property uid: str¶

Hex UUID of the run.

Type:: str

class resonance.api.data.catalog.LazyImageSequence(conn: Connection, refs: list[dict[str, Any]], zarr_store_path: Path)[source]¶

Bases: object

Lazy accessor for detector images referenced via Zarr.

Images are not loaded until explicitly indexed. Each image is stored as a frame in a Zarr array on the filesystem; this class holds the reference metadata and defers loading.

Parameters:

conn (sqlite3.Connection) – Active connection to the beamtime database.
refs (list[dict[str, Any]]) – List of deserialized rows from the image_refs table.
zarr_store_path (Path) – Path to the Zarr store directory containing detector arrays.

_conn¶

Shared database connection from the parent Catalog.

Type:: sqlite3.Connection

_refs¶

Image reference metadata, one entry per frame.

Type:: list[dict[str, Any]]

_zarr_store_path¶

Path to the Zarr store directory.

Type:: Path

__getitem__(idx: int | slice) → ndarray[source]¶

Load one or more detector images from the Zarr store.

Parameters:

idx (int or slice) – Frame index or slice.

Returns:

A single (shape_x, shape_y) array for int index, or a stacked (n, shape_x, shape_y) array for slice index.

Return type:

np.ndarray

Raises:

IndexError – If idx is out of range.
TypeError – If idx is not int or slice.

property shape: tuple[int, int, int]¶

(n_frames, shape_x, shape_y).

Type:: tuple[int, int, int]

Schema helpers:

resonance.api.data.schema.create_beamtime_schema(conn: sqlite3.Connection) → None[source]¶

Create all per-beamtime tables, indexes, and pragmas.

Parameters:: conn (sqlite3.Connection) – Open connection to the beamtime SQLite database file.
Raises:: sqlite3.DatabaseError – If DDL execution fails due to a malformed database or I/O error.

Notes

Each beamtime session is stored in its own SQLite .db file alongside a Zarr store directory. The image_refs table holds references into the Zarr array (group path and frame index) rather than binary BLOBs, keeping the SQLite file small and enabling memory-mapped array access. Foreign keys are enforced on every connection via PRAGMA foreign_keys = ON, which must be re-applied per connection because SQLite resets it on open.

resonance.api.data.schema.create_index_schema(conn: sqlite3.Connection) → None[source]¶

Create master index tables, indexes, and pragmas.

Parameters:: conn (sqlite3.Connection) – Open connection to the master index SQLite database file.
Raises:: sqlite3.DatabaseError – If DDL execution fails due to a malformed database or I/O error.

Notes

The master index database aggregates metadata from all per-beamtime databases into three tables: researchers, beamtimes, and runs_index. This allows cross-beamtime queries without opening individual session databases. The runs_index table mirrors key fields from the per-beamtime runs table, identified by the same uid. Foreign keys are enforced via PRAGMA foreign_keys = ON, which must be re-applied per connection.

resonance.api.data.schema.migrate_beamtime_schema(conn: sqlite3.Connection, target_version: int = 2) → None[source]¶

Apply pending schema migrations up to target_version.

Parameters:

conn (sqlite3.Connection) – Open connection to the beamtime SQLite database file.
target_version (int, optional) – Schema version to migrate to. Defaults to BEAMTIME_SCHEMA_VERSION.

Raises:

RuntimeError – If target_version is less than the current schema version (downgrade not supported).

resonance.api.data.schema.migrate_index_schema(conn: sqlite3.Connection, target_version: int = 1) → None[source]¶

Apply pending index schema migrations up to target_version.

Parameters:

conn (sqlite3.Connection) – Open connection to the master index SQLite database file.
target_version (int, optional) – Schema version to migrate to. Defaults to INDEX_SCHEMA_VERSION.

Raises:

RuntimeError – If the database’s current user_version is greater than target_version, indicating a downgrade attempt or a database written by a newer version of this software.
NotImplementedError – If migrations are required (current_version < target_version) but no migration path has been implemented yet.

Notes

Migration steps are applied sequentially from current_version + 1 up to target_version. Version 1 is the initial schema; future versions will add individual ALTER TABLE or data-transform statements here.