Skip to content

Base Models

mosaicolabs.models.core.BaseModel

Bases: BaseModel

The root base class for SDK data models.

It inherits from pydantic.BaseModel to provide runtime type checking and initialization logic. It adds a hook for defining the corresponding PyArrow structure (__msco_pyarrow_struct__), enabling the SDK to auto-generate Flight schemas.

Note

This class has been added mainly for wrapping pydantic, toward future implementation where other fields mapping and checks are implemented

mosaicolabs.models.core.MosaicoType

Collection of Annotated type aliases mapping Python primitives to their PyArrow counterparts.

Each class attribute is an Annotated[PythonType, pa.DataType] alias. When used as a field annotation in a Serializable subclass, the embedded pa.DataType is extracted by _build_ontology_struct at class-definition time to derive the __msco_pyarrow_struct__ automatically — no manual schema declaration required.

For Arrow types not covered by the built-in aliases, fall back to a raw Annotated[T, pa.SomeType()] annotation; the schema builder resolves it transparently.

Scalar aliases:

Alias Python type Arrow type
MosaicoType.uint8 int pa.uint8()
MosaicoType.int8 int pa.int8()
MosaicoType.uint16 int pa.uint16()
MosaicoType.int16 int pa.int16()
MosaicoType.uint32 int pa.uint32()
MosaicoType.int32 int pa.int32()
MosaicoType.uint64 int pa.uint64()
MosaicoType.int64 int pa.int64()
MosaicoType.float16 float pa.float16()
MosaicoType.float32 float pa.float32()
MosaicoType.float64 float pa.float64()
MosaicoType.bool bool pa.bool_()
MosaicoType.string str pa.string()
MosaicoType.binary bytes pa.binary()

annotate staticmethod

annotate(py_type, pa_type)

Creates a type metadata binding between a Python type and a Pyarrow DataType.

This method uses Python's Annotated to wrap a standard type with specific Pyarrow schema information. This allows Pydantic models to correctly serialize and deserialize data into the desired Pyarrow format.

Parameters:

Name Type Description Default
py_type Type

The native Python type (e.g., int, str, or a Pydantic model).

required
pa_type DataType

The corresponding Pyarrow data type or logical type.

required

Returns:

Name Type Description
Annotated Annotated

A type hint containing the Python type and Pyarrow metadata.

list_ staticmethod

list_(source_type, list_size=None)

Build an Annotated[list, pa.list_(...)] type alias for list fields.

Accepts either a MosaicoType alias (i.e. any type carrying __metadata__ with a pa.DataType) or a raw Python primitive present in BASE_MAPPING (int, float, str, bool, bytes).

Parameters:

Name Type Description Default
source_type Any

A MosaicoType alias or a raw Python primitive type whose PyArrow equivalent is defined in BASE_MAPPING.

required
list_size Optional[int]

If provided, produces a fixed-size Arrow list (pa.list_(type, list_size)). If None, produces a variable-length Arrow list (pa.list_(type)).

None

Returns:

Name Type Description
Any Any

An Annotated[list, pa.ListType] alias ready to be used as a

Any

field annotation in a Serializable subclass.

Raises:

Type Description
ValueError

If source_type does not resolve to a valid pa.DataType.

matrix staticmethod

matrix(source_type, rows=None, cols=None)

Build an Annotated[list, pa.list_(...)] type alias for 2D matrix fields.

Composes two nested MosaicoType.list_() calls to represent a matrix of shape (rows, cols). Both dimensions are optional: if None, the dimension is variable-length; if provided, it is fixed-size.

Parameters:

Name Type Description Default
source_type Any

A MosaicoType alias or a raw Python primitive type whose PyArrow equivalent is defined in BASE_MAPPING.

required
rows Optional[int]

If provided, the outer list is fixed-size. If None, the outer list is variable-length.

None
cols Optional[int]

If provided, the inner list is fixed-size. If None, the inner list is variable-length.

None

Returns:

Type Description
Annotated

Annotated[list, pa.ListType]: An Annotated[list, pa.ListType] alias representing a 2D matrix, ready to be used as a field annotation in a Serializable subclass.

Raises:

Type Description
ValueError

If source_type does not resolve to a valid pa.DataType.

tensor3d staticmethod

tensor3d(source_type, depth=None, rows=None, cols=None)

Build an Annotated[list, pa.list_(...)] type alias for 3D tensor fields.

Composes MosaicoType.matrix() and MosaicoType.list_() to represent a tensor of shape (depth, rows, cols). All dimensions are optional: if None, the dimension is variable-length; if provided, it is fixed-size.

Parameters:

Name Type Description Default
source_type Any

A MosaicoType alias or a raw Python primitive type whose PyArrow equivalent is defined in BASE_MAPPING.

required
depth Optional[int]

If provided, the outer list is fixed-size. If None, the outer list is variable-length.

None
rows Optional[int]

If provided, the matrix rows are fixed-size. If None, the matrix rows are variable-length.

None
cols Optional[int]

If provided, the matrix cols are fixed-size. If None, the matrix cols are variable-length.

None

Returns:

Type Description
Annotated

Annotated[list, pa.ListType]: An Annotated[list, pa.ListType] alias representing a 3D tensor, ready to be used as a field annotation in a Serializable subclass.

Raises:

Type Description
ValueError

If source_type does not resolve to a valid pa.DataType.

mosaicolabs.models.core.MosaicoField

MosaicoField(
    nullable=False, default=..., description=None, **kwargs
)

Factory for Pydantic FieldInfo instances carrying Mosaico-specific Arrow metadata.

Acts as a drop-in replacement for pydantic.Field within Serializable subclasses. The nullable flag is stored in json_schema_extra and consumed by _build_ontology_struct when deriving the __msco_pyarrow_struct__ at class-definition time.

Parameters:

Name Type Description Default
nullable bool

Whether the corresponding Arrow field should be declared as nullable in the generated pa.struct. Defaults to False.

False
default Any | EllipsisType

Default value for the field. Use ... (the default) to mark the field as required. Any other value makes the field optional and sets its fallback.

...
description Optional[str]

Human-readable description of the field, forwarded to Pydantic and surfaced in the JSON Schema output.

None
**kwargs Any

Additional keyword arguments forwarded verbatim to pydantic.Field.

{}

Returns:

Name Type Description
Any Any

A pydantic.FieldInfo instance with Mosaico nullability metadata attached.

mosaicolabs.models.core.Message

Bases: BaseModel

The universal transport envelope for Mosaico data.

The Message class wraps a polymorphic Serializable payload with its ingestion timestamps (record time).

Attributes:

Name Type Description
timestamp_ns int

Ingestion timestamp in nanoseconds (record time). This represents the time at which the message was received and persisted by the recording system (e.g., rosbag, parquet writer, logging pipeline, or database).

data Serializable

The actual ontology data payload (e.g., an IMU or GPS instance).

Querying with the .Q Proxy

When constructing a QueryOntologyCatalog, the Message attributes are fully queryable.

Field Access Path Queryable Type Supported Operators
<Model>.Q.timestamp_ns Numeric .eq(), .lt(), .gt(), .leq(), .geq(), .in_(), .between(), .outside()
Universal Compatibility

The <Model> placeholder represents any Mosaico ontology class (e.g., IMU, GPS, Floating64) or any custom user-defined class that is a subclass of Serializable.

Example
from mosaicolabs import MosaicoClient, IMU, Floating64, QueryOntologyCatalog

with MosaicoClient.connect("localhost", 6726) as client:
    # Filter IMU data by a specific acquisition second
    qresponse = client.query(
        QueryOntologyCatalog(IMU.Q.timestamp_ns.lt(1770282868))
    )

    # Inspect the response
    if qresponse is not None:
        # Results are automatically grouped by Sequence for easier data management
        for item in qresponse:
            print(f"Sequence: {item.sequence.name}")
            print(f"Topics: {[topic.name for topic in item.topics]}")

data instance-attribute

data

The actual ontology data payload (e.g., an IMU or GPS instance).

timestamp_ns instance-attribute

timestamp_ns

Ingestion timestamp in nanoseconds (record time).

This represents the time at which this message was received and persisted by the recording system (e.g., rosbag, parquet writer, logging pipeline, or database).

This timestamp reflects infrastructure timing and may include:

  • transport delay
  • middleware delay
  • serialization/deserialization delay
  • scheduling delay

It does NOT represent when the sensor measurement occurred.

Typical usage:

  • latency measurement
  • debugging transport or pipeline delays
  • ordering messages by arrival time
Querying with the .Q Proxy

The timestamp_ns field is queryable using the .Q proxy.

Field Access Path Queryable Type Supported Operators
<Model>.Q.timestamp_ns Numeric .eq(), .lt(), .gt(), .leq(), .geq(), .in_(), .between(), .outside()

The <Model> placeholder represents any Mosaico ontology class (e.g., IMU, GPS, Floating64) or any custom user-defined class that is a subclass of Serializable

Example
from mosaicolabs import MosaicoClient, IMU, QueryOntologyCatalog

with MosaicoClient.connect("localhost", 6726) as client:
    # Filter IMU data by a specific recording second
    qresponse = client.query(
        QueryOntologyCatalog(IMU.Q.timestamp_ns.lt(1770282868))
    )

    # Inspect the response
    if qresponse is not None:
        # Results are automatically grouped by Sequence for easier data management
        for item in qresponse:
            print(f"Sequence: {item.sequence.name}")
            print(f"Topics: {[topic.name for topic in item.topics]}")

model_post_init

model_post_init(context)

Validates the message structure after initialization.

Ensures that there are no field name collisions between the envelope (e.g., timestamp_ns) and the data payload.

ontology_type

ontology_type()

Retrieves the class type of the ontology object stored in the data field.

ontology_tag

ontology_tag()

Returns the unique ontology tag name associated with the object in the data field.

get_data

get_data(target_type)

Safe, type-hinted accessor for the data payload.

Parameters:

Name Type Description Default
target_type Type[TSerializable]

The expected Serializable subclass type.

required

Returns:

Type Description
Optional[TSerializable]

Optional[TSerializable]: The data object cast to the requested type or None if cannot be casted.

Example
# Get the IMU data from the message
image_data = message.get_data(Image)
print(f"Timestamp: {message.timestamp_ns}")
print(f"Image size: {image_data.height}x{image_data.width}")
# Show the image
image_data.to_pillow().show()

# Get the Floating64 data from the message
floating64_data = message.get_data(Floating64)
print(f"Timestamp: {message.timestamp_ns}")
print(f"Data value: {floating64_data.data}")

from_dataframe_row staticmethod

from_dataframe_row(
    row, topic_name, timestamp_column_name="timestamp_ns"
)

Reconstructs a Message object from a flattened DataFrame row.

In the Mosaico Data Platform, DataFrames represent topics using a nested naming convention: {topic}.{tag}.{field}. This method performs Smart Reconstruction by:

  1. Topic Validation: Verifying if any columns associated with the topic_name exist in the row.
  2. Tag Inference: Inspecting the column headers to automatically determine the original ontology tag (e.g., "imu").
  3. Data Extraction: Stripping prefixes and re-nesting the flat columns into their original dictionary structures.
  4. Type Casting: Re-instantiating the specific Serializable subclass and wrapping it in a Message envelope.

Parameters:

Name Type Description Default
row Series

A single row from a Pandas DataFrame, representing a point in time across one or more topics.

required
topic_name str

The name of the specific topic to extract from the row.

required
timestamp_column_name str

The name of the column containing the timestamp.

'timestamp_ns'

Returns:

Type Description
Optional[Message]

Optional[Message]: A reconstructed Message instance containing the typed ontology data, or None if the topic is not present or the data is incomplete.

Raises:

Type Description
ValueError

If the ontology tag for the topic could not be inferred from the row's columns.

Example
# Obtain a dataframe with DataFrameExtractor
from mosaicolabs import MosaicoClient, IMU, Image
from mosaicolabs.ml import DataFrameExtractor, SyncTransformer

with MosaicoClient.connect("localhost", 6726) as client:
    sequence_handler = client.sequence_handler("example_sequence")
    for df in DataFrameExtractor(sequence_handler).to_pandas_chunks(
        topics = ["/front/imu", "/front/camera/image_raw"]
    ):
        # Do something with the dataframe.
        # e.g. reconstruct the image message from a dataframe row
        image_msg = Message.from_dataframe_row(
            row=df, topic_name="/front/camera/image_raw"
        )
        image_data = image_msg.get_data(Image)
        # Show the image
        image_data.to_pillow().show()
        # ...

mosaicolabs.models.core.Serializable

Bases: BaseModel, _QueryProxyMixin

The base class for all Mosaico ontology data payloads.

This class serves as the root for every sensor and data type in the Mosaico ecosystem. By inheriting from Serializable, data models are automatically compatible with the platform's storage, querying, and serialization engines.

Dynamic Attributes Injection

When you define a subclass, several key attributes are automatically managed or required. Understanding these is essential for customizing how your data is treated by the platform:

  • __serialization_format__: Determines the batching strategy and storage optimization.

    • Role: It tells the SequenceWriter whether to flush data based on byte size (optimal for heavy data like Images) or record count (optimal for light telemetry like IMU).
    • Default: SerializationFormat.Default.
  • __ontology_tag__: The string identifier for the class as known to the Mosaico platform (e.g., "imu", "gps_raw").

    • Role: This is the tag sent to the server (topic creation) and embedded in .Q query paths; it's what ontology_tag() returns.
    • Generation: If not explicitly provided, it is auto-generated by converting the class name from CamelCase to snake_case.
    • Note: Unlike a Python registry key, this value is not guaranteed unique per class — see __registry_key__ below.
  • __registry_key__: The internal key this class is stored under in the SDK's local class registry.

    • Role: Guarantees a collision-free lookup key per Python class, independent of __ontology_tag__. Defaults to __ontology_tag__ itself, so for every hand-authored class the two are identical.
    • When they differ: dynamically-resolved (Unmodeled) classes can end up sharing one __ontology_tag__ across multiple schema shapes (e.g. two versions of the same message type). In that case, only the first-seen schema keeps __registry_key__ == __ontology_tag__; subsequent schema variants get a distinct, fingerprint-suffixed __registry_key__ while still reporting the same __ontology_tag__ to the platform.
  • __class_type__: A reference to the concrete class itself.

    • Role: Injected during initialization to facilitate polymorphic instantiation and safe type-checking when extracting data from a Message.
Requirements for Custom Ontologies

To create a valid custom ontology, your subclass must:

  1. Inherit from Serializable.
  2. Define the attributes using MosaicoType and MosaicoField
Automatic Registration

Any subclass of Serializable is automatically registered in the global Mosaico registry upon definition. This enables the use of the factory methods and the .Q query proxy immediately.

is_registered classmethod

is_registered()

Checks if a class is registered.

Returns:

Name Type Description
bool bool

True if registered.

ontology_tag classmethod

ontology_tag()

Retrieves the unique identifier (tag) for the current ontology class, automatically generated during class definition.

This method provides the string key used by the Mosaico platform to identify and route specific data types within the ontology registry. It abstracts away the internal naming conventions, ensuring that you always use the correct identifier for queries and serialization.

Returns:

Name Type Description
str str

The registered string tag for this class (e.g., "imu", "gps").

Raises:

Type Description
Exception

If the class was not properly initialized via __pydantic_init_subclass__.

Practical Application: Topic Filtering

This method is particularly useful when constructing QueryTopic requests. By using the convenience method QueryTopic.with_ontology_tag(), you can filter topics by data type without hardcoding strings that might change.

Example:

from mosaicolabs import MosaicoClient, Topic, IMU, QueryTopic

with MosaicoClient.connect("localhost", 6726) as client:
    # Filter for a specific data value (using constructor)
    qresponse = client.query(
        QueryTopic(
            Topic.with_ontology_tag(IMU.ontology_tag()),
        )
    )

    # Inspect the response
    if qresponse is not None:
        # Results are automatically grouped by Sequence for easier data management
        for item in qresponse:
            print(f"Sequence: {item.sequence.name}")
            print(f"Topics: {[topic.name for topic in item.topics]}")

mosaicolabs.models.core.unmodeled.Unmodeled

Bases: Serializable

Base class for ontology data that has no hand-authored Python class.

Where a normal Serializable subclass (e.g. IMU) declares one typed field per schema field, Unmodeled instead stores the entire payload in a single generic raw_data dict, and carries its actual Arrow schema as data (__msco_pyarrow_struct__) rather than deriving it from typed class fields. This lets a single class shape represent any ontology schema, decided entirely at runtime.

Not meant to be subclassed directly

Don't subclass Unmodeled by hand. Instead, use the make_unmodeled_ontology_class factory (or, in most cases, the higher-level resolve_ontology_class helper) to generate a properly-configured subclass for a specific ontology tag and pyarrow schema.

Validation

Every Unmodeled instance validates raw_data against the class's declared __msco_pyarrow_struct__ schema at construction time - missing required fields, unknown fields, and nested-object type mismatches all raise a ValueError immediately, rather than surfacing later as an opaque error during Arrow serialization.

Querying with the .Q Proxy

Classes generated via make_unmodeled_ontology_class are still fully queryable via the .Q proxy, exactly like a hand-authored ontology, since the proxy is built from the class's pyarrow schema rather than its Python field declarations.

raw_data instance-attribute

raw_data

The full ontology payload, keyed by field name exactly as declared in the class's __msco_pyarrow_struct__ schema. Nested struct fields are represented as nested dicts (e.g. {"gyro": {"x": 1.0, "y": 2.0, "z": 3.0}}).

model_post_init

model_post_init(context)

Validates raw_data against the class's declared pyarrow schema.

Parameters:

Name Type Description Default
context Any

The Pydantic validation context passed by the base class.

required

Raises:

Type Description
ValueError

If raw_data is missing a required field, contains a field not present in the schema, or has a nested field whose value isn't a dict where the schema expects a nested struct. All mismatches found are reported together in one exception.

is_registered classmethod

is_registered()

Checks if a class is registered.

Returns:

Name Type Description
bool bool

True if registered.

ontology_tag classmethod

ontology_tag()

Retrieves the unique identifier (tag) for the current ontology class, automatically generated during class definition.

This method provides the string key used by the Mosaico platform to identify and route specific data types within the ontology registry. It abstracts away the internal naming conventions, ensuring that you always use the correct identifier for queries and serialization.

Returns:

Name Type Description
str str

The registered string tag for this class (e.g., "imu", "gps").

Raises:

Type Description
Exception

If the class was not properly initialized via __pydantic_init_subclass__.

Practical Application: Topic Filtering

This method is particularly useful when constructing QueryTopic requests. By using the convenience method QueryTopic.with_ontology_tag(), you can filter topics by data type without hardcoding strings that might change.

Example:

from mosaicolabs import MosaicoClient, Topic, IMU, QueryTopic

with MosaicoClient.connect("localhost", 6726) as client:
    # Filter for a specific data value (using constructor)
    qresponse = client.query(
        QueryTopic(
            Topic.with_ontology_tag(IMU.ontology_tag()),
        )
    )

    # Inspect the response
    if qresponse is not None:
        # Results are automatically grouped by Sequence for easier data management
        for item in qresponse:
            print(f"Sequence: {item.sequence.name}")
            print(f"Topics: {[topic.name for topic in item.topics]}")

mosaicolabs.models.core.unmodeled.make_unmodeled_ontology_class

make_unmodeled_ontology_class(
    class_name,
    ontology_tag,
    serialization_format,
    pyarrow_schema,
    registry_key=None,
)

Dynamically creates an Unmodeled subclass for a specific pyarrow schema.

This is the factory that turns an arbitrary Arrow schema into a class the SDK can serialize, ingest, query and retrieve just like any hand-authored ontology - the schema is attached to the generated class as data (__msco_pyarrow_struct__) instead of being derived from Python field declarations.

registry_key is for advanced use only

Most callers should omit registry_key entirely. It exists so that resolve_ontology_class can create a schema variant of an existing ontology_tag: a second class reporting the same ontology_tag to the platform (so it remains discoverable under one consistent tag) while still occupying a distinct, collision-free key in the SDK's local class registry. See Serializable.__registry_key__ for the full rationale.

Schema generation is intentionally skipped

The returned class is created with skip_schema_generation=True, so pyarrow_schema is used verbatim as __msco_pyarrow_struct__ rather than being (re)derived from Unmodeled.raw_data's Dict[str, Any] annotation, which wouldn't produce a useful schema on its own. skip_query_proxy_ingestion is left at its default (False), so the .Q query proxy is still generated from pyarrow_schema.

Example
import pyarrow as pa
from mosaicolabs.enum import SerializationFormat
from mosaicolabs.models.core.unmodeled import make_unmodeled_ontology_class

UnmodeledGyro = make_unmodeled_ontology_class(
    class_name="UnmodeledGyro",
    ontology_tag="gyro_raw",
    serialization_format=SerializationFormat.Default,
    pyarrow_schema=pa.struct([
        pa.field("gyro", pa.struct([
            pa.field("x", pa.float32()),
            pa.field("y", pa.float32()),
            pa.field("z", pa.float32()),
        ])),
    ]),
)

# Fully usable like any other ontology class, e.g.:
# topic_writer.push(Message(timestamp_ns=..., data=UnmodeledGyro(
#     raw_data={"gyro": {"x": 0.1, "y": 0.0, "z": -0.2}}
# )))

Parameters:

Name Type Description Default
class_name str

The Python class name assigned to the generated class (e.g. shown in repr() and error messages).

required
ontology_tag Optional[str]

The unique ontology identifier to register the class under. If None, it's auto-generated from class_name, matching the behavior of a normal Serializable subclass.

required
serialization_format SerializationFormat

The batching/serialization strategy for topics using this ontology (see Serializable.__serialization_format__).

required
pyarrow_schema StructType

The Arrow struct schema describing the ontology's data payload, used verbatim as the class's __msco_pyarrow_struct__.

required
registry_key Optional[str]

Advanced/internal use - see the note above. Defaults to None, meaning the class's local registry key is ontology_tag itself (the common case for every direct caller of this factory).

None

Returns:

Type Description
Type[Unmodeled]

Type[Unmodeled]: A new Unmodeled subclass, already registered in the Serializable factory under registry_key (or ontology_tag, if registry_key is omitted), and reporting ontology_tag to the platform either way.

Raises:

Type Description
ValueError

If the resolved registry key is already registered for a different class.

mosaicolabs.models.core.helpers.resolve_ontology_class

resolve_ontology_class(
    *,
    ontology_tag,
    schema=None,
    schema_fingerprint=None,
    serialization_format=None,
)

Resolves an ontology tag to a concrete Serializable class, creating a dynamic Unmodeled fallback class on demand when no hand-authored class is registered for the tag.

Schema Variants

A single tag can end up associated with more than one schema shape within a single process (e.g. two rosbags recorded with different versions of the same ROS message type, both mapped to the same inferred ontology tag). When the schema passed in doesn't match the one already registered for ontology_tag, a distinct variant class is resolved (or created) instead of silently reusing the wrong schema. The variant still reports the same ontology_tag to the platform (so all of its data stays discoverable under one consistent tag); only its SDK-local __registry_key__ differs, deterministically derived as f"{ontology_tag}__{fingerprint}".

Parameters:

Name Type Description Default
ontology_tag str

The ontology identifier to resolve. If a Serializable class is already registered under this tag, it's returned directly (subject to the schema-variant check above); otherwise a dynamic Unmodeled class is created and registered under it.

required
schema Optional[StructType]

The pyarrow schema of the incoming data. Required when ontology_tag isn't already registered, since it's needed to build the fallback class. When provided for an already-registered tag, it's compared against the registered schema to detect drift.

None
schema_fingerprint Optional[str]

The fingerprint of schema, if the caller already computed it (e.g. once at stream-connect time, since the schema is invariant for the life of a stream). Avoids re-hashing schema on every call. Computed from schema on demand if omitted.

None
serialization_format Optional[SerializationFormat]

The serialization format to use if a dynamic class needs to be created. Defaults to SerializationFormat.Default if omitted.

None

Returns:

Type Description
Type[Serializable]

Type[Serializable]: The resolved Serializable class: the already-registered class for ontology_tag, a newly created Unmodeled fallback class, or a distinct schema-variant class, depending on the case above.

Raises:

Type Description
ValueError

If ontology_tag isn't registered and no schema is provided to build a fallback class from.