Base Models
mosaicolabs.models.core.BaseModel ¶
Bases: BaseModel
The root base class for SDK data models.
It inherits from pydantic.BaseModel to provide runtime type checking and
initialization logic. It adds a hook for defining the corresponding
PyArrow structure (__msco_pyarrow_struct__), enabling the SDK to auto-generate
Flight schemas.
Note
This class has been added mainly for wrapping pydantic, toward future implementation where other fields mapping and checks are implemented
mosaicolabs.models.core.MosaicoType ¶
Collection of Annotated type aliases mapping Python primitives to
their PyArrow counterparts.
Each class attribute is an Annotated[PythonType, pa.DataType] alias.
When used as a field annotation in a Serializable subclass, the
embedded pa.DataType is extracted by _build_ontology_struct at
class-definition time to derive the __msco_pyarrow_struct__
automatically — no manual schema declaration required.
For Arrow types not covered by the built-in aliases, fall back to a raw
Annotated[T, pa.SomeType()] annotation; the schema builder resolves
it transparently.
Scalar aliases:
| Alias | Python type | Arrow type |
|---|---|---|
MosaicoType.uint8 |
int |
pa.uint8() |
MosaicoType.int8 |
int |
pa.int8() |
MosaicoType.uint16 |
int |
pa.uint16() |
MosaicoType.int16 |
int |
pa.int16() |
MosaicoType.uint32 |
int |
pa.uint32() |
MosaicoType.int32 |
int |
pa.int32() |
MosaicoType.uint64 |
int |
pa.uint64() |
MosaicoType.int64 |
int |
pa.int64() |
MosaicoType.float16 |
float |
pa.float16() |
MosaicoType.float32 |
float |
pa.float32() |
MosaicoType.float64 |
float |
pa.float64() |
MosaicoType.bool |
bool |
pa.bool_() |
MosaicoType.string |
str |
pa.string() |
MosaicoType.binary |
bytes |
pa.binary() |
annotate
staticmethod
¶
Creates a type metadata binding between a Python type and a Pyarrow DataType.
This method uses Python's Annotated to wrap a standard type with specific
Pyarrow schema information. This allows Pydantic models to correctly
serialize and deserialize data into the desired Pyarrow format.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
py_type
|
Type
|
The native Python type (e.g., int, str, or a Pydantic model). |
required |
pa_type
|
DataType
|
The corresponding Pyarrow data type or logical type. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Annotated |
Annotated
|
A type hint containing the Python type and Pyarrow metadata. |
list_
staticmethod
¶
Build an Annotated[list, pa.list_(...)] type alias for list fields.
Accepts either a MosaicoType alias (i.e. any type carrying
__metadata__ with a pa.DataType) or a raw Python primitive
present in BASE_MAPPING (int, float, str, bool,
bytes).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source_type
|
Any
|
A |
required |
list_size
|
Optional[int]
|
If provided, produces a fixed-size Arrow list
( |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
Any |
Any
|
An |
Any
|
field annotation in a |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
matrix
staticmethod
¶
Build an Annotated[list, pa.list_(...)] type alias for 2D matrix fields.
Composes two nested MosaicoType.list_() calls to represent a matrix
of shape (rows, cols). Both dimensions are optional: if None,
the dimension is variable-length; if provided, it is fixed-size.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source_type
|
Any
|
A |
required |
rows
|
Optional[int]
|
If provided, the outer list is fixed-size. If |
None
|
cols
|
Optional[int]
|
If provided, the inner list is fixed-size. If |
None
|
Returns:
| Type | Description |
|---|---|
Annotated
|
Annotated[list, pa.ListType]: An |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
tensor3d
staticmethod
¶
Build an Annotated[list, pa.list_(...)] type alias for 3D tensor fields.
Composes MosaicoType.matrix() and MosaicoType.list_() to represent
a tensor of shape (depth, rows, cols). All dimensions are optional:
if None, the dimension is variable-length; if provided, it is fixed-size.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source_type
|
Any
|
A |
required |
depth
|
Optional[int]
|
If provided, the outer list is fixed-size. If |
None
|
rows
|
Optional[int]
|
If provided, the matrix rows are fixed-size. If |
None
|
cols
|
Optional[int]
|
If provided, the matrix cols are fixed-size. If |
None
|
Returns:
| Type | Description |
|---|---|
Annotated
|
Annotated[list, pa.ListType]: An |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
mosaicolabs.models.core.MosaicoField ¶
Factory for Pydantic FieldInfo instances carrying Mosaico-specific
Arrow metadata.
Acts as a drop-in replacement for pydantic.Field within
Serializable subclasses. The nullable flag is stored in
json_schema_extra and consumed by _build_ontology_struct when
deriving the __msco_pyarrow_struct__ at class-definition time.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
nullable
|
bool
|
Whether the corresponding Arrow field should be declared
as nullable in the generated |
False
|
default
|
Any | EllipsisType
|
Default value for the field. Use |
...
|
description
|
Optional[str]
|
Human-readable description of the field, forwarded to Pydantic and surfaced in the JSON Schema output. |
None
|
**kwargs
|
Any
|
Additional keyword arguments forwarded verbatim to
|
{}
|
Returns:
| Name | Type | Description |
|---|---|---|
Any |
Any
|
A |
mosaicolabs.models.core.Message ¶
Bases: BaseModel
The universal transport envelope for Mosaico data.
The Message class wraps a polymorphic Serializable
payload with its ingestion timestamps (record time).
Attributes:
| Name | Type | Description |
|---|---|---|
timestamp_ns |
int
|
Ingestion timestamp in nanoseconds (record time). This represents the time at which the message was received and persisted by the recording system (e.g., rosbag, parquet writer, logging pipeline, or database). |
data |
Serializable
|
The actual ontology data payload (e.g., an IMU or GPS instance). |
Querying with the .Q Proxy¶
When constructing a QueryOntologyCatalog,
the Message attributes are fully queryable.
| Field Access Path | Queryable Type | Supported Operators |
|---|---|---|
<Model>.Q.timestamp_ns |
Numeric |
.eq(), .lt(), .gt(), .leq(), .geq(), .in_(), .between(), .outside() |
Universal Compatibility
The <Model> placeholder represents any Mosaico ontology class (e.g., IMU, GPS, Floating64)
or any custom user-defined class that is a subclass of Serializable.
Example
from mosaicolabs import MosaicoClient, IMU, Floating64, QueryOntologyCatalog
with MosaicoClient.connect("localhost", 6726) as client:
# Filter IMU data by a specific acquisition second
qresponse = client.query(
QueryOntologyCatalog(IMU.Q.timestamp_ns.lt(1770282868))
)
# Inspect the response
if qresponse is not None:
# Results are automatically grouped by Sequence for easier data management
for item in qresponse:
print(f"Sequence: {item.sequence.name}")
print(f"Topics: {[topic.name for topic in item.topics]}")
timestamp_ns
instance-attribute
¶
Ingestion timestamp in nanoseconds (record time).
This represents the time at which this message was received and persisted by the recording system (e.g., rosbag, parquet writer, logging pipeline, or database).
This timestamp reflects infrastructure timing and may include:
- transport delay
- middleware delay
- serialization/deserialization delay
- scheduling delay
It does NOT represent when the sensor measurement occurred.
Typical usage:
- latency measurement
- debugging transport or pipeline delays
- ordering messages by arrival time
Querying with the .Q Proxy¶
The timestamp_ns field is queryable using the .Q proxy.
| Field Access Path | Queryable Type | Supported Operators |
|---|---|---|
<Model>.Q.timestamp_ns |
Numeric |
.eq(), .lt(), .gt(), .leq(), .geq(), .in_(), .between(), .outside() |
The <Model> placeholder represents any Mosaico ontology class (e.g., IMU, GPS, Floating64)
or any custom user-defined class that is a subclass of Serializable
Example
from mosaicolabs import MosaicoClient, IMU, QueryOntologyCatalog
with MosaicoClient.connect("localhost", 6726) as client:
# Filter IMU data by a specific recording second
qresponse = client.query(
QueryOntologyCatalog(IMU.Q.timestamp_ns.lt(1770282868))
)
# Inspect the response
if qresponse is not None:
# Results are automatically grouped by Sequence for easier data management
for item in qresponse:
print(f"Sequence: {item.sequence.name}")
print(f"Topics: {[topic.name for topic in item.topics]}")
model_post_init ¶
Validates the message structure after initialization.
Ensures that there are no field name collisions between the envelope
(e.g., timestamp_ns) and the data payload.
ontology_type ¶
Retrieves the class type of the ontology object stored in the data field.
ontology_tag ¶
Returns the unique ontology tag name associated with the object in the data field.
get_data ¶
Safe, type-hinted accessor for the data payload.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
target_type
|
Type[TSerializable]
|
The expected |
required |
Returns:
| Type | Description |
|---|---|
Optional[TSerializable]
|
Optional[TSerializable]: The data object cast to the requested type or None if cannot be casted. |
Example
# Get the IMU data from the message
image_data = message.get_data(Image)
print(f"Timestamp: {message.timestamp_ns}")
print(f"Image size: {image_data.height}x{image_data.width}")
# Show the image
image_data.to_pillow().show()
# Get the Floating64 data from the message
floating64_data = message.get_data(Floating64)
print(f"Timestamp: {message.timestamp_ns}")
print(f"Data value: {floating64_data.data}")
from_dataframe_row
staticmethod
¶
Reconstructs a Message object from a flattened DataFrame row.
In the Mosaico Data Platform, DataFrames represent topics using a nested naming
convention: {topic}.{tag}.{field}. This method performs
Smart Reconstruction by:
- Topic Validation: Verifying if any columns associated with the
topic_nameexist in the row. - Tag Inference: Inspecting the column headers to automatically determine
the original ontology tag (e.g.,
"imu"). - Data Extraction: Stripping prefixes and re-nesting the flat columns into their original dictionary structures.
- Type Casting: Re-instantiating the specific
Serializablesubclass and wrapping it in aMessageenvelope.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
row
|
Series
|
A single row from a Pandas DataFrame, representing a point in time across one or more topics. |
required |
topic_name
|
str
|
The name of the specific topic to extract from the row. |
required |
timestamp_column_name
|
str
|
The name of the column containing the timestamp. |
'timestamp_ns'
|
Returns:
| Type | Description |
|---|---|
Optional[Message]
|
Optional[Message]: A reconstructed |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the ontology tag for the topic could not be inferred from the row's columns. |
Example
# Obtain a dataframe with DataFrameExtractor
from mosaicolabs import MosaicoClient, IMU, Image
from mosaicolabs.ml import DataFrameExtractor, SyncTransformer
with MosaicoClient.connect("localhost", 6726) as client:
sequence_handler = client.sequence_handler("example_sequence")
for df in DataFrameExtractor(sequence_handler).to_pandas_chunks(
topics = ["/front/imu", "/front/camera/image_raw"]
):
# Do something with the dataframe.
# e.g. reconstruct the image message from a dataframe row
image_msg = Message.from_dataframe_row(
row=df, topic_name="/front/camera/image_raw"
)
image_data = image_msg.get_data(Image)
# Show the image
image_data.to_pillow().show()
# ...
mosaicolabs.models.core.Serializable ¶
Bases: BaseModel, _QueryProxyMixin
The base class for all Mosaico ontology data payloads.
This class serves as the root for every sensor and data type in the Mosaico ecosystem.
By inheriting from Serializable, data models are automatically compatible with the platform's storage,
querying, and serialization engines.
Dynamic Attributes Injection¶
When you define a subclass, several key attributes are automatically managed or required. Understanding these is essential for customizing how your data is treated by the platform:
-
__serialization_format__: Determines the batching strategy and storage optimization.- Role: It tells the
SequenceWriterwhether to flush data based on byte size (optimal for heavy data likeImages) or record count (optimal for light telemetry likeIMU). - Default:
SerializationFormat.Default.
- Role: It tells the
-
__ontology_tag__: The string identifier for the class as known to the Mosaico platform (e.g.,"imu","gps_raw").- Role: This is the tag sent to the server (topic creation) and embedded in
.Qquery paths; it's whatontology_tag()returns. - Generation: If not explicitly provided, it is auto-generated by converting the class name from
CamelCasetosnake_case. - Note: Unlike a Python registry key, this value is not guaranteed unique per class — see
__registry_key__below.
- Role: This is the tag sent to the server (topic creation) and embedded in
-
__registry_key__: The internal key this class is stored under in the SDK's local class registry.- Role: Guarantees a collision-free lookup key per Python class, independent of
__ontology_tag__. Defaults to__ontology_tag__itself, so for every hand-authored class the two are identical. - When they differ: dynamically-resolved (
Unmodeled) classes can end up sharing one__ontology_tag__across multiple schema shapes (e.g. two versions of the same message type). In that case, only the first-seen schema keeps__registry_key__ == __ontology_tag__; subsequent schema variants get a distinct, fingerprint-suffixed__registry_key__while still reporting the same__ontology_tag__to the platform.
- Role: Guarantees a collision-free lookup key per Python class, independent of
-
__class_type__: A reference to the concrete class itself.- Role: Injected during initialization to facilitate polymorphic instantiation and safe type-checking when extracting data from a
Message.
- Role: Injected during initialization to facilitate polymorphic instantiation and safe type-checking when extracting data from a
Requirements for Custom Ontologies¶
To create a valid custom ontology, your subclass must:
- Inherit from
Serializable. - Define the attributes using
MosaicoTypeandMosaicoField
Automatic Registration
Any subclass of Serializable is automatically registered in the global Mosaico registry upon definition. This enables the use of the factory methods and the .Q query proxy immediately.
is_registered
classmethod
¶
Checks if a class is registered.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if registered. |
ontology_tag
classmethod
¶
Retrieves the unique identifier (tag) for the current ontology class, automatically generated during class definition.
This method provides the string key used by the Mosaico platform to identify and route specific data types within the ontology registry. It abstracts away the internal naming conventions, ensuring that you always use the correct identifier for queries and serialization.
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
The registered string tag for this class (e.g., |
Raises:
| Type | Description |
|---|---|
Exception
|
If the class was not properly initialized via |
Practical Application: Topic Filtering
This method is particularly useful when constructing QueryTopic
requests. By using the convenience method QueryTopic.with_ontology_tag(),
you can filter topics by data type without hardcoding strings that might change.
Example:
from mosaicolabs import MosaicoClient, Topic, IMU, QueryTopic
with MosaicoClient.connect("localhost", 6726) as client:
# Filter for a specific data value (using constructor)
qresponse = client.query(
QueryTopic(
Topic.with_ontology_tag(IMU.ontology_tag()),
)
)
# Inspect the response
if qresponse is not None:
# Results are automatically grouped by Sequence for easier data management
for item in qresponse:
print(f"Sequence: {item.sequence.name}")
print(f"Topics: {[topic.name for topic in item.topics]}")
mosaicolabs.models.core.unmodeled.Unmodeled ¶
Bases: Serializable
Base class for ontology data that has no hand-authored Python class.
Where a normal Serializable
subclass (e.g. IMU) declares one typed field per schema field, Unmodeled
instead stores the entire payload in a single generic
raw_data dict, and
carries its actual Arrow schema as data (__msco_pyarrow_struct__) rather
than deriving it from typed class fields. This lets a single class shape
represent any ontology schema, decided entirely at runtime.
Not meant to be subclassed directly
Don't subclass Unmodeled by hand. Instead, use the
make_unmodeled_ontology_class
factory (or, in most cases, the higher-level
resolve_ontology_class
helper) to generate a properly-configured subclass for a specific
ontology tag and pyarrow schema.
Validation¶
Every Unmodeled instance validates
raw_data against
the class's declared __msco_pyarrow_struct__ schema at construction time -
missing required fields, unknown fields, and nested-object type mismatches
all raise a ValueError immediately, rather than surfacing later as an
opaque error during Arrow serialization.
Querying with the .Q Proxy¶
Classes generated via make_unmodeled_ontology_class are still fully
queryable via the .Q proxy, exactly like a hand-authored ontology, since
the proxy is built from the class's pyarrow schema rather than its Python
field declarations.
raw_data
instance-attribute
¶
The full ontology payload, keyed by field name exactly as declared in the
class's __msco_pyarrow_struct__ schema. Nested struct fields are
represented as nested dicts (e.g. {"gyro": {"x": 1.0, "y": 2.0, "z": 3.0}}).
model_post_init ¶
Validates raw_data against the class's declared pyarrow schema.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
context
|
Any
|
The Pydantic validation context passed by the base class. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
is_registered
classmethod
¶
Checks if a class is registered.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if registered. |
ontology_tag
classmethod
¶
Retrieves the unique identifier (tag) for the current ontology class, automatically generated during class definition.
This method provides the string key used by the Mosaico platform to identify and route specific data types within the ontology registry. It abstracts away the internal naming conventions, ensuring that you always use the correct identifier for queries and serialization.
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
The registered string tag for this class (e.g., |
Raises:
| Type | Description |
|---|---|
Exception
|
If the class was not properly initialized via |
Practical Application: Topic Filtering
This method is particularly useful when constructing QueryTopic
requests. By using the convenience method QueryTopic.with_ontology_tag(),
you can filter topics by data type without hardcoding strings that might change.
Example:
from mosaicolabs import MosaicoClient, Topic, IMU, QueryTopic
with MosaicoClient.connect("localhost", 6726) as client:
# Filter for a specific data value (using constructor)
qresponse = client.query(
QueryTopic(
Topic.with_ontology_tag(IMU.ontology_tag()),
)
)
# Inspect the response
if qresponse is not None:
# Results are automatically grouped by Sequence for easier data management
for item in qresponse:
print(f"Sequence: {item.sequence.name}")
print(f"Topics: {[topic.name for topic in item.topics]}")
mosaicolabs.models.core.unmodeled.make_unmodeled_ontology_class ¶
make_unmodeled_ontology_class(
class_name,
ontology_tag,
serialization_format,
pyarrow_schema,
registry_key=None,
)
Dynamically creates an Unmodeled
subclass for a specific pyarrow schema.
This is the factory that turns an arbitrary Arrow schema into a class the
SDK can serialize, ingest, query and retrieve just like any hand-authored
ontology - the schema is attached to the generated class as data
(__msco_pyarrow_struct__) instead of being derived from Python field
declarations.
registry_key is for advanced use only
Most callers should omit registry_key entirely. It exists so that
resolve_ontology_class
can create a schema variant of an existing ontology_tag: a second
class reporting the same ontology_tag to the platform (so it remains
discoverable under one consistent tag) while still occupying a
distinct, collision-free key in the SDK's local class registry. See
Serializable.__registry_key__
for the full rationale.
Schema generation is intentionally skipped
The returned class is created with skip_schema_generation=True, so
pyarrow_schema is used verbatim as __msco_pyarrow_struct__ rather
than being (re)derived from Unmodeled.raw_data's Dict[str, Any]
annotation, which wouldn't produce a useful schema on its own.
skip_query_proxy_ingestion is left at its default (False), so the
.Q query proxy is still generated from pyarrow_schema.
Example
import pyarrow as pa
from mosaicolabs.enum import SerializationFormat
from mosaicolabs.models.core.unmodeled import make_unmodeled_ontology_class
UnmodeledGyro = make_unmodeled_ontology_class(
class_name="UnmodeledGyro",
ontology_tag="gyro_raw",
serialization_format=SerializationFormat.Default,
pyarrow_schema=pa.struct([
pa.field("gyro", pa.struct([
pa.field("x", pa.float32()),
pa.field("y", pa.float32()),
pa.field("z", pa.float32()),
])),
]),
)
# Fully usable like any other ontology class, e.g.:
# topic_writer.push(Message(timestamp_ns=..., data=UnmodeledGyro(
# raw_data={"gyro": {"x": 0.1, "y": 0.0, "z": -0.2}}
# )))
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
class_name
|
str
|
The Python class name assigned to the generated class
(e.g. shown in |
required |
ontology_tag
|
Optional[str]
|
The unique ontology identifier to register the class
under. If |
required |
serialization_format
|
SerializationFormat
|
The batching/serialization strategy for topics
using this ontology (see
|
required |
pyarrow_schema
|
StructType
|
The Arrow struct schema describing the ontology's data
payload, used verbatim as the class's |
required |
registry_key
|
Optional[str]
|
Advanced/internal use - see the note above. Defaults to
|
None
|
Returns:
| Type | Description |
|---|---|
Type[Unmodeled]
|
Type[Unmodeled]: A new |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the resolved registry key is already registered for a different class. |
mosaicolabs.models.core.helpers.resolve_ontology_class ¶
resolve_ontology_class(
*,
ontology_tag,
schema=None,
schema_fingerprint=None,
serialization_format=None,
)
Resolves an ontology tag to a concrete Serializable class, creating a
dynamic Unmodeled fallback class on demand when no hand-authored class is
registered for the tag.
Schema Variants¶
A single tag can end up associated with more than one schema shape within a
single process (e.g. two rosbags recorded with different versions of the same
ROS message type, both mapped to the same inferred ontology tag). When the
schema passed in doesn't match the one already registered for ontology_tag,
a distinct variant class is resolved (or created) instead of silently reusing
the wrong schema. The variant still reports the same ontology_tag to the
platform (so all of its data stays discoverable under one consistent tag);
only its SDK-local __registry_key__ differs, deterministically derived as
f"{ontology_tag}__{fingerprint}".
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ontology_tag
|
str
|
The ontology identifier to resolve. If a |
required |
schema
|
Optional[StructType]
|
The pyarrow schema of the incoming data. Required when
|
None
|
schema_fingerprint
|
Optional[str]
|
The fingerprint of |
None
|
serialization_format
|
Optional[SerializationFormat]
|
The serialization format to use if a dynamic
class needs to be created. Defaults to |
None
|
Returns:
| Type | Description |
|---|---|
Type[Serializable]
|
Type[Serializable]: The resolved |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |