ord_schema#

Generic helpers for ord_schema, including common message types.

Subpackages#

ord_schema.atomic_io#

Atomic-write helpers.

Centralizes the write-to-tmp-then-rename pattern used throughout ord_schema so partial files never appear at a destination path on crash, exception, or interrupt.

ord_schema.atomic_io.atomic_path(path: str | PathLike[str]) → Iterator[str]#

Yields a sibling temp path; renames it onto path on clean exit.

Usage:
with atomic_path(dest) as tmp, open(tmp, “wb”) as f:

f.write(…)

On clean exit the temp path is renamed onto path via os.replace, which is atomic on POSIX when source and destination live on the same filesystem (the temp is a sibling, so this holds). On any exception (including KeyboardInterrupt) the temp is removed best-effort and the original path is left untouched.

The temp is created via tempfile.mkstemp in the destination’s directory with a unique random suffix, so concurrent writers to the same path do not collide on a fixed sentinel name. The temp file is created (empty) immediately on entry; callers typically open it for writing and overwrite it.

ord_schema.datasets#

Whole-dataset file I/O, dispatching on filename suffix.

The entry point for reading and writing a complete Dataset, whatever it is serialized as. .parquet reads back as a parquet.DatasetView, which streams from the file instead of materializing it; every other suffix reads back as a dataset_pb2.Dataset through message_helpers single-message I/O.

ord_schema.datasets.load_dataset(filename: str | PathLike[str], *, as_dataset: Literal[False] = False) → Dataset | DatasetView#
ord_schema.datasets.load_dataset(filename: str | PathLike[str], *, as_dataset: Literal[True]) → Dataset
ord_schema.datasets.load_dataset(filename: str | PathLike[str], *, as_dataset: bool) → Dataset | DatasetView

Loads a Dataset from disk, dispatching on filename suffix.

A Parquet dataset reads back as a parquet.DatasetView, which takes its scalars and row count from the file footer and reads Reactions on demand. The view is a read-only stand-in for the message: iteration, len, emptiness, indexing, and slicing over reactions behave the same, so code that only reads needs no change. Materializing every Reaction is the exception rather than the default, which matters because published datasets can be large.

Other suffixes have no streaming form and read back as a real message.

Parameters:
  • filename – Path to a serialized Dataset (.parquet or any suffix understood by message_helpers.load_message).

  • as_dataset – If True, always return a dataset_pb2.Dataset, materializing a Parquet dataset in full. Use this when the caller needs the protobuf surface – serialization, JSON conversion, mutation – and cannot know which format it was handed; a caller that knows it holds a view can call to_proto instead.

Returns:

A parquet.DatasetView for .parquet unless as_dataset is set, and a dataset_pb2.Dataset otherwise.

ord_schema.datasets.save_dataset(dataset: Dataset, filename: str | PathLike[str]) → None#

Writes a Dataset to disk, dispatching on filename suffix.

.parquet routes to parquet.save_dataset; other suffixes go through message_helpers.save_message.

Parameters:
  • dataset – Dataset message to write.

  • filename – Output path; its suffix selects the serialization.

ord_schema.huggingface#

Fetches datasets and derived artifacts from Hugging Face.

Source datasets come from the ord-data mirror. The derived artifacts of ord_schema.artifacts live in a repository of their own, ord-artifacts, so that rebuilding them leaves the mirror’s history alone; they are published by ord_schema.scripts.upload_artifacts.

Importing this module requires the optional huggingface extra (pip install 'ord- schema[huggingface]'); huggingface_hub is not a default dependency because it pulls in a heavy chain (typer, fsspec, the hf-xet native binary).

class ord_schema.huggingface.FetchedArtifacts(revision: str, projections: Path, structures: Path, arrivals: Path, pivots: Path, occurrences: Path)#

Bases: object

Where fetch_artifacts put each artifact, in the shape Corpus reads.

Variables:
  • revision (str) – The commit the artifacts were fetched from.

  • projections (pathlib.Path) – Directory holding the projections.

  • structures (pathlib.Path) – Directory holding the structures artifacts.

  • arrivals (pathlib.Path) – Directory holding the arrivals artifacts.

  • pivots (pathlib.Path) – Directory holding one subdirectory per pivoted level.

  • occurrences (pathlib.Path) – Directory holding one subdirectory per indexed path.

arrivals: Path#
property arrivals_pattern: str#

A glob matching every arrivals artifact, for Corpus.

occurrences: Path#
pivots: Path#
property projection_pattern: str#

A glob matching every projection, for Corpus.

projections: Path#
revision: str#
structures: Path#
property structures_pattern: str#

A glob matching every structures artifact, for Corpus.

ord_schema.huggingface.fetch_artifacts(local_dir: str | PathLike[str], *, revision: str = 'main', pivot_levels: Collection[str] | None = None) → FetchedArtifacts#

Downloads the derived artifacts ord-artifacts holds, if this library reads them.

The repository holds one build at a time, laid out as a local tree. A Corpus refuses artifacts whose schema version or RDKit differs from its own, so the build’s stamps are read in place first and a mismatch is refused before anything is downloaded: a tree holding a build this library reads is not overwritten with one it cannot. An earlier build is fetched by passing the commit that holds it. Files are written under local_dir rather than linked into the Hugging Face cache, because a Corpus confined to its own directories refuses a link leading out of them.

Repeat calls download only what changed, and delete the Parquet files under what they fetched that the build no longer holds – a dataset removed upstream would otherwise stay in every local tree that once fetched it. Pivot levels already held under local_dir are refreshed with the requested ones, since a Corpus refuses a level left stale beside refreshed projections.

Parameters:
  • local_dir – Directory to download beneath; the repository’s layout is kept.

  • revision – Branch, tag, or commit SHA to download (default: "main"). A branch is resolved to its commit first, so the build is checked and downloaded from the same one.

  • pivot_levels – The pivoted levels to fetch. None fetches every level, which is what Corpus(require_pivots=True) needs; a level left out is built in process when a query first ranges over it.

Returns:

The commit fetched, and where each artifact landed.

Raises:
  • LookupError – If the revision holds no artifacts of some kind, holds a build of another artifact schema version or RDKit, or lacks a requested pivot level. The message names what it does hold.

  • ValueError – If a file the build no longer holds sits in a directory reached through a link beneath local_dir, which is not this function’s to delete.

ord_schema.huggingface.fetch_dataset(dataset_id: str, *, revision: str = 'main', cache_dir: str | None = None, local_dir: str | None = None) → str#

Downloads a dataset file from the ord-data Hugging Face mirror.

Prefers the Parquet serialization and falls back to the legacy .pb.gz binary format for datasets that have not been converted yet. The file is not parsed; dispatch on the returned suffix to choose a reader, e.g. parquet.load_dataset/parquet.DatasetView for .parquet and load_message for .pb.gz.

Downloads are cached and content-verified by huggingface_hub: repeat calls reuse the cached file (re-validating against the remote for moving revisions like "main") instead of re-downloading. By default the cache lives under ~/.cache/huggingface/hub; set $HF_HOME (or pass cache_dir) to relocate it, or set $HF_HUB_OFFLINE=1 to reuse an already-warm cache without network access.

Parameters:
  • dataset_id – Dataset ID.

  • revision – Branch, tag, or commit SHA to download (default: "main").

  • cache_dir – Override the Hugging Face cache directory. Files are stored content-addressed and returned via a symlink into the cache.

  • local_dir – If set, materialize the file as a real (non-symlink) copy under this directory instead of returning a cache symlink; useful for staging files into a project tree.

Returns:

Local filesystem path to the downloaded dataset file.

Raises:
  • RuntimeError – If no file exists for the dataset.

  • ValueError – If the dataset ID is invalid.

ord_schema.huggingface.parquet_files(directory: Path) → list[Path]#

Returns the Parquet files under directory the way a Corpus finds them.

Through glob, as the readers do, so a linked shard directory is descended and a dotted file is passed over.

Parameters:

directory – The directory to search beneath.

Returns:

The paths, sorted. Empty if the directory does not exist.

ord_schema.huggingface.parquet_pattern(directory: Path) → str#

Returns a recursive glob for the Parquet files under directory, escaped.

ord_schema.huggingface.published_build(commit: str, published: Iterable[str]) → Stamps | None#

Returns the stamps of the build ord-artifacts holds at commit, read in place.

Every file of a published build carries the same artifact schema version and RDKit version, which upload_artifacts checks, so one footer speaks for the build. It is an occurrences file’s, whose few columns make for a small footer, and nothing else of the file is downloaded.

Parameters:
  • commit – The commit SHA published was listed at.

  • published – Every repository path the commit holds.

Returns:

The stamps, or None if the commit holds no occurrences artifacts.

ord_schema.huggingface.resolve_revision(revision: str) → str#

Returns the commit revision names in ord-artifacts.

Parameters:

revision – Branch, tag, or commit SHA.

Returns:

The commit SHA.

Raises:

LookupError – If the Hub reports no commit for it.

ord_schema.logging#

Logging utilities.

ord_schema.logging.get_logger(name: str, level: int = 20) → Logger#

Creates a Logger.

ord_schema.logging.silence_rdkit_logs(pattern: str = 'rdApp.*') → None#

Disables noisy RDKit logs.

ord_schema.message_helpers#

Helper functions for constructing Protocol Buffer messages.

class ord_schema.message_helpers.MessageFormat(value)#

Bases: Enum

Input/output types for protocol buffer messages.

BINARY/BINPB and PBTXT/TXTPB pairs use the same wire format; the second of each pair is the newer canonical suffix recommended by protobuf.dev.

BINARY = '.pb'#
BINPB = '.binpb'#
JSON = '.json'#
PBTXT = '.pbtxt'#
TXTPB = '.txtpb'#
ord_schema.message_helpers.build_compound(smiles: str | None = None, name: str | None = None, amount: str | None = None, role: str | None = None, is_limiting: bool | None = None, prep: str | None = None, prep_details: str | None = None, vendor: str | None = None) → Compound#

Builds a Compound message with the most common fields.

Parameters:
  • smiles – Text compound SMILES, recorded as CXSMILES if it has an extension block.

  • name – Text compound name.

  • amount – Text amount string, e.g. ‘1.25 g’.

  • role – Text reaction role. Must match a value in ReactionRoleType.

  • is_limiting – Boolean whether this compound is limiting for the reaction.

  • prep – Text compound preparation type. Must match a value in PreparationType.

  • prep_details – Text compound preparation details. If provided, prep is required.

  • vendor – Text compound vendor/supplier.

Returns:

Compound message.

Raises:
  • KeyError – if role or prep does not match a supported enum value.

  • TypeError – if amount units are not supported.

  • ValueError – if prep_details is provided and prep is None.

ord_schema.message_helpers.build_data(filename: str, description: str) → Data#

Reads raw data from a file and creates a Data message.

Parameters:
  • filename – Text filename.

  • description – Text description of the data.

Returns:

Data message.

ord_schema.message_helpers.canonical_smiles(mol: Mol) → str#

Returns a canonical SMILES, keeping the bonding plain SMILES cannot express.

Plain SMILES cannot express AND/OR stereo groups, so writing one would silently assert a more specific structure than the source recorded. Coordinate bonds are kept for the same reason: dropping the |C:...| block leaves a dative bond written as a plain single bond, which is a different molecule and not recoverable by canonicalizing again. Presentation fields – atom labels, coordinates – are omitted.

Parameters:

mol – RDKit Mol.

Returns:

Canonical SMILES, with a |a:...|/|o...|/|C:...| block only for molecules that have enhanced stereochemistry or coordinate bonds. Others match Chem.MolToSmiles exactly; molecules with dative bonds do not, since MolToSmiles writes those inline as -> instead.

ord_schema.message_helpers.canonical_smiles_for_identifier(identifier_type: int, value: str) → str | None#

Canonicalizes a structural identifier, or returns None if it will not parse.

Parameters:
  • identifier_type – CompoundIdentifier type enum value.

  • value – The identifier value.

Returns:

Canonical SMILES, or None if value is not a valid identifier of that type. Returns None for types no Mol can be built from.

ord_schema.message_helpers.check_compound_identifiers(compound: Compound | ProductCompound) → None#

Verifies that structural compound identifiers are consistent.

Compared through canonical_smiles(), so identifiers that disagree about enhanced stereochemistry are inconsistent: they assert different things about the molecule even though they share a skeleton.

Parameters:

compound – reaction_pb2.Compound message.

Raises:

ValueError – If structural identifiers are not consistent or are invalid.

ord_schema.message_helpers.create_message(message_name: str) → Message#

Converts a message name into an instantiation of that class.

The message belongs to the reaction_pb2 module.

Parameters:

message_name – Text name of a message field. For example, “Reaction” or “TemperatureConditions.Measurement”.

Returns:

Initialized message of the requested type.

Raises:

ValueError if the name cannot be resolved. –

ord_schema.message_helpers.derived_reaction_smiles(reaction: Reaction) → str | None#

Returns the reaction SMILES that derived artifacts store, or None.

A recorded REACTION_CXSMILES or REACTION_SMILES wins over generating one, because it carries atom mapping, which generation cannot reconstruct. It is normalized, not copied: agents removed and the result canonicalized, so agent placement and atom ordering stop deciding whether two reactions look alike. A recorded value RDKit cannot read, or that survives as a half reaction, falls through to generation.

Either way the result carries no agents. An empty agent block is idiomatic, and excluding agents keeps a reagent, solvent, or catalyst recorded only by name – routine for ligands – from deciding whether the reaction gets a SMILES at all. Generation is otherwise strict: every reactant and product must be readable, since a SMILES silently missing one describes a different reaction and nothing marks it.

Parameters:

reaction – Reaction message.

Returns:

Canonical reactants>>products SMILES, or None if nothing recorded can be read and nothing complete can be generated.

ord_schema.message_helpers.find_submessages(message: Message, submessage_type: type[MessageType]) → list[MessageType]#

Recursively finds all submessages of a specified type.

Parameters:
  • message – Protocol buffer.

  • submessage_type – Protocol buffer type.

Returns:

List of messages.

Raises:

TypeError – if submessage_type is not a protocol buffer type.

ord_schema.message_helpers.generate_reaction_smiles(message: Reaction, *, allow_incomplete: bool = True, allow_unspecified_roles: bool = True, include_agents: bool = True) → str#

Builds a reaction SMILES from a Reaction’s components, ignoring its identifiers.

Parameters:
  • message – reaction_pb2.Reaction message.

  • allow_incomplete – Whether to omit components with no readable structure rather than raising. Only components the result would carry are considered, so an unreadable agent is irrelevant when include_agents is False. Tolerates missing components but not a missing half: RDKit calls a reaction with no reactants or no products an error however it got that way.

  • allow_unspecified_roles – If True, components with the UNSPECIFIED reaction role are treated as reactants and products.

  • include_agents – Whether to emit the middle >agents> block. Reagents, solvents, and catalysts are dropped when False, which leaves the SMILES describing the transformation alone.

Returns:

Canonical reaction CXSMILES, so enhanced stereochemistry recorded on a component survives into the reaction.

Raises:

ValueError – If a component the result would carry has no readable structure and allow_incomplete is False, if there is no reactant or no product, or if RDKit reports errors in the assembled reaction.

ord_schema.message_helpers.get_compound_identifier(compound: Compound | ProductCompound, identifier_type: <google.protobuf.internal.enum_type_wrapper.EnumTypeWrapper object at 0x7265551adb50>) → str | None#

Returns the value of a compound identifier if it exists.

If multiple identifiers of that type exist, only the first is returned.

Parameters:
  • compound – Compound message.

  • identifier_type – The CompoundIdentifier type to retrieve the value of.

Returns:

Identifier value or None if the identifier is not defined.

ord_schema.message_helpers.get_compound_molblock(compound: Compound | ProductCompound) → str | None#

Returns the value of the compound’s MOLBLOCK identifier if it exists.

Parameters:

compound – Compound message.

Returns:

MOLBLOCK string or None if the compound has no MOLBLOCK identifier.

ord_schema.message_helpers.get_compound_name(compound: Compound) → str | None#

Returns the value of the compound’s NAME identifier if it exists.

Parameters:

compound – Compound message.

Returns:

NAME string or None if the compound has no NAME identifier.

ord_schema.message_helpers.get_compound_smiles(compound: Compound | ProductCompound) → str | None#

Returns the value of the compound’s SMILES identifier if it exists.

Parameters:

compound – Compound message.

Returns:

SMILES string or None if the compound has no SMILES identifier.

ord_schema.message_helpers.get_product_yield(product: ProductCompound, as_measurement: bool = False) → ProductMeasurement | float | None#

Returns the value of a product’s yield if it is defined.

If multiple measurements of type YIELD exist, only the first is returned.

A yield recorded as float_value, amount, or text has no percentage to report: reading one as a percentage would assume a scale the source did not state. Pass as_measurement to reach those. views applies the same rule across every outcome and counts what it drops.

Parameters:
  • product – ProductCompound message.

  • as_measurement – Whether to return the full ProductMeasurement that corresponds to the yield measurement. Defaults to False.

Returns:

The ProductMeasurement when as_measurement is set. Otherwise the yield as a percentage, or None when the product records no yield or records one that is not a percentage.

ord_schema.message_helpers.get_reaction_smiles(message: Reaction, generate_if_missing: bool = False, allow_incomplete: bool = True, allow_unspecified_roles: bool = True) → str | None#

Fetches or generates a reaction SMILES.

A stored REACTION_CXSMILES identifier is returned whole, extension block and all. The block is chemistry the source recorded – enhanced stereochemistry, fragment grouping – and RDKit reads it, so there is nothing to gain by dropping it. split_cxsmiles_extension separates the two parts for a caller that needs the bare SMILES, e.g. to look for atom maps.

Parameters:
  • message – reaction_pb2.Reaction message.

  • generate_if_missing – Whether to generate a reaction SMILES from the inputs and outputs if one is not defined explicitly.

  • allow_incomplete – Boolean whether to allow “incomplete” reaction SMILES that do not include all components (e.g. if a component does not have a structural identifier).

  • allow_unspecified_roles – If True, reactants and products with the UNSPECIFIED reaction role will be included when generating a reaction SMILES.

Returns:

Text reaction SMILES, or None.

Raises:

ValueError – If the reaction contains errors.

ord_schema.message_helpers.has_transition_metal(mol: Mol) → bool#

Determines if a molecule contains a transition metal.

Parameters:

mol – The molecule in question. Should be of type rdkit.Chem.rdchem.Mol

Returns:

Boolean for whether the molecule has a transition metal.

ord_schema.message_helpers.id_filename(filename: str) → str#

Converts a filename into a relative path for the repository.

Parameters:

filename – Text basename including an ID.

Returns:

Text filename relative to the root of the repository.

ord_schema.message_helpers.identifier_parses_unsanitized(identifier_type: int, value: str) → bool#

Tests whether an identifier parses at all once sanitization is skipped.

Separate from canonical_smiles_for_identifier() so the extra parse happens only for the identifiers that failed, which are rare.

Parameters:
  • identifier_type – CompoundIdentifier type enum value.

  • value – The identifier value.

Returns:

True if value parses with sanitize=False.

ord_schema.message_helpers.is_transition_metal(atom: Atom) → bool#

Determines if an atom is a transition metal.

Parameters:

atom – The atom in question. Should be of type rdkit.Chem.rdchem.Atom

Returns:

Boolean for whether the atom is a transition metal.

ord_schema.message_helpers.load_message(filename: str | PathLike[str], message_type: type[MessageType]) → MessageType#

Loads a protocol buffer message from a file.

Parameters:
  • filename – Filename (str or path-like) containing a serialized message.

  • message_type – Message subclass.

Returns:

Message object.

Raises:

ValueError – if the message cannot be parsed, if filename is a Parquet dataset (use datasets.load_dataset), or if the format is unsupported.

ord_schema.message_helpers.message_to_row(message: Message, trace: tuple[str, ...] | None = None) → dict[str, str | bytes | float | int | bool]#

Converts a proto into a flat dictionary mapping fields to values.

The keys indicate any nesting; for instance a proto that looks like this:

value: {
  subvalue: 5
}

will show up as {'value.subvalue': 5} in the dict.

Parameters:
  • message – Proto to convert.

  • trace – Tuple of strings; the trace of nested field names.

Returns:

Dict mapping string field names to scalar value types.

ord_schema.message_helpers.messages_to_dataframe(messages: Iterable[Message], drop_constant_columns: bool = False) → DataFrame#

Converts a list of protos to a pandas DataFrame.

Parameters:
  • messages – List of protos.

  • drop_constant_columns – Whether to drop columns that have the same value for all rows.

Returns:

DataFrame.

ord_schema.message_helpers.mol_from_compound(compound: Compound | ProductCompound, return_identifier: bool = False) → Mol | tuple[Mol, CompoundIdentifier]#

Creates an RDKit Mol from a Compound message.

Identifiers are tried in structural_identifiers() order, so this and smiles_from_compound() agree about which one describes the compound, and one malformed identifier does not mask a readable one recorded alongside it.

Parameters:
  • compound – reaction_pb2.Compound message.

  • return_identifier – If True, return the CompoundIdentifier used to create the Mol.

Returns:

The RDKit Mol; with return_identifier, a tuple of the Mol and the CompoundIdentifier used to create it.

Raises:

ValueError – If no structural identifier reads as a Mol. Unlike smiles_from_compound(), which reports that as None, callers here want a Mol in hand and have nothing to do with its absence.

ord_schema.message_helpers.molblock_from_compound(compound: Compound | ProductCompound) → str#

Fetches or generates a MolBlock identifier for a compound.

A recorded MOLBLOCK is returned as deposited, keeping the atom coordinates that are the reason to record one, but only once it parses: an unreadable value is not a MolBlock, and returning it verbatim would hand the caller something no MolBlock reader accepts while a usable structure sat in another identifier.

That check skips sanitization, because the question is whether the value is a MolBlock rather than whether it describes a sanitizable molecule. ORD carries structures RDKit reads but will not sanitize – organometallics and charged-N rings, which validation warns about rather than rejecting – and regenerating those from another identifier would discard the coordinates for a value that was fine.

Parameters:

compound – reaction_pb2.Compound or ProductCompound message.

Returns:

MolBlock identifier, generated from the best structural identifier (see structural_identifiers()) when none is recorded or the recorded one cannot be read.

Return type:

molblock

Raises:

ValueError – if no structural identifier reads as a Mol.

ord_schema.message_helpers.parse_doi(doi: str) → str#

Parses a DOI from e.g. a URL.

Parameters:

doi – DOI string.

Returns:

The (possibly trimmed) DOI.

Raises:

ValueError – if the DOI cannot be parsed.

ord_schema.message_helpers.reaction_from_smiles(reaction_smiles: str) → Reaction#

Builds a Reaction by splitting a reaction SMILES.

Components are written through canonical_smiles(), so enhanced stereochemistry and coordinate bonds carried by the input reach the components that have them.

Parameters:

reaction_smiles – A reaction SMILES or CXSMILES.

Returns:

A Reaction with one input holding the reactants and agents, and one outcome holding the products. Amounts are unmeasured; only structure is recovered.

ord_schema.message_helpers.reaction_smiles_without_agents(reaction_smiles: str) → str | None#

Returns a reaction SMILES with its agent block removed, or None if unreadable.

Splitting the string on > would corrupt a CXSMILES extension, which indexes atoms positionally: every index in the block would point at the wrong atom once the agents are gone. Rebuilding through RDKit recomputes them.

The surviving templates are added in canonical order because RDKit numbers the block against the order they were added and then writes them sorted; adding them in any other order leaves a stereo group marking whichever atom lands at that index.

Atom mapping survives, being part of the atoms themselves, and so does enhanced stereochemistry. Fragment grouping does not, and canonical atom ordering replaces whatever the source used.

Parameters:

reaction_smiles – A reaction SMILES or CXSMILES, with or without agents.

Returns:

Canonical reactants>>products CXSMILES. None if RDKit cannot read the input, reports errors in it, or it has no reactant or no product once agents are gone: a half reaction is not a restatement of the reaction the source recorded.

ord_schema.message_helpers.safe_update(target: dict, update: Mapping) → None#

Checks that update will not clobber any keys in target.

ord_schema.message_helpers.save_message(message: Message, filename: str | PathLike[str]) → None#

Writes a protocol buffer message to disk.

Parameters:
  • message – Protocol buffer message.

  • filename – Output filename (str or path-like).

Raises:

ValueError – if filename does not have the expected suffix.

ord_schema.message_helpers.set_compound_identifier(compound: Compound, identifier_type: <google.protobuf.internal.enum_type_wrapper.EnumTypeWrapper object at 0x7265551adb50>, value: str) → CompoundIdentifier#

Sets the value of a compound identifier if it exists or creates one.

If multiple identifiers of that type exist, only the first is overwritten.

Parameters:
  • compound – Compound message.

  • identifier_type – The CompoundIdentifier type to retrieve the value of.

  • value – The value to set.

Returns:

The compound identifier that was modified or created.

ord_schema.message_helpers.set_compound_molblock(compound: Compound, value: str) → CompoundIdentifier#

Sets the value of the compound’s MOLBLOCK identifier if it exists or creates one.

Parameters:
  • compound – Compound message.

  • value – The value to set.

Returns:

The compound identifier that was modified or created.

ord_schema.message_helpers.set_compound_name(compound: Compound, value: str) → CompoundIdentifier#

Sets the value of the compound’s NAME identifier if it exists or creates one.

Parameters:
  • compound – Compound message.

  • value – The value to set.

Returns:

The compound identifier that was modified or created.

ord_schema.message_helpers.set_compound_smiles(compound: Compound, value: str) → CompoundIdentifier#

Sets the value of the compound’s SMILES identifier if it exists or creates one.

Parameters:
  • compound – Compound message.

  • value – The value to set.

Returns:

The compound identifier that was modified or created.

ord_schema.message_helpers.set_dative_bonds(mol: Mol, from_atoms: tuple[str, ...] = ('N', 'P')) → Mol#

Converts metal-ligand bonds to dative.

Replaces some single bonds between metals and atoms with atomic numbers in fromAtoms with dative bonds. For all atoms except carbon, the replacement is only done if the atom has “too many” bonds. To handle metal-carbene complexes, metal-carbon bonds are converted to dative if the sum of the explicit and implicit valence of the carbon atom does not equal its default valence, 4.

Parameters:
  • mol – The molecule to be converted.

  • from_atoms – tuple of atomic symbols corresponding to atom types that should have atom-metal bonds converted to dative. Default is N and P

Returns:

The modified molecule.

ord_schema.message_helpers.set_solute_moles(solute: Compound, solvents: Sequence[Compound], concentration: str, overwrite: bool = False) → list[Compound]#

Helps define components for stock solution inputs.

Handles a single solute and one or more solvent compounds.

Parameters:
  • solute – Compound with identifiers, roles, etc.; this argument is modified in place to define an amount in moles.

  • solvents – list of Compounds each with defined volume.

  • concentration – string defining solute concentration.

  • overwrite – whether to overwrite an existing solute amount if defined. Defaults to False

Raises:
  • ValueError – if any solvent does not have a defined volume.

  • ValueError – if the solute has an existing amount field and overwrite is set to False.

Returns:

List of Compounds to assign to a repeated components field.

ord_schema.message_helpers.smiles_from_compound(compound: Compound | ProductCompound) → str | None#

Returns canonical SMILES for a compound, preferring its CXSMILES form.

Identifiers are tried in structural_identifiers() order, so this and mol_from_compound() agree about which one describes the compound.

A compound with nothing readable is an ordinary state in ORD – ligands and reagents are routinely recorded by name alone – so it reads as None rather than raising.

Parameters:

compound – Compound or ProductCompound message.

Returns:

Canonical SMILES, carrying an enhanced-stereochemistry block where the structure has one (see canonical_smiles()), or None if the compound records no structure any loader can read.

ord_schema.message_helpers.smiles_identifier_type(smiles: str) → str#

Returns the CompoundIdentifier type a SMILES value belongs under.

Validation warns about an extension block recorded under SMILES.

Parameters:

smiles – A SMILES or CXSMILES string.

Returns:

"CXSMILES" if the value has an extension block, otherwise "SMILES".

ord_schema.message_helpers.split_cxsmiles_extension(value: str) → tuple[str, str | None]#

Splits a CXSMILES extension block off a value, if it has one.

The block follows whitespace and is introduced by |. Whitespace alone does not identify one – records exist whose SMILES carry trailing non-breaking spaces – so anything else is returned intact rather than truncated at the space.

Parameters:

value – A SMILES or CXSMILES string.

Returns:

(smiles, extension), where extension is None when there is no block.

ord_schema.message_helpers.structural_identifiers(compound: Compound | ProductCompound) → Iterator[CompoundIdentifier]#

Yields a compound’s structural identifiers, best first.

CXSMILES comes before SMILES because it is a superset and RDKit reads either, so preferring it keeps enhanced stereochemistry: a plain SMILES would assert one configuration where the source recorded a group. Everything else follows in message order.

Yielding rather than picking one lets a caller keep trying, so a malformed value does not hide a good identifier behind it. Nothing here parses the values, so a yielded identifier is a candidate rather than a promise.

Parameters:

compound – Compound or ProductCompound message.

Yields:

Each identifier with a non-empty value whose type a Mol can be built from.

ord_schema.message_helpers.validate_reaction_smiles(reaction_smiles: str) → None#

Validates reaction SMILES.

Prefer _validate_reaction() where a parsed reaction is already at hand; this is for callers holding only the string, e.g. checking a recorded identifier.

Parameters:

reaction_smiles – Text reaction SMILES.

Raises:

ValueError – If the reaction contains errors.

ord_schema.parquet#

Parquet-based serialization for Dataset messages.

Each Parquet file is a serialized Dataset:

  • Rows are Reaction messages with columns reaction_id (string) and reaction (serialized wire-format bytes).

  • The Dataset scalar fields (dataset_id, name, description) are stored in the Parquet footer key-value metadata under ord.* keys.

  • Dataset.reaction_ids is not persisted — the reaction_id column is the source of truth.

This layout supports random access (by row group) and streaming iteration without loading the full dataset into memory.

class ord_schema.parquet.DatasetView(path: str | PathLike[str])#

Bases: object

A read-only, streaming view of a Parquet-serialized Dataset.

Quacks like a dataset_pb2.Dataset for read-only access: name, description, and dataset_id come from the Parquet footer, and reactions supports iteration, length, emptiness, and indexing like a list while reading on demand: a fresh read per iteration, length from the footer, one row group per index. reactions is a read-only property so accidental rebinding raises; the scalars are plain writable attributes, which updates.assign_dataset_id relies on. The protobuf message methods (SerializeToString, CopyFrom) are absent and no write reaches the file; to_proto materializes a real message.

Two things differ from a real message and are worth knowing before substituting one for the other. Indexing returns a freshly deserialized Reaction rather than a live sub-message, so reactions[0] is reactions[0] is False and mutating the result changes nothing; and reactions does not compare equal to a list, so compare against list(view.reactions) or view.reactions[:].

The backing file must not be replaced while a view is open: the row count is read once at construction, and DatasetWriter publishes by renaming onto its destination. Reads that can detect the mismatch raise and name the path; iteration reads whatever the path points at.

reaction_ids is always empty by design: in the schema it names reactions stored outside the Dataset and is mutually exclusive with reactions, so serving it from the reaction_id column would make every Parquet dataset fail validation. Use iter_reaction_ids for the IDs of the reactions this dataset does contain, or get_reaction to look one up.

get_reaction(reaction_id: str) → Reaction#

Reads a single Reaction by ID.

The first call reads the reaction_id column to build an ID-to-position index; later calls are a dict lookup plus one row-group read. Reading the whole column costs far less than deserializing the Reactions, so this beats a scan as soon as more than one lookup is made. Where a reaction ID appears more than once, the first occurrence wins.

Parameters:

reaction_id – ID of the Reaction to read.

Returns:

A freshly deserialized Reaction.

Raises:

KeyError – If reaction_id is not present in the file.

iter_reaction_ids() → Iterator[str]#

Streams reaction_id values without deserializing any Reaction.

Reads only the reaction_id column, a row group at a time, so this stays cheap on very large files. Order matches iter_reactions.

Yields:

Each reaction ID, in file order.

iter_reactions(reaction_ids: Iterable[str] | None = None, *, row_group: int | None = None) → Iterator[tuple[str, Reaction]]#

Streams (reaction_id, Reaction) pairs from the backing file.

Iterating reactions is the same read with the IDs dropped; use this when the ID is wanted alongside the message, or to restrict the read.

Parameters:
  • reaction_ids – Optional iterable of reaction IDs to restrict iteration to. IDs not present in the file are silently skipped. Filtering is applied row-wise; row groups are still read in full. Pass None (the default) to iterate all reactions; an explicitly empty iterable raises ValueError since it is almost always a bug.

  • row_group – Optional row-group index. When set, only that row group is read; this is the unit of parallelism for callers that fan out across row groups (see num_row_groups).

Yields:

Each (reaction_id, Reaction) pair, in file order.

Raises:
  • ValueError – If reaction_ids is provided but empty.

  • IndexError – If row_group is out of range.

md5() → str#

Returns a content hash, streamed a row group at a time.

The hash is fed in this order:

  • name=<value>\n, description=<value>\n, dataset_id=<value>\n (omitted when unset). These come from this view, so a scalar assigned through updates.assign_dataset_id changes the hash before it reaches the file.

  • For each Reaction in iteration order: the 8-byte big-endian length of the serialized Reaction blob, then the blob bytes. The reaction_id is not hashed separately since it is already inside the blob (field 1 of Reaction); the length prefix disambiguates blob boundaries.

Different from md5(Dataset.SerializeToString(deterministic=True)) – re-ingesting an existing .pb.gz dataset as .parquet will look like a content change once. That is correct: the on-disk bytes really did change, and the hash is only a cheap “did the content change since I last saw it?” indicator. The scheme is decoupled from the Parquet file layout (row group sizes, compression) so the same logical content rewritten with different writer settings still hashes the same.

Returns:

The MD5 hex digest.

property num_row_groups: int#

Number of row groups, the unit of parallelism for iter_reactions.

property path: str#

Path to the backing Parquet file.

property reactions: _ReactionStream#

Re-iterable stream of Reactions read from the backing file.

to_proto() → Dataset#

Materializes a real Dataset message, deserializing every Reaction.

Escape hatch for the operations a view cannot serve: serialization, JSON conversion, mutation, and helpers that require a protobuf message. Peak memory scales with the whole dataset, so prefer iterating reactions when the message itself is not required.

Returns:

A Dataset whose Reactions are streamed from the backing file and whose scalars are taken from this view, so an assignment such as updates.assign_dataset_id reaches the message.

class ord_schema.parquet.DatasetWriter(path: str | PathLike[str], *, name: str, description: str, dataset_id: str | None = None, compression: str = 'zstd', row_group_size: int = 1000)#

Bases: object

Streaming writer for a Parquet dataset.

Reactions are buffered and flushed to a new row group every row_group_size rows, so peak memory stays bounded regardless of the total number of reactions written.

name and description are required; dataset_id is optional because it is assigned during the submission process. Zero reactions is permitted: a streaming writer only learns the count at close, where failing would be too late to help. save_dataset has the whole message and rejects an empty one up front.

Example

with DatasetWriter(path, name=”big dataset”, description=”…”) as writer:
for reaction in source:

writer.write(reaction)

close() → None#

Flushes the final row group, closes the writer, and publishes the file.

On success the temp file is renamed onto the destination via os.replace. On any exception during flush or close (including KeyboardInterrupt) the temp is removed best-effort and the destination is left untouched; the original error propagates and any secondary close error is suppressed. Idempotent thereafter.

write(reaction: Reaction) → None#

Buffers a Reaction; flushes a row group when the buffer is full.

Parameters:

reaction – Reaction to append. Serialized immediately, so later mutation of the message does not affect what is written.

write_all(reactions: Iterable[Reaction]) → None#

Writes an iterable of Reactions.

Parameters:

reactions – Reactions to append, in order.

ord_schema.parquet.load_dataset(path: str | PathLike[str]) → Dataset#

Reads a full Dataset from a Parquet file.

Every Reaction is deserialized into memory. Prefer DatasetView, which reads the same file without materializing it, whenever a real message is not required.

Parameters:

path – Path to the Parquet-serialized Dataset.

Returns:

The Dataset message.

ord_schema.parquet.save_dataset(dataset: Dataset, path: str | PathLike[str], *, compression: str = 'zstd', row_group_size: int = 1000) → None#

Writes a Dataset as a Parquet file.

Dataset.reaction_ids is not persisted; the reaction_id column derived from Dataset.reactions is the source of truth.

Parameters:
  • dataset – Dataset message. Must have at least one Reaction.

  • path – Output filename.

  • compression – Parquet column compression codec.

  • row_group_size – Maximum number of rows per row group. Smaller groups give finer-grained random access; larger groups compress better.

Raises:

ValueError – If dataset.reactions is empty or required scalar fields (name, description) are unset.

ord_schema.resolvers#

Name/string resolution to structured messages or identifiers.

ord_schema.resolvers.canonicalize_smiles(smiles: str) → str#

Canonicalizes a SMILES string, raising if it will not parse.

A thin wrapper over message_helpers.canonical_smiles(), so a resolved structure is written the same way as one derived from a Compound. Callers holding a Mol should use that directly; this exists for the string-in, string-out case.

Parameters:

smiles – SMILES string, which may carry a CXSMILES extension block.

Returns:

Canonical SMILES, carrying a block where the structure has enhanced stereochemistry or coordinate bonds.

Raises:

ValueError – If the SMILES cannot be parsed by RDKit.

ord_schema.resolvers.resolve_input(input_string: str) → ReactionInput#

Resolves a text-based description of an input.

Supported formats:
  1. [AMOUNT] of [NAME]

  2. [AMOUNT] of [CONCENTRATION] [SOLUTE] in [SOLVENT]

Parameters:

input_string – String describing the input.

Returns:

ReactionInput message.

Raises:

ValueError – if the string cannot be parsed properly.

ord_schema.resolvers.resolve_name(value_type: str, value: str) → tuple[str, str]#

Resolves compound identifiers to SMILES via multiple APIs.

Resolvers are tried in order until one answers. Any of them failing falls through to the next, so only an exhausted chain is a failure.

Parameters:
  • value_type – The kind of identifier being resolved, e.g. “name”.

  • value – The identifier to resolve.

Returns:

A tuple of SMILES and the name of the resolver that produced it.

Raises:

ValueError – If no resolver returns a structure.

ord_schema.resolvers.resolve_names(message: Message) → bool#

Attempts to resolve compound NAME identifiers to SMILES.

When a NAME identifier is resolved, a SMILES identifier is added to the list of identifiers for that compound. The first success ends work on that Compound; any remaining NAME identifiers on it are left unresolved.

A compound already carrying an identifier a Mol can be built from is skipped. Coordinates alone do not count, since nothing in this library reads them into a structure, so a compound recorded as XYZ plus a name is worth resolving.

Parameters:

message – Protocol buffer tree containing Compound submessages (e.g. Reaction or ReactionInput).

Returns:

Boolean whether message was modified.

ord_schema.templating#

Functions for creating Datasets by enumerating a template with a spreadsheet.

The templating code has specific expectations for how the reaction pbtxt and spreadsheet are defined, namely that placeholder values in the pbtxt begin and end with a “$” (dollar sign) and that these match a unique column header in the spreadsheet file.

ord_schema.templating.generate_dataset(name: str, description: str, template_string: str, df: DataFrame, validate: bool = True) → Dataset#

Generates a Dataset by enumerating a template reaction.

Parameters:
  • name – Dataset name.

  • description – Dataset description.

  • template_string – The contents of a Reaction pbtxt where placeholder values to be replaced are defined between dollar signs. For example, a SMILES identifier value could be “$product_smiles$”. Placeholders may only use letters, numbers, and underscores.

  • df – Pandas Dataframe where each row corresponds to one reaction and column names match placeholders in the template_string.

  • validate – If True, validate each enumerated Reaction and raise on the first one with errors.

Returns:

A Dataset message.

Raises:
  • ValueError – If there is no match for a placeholder string in df.

  • ValueError – If validate is True and there are validation errors when validating an enumerated Reaction message.

ord_schema.templating.load_spreadsheet(file_name_or_buffer: str | BinaryIO, suffix: str | None = None) → DataFrame#

Reads a {csv, xls, xlsx} spreadsheet file.

Parameters:
  • file_name_or_buffer – Filename, or an open binary buffer. A buffer requires suffix, since there is no name to read it from.

  • suffix – Filename suffix, used to determine the data encoding. Anything other than “.xls” or “.xlsx” is read as CSV.

Returns:

DataFrame containing the reaction spreadsheet data.

ord_schema.units#

Helpers for translating strings with units.

class ord_schema.units.UnitResolver(unit_synonyms: dict[type[Concentration | Current | FlowRate | Length | Mass | Moles | Pressure | Temperature | Time | Voltage | Volume | Wavelength], dict[int, list[str]]] | None = None, forbidden_units: dict[str, str] | None = None)#

Bases: object

Resolver class for translating value+unit strings into messages.

convert(message: Concentration | Current | FlowRate | Length | Mass | Moles | Pressure | Temperature | Time | Voltage | Volume | Wavelength, new_units: str | int) → Concentration | Current | FlowRate | Length | Mass | Moles | Pressure | Temperature | Time | Voltage | Volume | Wavelength#

Converts a united message into another of the same type with different units.

Parameters:
  • message – a message with units, e.g., Mass, Length.

  • new_units – the desired units of the new message, expressed either as a string or an integer (ENUM value). Use of a string is recommended due to the ambiguity of using ENUM values; for example, Mass.GRAM == Time.MINUTE.

Returns:

A new message with units, e.g., Mass, Length.

resolve(string: str, allow_range: bool = False) → Concentration | Current | FlowRate | Length | Mass | Moles | Pressure | Temperature | Time | Voltage | Volume | Wavelength#

Resolves a string into a message containing a value with units.

Parameters:
  • string – The string to parse; must contain a numeric value and a string unit. For example: “1.25 h”.

  • allow_range – If True, ranges like “1-2 h” can be provided and the average value will be reported along with the standard deviation.

Returns:

Message containing a numeric value with units listed in the schema.

Raises:

ValueError – if string does not contain a value with units, or if the value is invalid.

resolve_unit(string_unit: str) → tuple[type[Concentration | Current | FlowRate | Length | Mass | Moles | Pressure | Temperature | Time | Voltage | Volume | Wavelength], int]#

Resolves a unit string into its message type and unit ENUM value.

Parameters:

string_unit – The string unit to parse; for example: “gram”.

Returns:

Tuple containing the message type and unit ENUM value.

Raises:

KeyError – if string unit cannot be parsed.

ord_schema.units.canonical_precision(message: Concentration | Current | FlowRate | Length | Mass | Moles | Pressure | Temperature | Time | Voltage | Volume | Wavelength, target: str) → float | None#

Converts a united message’s precision to target units, or returns None.

Precision is recorded in the same units as the value, so it converts with the value and is null wherever the value is – a column named for target has nowhere to say its uncertainty is in different units, and an uncertainty published beside a null reads as a measurement nobody took but everybody bounded.

Parameters:
  • message – A united message, e.g. Temperature or Mass.

  • target – Unit to convert to, spelled as the resolver understands it, e.g. “K”.

Returns:

The converted precision, or None when the message records no precision, no value to attach it to, no units, or units that cannot be converted to target. Also None where the conversion declines to state one: an inverted interval reaching zero is unbounded, and no number describes it.

ord_schema.units.canonical_value(message: Concentration | Current | FlowRate | Length | Mass | Moles | Pressure | Temperature | Time | Voltage | Volume | Wavelength, target: str) → float | None#

Converts a united message to target units, or returns None if it cannot.

Null beats raising for a derived column: the column is named for target, so an unconverted value has nowhere to say it is in different units, and a number there would be read as target.

Parameters:
  • message – A united message, e.g. Temperature or Mass.

  • target – Unit to convert to, spelled as the resolver understands it, e.g. “K”.

Returns:

The converted value, or None when the message records no value, records no units, or records units that cannot be converted to target. A wavenumber of zero is in the last group: converting it to a wavelength divides by it.

ord_schema.units.compute_solute_quantity(volume: Volume, concentration: Concentration) → Amount#

Computes the quantity of a solute, given volume and concentration.

ord_schema.units.format_message(message: Concentration | Current | FlowRate | Length | Mass | Moles | Pressure | Temperature | Time | Voltage | Volume | Wavelength) → str | None#

Formats a united message into a string.

Parameters:

message – a message with units, e.g., Mass, Length.

Returns:

A string describing the value, e.g., “5.0 (± 0.1) mL” using the

first unit synonym listed in _UNIT_SYNONYMS.

ord_schema.updates#

Automated updates for Reaction messages.

ord_schema.updates.apply_cross_reference_substitutions(reaction: Reaction, id_substitutions: dict[str, str]) → None#

Rewrites reaction_ids referenced in reaction via the substitution map.

ord_schema.updates.apply_reaction_updates(reaction: Reaction, *, new_id: str | None) → bool#

Applies per-reaction updates in place using a pre-computed reaction ID.

Splitting ID generation out of this function lets a streaming caller allocate IDs in a cheap pre-pass (e.g. from a Parquet reaction_id column) and inject them here without re-deriving them.

Parameters:
  • reaction – Reaction message to mutate.

  • new_id – Pre-computed reaction_id to assign, or None to leave the existing ID untouched.

Returns:

True if the reaction was modified.

ord_schema.updates.assign_dataset_id(dataset: Dataset | DatasetView) → str#

Assigns a canonical dataset_id if missing or non-canonical.

Mutates dataset.dataset_id in place. Works for both Dataset and DatasetView (which exposes dataset_id as a writable attribute).

Returns:

The (possibly newly-assigned) dataset_id.

ord_schema.updates.assign_id_substitutions(old_ids: Iterable[str]) → tuple[list[str | None], dict[str, str]]#

Pre-allocates canonical reaction IDs for a sequence of old IDs.

A reaction’s ID is replaced when the existing one is missing or does not match the canonical ord-{32 hex} pattern. Cross-reference rewriting only applies to old IDs that were non-empty (i.e., user-supplied placeholders); reactions whose old ID was empty get a new ID but no substitution entry, since nothing else could have referenced them.

NOTE(kearnes): This does not check for the case where a Dataset is edited and reaction_id values are changed inappropriately. This will need to be either (1) caught in review or (2) found by a complex check of the diff.

Parameters:

old_ids – Reaction IDs in the order they appear in the dataset.

Returns:

List parallel to old_ids; entry is the new reaction_id

to assign, or None if the old ID was already canonical.

id_substitutions: Map of old_id -> new_id for entries where the

old ID was a non-empty placeholder. Used to rewrite cross-references.

Return type:

new_ids

ord_schema.updates.update_dataset(dataset: Dataset) → None#

Updates a Dataset message.

Current updates:
  • Sets dataset_id if not already canonical.

  • Sets reaction_id on each Reaction if not already canonical, and appends a record_modified provenance event for any modified Reaction.

  • Rewrites reaction_id cross-references between Reactions in the dataset.

Parameters:

dataset – dataset_pb2.Dataset message.

Raises:

KeyError – if the dataset has not been validated and there exists a cross-referenced reaction_id in any Reaction that is not defined elsewhere in the Dataset.

ord_schema.updates.update_parquet_dataset(input_path: str | PathLike[str], output_path: str | PathLike[str], *, dataset_id: str) → None#

Applies update_dataset to Parquet input_path, writing to output_path.

Two passes over input_path:

  • Pass 1 reads only the reaction_id column (no Reaction decode) to pre-allocate canonical reaction IDs and build the cross-reference map.

  • Pass 2 streams full Reactions, applies per-reaction updates and cross-reference rewrites, and writes them via DatasetWriter.

Peak memory is bounded by one row group plus the ID maps. The caller is responsible for choosing output_path based on the resolved dataset_id (call assign_dataset_id on the input header first to learn it) and for any atomic-rename / validation dance — keeping the rename outside lets the caller validate the written file before publishing it.

Parameters:
  • input_path – Path to the input Parquet dataset.

  • output_path – Path to write the updated Parquet dataset to.

  • dataset_id – Resolved dataset_id to write into the output footer.

ord_schema.validations#

Helpers validating specific Message types.

class ord_schema.validations.DatasetCrossRefState(defined_ids: set[str] = <factory>, referenced_ids: set[str] = <factory>, duplicate_count: int = 0, self_reference_count: int = 0)#

Bases: object

Aggregated cross-reference observations for a Dataset.

A worker validating a slice of reactions feeds each one into observe and returns the resulting state. The master process merges the per-slice states with merge and then report records a finding per duplicate occurrence, per self-reference, and one summary finding if any referenced reaction_ids are undefined. This keeps the streaming path behaviorally equivalent to the in-memory path.

defined_ids: set[str]#
duplicate_count: int = 0#
merge(other: DatasetCrossRefState) → None#

Merges another state into this one, counting cross-slice duplicate IDs.

observe(reaction: Reaction) → None#

Records one reaction’s defined ID, referenced IDs, and self-references.

referenced_ids: set[str]#
report(context: ValidationContext) → None#

Reports duplicate IDs, self-references, and undefined references.

Parameters:

context – Where the findings are recorded.

self_reference_count: int = 0#
class ord_schema.validations.Severity(value)#

Bases: IntEnum

How serious a validation finding is.

Ordered, so a caller can threshold on >= Severity.ERROR rather than enumerating members.

ERROR = 2#
INFO = 0#
WARNING = 1#
class ord_schema.validations.ValidationContext(options: ValidationOptions = <factory>, findings: list[tuple[str, ~ord_schema.validations.Severity]]=<factory>)#

Bases: object

Where a validator reports to, and the toggles it validates under.

Passed to every validator rather than held in module state, so what a call can see and affect is visible in its signature. validate_message reads back one message’s findings by slicing off whatever its validator appended.

error(message: str) → None#

Records a finding that makes the message invalid.

findings: list[tuple[str, Severity]]#
info(message: str) → None#

Records a finding that needs no action, such as a suggested addition.

options: ValidationOptions#
warn(message: str) → None#

Records a finding worth surfacing that does not fail validation.

exception ord_schema.validations.ValidationError#

Bases: Exception

Raised when validation is asked to fail on invalid data.

A finding’s severity is Severity; this is only the exception that carries an error out to the caller, via raise_on_error or validate_datasets.

class ord_schema.validations.ValidationOptions(validate_ids: bool = False, require_provenance: bool = True, allow_reaction_smiles_only: bool = True)#

Bases: object

Options for message validation.

allow_reaction_smiles_only: bool = True#
require_provenance: bool = True#
validate_ids: bool = False#
class ord_schema.validations.ValidationOutput(errors: list[str] = <factory>, warnings: list[str] = <factory>, info: list[str] = <factory>)#

Bases: object

Validation output: errors, warnings, and info.

errors: list[str]#
extend(other: ValidationOutput) → None#

Appends the findings from another output to this one.

info: list[str]#
warnings: list[str]#
ord_schema.validations.get_referenced_reaction_ids(message: Reaction) → set[str]#

Return the set of reaction IDs that are referenced in a Reaction.

ord_schema.validations.has_atom_mapping(smiles: str) → bool#

Returns whether a SMILES string contains atom-map numbers.

ord_schema.validations.is_empty(message: Message) → bool#

Returns whether the given message is empty.

ord_schema.validations.is_url(value: str) → bool#

Returns whether a string looks like an http(s) URL with a host.

ord_schema.validations.is_valid_dataset_id(dataset_id: str) → bool#

Returns whether a dataset ID matches the ord_dataset-<32 hex digits> format.

ord_schema.validations.is_valid_orcid(orcid: str) → bool#

Returns whether an ORCID is well-formed, including its checksum.

The final character is an ISO 7064 MOD 11-2 check digit over the preceding 15 digits; see https://support.orcid.org/hc/en-us/articles/360006897674.

Parameters:

orcid – ORCID string, expected as 0000-0000-0000-0000.

Returns:

True if orcid is well-formed and the checksum is correct.

ord_schema.validations.is_valid_reaction_id(reaction_id: str) → bool#

Returns whether a reaction ID matches the ord-<32 hex digits> format.

ord_schema.validations.parse_date_time(value: str) → date | None#

Parses a DateTime value: a date, or a time with or without its UTC offset.

The one definition of the shapes a DateTime may take: validation accepts exactly the values this parses, and the artifact projection reads times through it.

Memoized: a DateTime string is parsed once by this module’s per-message walk and again by _validate_reaction_provenance() to order the timestamps, and record timestamps repeat across the reactions of a dataset. Caching is safe here because date and datetime are immutable.

Parameters:

value – The DateTime value.

Returns:

A date for YYYY-MM-DD; a datetime for YYYY-MM-DDTHH:MM:SS[.ffffff], aware when Z or ±HH:MM follows and naive otherwise; or None for anything else, including a time whose UTC instant falls outside the years datetime holds.

ord_schema.validations.reaction_has_internal_standard(message: Reaction) → bool#

Whether any reaction component uses the internal standard role.

ord_schema.validations.reaction_has_limiting_component(message: Reaction) → bool#

Whether any reaction input compound is limiting.

ord_schema.validations.reaction_needs_internal_standard(message: Reaction) → bool#

Whether any analysis uses an internal standard.

ord_schema.validations.validate_dataset_streaming(*, context: ValidationContext, name: str, description: str, dataset_id: str, reaction_ids: list[str], has_reactions: bool, state: DatasetCrossRefState) → None#

Dataset-level validation for callers that have already streamed reactions.

Equivalent to _validate_dataset for a Dataset whose reactions have been iterated in slices (e.g., per Parquet row group) by upstream workers, with each worker contributing a DatasetCrossRefState that the caller has merged. has_reactions should reflect the source’s row count (e.g., len(parquet.DatasetView(...).reactions) for parquet); inferring it from state would misclassify reactions without reaction_ids or references as empty. Pass reaction_ids=[] for the typical streaming case (parquet does not persist Dataset.reaction_ids).

Validates under context.options; there is no separate options argument, so this and _validate_dataset cannot disagree about which toggles are in force.

ord_schema.validations.validate_datasets(datasets: Mapping[str, Dataset | DatasetView], write_errors: bool = False, options: ValidationOptions | None = None) → None#

Runs validation for a set of datasets.

Parameters:
  • datasets – Dict mapping text filenames to Dataset protos.

  • write_errors – If True, errors are written to disk.

  • options – ValidationOptions.

Raises:

ValidationError – if any Dataset does not pass validation.

ord_schema.validations.validate_message(message: Message, recurse: bool = True, raise_on_error: bool = True, options: ValidationOptions | None = None, trace: tuple[str, ...] | None = None, context: ValidationContext | None = None) → ValidationOutput#

Template function for validating custom messages in the reaction_pb2.

Messages are not validated to check enum values, since these are enforced by the schema. Instead, we only check for validity of items that cannot be enforced in the schema (e.g., non-negativity of certain measurements, consistency of cross-referenced keys).

The message may be modified in place with any unambiguous changes needed to ensure validity.

Parameters:
  • message – A message to validate.

  • recurse – If True, also validate submessages, meaning fields that are themselves messages.

  • raise_on_error – If True, raises a ValidationError exception when errors are encountered. If False, the user must manually check the return value to identify validation errors.

  • options – Toggles for the checks that are not always applied; see ValidationOptions.

  • trace – Tuple containing a string “stack trace” to track the position of the current message relative to the recursion root.

  • context – Where findings are reported, and the options they are validated under. Created for the root call and threaded through the recursion; callers do not normally pass one. Supplying both this and options validates under context.options and ignores options.

Returns:

Errors, warnings, and info accumulated over the message and, when recursing, its submessages.

Raises:

ValidationError – If any fields are invalid.