Fusion data structure and IMAS concepts

VAFT stores every VEST shot — raw diagnostics, processed signals, equilibria, kinetic profiles — in the IMAS data model. There is no VEST-specific container format: if you know the IMAS Data Dictionary, you already know how to read VEST data.

The standard representation complements the native scientific artifacts (gEQDSK files, solver outputs, camera and diagnostic files); it does not replace them. Native artifacts are stored alongside the IMAS database:

Two libraries are involved, and it is worth keeping them straight.

In practice: you work with an ODS, and IMAS is the format it is persisted in. vaft.imas is the bridge between the two.

flowchart LR
    RAW[VEST raw DAQ / SQL] -->|vaft.machine_mapping| ODS
    ODS[omas.ODS in memory] -->|vaft.imas.save| H5[IMAS HDF5 images: master.h5 + equilibrium.h5 + ...]
    H5 -->|vaft.omas.load| ODS
    H5 -->|imas.DBEntry.get| IDS[IDSToplevel: native IMAS-Python]
    H5 -->|hsload / hsget| HSDS[(VEST HSDS server)]
    ODS -->|save_omas_json / save_omas_nc| FILES[.json / .nc]

Anchor notebooks for this page:

• read_and_convert_data_structure.ipynb — walking an ODS tree.
• imas_omas_data_conversion.ipynb — ODS ↔ IMAS AL5 round-trips.

IDSs available in the VEST database

A VEST shot is a set of IDSs. Which ones exist depends on whether the quantity was measured or computed.

Experimental

dataset_description · magnetics · tf · pf_active · barometry · spectrometer_uv · thomson_scattering · charge_exchange

Modelling

wall · em_coupling · pf_passive · pf_plasma (a reconstruction’s plasma current as elements; see vaft.omas.pf_plasma) · equilibrium (EFIT/CHEASE) · core_profiles · mhd_linear (DCON/RDCON)

Not every shot carries every IDS. Check before you index:

import vaft

ods = vaft.database.load(39915)
print(list(ods.keys()))
# ['coils_non_axisymmetric', 'dataset_description', 'em_coupling', 'equilibrium',
#  'magnetics', 'pf_active', 'pf_passive', 'spectrometer_uv', 'tf', 'wall']

if 'thomson_scattering' in ods:
    n_e = ods['thomson_scattering.channel.0.n_e.data']

Navigating IMAS paths in an ODS

An ODS is addressed by path strings — the IMAS DD path, with array indices written as plain integers. These three forms are equivalent:

ods['magnetics.ip.0.data']          # flat path string (idiomatic)
ods['magnetics']['ip'][0]['data']   # step by step
ods['magnetics.ip'][0]['data']      # mixed

The essential moves, all taken from read_and_convert_data_structure.ipynb:

import vaft

ods = vaft.omas.sample_ods()           # packaged shot 39915 — no server needed

list(ods.keys())                       # which IDSs are present
list(ods['equilibrium'].keys())        # what is inside one IDS
ods['equilibrium.time']                # the IDS time base (ndarray)
len(ods['equilibrium.time_slice'])     # number of reconstructed slices
list(ods['equilibrium.time_slice.0'].keys())
ods['equilibrium.time_slice.0.profiles_1d.volume'][-1]     # plasma volume at the boundary
ods['equilibrium.time_slice.0.global_quantities.ip']

ODS.paths() returns every filled leaf as a list of path components. It is the workhorse for programmatic traversal, and it is what vaft.omas.shift_time and the low-level vaft.imas.omas_imas.save_omas_imas iterate over internally:

for path in ods.paths():
    if path[0] == 'magnetics' and path[-1] == 'data':
        print('.'.join(str(p) for p in path))
# magnetics.flux_loop.0.flux.data
# magnetics.b_field_pol_probe.0.field.data
# magnetics.ip.0.data
# ...

To dump the whole tree, use the recursive helper from the notebook:

def print_hierarchy(d, prefix=""):
    try:
        keys = d.keys()
    except AttributeError:
        return
    for k in keys:
        new_prefix = f"{prefix}.{k}" if prefix else k
        print(new_prefix)
        try:
            print_hierarchy(d[k], new_prefix)
        except Exception:
            pass

print_hierarchy(ods)

Time bases

Each IDS carries its own time base. ODS.time(<ids>) resolves it for you, which matters because different diagnostics are digitised at different rates:

t_mag  = ods.time('magnetics')        # DAQ base of the magnetics IDS
t_spec = ods.time('spectrometer_uv')
ip     = ods['magnetics.ip.0.data']   # same length as t_mag

Never assume two IDSs share a grid. Interpolate, or use vaft.omas.find_matching_time_indices(ods, time_slice=...), which returns (cp_idx, equil_idx, time) after verifying that the selected core_profiles slice and the matched equilibrium slice refer to the same instant. It raises ValueError when the closest equilibrium time is farther away than atol (default 1 µs) — a deliberate refusal to silently pair kinetic profiles with the wrong equilibrium.

Shot metadata (dataset_description)

Provenance lives in the dataset_description IDS.

import vaft

ods = vaft.omas.sample_ods()
vaft.omas.find_shotnumber(ods)            # -> 39915  (reads data_entry.pulse)
vaft.omas.print_info(ods)                 # metadata header, then one line per IDS with its sub-keys
vaft.omas.print_info(ods, 'magnetics')    # channel counts inside one IDS

vaft.omas.classify_shot(ods) labels a shot 'Plasma', 'BD failure' or 'Vacuum', decided from the shared plasma timing and the gas response, in this order: a plasma-current pulse found by vaft.omas.plasma_timing → 'Plasma'; a current that could not be judged at all (ip_unusable, a condemned channel) with a window the light saw → 'Plasma' too, flagged; otherwise a barometry pressure response (is_signal_active over the finite samples of the whole record, the puff precedes the analysis window) or an optical window → 'BD failure' (the shot was attempted: a puff, or a flash without current); otherwise 'Vacuum'. vaft.omas.shot_class.shot_class(ods) returns the record behind the string: the label, decided_by (ip_pulse, pressure_response, optical_window, none), the three checks, the timing summary and flags such as barometry_absent (a product without barometry is judged on the light alone). A product without magnetics.ip cannot be classified either way and raises PlasmaTimingError; on the packaged samples every shot is 'Plasma' by its current pulse. halpha_threshold is no longer read.

The underlying primitive is scale-free: it compares the trace’s variance and its mean |Δx|, both divided by the trace’s mean absolute level, against relative thresholds, so it needs no knowledge of the signal’s units. A trace held at any non-zero level with noise far below that level is inactive; the converse caveat is that a zero-mean noise trace has no level to be small against and reads active at any amplitude, so apply a known noise floor before asking (issue #463).

import vaft

ods = vaft.omas.sample_ods()

halpha = ods['spectrometer_uv.channel.0.processed_line.0.intensity.data']
vaft.process.is_signal_active(halpha, verbose=True)   # -> True  (H-alpha fired: breakdown occurred)
# Variance ratio: 2.918e+00 (thresh=1.000e-02)
# Mean |Δx| ratio: 2.276e-01 (thresh=1.000e-02)

Compose your own classifier on top of it, guarding each IDS before you index it:

import numpy as np
import vaft

def classify(ods, var_thresh=1e-2, change_thresh=1e-2):
    def active(path):
        if path.split('.')[0] not in ods:
            return None                       # IDS absent — undecidable
        return vaft.process.is_signal_active(
            ods[path], var_ratio_thresh=var_thresh, change_ratio_thresh=change_thresh)

    if active('barometry.gauge.0.pressure.data') is False:
        return 'Vacuum'
    if not active('spectrometer_uv.channel.0.processed_line.0.intensity.data'):
        return 'BD failure'
    if 'magnetics' in ods and np.max(ods['magnetics.ip.0.data']) <= 0:
        return 'BD failure'
    return 'Plasma'

classify(vaft.omas.sample_ods())     # -> 'Plasma'

When you build an ODS yourself (for example from the raw DAQ), populate the metadata with the canonical builder rather than by hand:

from omas import ODS
import vaft

ods = ODS()
vaft.machine_mapping.vfit_dataset_description(ods, shot=39915, run=0,
                                              machine="VEST", pulse_type="pulse")

vaft.database.load_ods back-fills user, pulse and run with setdefault after a download, so a shot loaded from HSDS always carries at least those three.

Sample ODS / ODC data (works offline)

VAFT ships real VEST shots inside the package, so the snippets on this page run without HSDS credentials — with the caveat that the packaged shots do not carry every IDS. Shot 39915 holds:

['barometry', 'dataset_description', 'equilibrium', 'magnetics',
 'pf_active', 'pf_passive', 'spectrometer_uv', 'tf', 'wall']

There is no thomson_scattering or core_profiles in it, so anything keyed on those IDSs needs a shot pulled from the database.

import vaft

ods = vaft.omas.sample_ods()    # ODS  — shot 39915
gf  = vaft.omas.sample_gfile()  # GEQDSK — packaged EFIT g-file for shot 39915

An ODC (OMAS Data Collection) is a dict of ODSs — the natural container for a multi-shot study. There is no packaged ODC helper; build one from the registered reference samples:

from omas import ODC

odc = ODC()
for key, shot in enumerate(vaft.data.available_samples()):   # (39915, 41524, 41672)
    odc[key] = vaft.omas.load(vaft.data.sample(shot))

for key, one_ods in odc.items():
    print(key, vaft.omas.find_shotnumber(one_ods), len(one_ods['magnetics.time']))

vaft.omas.odc_or_ods_check(x) normalises either input to an ODC (a bare ODS is wrapped under key '0'). That is how the multi-shot helpers accept both types.

Reference shots are reached through vaft.data.sample(), which resolves the registered artifact for the adapter you ask for. Other packaged fixtures are reached through vaft.data.resources.data_path(), whose paths are category-prefixed; flat calls such as data_path("39915.json") are intentionally unsupported.

Loading a packaged shot explicitly — this is what sample_ods() does under the hood:

from vaft.database._local import load_ods

ods, _ = load_ods(vaft.data.sample(39915, representation="omas"), imas_version="3.41.0")

Pinning imas_version is deliberate: the packaged files predate the current DD and would otherwise be rejected on load.

ODS ↔ IMAS-Python (AL5)

vaft.imas is a hardened fork of OMAS’s omas_imas module. It exists because stock OMAS targets the AL4 stack, whereas the open-source IMAS distribution (IMAS-Python + imas_core) is AL5:

• AL4 addressed a data entry by user / machine / pulse / run under a fixed backend root. AL5 addresses it by URI — imas:hdf5?path=/any/directory — with a mode ('r', 'w', 'x', 'a').
• AL4’s DBEntry.create() returned a tuple; AL5’s returns None.
• imasdef moved from the imas package into imas_core.

vaft.imas handles all three, and pins the DD version used for conversion:

from vaft.imas import IMAS_DD_VERSION_CONVERSION
print(IMAS_DD_VERSION_CONVERSION)   # '3.41.0'

Override it with the IMAS_DD_VERSION_CONVERSION environment variable if you must, but 3.41.0 is the version OMAS is validated against and the version the VEST HSDS images are written with. Note that dataset_description was removed in newer DD releases — vaft.imas.omas_imas.IMAS_REMOVED_IDS records that — so round-trip checks use equilibrium (or summary), never dataset_description.

Write an ODS to an AL5 HDF5 entry

vaft.imas.save is the public writer. Hand it an ODS — or a native IDSToplevel, or an IMASHandle — and a destination path. A directory is written as an AL5 HDF5 image set (master.h5 plus one file per IDS); a path ending in .nc goes through IMAS-Python’s netCDF backend.

import tempfile
from omas import ODS
import vaft

ods = ODS()
ods['equilibrium.ids_properties.homogeneous_time'] = 2
ods['equilibrium.ids_properties.comment'] = 'testing'
ods['equilibrium.time'] = [0.01]

entry_dir = tempfile.mkdtemp(prefix='imas_step1_')
vaft.imas.save(ods, entry_dir)

save returns the path it wrote.

Key arguments:

Point save at a fresh directory. Writing into one that already holds a master.h5 replaces that entry rather than appending to it.

Read it back

ods_loaded = vaft.omas.load(entry_dir)
print(ods_loaded['equilibrium.time'])     # [0.01]

vaft.omas.load detects the artifact kind and returns an ODS. To reach the native IMAS objects instead, with no OMAS in between, vaft.imas.load opens the same entry as a context manager around an AL5 DBEntry:

with vaft.imas.load(entry_dir) as dbentry:
    equilibrium = dbentry.get('equilibrium', 0)

The OMAS-derived low-level functions underneath — save_omas_imas, load_omas_imas, imas_open_uri, imas_open, imas_get, imas_set — still exist under vaft.imas.omas_imas, and they still take uri=, new=, paths=, time= and consistency_check=. They are deliberately not re-exported from vaft.imas: prefer save / load unless you need to build the URI yourself or restrict a read to specific paths.

Restricting a read is worth knowing about. With paths=None, load_omas_imas asks the entry for every IDS in the DD, so IDSs that were never written (amns_data, …) get probed and skipped one at a time. Passing the paths you wrote keeps the load fast and the log readable.

What survives on code.parameters

code.parameters is a STR_0D in the Data Dictionary — one string per IDS, XML by IMAS habit — and VAFT puts real provenance in it: EFIT’s collection status, CHEASE’s comparison metrics, the DCON solver fragments, the coupling basis identity, the time-convention memo. What arrives at the other end depends on the shape you wrote, and the shapes are not equivalent.

The low-level vaft.imas.omas_imas.save_omas_imas returns the paths it actually wrote, and that list is the only machine-readable evidence a caller gets. A discarded code.parameters is simply absent from it: no exception, no return code, one WARNING: … is not part of IMAS line on stderr that a pipeline log buries. This is how #380 lost 4096 paths of EFIT provenance between the local product and its HSDS replica without anything reporting a failure.

What can be written is decided at the write, by vaft.imas.code_parameters.as_entry_payload: the leaves an entry can carry are kept, anything it cannot is left on the local product, and a parameters_cache_omitted leaf names what stayed behind. So an EFIT product’s declared COCOS index travels and its per-slice parser cache does not, from the same field, and a reader of the entry is told which — where before, a nested block took every leaf beside it down in silence (#642).

The cache stays behind because it cannot be carried, not merely because it is large: the XML encoder has no array representation, so an array put through the envelope returns as the repr of its elements joined by spaces. Present, named and useless is worse than absent and declared.

vaft.omas.load still promotes a flat, leaf-only block back to a CodeParameters on the way out of a stage product (#478); that now decides what a local reader sees rather than whether the block survives.

test/test_code_parameters_contract.py measures every row through the real Access Layer, so the table stays true rather than being a claim about a version of OMAS somebody once used.

Two consequences are worth stating on their own.

The round trip is not symmetric, and which form you get depends on the document. The loader parses anything XML-shaped, so a one-fragment envelope comes back as a tree — where attribute keys carry an @ prefix and no regular expression over the string will find them. Two fragments do not: OMAS’s decoder cannot represent repeated sibling elements, raises internally, and leaves the string alone. A single-module mhd_linear shot therefore returns a tree and a multi-module one returns the string, from the same field of the same IDS. A consumer must accept both forms whatever it knows about the producer; vaft.plot.backend.recipes._mhd_linear_radial_stride and vaft.plot.backend.convention both do, and say why.

Values inside the envelope are reinterpreted. CodeParameters.from_string runs every text node through ast.literal_eval, so the EFIT case label "039915.00316" returns as the float 39915.00316, and a whitespace-separated numeric string returns as an array. Arbitrary keys and arbitrary values therefore belong inside a payload the XML loader cannot parse — which in this repository means JSON.

The last two rows used to read “nothing”: promotion was all-or-nothing on the block, so an EFIT product lost the COCOS index it declares on the way to a replica — the leaf was not nested, it was merely standing next to something that is (#642). Replicas written before that fix carry no equilibrium.code.parameters at all and need re-replicating for their declaration to appear.

Writing it

Three patterns are sanctioned, each with a stage that shows it:

• A JSON string — generate_efit_ods.py, generate_chease_ods.py. Use sort_keys=True, so two runs over equal inputs are byte-identical and replication does not re-send an unchanged product.
• XML fragments inside one <parameters> envelope — vaft.machine_mapping.mhd_linear, core_sources, coils_non_axisymmetric, gpec_ideal. Each solver appends its fragment by splicing on the trailing </parameters>, so no fragment’s text may contain that literal: one unescaped copy closes the document early and everything appended afterwards lands outside the root element, where a parser refuses it.
• key=value lines — em_coupling, for a short flat record read by eye as often as by code.

A nested sub-path is not forbidden — EFIT’s parser cache is genuinely useful and genuinely local — but it must be a decision somebody wrote down. test/test_code_parameters_writers.py scans vaft/, workflow/ and test/ and fails on a nested path whose file is not in its allow-list, with the role it plays and the reason that data may stop at the FileDB.

Budgeting a large payload

Stage products are gzipped (vaft.database.filedb.OMAS_PRODUCT_SUFFIX is .json.gz since #813), and nothing anywhere guards their size. Do not budget against the file on disk. A code.parameters payload is quote-heavy XML-escaped text, which is exactly what gzip is best at, so a payload that inflates several-fold on escaping can leave the product barely larger — the file size stops tracking the cost entirely. The budget is raw bytes of the finished field string, measured after XML escaping, because that is what the Access Layer serializes and what HSDS stores: neither sees the local container. For scale, the packaged whole-shot sample vaft/data/samples/39915/omas.json.gz is about 10 MB gzipped and carries no mhd_linear at all — now directly comparable to a stage product, which is stored the same way.

Converting a real VEST shot

The full flow, from imas_omas_data_conversion.ipynb:

import tempfile
import vaft

ods_from_legacy = vaft.omas.sample_ods()          # shot 39915

# Legacy files can carry coordinate-inconsistent IDSs: drop them before conversion
for drop_ids in ['em_coupling', 'magnetics']:
    if drop_ids in ods_from_legacy:
        del ods_from_legacy[drop_ids]

entry_dir = tempfile.mkdtemp(prefix='imas_step3_')

vaft.imas.save(ods_from_legacy, entry_dir)
ods_round_trip = vaft.omas.load(entry_dir)

print('Saved IDSs:', sorted(ods_from_legacy.keys()))
print('Loaded IDSs:', list(ods_round_trip.keys()))

Now look at what landed on disk. This is the layout the VEST HSDS server stores per shot:

<entry_dir>/
    master.h5
    equilibrium.h5
    wall.h5
    pf_active.h5
    ...

master.h5 is the aggregator: it externally links every <ids>.h5. IMAS-Core refuses to open the entry when a linked file is missing, which is why vaft.database.ids.load always downloads master.h5 plus every file it links, even for a single-IDS request.

Verify with the native IMAS-Python API

An OMAS round-trip only proves that OMAS can read what OMAS wrote. To prove the entry is genuinely valid IMAS, open it with the Access Layer directly:

import imas
from vaft.imas import IMAS_DD_VERSION_CONVERSION

with imas.DBEntry('imas:hdf5?path=' + entry_dir, 'r', dd_version=IMAS_DD_VERSION_CONVERSION) as dbentry:
    eq = dbentry.get('equilibrium', 0)          # -> IDSToplevel
    print(eq.ids_properties.homogeneous_time)
    print(len(eq.time), eq.time[0])

Pass the same dd_version you saved with, or get() cannot interpret the layout.

Creating an IDS from scratch, with no OMAS involved at all:

import imas

factory = imas.IDSFactory()
equilibrium = factory.equilibrium()
equilibrium.ids_properties.homogeneous_time = imas.ids_defs.IDS_TIME_MODE_HOMOGENEOUS
equilibrium.ids_properties.comment = "testing"
equilibrium.time = [0.01]

with imas.DBEntry("imas:hdf5?path=my_entry", "w") as dbentry:
    dbentry.put(equilibrium)

IDSFactory knows the DD and mints empty IDSs; DBEntry is the I/O handle for one entry; one DBEntry holds many IDSs, each identified by name and occurrence. That is the whole IMAS object model.

NetCDF export

The HDF5 backend needs imas_core. The NetCDF backend does not — IMAS-Python writes it natively, which makes .nc the format of choice for shipping a self-contained entry to a collaborator:

import imas

with imas.DBEntry("vest_39915.nc", "w") as dbentry:
    dbentry.put(equilibrium)

The repository sample vaft.data.sample(39915, representation="imas") is exactly such a container (it is not in the PyPI distribution, which ships only the OMAS form of 39915).

If you are staying inside OMAS, its own NetCDF backend serialises a whole ODS (all IDSs at once):

from omas import save_omas_nc

save_omas_nc(ods, 'ods_39915.nc')

Round-tripping through the VEST database

The database layer wraps the conversion above: vaft.database.load_ods downloads a shot’s IMAS images from HSDS and hands you an ODS; vaft.database.save_ods does the reverse.

import vaft

ods = vaft.database.load(39915)                             # ODS from the default source, "main"
ods = vaft.database.load_ods(39915, paths=['magnetics'])    # only one IDS
ods = vaft.database.load_ods(39915, time=0.325)             # single time slice
ods_list = vaft.database.load_ods([39915, 41524, 41672])    # list in, list out

If the IMAS images are already on disk — for instance the entry_dir you just wrote — skip HSDS entirely:

ods = vaft.database.load_ods(39915, path=entry_dir)   # directory must contain master.h5

For native IDS objects instead of an ODS, go through the IDS pair. vaft.database.load returns an ODS and has no ids_name argument; its second positional argument is source.

equilibrium = vaft.database.ids.load(2, "equilibrium", dd_version="3.41.0")
# or, for a whole shot as native IMAS: vaft.database.load(2, representation="imas")

Symmetrically, vaft.database.save / save_ods take an ODS only; a native IDSToplevel must go through vaft.database.ids.save. Writing to the shared server is admin-restricted, but env="local" writes the IMAS images to disk and returns the local directory:

local_dir = vaft.database.save_ods(ods, 39915, env="local")

See the Quick start guide for HSDS credentials and the h5pyd / hsget / hsload prerequisites.

Time-convention handling

VEST diagnostics are digitised on a DAQ clock whose $t=0$ is the trigger, not any physics event. Comparing shots on that clock is meaningless: breakdown happens tens of milliseconds later, and when it happens varies from shot to shot. VAFT therefore lets you re-reference an entire ODS to a physical event.

Four conventions are defined. Every origin is a measured event on the DAQ clock, found by the shared timing modules with provenance (issue #409):

import logging

from omas import ODC

import vaft

logging.basicConfig(level=logging.INFO)          # the shift is logged, not printed
odc = ODC()
odc['0'] = vaft.omas.sample_ods()                # packaged 39915
vaft.omas.change_time_convention(odc, convention='breakdown')
# INFO:vaft.omas.general:[0] shift -0.30632 s  (daq -> breakdown)

The shift differs from shot to shot (39915 breaks down at 0.3063 s, 41524 at 0.3146 s, 41672 at 0.3122 s on the DAQ clock) — which is exactly why the raw clock is useless for comparing shots.

change_time_convention(odc_or_ods, convention='vloop') accepts an ODS or an ODC (a bare ODS is wrapped internally, and an ODC is returned). On the first call it derives the origins on the product’s DAQ clock and records them under summary.code.parameters — a flat, leaf-only block, which is the shape that survives replication (What survives on code.parameters) — with their sources:

params = odc['0']['summary.code.parameters']
params['time_convention']         # 'breakdown'
params['vloop_onset']             # seconds, on the ORIGINAL daq clock (absent when the event was not found)
params['ip_onset']
params['breakdown_onset']
params['onset_method']            # 'plasma_timing/discharge_timing'
params['breakdown_onset_source']  # 'h_alpha_primary', 'h_alpha_fast', 'ip_principal'
params['vloop_onset_source']      # 'magnetics.flux_loop.5 dflux_dt zero crossing after the ohmic excursion'
params['vloop_onset_reason']      # present instead of the origin when the event was not found
params['onset_flags']             # ';'-joined, e.g. 'vloop:approached_without_crossing'

The finders and every consumer of the plasma window (the default time window of a time plot, the vacuum-flux plot, the shot overview) work on the product’s own clock: the analysis span is stated in DAQ seconds and displaced by the recorded origin, so a product already shifted to 'breakdown' reports its plasma onset at 0 s. A product that declares a convention without recording its origin cannot be placed and raises.

Because the onsets are stored, conversions are composable and reversible: call it again with a different convention and the shift is computed from the recorded originals, never re-derived from already-shifted data. An origin that was not found is simply absent, its *_onset_reason says why, and asking for its convention raises ValueError with that reason. Nothing is written when no origin at all can be derived (a shifted product whose memo was lost): the call raises and the product is left as it was. A product carrying the memo of the earlier finders (no onset_method) is re-derived when it is still on the daq clock and flagged legacy_memo_rederived; one already shifted with those origins keeps them — they are what makes its shift reversible — with onset_method = 'legacy' and the flag legacy_origins_retained.

vaft.omas.change_time_convention(odc, convention='ip')     # breakdown → ip
vaft.omas.change_time_convention(odc, convention='daq')    # back to the raw clock

The underlying primitive is vaft.omas.shift_time(one_ods, time_shift). It is deliberately conservative: it walks ods.paths() and shifts a leaf only when the last component of the path is exactly time, onset or offset, and it never touches anything under summary.code.parameters. That narrowness is the point — a looser rule corrupts data by shifting fields such as magnetics.ip.0.data whose path merely contains a time-like word.

Individual onsets are available directly; they are meaningful on a product in the daq convention, which is why the conversion derives them once:

vaft.omas.find_vloop_onset(ods)
vaft.omas.find_ip_onset(ods)
vaft.omas.find_breakdown_onset(ods)
vaft.omas.find_pulse_duration(ods)     # plasma offset - onset, gaps included
vaft.omas.find_pf_active_onset(ods)    # one entry per coil, nan for a coil that did not fire
vaft.omas.find_max_ip(ods)             # representative peak Ip inside the plasma window
vaft.omas.find_bt(ods)                 # mean toroidal field over the plasma window

The unit is the one main discharge. Both detectors keep only the segment holding their own maximum (principal_only in the plasma_timing policy), so a window is one run and its extent is a duration. Before #842 the light rule returned the envelope of every accepted segment, so a record of scattered flashes reported one long window that was mostly gap — and opened it at the first flash, which the plasma current said was the discharge in only 11 of 35 shots. A record whose brightest feature is a spike the segment tests refused now yields no light window at all rather than a window built from lesser bursts.

plasma_timing(ods).duty_cycle is how much of a window is above threshold; the shot overview carries it as pulse_duty_cycle beside pulse_duration_s, and the onset corpus keeps it per detector. It is 1 everywhere in the corpus today, which is the point: it is the check that the unit has not slipped back (#752).

The records behind them carry the evidence: vaft.omas.plasma_timing.plasma_timing(ods) (the window, its source, the light/current agreement) and vaft.omas.discharge_timing.discharge_timing(ods) (every coil’s onset, the ohmic onset, the loop-voltage excursion and its crossing, and the actuators that are not mapped).

Fix the time convention before you compare shots or overlay traces. Mixing an ODS on the daq clock with one on the breakdown clock in the same figure is the easiest way to produce a wrong result that still looks plausible.

Combining ODSs

A shot is assembled piecewise by the pipeline — diagnostics mapped first, equilibrium reconstructed later, kinetic profiles fitted last — and the pieces have to end up in one ODS before it can be written back as a single IMAS entry.

vaft.omas.combine_ods(ods_list) is the function nominally for this, but it does not merge more than one ODS. Its loop calls combined_ods.update(ods) and then breaks on the first success, so only ods_list[0] ever lands:

merged = vaft.omas.combine_ods([ods_equilibrium, ods_magnetics, ods_wall])
list(merged.keys())
# ['equilibrium']          <- magnetics and wall silently dropped

Its error-recovery branch is dead code too: on an IMAS-validity failure it increments an attempt_count that is never initialised (NameError) and continues to the next ODS rather than retrying the current one.

Merge with ODS.update() directly instead — that is the primitive combine_ods was built on, and in a plain loop it does the whole job:

from omas import ODS

merged = ODS()
for one in [ods_equilibrium, ods_magnetics, ods_wall]:
    merged.update(one)

list(merged.keys())
# ['equilibrium', 'magnetics', 'wall']

If a source ODS carries a coordinate-inconsistent IDS, drop it before the update (the same guard the conversion example above uses) rather than relying on combine_ods to recover:

for drop_ids in ['em_coupling', 'magnetics']:
    if drop_ids in one:
        del one[drop_ids]

Reference

Source: vaft/imas/omas_imas.py · vaft/omas/general.py · vaft/omas/sample.py · vaft/process/signal_processing.py · vaft/database/ods.py · vaft/database/ids.py

More runnable examples: Examples.