Process reference: signal_processing

Generated from the docstrings of vaft.process.signal_processing: 27 public functions. Each entry is what vaft.process.describe("signal_processing.<name>") prints. All process categories

Overview

Everything here takes NumPy arrays and returns NumPy arrays. Nothing reads an ODS, nothing knows what the waveform is, and nothing embeds a VEST constant except where the function’s name says so (vest_*) and its Applicability section says why. The VEST source selection, calibration and processing windows that decide which waveform to condition, and with what numbers, live in vaft.machine_mapping and vest.yaml.

Four kinds of operation:

  • Filtering – smooth, butterworth_lowpass, butterworth_bandpass, detrend_moving_average, anti_alias_filter, vest_coil_current_noise_reduction.
  • Baseline – define_baseline selects the quiet samples, subtract_baseline fits one of linear_baseline, quadratic_baseline, exp_baseline or a spline through them and removes it.
  • Saturation repair – detect_clipped_samples, repair_clipped_interval.
  • Timebase and activity – describe_time_grid, resample_to_time, detect_active_window, signal_on_offset, is_signal_active, infer_signal_orientation, line_average_density, and the legacy process_signal wrapper.
SymbolMeaningUnit
ttimes
dtsample spacings
f_ssample rate, ``1 / dt``Hz
f_NNyquist frequency, ``f_s / 2``Hz
f_cfilter cutoffHz
xthe waveform, in whatever unit it hasany

Two filtering conventions coexist and each function says which it uses. Causal (scipy.signal.lfilter) delays every feature by the filter’s group delay and is the convention of the validated VEST SXR viewer, where matched delay between channels matters more than zero phase. Zero-phase (scipy.signal.filtfilt, forward then backward) moves nothing in time and is mandatory wherever an onset time is read off the result. butterworth_lowpass defaults to causal, everything else to zero-phase.

Functions

VEST_CoilCurrentNoiseReductionDeprecated compatibility wrapper for the snake-case function. Deprecated
anti_alias_filterZero-phase FIR low-pass applied on the source grid, ahead of a rate cut. Convention
butterworth_bandpassButterworth band-pass filter along the last axis. Convention
butterworth_lowpassButterworth low-pass filter along the last axis. Convention
define_baselineSelect the baseline samples around a signal’s onset and optional offset.
describe_time_gridMeasure a timebase’s spacing, uniformity and monotonicity. Convention
detect_active_windowReturn the active time window that contains a signal’s main peak.
detect_clipped_samplesReturn a boolean mask of samples sitting at an acquisition limit.
detrend_moving_averageSubtract a centered moving-average trend along the last axis. Convention
exp_baselineExponential model for baseline fitting: y = a * exp(b * x) + c.
fir_filterLinear-phase FIR filter along the last axis. Convention
fir_filter_coefficientsWindow-method FIR coefficients for a linear-phase filter. Convention
infer_signal_orientationInfer the dominant sign of signal from its active region. Convention
is_signal_activeDecide whether a trace carries a signal or is flat.
line_average_densityReturn line-average electron density given an explicit chord length. Convention
linear_baselineLinear model for baseline fitting: y = a * x + b.
process_signalLegacy conditioning wrapper: crop, resample, then filter. Convention
quadratic_baselineQuadratic model for baseline fitting: y = a * x^2 + b * x + c.
repair_clipped_intervalReconstruct samples saturated at an acquisition limit by interpolation.
resample_to_timeProject a signal onto target_time, low-passing first if that is a rate cut. Convention
signal_on_offsetSmooth a waveform, then return the window in which it is above a threshold. Convention
signal_onoffsetDeprecated compatibility wrapper for signal_on_offset. Deprecated
smoothApply MATLAB-like moving-average smoothing with edge tapering. Convention
subtract_baselineFit a baseline through the selected samples and subtract it everywhere.
vest_coil_current_noise_reductionSuppress point spikes in coil current traces. Convention
vfit_signal_start_endDeprecated compatibility wrapper for detect_active_window. Deprecated
vfit_signal_startendDeprecated compatibility wrapper for detect_active_window. Deprecated

VEST_CoilCurrentNoiseReduction

Deprecated source
VEST_CoilCurrentNoiseReduction(data)
Show source vaft/process/signal_processing.py, lines 1998–2006
def VEST_CoilCurrentNoiseReduction(data):  # noqa: N802
    """Deprecated compatibility wrapper for the snake-case function."""
    warnings.warn(
        "VEST_CoilCurrentNoiseReduction() is deprecated; use "
        "vest_coil_current_noise_reduction().",
        DeprecationWarning,
        stacklevel=2,
    )
    return vest_coil_current_noise_reduction(data)

Deprecated compatibility wrapper for the snake-case function.

anti_alias_filter

Machine-independentConvention-sensitive source
anti_alias_filter(values, *, source_rate, cutoff_hz, stopband_hz=None, numtaps=None, axis=-1, nan_policy='segment', return_mask=False)

Zero-phase FIR low-pass applied on the source grid, ahead of a rate cut.

This is the half of a downsample that np.interp cannot do for you. Filtering after the rate has already been reduced is too late: whatever folded is now indistinguishable from real in-band signal.

NameTypeUnitDescription
valuesarray_likeany

The waveform or a stack of them; the time axis is axis.

source_ratefloatHz

Sample rate of values.

cutoff_hzfloatHz

Passband edge, strictly inside (0, source_rate / 2).

stopband_hzfloat or None, optionalHz

Frequency by which the response must be down – for a downsample, the target Nyquist. Sets the transition band and so the tap count.

numtapsint or None, optional-

Override the designed tap count; forced odd.

axisint, optional-

The time axis of values.

nan_policy{"segment", "error", "ignore"}, optional-

"segment" filters each maximal finite run on its own; "error" refuses non-finite samples; "ignore" hands them to filtfilt, which smears them across the record.

return_maskbool, optional-

Also return which samples the filter actually covered.

Returns
—np.ndarrayany

The filtered waveform, same shape as values.

—np.ndarray of bool-

Only with return_mask=True: True where filtfilt ran over the sample, False for a sample left unfiltered (a short record or finite run) and for a non-finite sample under "segment"; same shape as values.

Convention

Zero-phase, and load-bearing: filterscope intensity feeds onset detection, where a causal filter would move the onset by the group delay. firwin

  • filtfilt rather than a Butterworth or decimate’s Chebyshev, because it is the idiom the VEST mappers already use (machine_mapping/tf.py, machine_mapping/pf_active.py, process/magnetics.py).

Raises

  • ResamplingError cutoff_hz or stopband_hz outside their bands, or a non-finite sample under nan_policy="error".
  • ValueError Unknown nan_policy.
Processing steps · Defaults · Applicability · Limitations · Provenance

Processing steps

  1. Size the filter: numtaps ~ 3.3 * source_rate / (stopband - cutoff), the Hamming-window rule [1], measured against the target Nyquist – not the source Nyquist, which is ten times higher for a VEST fast channel and would size the filter at nine taps instead of the three hundred the job needs. Forced odd so the FIR is Type I with integer group delay.
  2. Design it with scipy.signal.firwin (Hamming window, low-pass).
  3. Apply scipy.signal.filtfilt to each finite run long enough for it; shorter runs are passed through with a RuntimeWarning.

Defaults

stopband_hz = 1.25 * cutoff_hz is a numerical convenience consistent with resample_to_time’s default 0.8-of-Nyquist cutoff: it puts the stopband exactly at the target Nyquist. nan_policy = "segment" is a validated-workflow default – one NaN must not blank a whole record.

Applicability

Machine-independent.

Limitations

filtfilt needs 3 * numtaps + 1 samples; a record or finite run shorter than that is returned unfiltered rather than padded, and nothing is interpolated or invented in its place. One warning per call summarizes how many runs and samples were left unfiltered, so a caller can tell a single stray sample from most of the record; return_mask=True says which samples, so a caller can flag them rather than store them beside filtered values as if they were equivalent (#915). The filter assumes evenly spaced samples; on an irregular grid the design rate is a fiction, which resample_to_time refuses for you.

Provenance

  1. Hamming-window FIR design rule, N ~ 3.3 f_s / Delta f, e.g. Oppenheim & Schafer, Discrete-Time Signal Processing, window method.
  2. vaft/machine_mapping/tf.py and pf_active.py, the mappers whose established filtering idiom this generalizes.

butterworth_bandpass

Machine-independentConvention-sensitive source
butterworth_bandpass(data, low, high, fs, order=2, *, zero_phase=True)
Show source vaft/process/signal_processing.py, lines 513–576
def butterworth_bandpass(data, low: float, high: float, fs: float, order: int = 2,
                         *, zero_phase: bool = True) -> np.ndarray:
    """Butterworth band-pass filter along the last axis.

    Parameters
    ----------
    data : array_like
        The waveform or a stack of them; the time axis is the last [any].
    low : float
        Lower band edge [Hz].
    high : float
        Upper band edge, above ``low`` and below the Nyquist frequency [Hz].
    fs : float
        Sample rate [Hz].
    order : int, optional
        Filter order [-].
    zero_phase : bool, optional
        ``True`` for forward-backward ``filtfilt``; ``False`` for a causal
        ``lfilter`` [-].

    Returns
    -------
    np.ndarray
        The filtered waveform, same shape as ``data`` [any].

    Raises
    ------
    ValueError
        The band edges do not satisfy ``0 < low < high < fs / 2``.

    Defaults
    --------
    ``order = 2`` is a validated-workflow default shared with
    :func:`butterworth_lowpass`.  ``zero_phase = True`` is the opposite of the
    low-pass default because band-passed signals feed spectral and
    mode-number analysis, where phase across channels is the measurement.

    Convention
    ----------
    Zero-phase by default; see :func:`butterworth_lowpass` for the causal
    alternative and when each is right.

    Applicability
    -------------
    Machine-independent.

    Limitations
    -----------
    As :func:`butterworth_lowpass`: a record too short for ``filtfilt`` raises
    from SciPy.
    """
    values = np.asarray(data, dtype=float)
    nyquist = 0.5 * float(fs)
    if not 0.0 < float(low) < float(high) < nyquist:
        raise ValueError(
            f"band edges must satisfy 0 < low < high < {nyquist:g} Hz for fs={fs:g}; "
            f"got ({low!r}, {high!r})"
        )
    b, a = scipy_signal.butter(
        int(order), [float(low) / nyquist, float(high) / nyquist], btype="band"
    )
    if zero_phase:
        return scipy_signal.filtfilt(b, a, values, axis=-1)
    return scipy_signal.lfilter(b, a, values, axis=-1)

Butterworth band-pass filter along the last axis.

NameTypeUnitDescription
dataarray_likeany

The waveform or a stack of them; the time axis is the last.

lowfloatHz

Lower band edge.

highfloatHz

Upper band edge, above low and below the Nyquist frequency.

fsfloatHz

Sample rate.

orderint, optional-

Filter order.

zero_phasebool, optional-

True for forward-backward filtfilt; False for a causal lfilter.

Returns
—np.ndarrayany

The filtered waveform, same shape as data.

Convention

Zero-phase by default; see butterworth_lowpass for the causal alternative and when each is right.

Raises

  • ValueError The band edges do not satisfy 0 < low < high < fs / 2.
Defaults · Applicability · Limitations

Defaults

order = 2 is a validated-workflow default shared with butterworth_lowpass. zero_phase = True is the opposite of the low-pass default because band-passed signals feed spectral and mode-number analysis, where phase across channels is the measurement.

Applicability

Machine-independent.

Limitations

As butterworth_lowpass: a record too short for filtfilt raises from SciPy.

butterworth_lowpass

Machine-independentConvention-sensitive source
butterworth_lowpass(data, cutoff, fs, order=2, *, zero_phase=False)
Show source vaft/process/signal_processing.py, lines 449–510
def butterworth_lowpass(data, cutoff: float, fs: float, order: int = 2,
                        *, zero_phase: bool = False) -> np.ndarray:
    """Butterworth low-pass filter along the last axis.

    Parameters
    ----------
    data : array_like
        The waveform or a stack of them; the time axis is the last [any].
    cutoff : float
        Cutoff frequency, strictly between 0 and the Nyquist frequency [Hz].
    fs : float
        Sample rate [Hz].
    order : int, optional
        Filter order [-].
    zero_phase : bool, optional
        ``True`` for forward-backward ``filtfilt``; ``False`` for a causal
        ``lfilter`` [-].

    Returns
    -------
    np.ndarray
        The filtered waveform, same shape as ``data`` [any].

    Raises
    ------
    ValueError
        ``cutoff`` is not inside ``(0, fs / 2)``.

    Defaults
    --------
    ``order = 2`` and ``zero_phase = False`` are validated-workflow defaults:
    the values the VEST SXR viewer uses [SXR]_, kept so that
    :mod:`vaft.process.soft_x_rays` reproduces it.

    Convention
    ----------
    **Causal by default.**  With ``zero_phase=False`` every feature is delayed
    by the filter's group delay, identically on every channel, which is what
    ratio and reference arithmetic between SXR channels needs.  Pass
    ``zero_phase=True`` wherever a time is going to be read off the result --
    an onset, a peak -- because a causal filter moves it.

    Applicability
    -------------
    Machine-independent.  The defaults are VEST SXR practice but embed no VEST
    constant.

    Limitations
    -----------
    ``filtfilt`` needs a record longer than three times the filter's padding
    length and raises from SciPy otherwise; nothing here catches that.
    """
    values = np.asarray(data, dtype=float)
    nyquist = 0.5 * float(fs)
    if not 0.0 < float(cutoff) < nyquist:
        raise ValueError(
            f"cutoff must lie in (0, {nyquist:g}) Hz for fs={fs:g}; got {cutoff!r}"
        )
    b, a = scipy_signal.butter(int(order), float(cutoff) / nyquist, btype="low")
    if zero_phase:
        return scipy_signal.filtfilt(b, a, values, axis=-1)
    return scipy_signal.lfilter(b, a, values, axis=-1)

Butterworth low-pass filter along the last axis.

NameTypeUnitDescription
dataarray_likeany

The waveform or a stack of them; the time axis is the last.

cutofffloatHz

Cutoff frequency, strictly between 0 and the Nyquist frequency.

fsfloatHz

Sample rate.

orderint, optional-

Filter order.

zero_phasebool, optional-

True for forward-backward filtfilt; False for a causal lfilter.

Returns
—np.ndarrayany

The filtered waveform, same shape as data.

Convention

Causal by default. With zero_phase=False every feature is delayed by the filter’s group delay, identically on every channel, which is what ratio and reference arithmetic between SXR channels needs. Pass zero_phase=True wherever a time is going to be read off the result – an onset, a peak – because a causal filter moves it.

Raises

  • ValueError cutoff is not inside (0, fs / 2).
Defaults · Applicability · Limitations

Defaults

order = 2 and zero_phase = False are validated-workflow defaults: the values the VEST SXR viewer uses [SXR], kept so that vaft.process.soft_x_rays reproduces it.

Applicability

Machine-independent. The defaults are VEST SXR practice but embed no VEST constant.

Limitations

filtfilt needs a record longer than three times the filter’s padding length and raises from SciPy otherwise; nothing here catches that.

define_baseline

Machine-independent source
define_baseline(time, onset_time, onset_window, offset_time=None, offset_window=None)
Show source vaft/process/signal_processing.py, lines 1706–1772
def define_baseline(time, onset_time, onset_window, offset_time=None, offset_window=None):
    """Select the baseline samples around a signal's onset and optional offset.

    Converts ``onset_time`` (and ``offset_time``) to indices with
    ``np.searchsorted`` and returns the ``onset_window`` samples immediately
    before the onset, plus the ``offset_window`` samples immediately after the
    offset when both are given.  The result is what :func:`subtract_baseline`
    fits through.

    Parameters
    ----------
    time : np.ndarray
        Sample times, sorted [s].
    onset_time : float
        Time at which the signal begins to deviate from baseline [s].
    onset_window : int
        Number of samples before the onset to include [-].
    offset_time : float or None, optional
        Time at which the signal returns to baseline [s].
    offset_window : int or None, optional
        Number of samples after the offset to include; both offset arguments
        must be given for the offset region to be used [-].

    Returns
    -------
    np.ndarray
        Indices into ``time`` of the baseline samples, onset region first [-].

    Assumptions
    -----------
    The signal is genuinely quiet in the selected windows.  A window that
    reaches back into a previous pulse, or forward into pickup from a coil
    ramp, is fitted as if it were baseline.

    Applicability
    -------------
    Machine-independent.  The onset and offset times, and the window widths,
    are the caller's; on VEST they come from the magnetics processing window
    in ``vest.yaml`` via :mod:`vaft.process.magnetics`.

    Limitations
    -----------
    Windows are clipped to the record silently, so a window wider than the
    pre-onset record yields fewer samples than asked, with no warning.

    Provenance
    ----------
    .. [1] The legacy VEST EFIT magnetics chain [VFIT]_, where the baseline is
       taken before the discharge and after it and removed by a fit.
    """
    baseline_indices = []

    # Convert onset_time -> onset_idx
    onset_idx = np.searchsorted(time, onset_time)

    # Add onset window
    if onset_window > 0:
        start_idx = max(0, onset_idx - onset_window)
        baseline_indices.extend(range(start_idx, onset_idx))

    # Convert offset_time -> offset_idx if provided
    if offset_time is not None and offset_window is not None:
        offset_idx = np.searchsorted(time, offset_time)
        end_idx = min(len(time), offset_idx + offset_window)
        baseline_indices.extend(range(offset_idx, end_idx))

    return np.array(baseline_indices)

Select the baseline samples around a signal’s onset and optional offset.

Converts onset_time (and offset_time) to indices with np.searchsorted and returns the onset_window samples immediately before the onset, plus the offset_window samples immediately after the offset when both are given. The result is what subtract_baseline fits through.

NameTypeUnitDescription
timenp.ndarrays

Sample times, sorted.

onset_timefloats

Time at which the signal begins to deviate from baseline.

onset_windowint-

Number of samples before the onset to include.

offset_timefloat or None, optionals

Time at which the signal returns to baseline.

offset_windowint or None, optional-

Number of samples after the offset to include; both offset arguments must be given for the offset region to be used.

Returns
—np.ndarray-

Indices into time of the baseline samples, onset region first.

Assumptions · Applicability · Limitations · Provenance

Assumptions

The signal is genuinely quiet in the selected windows. A window that reaches back into a previous pulse, or forward into pickup from a coil ramp, is fitted as if it were baseline.

Applicability

Machine-independent. The onset and offset times, and the window widths, are the caller’s; on VEST they come from the magnetics processing window in vest.yaml via vaft.process.magnetics.

Limitations

Windows are clipped to the record silently, so a window wider than the pre-onset record yields fewer samples than asked, with no warning.

Provenance

  1. The legacy VEST EFIT magnetics chain [VFIT], where the baseline is taken before the discharge and after it and removed by a fit.

describe_time_grid

Machine-independentConvention-sensitive source
describe_time_grid(time, *, rtol=0.001)
Show source vaft/process/signal_processing.py, lines 1001–1060
def describe_time_grid(time, *, rtol: float = 1e-3) -> TimeGrid:
    """Measure a timebase's spacing, uniformity and monotonicity.

    Parameters
    ----------
    time : array_like
        Sample times, flattened [s].
    rtol : float, optional
        Largest ``|dt_i - median(dt)| / median(dt)`` for which the grid is
        still reported uniform [-].

    Returns
    -------
    TimeGrid
        ``n``, ``t0``, the median ``dt``, ``sample_rate``, whether the grid is
        ``uniform`` within ``rtol`` and ``strictly_increasing``, and the
        ``max_relative_jitter`` actually measured [any].

    Defaults
    --------
    ``rtol = 1e-3`` is a numerical convenience sized to an acquisition-era
    fact: VEST shots after 42190 store their timebase as ``linspace(0, span,
    n)``, so a nominally 4e-6 s grid is really ``span / (n - 1)`` =
    4.00016e-6 s.  That is uniform for every purpose here and a tight
    tolerance would only reject it.

    Convention
    ----------
    ``dt`` is the *median* spacing, robust to the single ragged interval a
    concatenated acquisition can leave behind.  ``dt`` and ``sample_rate``
    are ``nan`` for a single-sample grid, where neither is defined; an empty
    grid reports ``n = 0`` and ``nan`` throughout.

    Applicability
    -------------
    Machine-independent.
    """
    values = np.asarray(time, dtype=float).reshape(-1)
    n = int(values.size)
    if n == 0:
        return TimeGrid(0, float("nan"), float("nan"), float("nan"), False, False, float("nan"))
    t0 = float(values[0])
    if n == 1:
        return TimeGrid(1, t0, float("nan"), float("nan"), True, True, float("nan"))

    steps = np.diff(values)
    dt = float(np.median(steps))
    strictly_increasing = bool(np.all(steps > 0.0))
    if dt == 0.0 or not np.isfinite(dt):
        return TimeGrid(n, t0, dt, float("nan"), False, strictly_increasing, float("inf"))
    jitter = float(np.max(np.abs(steps - dt)) / abs(dt))
    return TimeGrid(
        n=n,
        t0=t0,
        dt=dt,
        sample_rate=1.0 / dt,
        uniform=bool(jitter <= float(rtol)),
        strictly_increasing=strictly_increasing,
        max_relative_jitter=jitter,
    )

Measure a timebase’s spacing, uniformity and monotonicity.

NameTypeUnitDescription
timearray_likes

Sample times, flattened.

rtolfloat, optional-

Largest |dt_i - median(dt)| / median(dt) for which the grid is still reported uniform.

Returns
—TimeGridany

n, t0, the median dt, sample_rate, whether the grid is uniform within rtol and strictly_increasing, and the max_relative_jitter actually measured.

Convention

dt is the median spacing, robust to the single ragged interval a concatenated acquisition can leave behind. dt and sample_rate are nan for a single-sample grid, where neither is defined; an empty grid reports n = 0 and nan throughout.

Defaults · Applicability

Defaults

rtol = 1e-3 is a numerical convenience sized to an acquisition-era fact: VEST shots after 42190 store their timebase as linspace(0, span, n), so a nominally 4e-6 s grid is really span / (n - 1) = 4.00016e-6 s. That is uniform for every purpose here and a tight tolerance would only reject it.

Applicability

Machine-independent.

detect_active_window

Machine-independent source
detect_active_window(time, signal, threshold=0.01)

Return the active time window that contains a signal’s main peak.

Finds the largest sample and walks outward from it in both directions while the signal stays at or above threshold; the window is the time of the first and last sample reached. If the peak itself is below threshold the whole record is returned.

NameTypeUnitDescription
timearray_likes

Sample times, one-dimensional.

signalarray_likeany

The waveform, one-dimensional, same length as time; already the physical quantity, not a raw digitizer voltage.

thresholdfloat, optionalany

Level at or above which a sample counts as active, in the units of signal.

Returns
—tuple of (float, float)s

(start, end) times of the active window.

Raises

  • ValueError The inputs are not one-dimensional, differ in length, or are empty.
Defaults · Assumptions · Applicability · Limitations · Provenance

Defaults

threshold = 0.01 is a legacy compatibility value from the donor vfit_signal_startend, which was applied to an H-alpha trace normalised to its own minimum. It is absolute, so a caller with a differently scaled signal must pass its own.

Assumptions

The signal has one dominant positive excursion and is at or above threshold throughout it. A dip below the threshold inside the pulse ends the window early.

Applicability

Machine-independent. VEST source selection and calibration belong in vaft.machine_mapping; the resulting physical signal is what is passed here. For a plasma onset with evidence and a verdict, use vaft.process.onset instead.

Limitations

The threshold is absolute, not relative to the peak, and the walk is not robust to noise: a single sample below threshold stops it.

Provenance

  1. vfit_signal_startend, the function nested in Function/vest_Halpha_tstart_tend.m of the legacy VEST workflow [VFIT]; kept here under the deprecated name vfit_signal_start_end.

detect_clipped_samples

Machine-independent source
detect_clipped_samples(data, *, clip_values, tolerance)
Show source vaft/process/signal_processing.py, lines 116–183
def detect_clipped_samples(data, *, clip_values, tolerance: float) -> np.ndarray:
    """Return a boolean mask of samples sitting at an acquisition limit.

    ``clip_values`` is a single signed level or a sequence of them, and the
    mask is their union: a sample is saturated when it lies within
    ``tolerance`` of *any* supplied level.  Real acquisition hardware rails on
    both sides and rarely symmetrically -- VEST's diamagnetic Rogowski channel
    is a signed 16-bit ADC over +/-5 V, so its rails are exactly ``-5.0`` and
    ``5 * 32767 / 32768`` [1]_.

    Detecting every level in one pass is not a convenience.  Where a waveform
    oscillates hard enough to hit both rails within a few samples, repairing
    one rail at a time would fit the reconstruction through samples still
    pinned at the other one.

    Parameters
    ----------
    data : array_like
        The waveform, raw or calibrated; the rails are in its units [any].
    clip_values : float or sequence of float
        The acquisition limit or limits, signed, in the units of ``data`` [any].
    tolerance : float
        Half-width of the band around each limit inside which a sample counts
        as saturated [any].

    Returns
    -------
    np.ndarray
        Boolean mask, ``True`` where the sample is saturated; same shape as
        ``data`` [-].

    Raises
    ------
    SignalRepairError
        ``clip_values`` is empty or non-finite, or ``tolerance`` is not a
        positive finite width.

    Assumptions
    -----------
    Saturation shows as a sample *at* the rail, not as a distorted sample near
    it.  A digitizer that soft-clips will not be detected by a narrow
    tolerance.

    Applicability
    -------------
    Machine-independent.  The caller supplies the rails and tolerance; this
    function infers nothing about the hardware.  The VEST values are read from
    ``vest.yaml`` by :mod:`vaft.machine_mapping`, never here.

    Provenance
    ----------
    .. [1] ``vest.yaml``, the ``diamagnetic_flux`` processing block, where the
       asymmetric 16-bit rails are derived; issue #285.
    """
    values = np.asarray(data, dtype=float)
    levels = np.atleast_1d(np.asarray(clip_values, dtype=float)).reshape(-1)
    if levels.size == 0:
        raise SignalRepairError("`clip_values` must contain at least one acquisition limit.")
    if not np.all(np.isfinite(levels)):
        raise SignalRepairError(f"`clip_values` must be finite, got {clip_values!r}.")
    width = float(tolerance)
    if not np.isfinite(width) or width <= 0.0:
        raise SignalRepairError(f"`tolerance` must be a positive finite width, got {tolerance!r}.")

    saturated = np.zeros(values.shape, dtype=bool)
    for level in levels:
        saturated |= np.abs(values - level) < width
    return saturated

Return a boolean mask of samples sitting at an acquisition limit.

clip_values is a single signed level or a sequence of them, and the mask is their union: a sample is saturated when it lies within tolerance of any supplied level. Real acquisition hardware rails on both sides and rarely symmetrically – VEST’s diamagnetic Rogowski channel is a signed 16-bit ADC over +/-5 V, so its rails are exactly -5.0 and 5 * 32767 / 32768 [1].

Detecting every level in one pass is not a convenience. Where a waveform oscillates hard enough to hit both rails within a few samples, repairing one rail at a time would fit the reconstruction through samples still pinned at the other one.

NameTypeUnitDescription
dataarray_likeany

The waveform, raw or calibrated; the rails are in its units.

clip_valuesfloat or sequence of floatany

The acquisition limit or limits, signed, in the units of data.

tolerancefloatany

Half-width of the band around each limit inside which a sample counts as saturated.

Returns
—np.ndarray-

Boolean mask, True where the sample is saturated; same shape as data.

Raises

  • SignalRepairError clip_values is empty or non-finite, or tolerance is not a positive finite width.
Assumptions · Applicability · Provenance

Assumptions

Saturation shows as a sample at the rail, not as a distorted sample near it. A digitizer that soft-clips will not be detected by a narrow tolerance.

Applicability

Machine-independent. The caller supplies the rails and tolerance; this function infers nothing about the hardware. The VEST values are read from vest.yaml by vaft.machine_mapping, never here.

Provenance

  1. vest.yaml, the diamagnetic_flux processing block, where the asymmetric 16-bit rails are derived; issue #285.

detrend_moving_average

Machine-independentConvention-sensitive source
detrend_moving_average(data, window_samples)
Show source vaft/process/signal_processing.py, lines 744–794
def detrend_moving_average(data, window_samples: int) -> np.ndarray:
    """Subtract a centered moving-average trend along the last axis.

    Parameters
    ----------
    data : array_like
        The waveform or a stack of them; the time axis is the last [any].
    window_samples : int
        Width of the trend window in samples; ``1`` or less removes nothing [-].

    Returns
    -------
    np.ndarray
        ``data`` minus its rolling mean, same shape [any].

    Convention
    ----------
    The trend is a centred rolling mean with ``min_periods=1`` semantics: the
    window at index ``i`` spans ``[i - w//2, i + (w-1)//2]`` clipped to the
    record, so edge windows shrink rather than yield NaN and the output has the
    input's length.  This is exactly ``pandas.Series.rolling(window,
    center=True, min_periods=1).mean()``, the convention of the validated VEST
    SXR viewer [SXR]_, reimplemented with cumulative sums so that
    :mod:`vaft.process.soft_x_rays` does not need pandas for it.

    Applicability
    -------------
    Machine-independent.

    Limitations
    -----------
    A centred window is zero-phase but the shrinking edge windows bias the
    first and last ``w/2`` samples toward the local level.  The trend is a
    plain mean: one large excursion inside the window pulls the trend toward
    it and appears, inverted, in the neighbours.
    """
    values = np.asarray(data, dtype=float)
    window = int(window_samples)
    if window <= 1 or values.shape[-1] == 0:
        return values - values  # zero trend removal, preserving shape/dtype

    length = values.shape[-1]
    cumsum = np.zeros(values.shape[:-1] + (length + 1,), dtype=float)
    np.cumsum(values, axis=-1, out=cumsum[..., 1:])
    # A centered pandas window of size w at index i spans
    # [i - w//2, i + (w-1)//2], clipped to the record.
    index = np.arange(length)
    start = np.clip(index - window // 2, 0, length)
    stop = np.clip(index + (window - 1) // 2 + 1, 0, length)
    trend = (cumsum[..., stop] - cumsum[..., start]) / (stop - start)
    return values - trend

Subtract a centered moving-average trend along the last axis.

NameTypeUnitDescription
dataarray_likeany

The waveform or a stack of them; the time axis is the last.

window_samplesint-

Width of the trend window in samples; 1 or less removes nothing.

Returns
—np.ndarrayany

data minus its rolling mean, same shape.

Convention

The trend is a centred rolling mean with min_periods=1 semantics: the window at index i spans [i - w//2, i + (w-1)//2] clipped to the record, so edge windows shrink rather than yield NaN and the output has the input’s length. This is exactly pandas.Series.rolling(window, center=True, min_periods=1).mean(), the convention of the validated VEST SXR viewer [SXR], reimplemented with cumulative sums so that vaft.process.soft_x_rays does not need pandas for it.

Applicability · Limitations

Applicability

Machine-independent.

Limitations

A centred window is zero-phase but the shrinking edge windows bias the first and last w/2 samples toward the local level. The trend is a plain mean: one large excursion inside the window pulls the trend toward it and appears, inverted, in the neighbours.

exp_baseline

Machine-independent source
exp_baseline(x, a, b, c)
Show source vaft/process/signal_processing.py, lines 1822–1845
def exp_baseline(x, a, b, c):
    """Exponential model for baseline fitting: ``y = a * exp(b * x) + c``.

    Parameters
    ----------
    x : array_like
        Abscissa, normally time [any].
    a : float
        Amplitude [any].
    b : float
        Rate; the sign decides growth or decay [any].
    c : float
        Offset [any].

    Returns
    -------
    np.ndarray
        ``a * exp(b * x) + c`` [any].

    Applicability
    -------------
    Machine-independent.  A fit model for :func:`subtract_baseline`.
    """
    return a * np.exp(b * x) + c

Exponential model for baseline fitting: y = a * exp(b * x) + c.

NameTypeUnitDescription
xarray_likeany

Abscissa, normally time.

afloatany

Amplitude.

bfloatany

Rate; the sign decides growth or decay.

cfloatany

Offset.

Returns
—np.ndarrayany

a * exp(b * x) + c.

Applicability

Applicability

Machine-independent. A fit model for subtract_baseline.

fir_filter

Machine-independentConvention-sensitive source
fir_filter(data, cutoff, fs, *, kind='lowpass', numtaps=None, window='hann', zero_phase=True)
Show source vaft/process/signal_processing.py, lines 668–741
def fir_filter(
    data,
    cutoff,
    fs: float,
    *,
    kind: str = "lowpass",
    numtaps: int | None = None,
    window: str = "hann",
    zero_phase: bool = True,
) -> np.ndarray:
    """Linear-phase FIR filter along the last axis.

    Parameters
    ----------
    data : array_like
        The waveform or a stack of them; the time axis is the last [any].
    cutoff : float or sequence of float
        Band edge, or the pair of edges for ``bandpass``/``bandstop`` [Hz].
    fs : float
        Sample rate [Hz].
    kind : str, optional
        ``lowpass``, ``highpass``, ``bandpass`` or ``bandstop`` [-].
    numtaps : int or None, optional
        Filter length; ``None`` derives it from ``fs``
        (:func:`fir_filter_coefficients`) [-].
    window : str, optional
        Window passed to :func:`scipy.signal.firwin` [-].
    zero_phase : bool, optional
        ``True`` for forward-backward ``filtfilt``; ``False`` for a causal
        ``lfilter`` that keeps the filter's own group delay [-].

    Returns
    -------
    np.ndarray
        The filtered waveform, same shape as ``data`` [any].

    Raises
    ------
    ValueError
        As :func:`fir_filter_coefficients`.

    Defaults
    --------
    ``zero_phase = True`` is a validated-workflow default matching
    :func:`butterworth_bandpass`: a filtered fluctuation signal feeds spectral
    and mode-number analysis, where phase across channels is the measurement.

    Convention
    ----------
    An FIR filter of this design is already linear-phase, so a causal
    application shifts every frequency by the same ``(numtaps - 1) / 2``
    samples and the delay can be removed exactly -- unlike an IIR filter, whose
    phase distortion is frequency-dependent and only ``filtfilt`` removes it.
    That is the reason to reach for FIR here rather than
    :func:`butterworth_bandpass`: the cost is a longer impulse response and
    more arithmetic per sample.

    Applicability
    -------------
    Machine-independent.

    Limitations
    -----------
    ``filtfilt`` needs a record longer than roughly three times the tap count
    and raises from SciPy otherwise; a one-millisecond default response is long
    at a high sample rate.
    """
    values = np.asarray(data, dtype=float)
    taps = fir_filter_coefficients(
        cutoff, fs, kind=kind, numtaps=numtaps, window=window
    )
    if zero_phase:
        return scipy_signal.filtfilt(taps, 1.0, values, axis=-1)
    return scipy_signal.lfilter(taps, 1.0, values, axis=-1)

Linear-phase FIR filter along the last axis.

NameTypeUnitDescription
dataarray_likeany

The waveform or a stack of them; the time axis is the last.

cutofffloat or sequence of floatHz

Band edge, or the pair of edges for bandpass/bandstop.

fsfloatHz

Sample rate.

kindstr, optional-

lowpass, highpass, bandpass or bandstop.

numtapsint or None, optional-

Filter length; None derives it from fs (fir_filter_coefficients).

windowstr, optional-

Window passed to scipy.signal.firwin.

zero_phasebool, optional-

True for forward-backward filtfilt; False for a causal lfilter that keeps the filter’s own group delay.

Returns
—np.ndarrayany

The filtered waveform, same shape as data.

Convention

An FIR filter of this design is already linear-phase, so a causal application shifts every frequency by the same (numtaps - 1) / 2 samples and the delay can be removed exactly – unlike an IIR filter, whose phase distortion is frequency-dependent and only filtfilt removes it. That is the reason to reach for FIR here rather than butterworth_bandpass: the cost is a longer impulse response and more arithmetic per sample.

Raises

  • ValueError As fir_filter_coefficients.
Defaults · Applicability · Limitations

Defaults

zero_phase = True is a validated-workflow default matching butterworth_bandpass: a filtered fluctuation signal feeds spectral and mode-number analysis, where phase across channels is the measurement.

Applicability

Machine-independent.

Limitations

filtfilt needs a record longer than roughly three times the tap count and raises from SciPy otherwise; a one-millisecond default response is long at a high sample rate.

fir_filter_coefficients

Machine-independentConvention-sensitive source
fir_filter_coefficients(cutoff, fs, *, kind='lowpass', numtaps=None, window='hann')

Window-method FIR coefficients for a linear-phase filter.

NameTypeUnitDescription
cutofffloat or sequence of floatHz

Band edge, or the pair of edges for bandpass/bandstop.

fsfloatHz

Sample rate.

kindstr, optional-

lowpass, highpass, bandpass or bandstop.

numtapsint or None, optional-

Filter length; None derives it from fs (see Defaults).

windowstr, optional-

Window passed to scipy.signal.firwin.

Returns
—np.ndarray-

numtaps coefficients of a symmetric (Type I) FIR filter.

Convention

The length is always odd, so the filter is Type I: symmetric with an integer group delay of (numtaps - 1) / 2 samples, and with a response that is not forced to zero at the Nyquist frequency – which a Type II (even-length) filter would be, making a high-pass or band-stop unusable.

Raises

  • ValueError kind is not one of the four supported responses, the number of band edges does not match it, or an edge is outside 0 < f < fs / 2.
Defaults · Applicability · Limitations

Defaults

numtaps = None and window = "hann" are validated-workflow defaults, taken from VEST’s fluctuation analysis. The length is an impulse response one millisecond long, int(fs * 1e-3) forced odd, which ties the transition width to the sample rate rather than to a fixed tap count.

Applicability

Machine-independent.

Limitations

The window method sets the transition width by the tap count alone; it does not meet a stated ripple specification the way an equiripple design would.

infer_signal_orientation

Machine-independentConvention-sensitive source
infer_signal_orientation(signal, *, mask=None, active_fraction=0.1, min_samples=8, agreement=0.6)

Infer the dominant sign of signal from its active region.

The region is mask when given, otherwise the samples whose magnitude reaches active_fraction of the signal’s peak magnitude – so the long zero stretches before and after a discharge do not vote. The verdict is the sign of the region’s median, resolved only when at least agreement of the region shares it and the region holds at least min_samples finite samples; otherwise it is unresolved and the multiplier is +1 (canonical), never a guess. abs is never applied to the data and no sample is flipped on its own.

NameTypeUnitDescription
signalarray_likeany

The waveform, one-dimensional; non-finite samples are ignored.

maskarray_like of bool or None, optional-

The region to vote over, instead of the magnitude rule.

active_fractionfloat, optional-

Fraction of the peak magnitude a sample must reach to be in the region

min_samplesint, optional-

Fewest finite samples the region may hold for a verdict.

agreementfloat, optional-

Smallest fraction of the region that must share the median’s sign.

Returns
—SignalOrientation-

multiplier (+1 or -1 when resolved, else +1), the median statistic, the region’s count, and a reason when unresolved.

Convention

The multiplier is meant to be applied to the whole signal once, for display or for a sign-convention check. It never rectifies samples.

Raises

  • ValueError signal is not one-dimensional.
Defaults · Applicability · Limitations · Provenance

Defaults

active_fraction = 0.1, min_samples = 8 and agreement = 0.6 are empirical estimates fixed when #307 was implemented; #307 sets the rule (median of an active region, never abs, never per-sample) but records no derivation for the three numbers.

Applicability

Machine-independent.

Limitations

A signal that is genuinely bipolar over its active region – a Mirnov fluctuation, a flux loop through a reversal – is correctly left unresolved, which the caller must handle rather than assume.

Provenance

  1. Issue #307, which established the rule and its defaults.

is_signal_active

Machine-independent source
is_signal_active(data, var_ratio_thresh=0.01, change_ratio_thresh=0.01, verbose=False)
Show source vaft/process/signal_processing.py, lines 2034–2101
def is_signal_active(
    data,
    var_ratio_thresh=1e-2,
    change_ratio_thresh=1e-2,
    verbose=False,
):
    """Decide whether a trace carries a signal or is flat.

    Both tests are relative to the trace's own level, so the verdict does not
    depend on the signal's units or gain.

    Parameters
    ----------
    data : array_like
        The waveform; fewer than two samples is never active [any].
    var_ratio_thresh : float, optional
        Threshold on the variance divided by the squared mean absolute level,
        the squared coefficient of variation [-].
    change_ratio_thresh : float, optional
        Threshold on the mean absolute sample-to-sample change divided by the
        mean absolute level [-].
    verbose : bool, optional
        Print both ratios [-].

    Returns
    -------
    bool
        ``True`` unless *both* ratios fall below their thresholds [-].

    Defaults
    --------
    Both thresholds at ``1e-2`` are legacy compatibility values; no derivation
    is recorded.

    Applicability
    -------------
    Machine-independent.

    Limitations
    -----------
    A trace held at a non-zero level with noise far below that level is
    inactive, whatever the level.  The converse is the caveat: the test is
    scale-free by design and has no absolute floor, so a zero-mean noise
    trace scores near 1 on both ratios and is reported active at any
    amplitude.  A caller that knows a noise floor must apply it before
    asking.  An identically zero trace is inactive.
    """
    data = np.asarray(data)

    if data.size < 2:
        return False

    variance = np.var(data)
    mean_abs_change = np.mean(np.abs(np.diff(data)))
    level = np.mean(np.abs(data))

    # Both ratios are relative to the magnitude of the trace (#463).  An
    # identically zero trace has no level to be relative to and is inactive.
    var_ratio = variance / level**2 if level > 0.0 else 0.0
    change_ratio = mean_abs_change / (level + 1e-12)

    if verbose:
        print(f"Variance ratio: {var_ratio:.3e} (thresh={var_ratio_thresh:.3e})")
        print(f"Mean |Δx| ratio: {change_ratio:.3e} (thresh={change_ratio_thresh:.3e})")

    if var_ratio < var_ratio_thresh and change_ratio < change_ratio_thresh:
        return False
    return True

Decide whether a trace carries a signal or is flat.

Both tests are relative to the trace’s own level, so the verdict does not depend on the signal’s units or gain.

NameTypeUnitDescription
dataarray_likeany

The waveform; fewer than two samples is never active.

var_ratio_threshfloat, optional-

Threshold on the variance divided by the squared mean absolute level, the squared coefficient of variation.

change_ratio_threshfloat, optional-

Threshold on the mean absolute sample-to-sample change divided by the mean absolute level.

verbosebool, optional-

Print both ratios.

Returns
—bool-

True unless both ratios fall below their thresholds.

Defaults · Applicability · Limitations

Defaults

Both thresholds at 1e-2 are legacy compatibility values; no derivation is recorded.

Applicability

Machine-independent.

Limitations

A trace held at a non-zero level with noise far below that level is inactive, whatever the level. The converse is the caveat: the test is scale-free by design and has no absolute floor, so a zero-mean noise trace scores near 1 on both ratios and is reported active at any amplitude. A caller that knows a noise floor must apply it before asking. An identically zero trace is inactive.

line_average_density

Machine-independentConvention-sensitive source
line_average_density(n_e_line, path_length_m)
Show source vaft/process/signal_processing.py, lines 334–368
def line_average_density(n_e_line, path_length_m: float) -> np.ndarray:
    """Return line-average electron density given an explicit chord length.

    Parameters
    ----------
    n_e_line : array_like
        Line-integrated electron density along the diagnostic chord [m^-2].
    path_length_m : float
        Length of the chord through the plasma [m].

    Returns
    -------
    np.ndarray
        Line-average electron density, ``n_e_line / path_length_m`` [m^-3].

    Raises
    ------
    ValueError
        ``path_length_m`` is not positive.

    Convention
    ----------
    Line-*integrated* in, line-*averaged* out.  An interferometer measures the
    former; the latter is what confinement scalings and Greenwald fractions
    want.  No calibration or geometry is inferred here -- the chord length is
    the caller's.

    Applicability
    -------------
    Machine-independent.  The chord length of a VEST interferometer channel
    comes from :mod:`vaft.machine_mapping`.
    """
    if path_length_m <= 0:
        raise ValueError(f"path_length_m must be positive, got {path_length_m}")
    return np.asarray(n_e_line, dtype=float) / float(path_length_m)

Return line-average electron density given an explicit chord length.

NameTypeUnitDescription
n_e_linearray_likem^-2

Line-integrated electron density along the diagnostic chord.

path_length_mfloatm

Length of the chord through the plasma.

Returns
—np.ndarraym^-3

Line-average electron density, n_e_line / path_length_m.

Convention

Line-integrated in, line-averaged out. An interferometer measures the former; the latter is what confinement scalings and Greenwald fractions want. No calibration or geometry is inferred here – the chord length is the caller’s.

Raises

  • ValueError path_length_m is not positive.
Applicability

Applicability

Machine-independent. The chord length of a VEST interferometer channel comes from vaft.machine_mapping.

linear_baseline

Machine-independent source
linear_baseline(x, a, b)
Show source vaft/process/signal_processing.py, lines 1774–1795
def linear_baseline(x, a, b):
    """Linear model for baseline fitting: ``y = a * x + b``.

    Parameters
    ----------
    x : array_like
        Abscissa, normally time [any].
    a : float
        Slope [any].
    b : float
        Intercept [any].

    Returns
    -------
    np.ndarray
        ``a * x + b`` [any].

    Applicability
    -------------
    Machine-independent.  A fit model for :func:`subtract_baseline`.
    """
    return a * x + b

Linear model for baseline fitting: y = a * x + b.

NameTypeUnitDescription
xarray_likeany

Abscissa, normally time.

afloatany

Slope.

bfloatany

Intercept.

Returns
—np.ndarrayany

a * x + b.

Applicability

Applicability

Machine-independent. A fit model for subtract_baseline.

process_signal

VEST-specificConvention-sensitive source
process_signal(time, data, options=None)

Legacy conditioning wrapper: crop, resample, then filter.

NameTypeUnitDescription
timearray_likes

Sample times, flattened.

dataarray_likeany

The waveform, flattened.

optionsdict or None, optional-

time_range (t_start, t_end) to crop to; resample with dt to write onto a uniform grid; filter_params with type ("lowpass", "highpass", "bandpass"), cutoff and order.

Returns
—tuple of (np.ndarray, np.ndarray)any

The conditioned (time, data).

Convention

Zero-phase (filtfilt) for the shaping filter. The sample rate for filter design is taken from the first two samples, not the median.

Raises

  • ValueError A filter_params cutoff outside (0, f_s / 2), a malformed band-pass cutoff, or an unknown filter type.
Processing steps · Defaults · Applicability · Limitations

Processing steps

  1. Crop to time_range.
  2. Resample onto arange(t[0], t[-1], dt) through resample_to_time, so a rate reduction anti-aliases on the input grid first.
  3. Apply a zero-phase Butterworth from filter_params. This is a shaping filter and its cutoff is interpreted against the output grid’s sample rate, not the input’s.

Defaults

dt = 4e-5 s is the VEST slow-DAQ spacing, a machine-specific setting. cutoff = 1000 Hz and order = 4 are legacy compatibility values whose origin is not recorded.

Applicability

VEST-specific. The defaults embed the VEST acquisition grid, and the option dictionary is the legacy call convention kept for existing workflows; new code should call resample_to_time and butterworth_lowpass directly.

Limitations

Kept for compatibility; its future is decided by the module-ownership audit tracked in #263.

quadratic_baseline

Machine-independent source
quadratic_baseline(x, a, b, c)
Show source vaft/process/signal_processing.py, lines 1797–1820
def quadratic_baseline(x, a, b, c):
    """Quadratic model for baseline fitting: ``y = a * x^2 + b * x + c``.

    Parameters
    ----------
    x : array_like
        Abscissa, normally time [any].
    a : float
        Quadratic coefficient [any].
    b : float
        Linear coefficient [any].
    c : float
        Constant [any].

    Returns
    -------
    np.ndarray
        ``a * x**2 + b * x + c`` [any].

    Applicability
    -------------
    Machine-independent.  A fit model for :func:`subtract_baseline`.
    """
    return a * x**2 + b * x + c

Quadratic model for baseline fitting: y = a * x^2 + b * x + c.

NameTypeUnitDescription
xarray_likeany

Abscissa, normally time.

afloatany

Quadratic coefficient.

bfloatany

Linear coefficient.

cfloatany

Constant.

Returns
—np.ndarrayany

a * x**2 + b * x + c.

Applicability

Applicability

Machine-independent. A fit model for subtract_baseline.

repair_clipped_interval

Machine-independent source
repair_clipped_interval(time, data, *, clip_value, tolerance, min_support=4, return_mask=False)

Reconstruct samples saturated at an acquisition limit by interpolation.

Samples within tolerance of clip_value are treated as saturated and replaced by a cubic spline fitted to the remaining samples on the physical time axis. Every unsaturated sample is preserved exactly.

clip_value may be a single signed level or a sequence of levels, in which case saturation is their union (see detect_clipped_samples).

With return_mask=True the saturation mask is returned alongside the repaired waveform, so a caller can report which samples it reconstructed instead of handing downstream consumers an unmarked mixture.

NameTypeUnitDescription
timearray_likes

Sample times, one-dimensional, same length as data.

dataarray_likeany

The waveform with saturated samples, one-dimensional.

clip_valuefloat or sequence of floatany

The acquisition limit or limits, signed, in the units of data.

tolerancefloatany

Half-width of the band around each limit inside which a sample counts as saturated.

min_supportint, optional-

Fewest unsaturated samples the spline may be fitted through.

return_maskbool, optional-

Also return the saturation mask.

Returns
—np.ndarray or tuple of (np.ndarray, np.ndarray)any

The repaired waveform, and with return_mask=True also the boolean saturation mask.

Raises

  • SignalRepairError The inputs differ in shape or are not one-dimensional; any sample is non-finite; the whole waveform is saturated; fewer than min_support unsaturated samples remain; or the saturated interval reaches either end of the record, which would require extrapolation.
Processing steps · Input semantics · Output semantics · Defaults · Assumptions · Applicability · Limitations · Provenance

Processing steps

  1. Mask the saturated samples with detect_clipped_samples.
  2. Refuse if the mask covers everything, leaves fewer than min_support samples, or touches either end of the record.
  3. Fit scipy.interpolate.CubicSpline through the unsaturated samples against time.
  4. Evaluate it at the saturated instants and write those samples back.

Input semantics

Raw or calibrated – the rails must be expressed in the same units as data. Unrepaired: a waveform that has already been filtered has smeared its rails and cannot be repaired here.

Output semantics

Repaired. The reconstructed samples are interpolated, not measured; the mask says which they are.

Defaults

min_support = 4 is a numerical convenience: a cubic spline needs four points to be determined at all, and this is the floor, not a quality bar.

Assumptions

The waveform is smooth on the scale of the saturated interval, so a cubic through its neighbours is a defensible estimate of what was lost. A saturated interval that hides a genuine fast feature is reconstructed as if the feature were absent.

Applicability

Machine-independent. Callers supply the limit and tolerance. On VEST the PF6 supply clips near -5000 A and the repair is enabled per acquisition era in vest.yaml, because PF6’s gain flips sign at shot 38110 and before that a -5000 A sample is ordinary data [1].

Limitations

Interpolation only: a saturated run at the start or end of the record is refused rather than extrapolated. Reconstructing more than a few samples in a row grows less defensible with every sample; there is no cap on that here, so the caller should look at the mask.

Provenance

  1. vest.yaml, pf_active processing saturation_repair and the per-era policy comment; the donor vest_pf.m applied the -5000 A repair unconditionally. Issues #195 and #285.

resample_to_time

Machine-independentConvention-sensitive source
resample_to_time(source_time, values, target_time, *, anti_alias='auto', cutoff_fraction=0.8, cutoff_hz=None, numtaps=None, min_ratio=1.05, extrapolate='clamp', nan_policy='segment', on_unsorted='error', axis=-1, return_filter_mask=False)

Project a signal onto target_time, low-passing first if that is a rate cut.

Use this instead of a bare np.interp anywhere a diagnostic is written onto a common time grid. Interpolation and downsampling look identical in source but are not the same operation: evaluating a signal at new instants is always fine, whereas reducing the sample rate discards the information needed to distinguish a high frequency from its alias, so the content above the new Nyquist has to be removed while it still can be.

NameTypeUnitDescription
source_timearray_likes

Sample times of values, flattened; must be strictly increasing under the default on_unsorted.

valuesarray_likeany

The waveform or a stack of them; the time axis is axis.

target_timearray_likes

Instants to evaluate at; need not be sorted.

anti_aliasbool or "auto", optional-

"auto" filters only for a genuine rate reduction; True always; False never.

cutoff_fractionfloat, optional-

Anti-alias cutoff as a fraction of the target Nyquist.

cutoff_hzfloat or None, optionalHz

Explicit cutoff, overriding cutoff_fraction.

numtapsint or None, optional-

Passed to anti_alias_filter.

min_ratiofloat, optional-

dt_target / dt_source at or below which "auto" does not filter.

extrapolate{"clamp", "nan", "error"}, optional-

What a target sample outside the source span gets.

nan_policystr, optional-

Passed to anti_alias_filter.

on_unsorted{"error", "sort"}, optional-

Refuse an unsorted source, or sort it and collapse duplicates.

axisint, optional-

The time axis of values.

return_filter_maskbool, optional-

Also return, on the source grid, which samples reached the target band-limited.

Returns
—np.ndarrayany

values on target_time; the time axis has len(target_time) samples.

—np.ndarray of bool-

Only with return_filter_mask=True (which requires on_unsorted="error"): per source sample, True if no filter was needed or anti_alias_filter covered it, False if a filter was needed but the sample was left unfiltered or is non-finite. It is a logical series: project it with anti_alias=False.

Convention

The cutoff is placed against the target grid’s spacing measured in time order, even when target_time is unsorted; the shuffled array would report a median dt tens of times too large and design a filter that eats the signal. An explicit cutoff_hz above the target Nyquist is honoured as a deliberate choice to keep more band and gets anti_alias_filter’s own transition instead.

Raises

  • ResamplingError Empty source; unsorted or duplicated source times under the default policy; a target outside the source span under extrapolate="error"; a forced filter whose cutoff exceeds the source Nyquist; or a filter requested on a non-uniform source grid.
  • ValueError An option outside its allowed values.
Processing steps · Output semantics · Defaults · Assumptions · Applicability · Limitations · Provenance

Processing steps

  1. Validate options; sort and de-duplicate the source if asked.
  2. Measure both grids with describe_time_grid.
  3. Decide whether to filter: anti_alias as given, or under "auto" whether dt_target / dt_source > min_ratio.
  4. If filtering, refuse a non-uniform source, place the cutoff at cutoff_fraction of the target Nyquist (or cutoff_hz), put the stopband at the target Nyquist, and run anti_alias_filter on the source grid.
  5. np.interp onto target_time with the extrapolate policy.

Output semantics

Interpolated. When no filter runs, the result is bit-for-bit np.interp – deliberately, so equal-rate call sites can adopt this primitive without moving a stored value. When one does, the result is band-limited to the target Nyquist and then interpolated.

Defaults

anti_alias = "auto" and min_ratio = 1.05 are validated-workflow defaults: the safe thing by default, the unsafe thing by name. cutoff_fraction = 0.8 is a numerical convenience – firwin sits at -6 dB at its cutoff and needs a transition band, so a passband edge below Nyquist puts everything that can fold into the stopband. extrapolate = "clamp" matches np.interp. on_unsorted = "error" because np.interp returns silent nonsense for an unsorted x.

Assumptions

values is a band-limited physical signal. A validity mask or any other logical series is not, and must be resampled with anti_alias=False.

Applicability

Machine-independent. The motivating case is VEST, which acquires at FAST_DT = 4e-6 s and SLOW_DT = 4e-5 s while every processed grid in vest.yaml is 4e-5 s, so a fast channel written onto a policy grid is a 10x decimation [1].

Limitations

A single-sample target has no Nyquist: "auto" interpolates with a warning and True raises unless cutoff_hz is given. A non-uniform source grid cannot be anti-aliased by a single-rate FIR and is refused; resample onto a uniform grid first, or pass anti_alias=False to accept a bare interpolation knowingly.

Provenance

  1. vaft.database.raw FAST_DT / SLOW_DT, and the processed time grids declared in vaft/machine_mapping/vest.yaml.

signal_on_offset

Machine-independentConvention-sensitive source
signal_on_offset(time, data, smooth_window=5, threshold=0.01, verbose=False)
Show source vaft/process/signal_processing.py, lines 1939–1995
def signal_on_offset(time, data, smooth_window=5, threshold=0.01, verbose=False):
    """Smooth a waveform, then return the window in which it is above a threshold.

    Parameters
    ----------
    time : array_like
        Sample times, one-dimensional [s].
    data : array_like
        The waveform, one-dimensional, same length as ``time`` [any].
    smooth_window : int, optional
        Savitzky-Golay window length in samples; must be odd and larger than
        the polynomial order, 3 [-].
    threshold : float, optional
        Level at or above which the smoothed signal counts as active, in the
        units of ``data`` [any].
    verbose : bool, optional
        Print the threshold [-].

    Returns
    -------
    tuple of (float, float)
        ``(onset, offset)`` times [s].

    Processing steps
    ----------------
    1. Smooth with ``scipy.signal.savgol_filter(data, smooth_window, 3)``.
    2. Hand the smoothed waveform to :func:`detect_active_window`.

    Defaults
    --------
    ``smooth_window = 5`` and the cubic polynomial are legacy compatibility
    values from the ported workflow, unrecorded beyond that.  ``threshold =
    0.01`` is inherited from :func:`detect_active_window`.

    Convention
    ----------
    Zero-phase: a Savitzky-Golay filter is a centred polynomial fit and moves
    nothing in time, so the returned onset is not delayed by the smoothing.

    Applicability
    -------------
    Machine-independent.  Callers in :mod:`vaft.omas.general` and
    :mod:`vaft.database` pass ``threshold=0.05`` for VEST H-alpha; the default
    is not a VEST policy.  For a plasma onset with evidence and a verdict, use
    :mod:`vaft.process.onset`.

    Limitations
    -----------
    Those of :func:`detect_active_window`: an absolute threshold and a walk
    that stops at the first sample below it.
    """

    if verbose:
        print("threshold for signal detection:", threshold)
    # Smooth the data
    smoothed = scipy_signal.savgol_filter(data, smooth_window, 3)
    return detect_active_window(time, smoothed, threshold=threshold)

Smooth a waveform, then return the window in which it is above a threshold.

NameTypeUnitDescription
timearray_likes

Sample times, one-dimensional.

dataarray_likeany

The waveform, one-dimensional, same length as time.

smooth_windowint, optional-

Savitzky-Golay window length in samples; must be odd and larger than the polynomial order, 3.

thresholdfloat, optionalany

Level at or above which the smoothed signal counts as active, in the units of data.

verbosebool, optional-

Print the threshold.

Returns
—tuple of (float, float)s

(onset, offset) times.

Convention

Zero-phase: a Savitzky-Golay filter is a centred polynomial fit and moves nothing in time, so the returned onset is not delayed by the smoothing.

Processing steps · Defaults · Applicability · Limitations

Processing steps

  1. Smooth with scipy.signal.savgol_filter(data, smooth_window, 3).
  2. Hand the smoothed waveform to detect_active_window.

Defaults

smooth_window = 5 and the cubic polynomial are legacy compatibility values from the ported workflow, unrecorded beyond that. threshold = 0.01 is inherited from detect_active_window.

Applicability

Machine-independent. Callers in vaft.omas.general and vaft.database pass threshold=0.05 for VEST H-alpha; the default is not a VEST policy. For a plasma onset with evidence and a verdict, use vaft.process.onset.

Limitations

Those of detect_active_window: an absolute threshold and a walk that stops at the first sample below it.

signal_onoffset

Deprecated source
signal_onoffset(time, data, smooth_window=5, threshold=0.01, verbose=False)
Show source vaft/process/signal_processing.py, lines 2019–2032
def signal_onoffset(time, data, smooth_window=5, threshold=0.01, verbose=False):
    """Deprecated compatibility wrapper for :func:`signal_on_offset`."""
    warnings.warn(
        "signal_onoffset() is deprecated; use signal_on_offset().",
        DeprecationWarning,
        stacklevel=2,
    )
    return signal_on_offset(
        time,
        data,
        smooth_window=smooth_window,
        threshold=threshold,
        verbose=verbose,
    )

Deprecated compatibility wrapper for signal_on_offset.

smooth

Machine-independentConvention-sensitive source
smooth(array, span)
Show source vaft/process/signal_processing.py, lines 371–446
def smooth(array, span: int) -> np.ndarray:
    """Apply MATLAB-like moving-average smoothing with edge tapering.

    A centred moving average of odd width ``span``.  At each end the window
    shrinks symmetrically -- widths 1, 3, 5, ... -- so the output has the
    input's length and no sample is padded, which is what MATLAB's ``smooth``
    does and what the ported VEST workflows expect.

    Parameters
    ----------
    array : array_like
        The waveform, one-dimensional [any].
    span : int
        Window width in samples; an even value is reduced by one, and a value
        wider than the record is clipped to it [-].

    Returns
    -------
    np.ndarray
        The smoothed waveform, same length as ``array`` [any].

    Raises
    ------
    ValueError
        ``array`` is not one-dimensional.

    Convention
    ----------
    Zero-phase: the window is centred, so features are not shifted in time.
    Edge samples are averaged over progressively narrower windows rather than
    over a padded record, which biases the ends less than reflection padding
    would but leaves them noisier than the interior.

    Applicability
    -------------
    Machine-independent.

    Provenance
    ----------
    .. [1] MATLAB ``smooth(y, span)`` with the default ``'moving'`` method;
       ported from the legacy VEST workflow [VFIT]_ with its semantics
       preserved so that results match the donor sample for sample.
    """
    values = np.asarray(array, dtype=float)
    if values.ndim != 1:
        raise ValueError("`array` must be one-dimensional.")
    if values.size == 0 or span <= 1:
        return values.copy()
    if span % 2 == 0:
        span -= 1
    span = min(span, values.size if values.size % 2 == 1 else values.size - 1)
    if span < 1:
        return values.copy()

    count = values.size
    out = np.zeros(count, dtype=float)
    half = (span - 1) // 2

    # O(N) vectorized implementation using cumulative sums.
    cumsum = np.zeros(count + 1, dtype=float)
    cumsum[1:] = np.cumsum(values)

    # Interior: full-width windows of size `span`.
    interior = np.arange(half, count - half)
    if interior.size:
        out[interior] = (cumsum[interior + half + 1] - cumsum[interior - half]) / span

    # Edge tapering: left and right sides use progressively narrower windows.
    if half > 0:
        left_idx = np.arange(half)
        widths = 2 * left_idx + 1
        out[left_idx] = (cumsum[widths] - cumsum[0]) / widths
        right_idx = count - 1 - left_idx
        out[right_idx] = (cumsum[count] - cumsum[count - widths]) / widths

    return out

Apply MATLAB-like moving-average smoothing with edge tapering.

A centred moving average of odd width span. At each end the window shrinks symmetrically – widths 1, 3, 5, … – so the output has the input’s length and no sample is padded, which is what MATLAB’s smooth does and what the ported VEST workflows expect.

NameTypeUnitDescription
arrayarray_likeany

The waveform, one-dimensional.

spanint-

Window width in samples; an even value is reduced by one, and a value wider than the record is clipped to it.

Returns
—np.ndarrayany

The smoothed waveform, same length as array.

Convention

Zero-phase: the window is centred, so features are not shifted in time. Edge samples are averaged over progressively narrower windows rather than over a padded record, which biases the ends less than reflection padding would but leaves them noisier than the interior.

Raises

  • ValueError array is not one-dimensional.
Applicability · Provenance

Applicability

Machine-independent.

Provenance

  1. MATLAB smooth(y, span) with the default 'moving' method; ported from the legacy VEST workflow [VFIT] with its semantics preserved so that results match the donor sample for sample.

subtract_baseline

Machine-independent source
subtract_baseline(time, signal, baseline_indices, fitting_opt='linear')

Fit a baseline through the selected samples and subtract it everywhere.

NameTypeUnitDescription
timenp.ndarrays

Sample times.

signalnp.ndarrayany

The waveform, same length as time.

baseline_indicesnp.ndarray-

Indices of the samples the baseline is fitted through, normally from define_baseline.

fitting_opt{"linear", "quadratic", "spline", "exp"}, optional-

The baseline model.

Returns
correctednp.ndarrayany

signal minus the fitted baseline.

baselinenp.ndarrayany

The fitted baseline evaluated on the whole time axis.

Raises

  • ValueError Unknown fitting_opt.
Processing steps · Output semantics · Defaults · Assumptions · Applicability · Limitations · Provenance

Processing steps

  1. Take time and signal at baseline_indices.
  2. Fit the chosen model through them: scipy.optimize.curve_fit for the parametric models, scipy.interpolate.UnivariateSpline(s=0) – an interpolating spline – for "spline".
  3. Evaluate the model on the full time axis and subtract.

Output semantics

Baseline-subtracted. The second return is the baseline itself, so the caller can plot or store what was removed.

Defaults

fitting_opt = "linear" is a legacy compatibility value: the donor workflow removed a linear drift, which is the right model for an integrator’s offset. The exponential fit uses maxfev = 10000, a numerical convenience.

Assumptions

The samples at baseline_indices contain no signal. The parametric models assume the drift has that form over the whole record, including the pulse, where it is extrapolated rather than measured.

Applicability

Machine-independent.

Limitations

"spline" interpolates the baseline samples exactly, so noise in them is reproduced in the baseline; with the onset and offset regions far apart it is a straight line between them in practice. "exp" can fail to converge and raises from SciPy. Nothing checks that baseline_indices is non-empty.

Provenance

  1. The legacy VEST EFIT magnetics chain [VFIT], where an integrated probe signal’s drift is removed by a fit through the pre- and post-discharge windows.

vest_coil_current_noise_reduction

VEST-specificConvention-sensitive source
vest_coil_current_noise_reduction(data)
Show source vaft/process/signal_processing.py, lines 797–863
def vest_coil_current_noise_reduction(data) -> np.ndarray:
    """Suppress point spikes in coil current traces.

    Walks the record and, wherever a sample's magnitude exceeds the mean of
    its two neighbours' magnitudes by more than a fixed margin, replaces it
    with the previous sample.  A port of the donor routine, kept sample for
    sample so that the legacy PF-current preprocessing reproduces.

    Parameters
    ----------
    data : array_like
        Coil current, one-dimensional; records shorter than three samples are
        returned unchanged [A].

    Returns
    -------
    np.ndarray
        The de-spiked current, same length [A].

    Processing steps
    ----------------
    1. For each interior sample (the first two and the last are never
       touched), compute ``|x[i]| - |(x[i+1] + x[i-1]) / 2|``.
    2. Where that exceeds ``0.001``, overwrite ``x[i]`` with the already
       processed ``x[i-1]``.

    Defaults
    --------
    The ``0.001`` margin is a legacy compatibility value from the donor
    ``VEST_CoilCurrentNoiseReduction.m`` [1]_.  It is absolute, in the units of
    ``data``, and its derivation is not recorded.  On a trace in amperes it
    flags any excursion above a milliamp, so in practice every spike; on a
    trace in kiloamperes it would let real noise through.

    Convention
    ----------
    Causal: a spike is replaced by its predecessor, so a genuine step is
    delayed by one sample at its leading edge.

    Applicability
    -------------
    VEST-specific.  Written for the VEST PF coil current traces as digitized by
    the legacy DAQ; the margin has no meaning for another machine's units or
    noise floor.

    Limitations
    -----------
    Only a single-sample spike is caught: two consecutive bad samples pass,
    because the second is compared against the first.  Sign is ignored in the
    test but preserved in the replacement.

    Provenance
    ----------
    .. [1] ``Function/VEST_CoilCurrentNoiseReduction.m`` in the legacy VEST
       workflow [VFIT]_; loop bounds and margin preserved.
    """
    values = np.asarray(data, dtype=float)
    if values.size < 3:
        return values.copy()

    smoothed = values.copy()
    for index_i in range(2, values.size - 1):
        local_ref = abs((values[index_i + 1] + values[index_i - 1]) / 2.0)
        diff = abs(values[index_i]) - local_ref
        if diff > 0.001:
            smoothed[index_i] = smoothed[index_i - 1]
    return smoothed

Suppress point spikes in coil current traces.

Walks the record and, wherever a sample’s magnitude exceeds the mean of its two neighbours’ magnitudes by more than a fixed margin, replaces it with the previous sample. A port of the donor routine, kept sample for sample so that the legacy PF-current preprocessing reproduces.

NameTypeUnitDescription
dataarray_likeA

Coil current, one-dimensional; records shorter than three samples are returned unchanged.

Returns
—np.ndarrayA

The de-spiked current, same length.

Convention

Causal: a spike is replaced by its predecessor, so a genuine step is delayed by one sample at its leading edge.

Processing steps · Defaults · Applicability · Limitations · Provenance

Processing steps

  1. For each interior sample (the first two and the last are never touched), compute |x[i]| - |(x[i+1] + x[i-1]) / 2|.
  2. Where that exceeds 0.001, overwrite x[i] with the already processed x[i-1].

Defaults

The 0.001 margin is a legacy compatibility value from the donor VEST_CoilCurrentNoiseReduction.m [1]. It is absolute, in the units of data, and its derivation is not recorded. On a trace in amperes it flags any excursion above a milliamp, so in practice every spike; on a trace in kiloamperes it would let real noise through.

Applicability

VEST-specific. Written for the VEST PF coil current traces as digitized by the legacy DAQ; the margin has no meaning for another machine’s units or noise floor.

Limitations

Only a single-sample spike is caught: two consecutive bad samples pass, because the second is compared against the first. Sign is ignored in the test but preserved in the replacement.

Provenance

  1. Function/VEST_CoilCurrentNoiseReduction.m in the legacy VEST workflow [VFIT]; loop bounds and margin preserved.

vfit_signal_start_end

Deprecated source
vfit_signal_start_end(time, data, threshold=0.01)
Show source vaft/process/signal_processing.py, lines 950–957
def vfit_signal_start_end(time, data, threshold: float = 0.01) -> tuple[float, float]:
    """Deprecated compatibility wrapper for :func:`detect_active_window`."""
    warnings.warn(
        "vfit_signal_start_end() is deprecated; use detect_active_window().",
        DeprecationWarning,
        stacklevel=2,
    )
    return detect_active_window(time, data, threshold=threshold)

Deprecated compatibility wrapper for detect_active_window.

vfit_signal_startend

Deprecated source
vfit_signal_startend(time, data, threshold=0.01)
Show source vaft/process/signal_processing.py, lines 2009–2016
def vfit_signal_startend(time, data, threshold: float = 0.01):
    """Deprecated compatibility wrapper for :func:`detect_active_window`."""
    warnings.warn(
        "vfit_signal_startend() is deprecated; use detect_active_window().",
        DeprecationWarning,
        stacklevel=2,
    )
    return detect_active_window(time, data, threshold=threshold)

Deprecated compatibility wrapper for detect_active_window.

results matching ""

    No results matching ""