Process reference: signal_processing
Generated from the docstrings of vaft.process.signal_processing:
27 public functions. Each entry is what
vaft.process.describe("signal_processing.<name>") prints.
All process categories
Overview
Everything here takes NumPy arrays and returns NumPy arrays. Nothing reads
an ODS, nothing knows what the waveform is, and nothing embeds a VEST
constant except where the function’s name says so (vest_*) and its
Applicability section says why. The VEST source selection, calibration
and processing windows that decide which waveform to condition, and with
what numbers, live in vaft.machine_mapping and vest.yaml.
Four kinds of operation:
- Filtering –
smooth,butterworth_lowpass,butterworth_bandpass,detrend_moving_average,anti_alias_filter,vest_coil_current_noise_reduction. - Baseline –
define_baselineselects the quiet samples,subtract_baselinefits one oflinear_baseline,quadratic_baseline,exp_baselineor a spline through them and removes it. - Saturation repair –
detect_clipped_samples,repair_clipped_interval. - Timebase and activity –
describe_time_grid,resample_to_time,detect_active_window,signal_on_offset,is_signal_active,infer_signal_orientation,line_average_density, and the legacyprocess_signalwrapper.
| Symbol | Meaning | Unit |
|---|---|---|
| t | time | s |
| dt | sample spacing | s |
| f_s | sample rate, ``1 / dt`` | Hz |
| f_N | Nyquist frequency, ``f_s / 2`` | Hz |
| f_c | filter cutoff | Hz |
| x | the waveform, in whatever unit it has | any |
Two filtering conventions coexist and each function says which it uses.
Causal (scipy.signal.lfilter) delays every feature by the filter’s
group delay and is the convention of the validated VEST SXR viewer, where
matched delay between channels matters more than zero phase. Zero-phase
(scipy.signal.filtfilt, forward then backward) moves nothing in time
and is mandatory wherever an onset time is read off the result.
butterworth_lowpass defaults to causal, everything else to
zero-phase.
Functions
VEST_CoilCurrentNoiseReduction | Deprecated compatibility wrapper for the snake-case function. Deprecated |
anti_alias_filter | Zero-phase FIR low-pass applied on the source grid, ahead of a rate cut. Convention |
butterworth_bandpass | Butterworth band-pass filter along the last axis. Convention |
butterworth_lowpass | Butterworth low-pass filter along the last axis. Convention |
define_baseline | Select the baseline samples around a signal’s onset and optional offset. |
describe_time_grid | Measure a timebase’s spacing, uniformity and monotonicity. Convention |
detect_active_window | Return the active time window that contains a signal’s main peak. |
detect_clipped_samples | Return a boolean mask of samples sitting at an acquisition limit. |
detrend_moving_average | Subtract a centered moving-average trend along the last axis. Convention |
exp_baseline | Exponential model for baseline fitting: y = a * exp(b * x) + c.
|
fir_filter | Linear-phase FIR filter along the last axis. Convention |
fir_filter_coefficients | Window-method FIR coefficients for a linear-phase filter. Convention |
infer_signal_orientation | Infer the dominant sign of signal from its active region.
Convention |
is_signal_active | Decide whether a trace carries a signal or is flat. |
line_average_density | Return line-average electron density given an explicit chord length. Convention |
linear_baseline | Linear model for baseline fitting: y = a * x + b.
|
process_signal | Legacy conditioning wrapper: crop, resample, then filter. Convention |
quadratic_baseline | Quadratic model for baseline fitting: y = a * x^2 + b * x + c.
|
repair_clipped_interval | Reconstruct samples saturated at an acquisition limit by interpolation. |
resample_to_time | Project a signal onto target_time, low-passing first if that is a rate cut.
Convention |
signal_on_offset | Smooth a waveform, then return the window in which it is above a threshold. Convention |
signal_onoffset | Deprecated compatibility wrapper for signal_on_offset.
Deprecated |
smooth | Apply MATLAB-like moving-average smoothing with edge tapering. Convention |
subtract_baseline | Fit a baseline through the selected samples and subtract it everywhere. |
vest_coil_current_noise_reduction | Suppress point spikes in coil current traces. Convention |
vfit_signal_start_end | Deprecated compatibility wrapper for detect_active_window.
Deprecated |
vfit_signal_startend | Deprecated compatibility wrapper for detect_active_window.
Deprecated |
No function matches.
VEST_CoilCurrentNoiseReduction
Deprecated
source
VEST_CoilCurrentNoiseReduction(data)
Show source vaft/process/signal_processing.py, lines 1998–2006
def VEST_CoilCurrentNoiseReduction(data): # noqa: N802
"""Deprecated compatibility wrapper for the snake-case function."""
warnings.warn(
"VEST_CoilCurrentNoiseReduction() is deprecated; use "
"vest_coil_current_noise_reduction().",
DeprecationWarning,
stacklevel=2,
)
return vest_coil_current_noise_reduction(data)Deprecated compatibility wrapper for the snake-case function.
anti_alias_filter
Machine-independentConvention-sensitive
source
anti_alias_filter(values, *, source_rate, cutoff_hz, stopband_hz=None, numtaps=None, axis=-1, nan_policy='segment', return_mask=False)
Zero-phase FIR low-pass applied on the source grid, ahead of a rate cut.
This is the half of a downsample that np.interp cannot do for you.
Filtering after the rate has already been reduced is too late: whatever
folded is now indistinguishable from real in-band signal.
| Name | Type | Unit | Description |
|---|---|---|---|
values | array_like | any | The waveform or a stack of them; the time axis is |
source_rate | float | Hz | Sample rate of |
cutoff_hz | float | Hz | Passband edge, strictly inside |
stopband_hz | float or None, optional | Hz | Frequency by which the response must be down – for a downsample, the target Nyquist. Sets the transition band and so the tap count. |
numtaps | int or None, optional | - | Override the designed tap count; forced odd. |
axis | int, optional | - | The time axis of |
nan_policy | {"segment", "error", "ignore"}, optional | - |
|
return_mask | bool, optional | - | Also return which samples the filter actually covered. |
| Returns | |||
| — | np.ndarray | any | The filtered waveform, same shape as |
| — | np.ndarray of bool | - | Only with |
Convention
Zero-phase, and load-bearing: filterscope intensity feeds onset detection,
where a causal filter would move the onset by the group delay. firwin
filtfiltrather than a Butterworth ordecimate’s Chebyshev, because it is the idiom the VEST mappers already use (machine_mapping/tf.py,machine_mapping/pf_active.py,process/magnetics.py).
Raises
ResamplingErrorcutoff_hzorstopband_hzoutside their bands, or a non-finite sample undernan_policy="error".ValueErrorUnknownnan_policy.
Processing steps · Defaults · Applicability · Limitations · Provenance
Processing steps
- Size the filter:
numtaps ~ 3.3 * source_rate / (stopband - cutoff), the Hamming-window rule [1], measured against the target Nyquist – not the source Nyquist, which is ten times higher for a VEST fast channel and would size the filter at nine taps instead of the three hundred the job needs. Forced odd so the FIR is Type I with integer group delay. - Design it with
scipy.signal.firwin(Hamming window, low-pass). - Apply
scipy.signal.filtfiltto each finite run long enough for it; shorter runs are passed through with aRuntimeWarning.
Defaults
stopband_hz = 1.25 * cutoff_hz is a numerical convenience consistent
with resample_to_time’s default 0.8-of-Nyquist cutoff: it puts
the stopband exactly at the target Nyquist. nan_policy = "segment" is
a validated-workflow default – one NaN must not blank a whole record.
Applicability
Machine-independent.
Limitations
filtfilt needs 3 * numtaps + 1 samples; a record or finite run
shorter than that is returned unfiltered rather than padded, and nothing is
interpolated or invented in its place. One warning per call summarizes how
many runs and samples were left unfiltered, so a caller can tell a single
stray sample from most of the record; return_mask=True says which
samples, so a caller can flag them rather than store them beside filtered
values as if they were equivalent (#915). The filter assumes evenly spaced
samples; on an irregular grid the design rate is a fiction, which
resample_to_time refuses for you.
Provenance
- Hamming-window FIR design rule,
N ~ 3.3 f_s / Delta f, e.g. Oppenheim & Schafer, Discrete-Time Signal Processing, window method. vaft/machine_mapping/tf.pyandpf_active.py, the mappers whose established filtering idiom this generalizes.
butterworth_bandpass
Machine-independentConvention-sensitive
source
butterworth_bandpass(data, low, high, fs, order=2, *, zero_phase=True)
Show source vaft/process/signal_processing.py, lines 513–576
def butterworth_bandpass(data, low: float, high: float, fs: float, order: int = 2,
*, zero_phase: bool = True) -> np.ndarray:
"""Butterworth band-pass filter along the last axis.
Parameters
----------
data : array_like
The waveform or a stack of them; the time axis is the last [any].
low : float
Lower band edge [Hz].
high : float
Upper band edge, above ``low`` and below the Nyquist frequency [Hz].
fs : float
Sample rate [Hz].
order : int, optional
Filter order [-].
zero_phase : bool, optional
``True`` for forward-backward ``filtfilt``; ``False`` for a causal
``lfilter`` [-].
Returns
-------
np.ndarray
The filtered waveform, same shape as ``data`` [any].
Raises
------
ValueError
The band edges do not satisfy ``0 < low < high < fs / 2``.
Defaults
--------
``order = 2`` is a validated-workflow default shared with
:func:`butterworth_lowpass`. ``zero_phase = True`` is the opposite of the
low-pass default because band-passed signals feed spectral and
mode-number analysis, where phase across channels is the measurement.
Convention
----------
Zero-phase by default; see :func:`butterworth_lowpass` for the causal
alternative and when each is right.
Applicability
-------------
Machine-independent.
Limitations
-----------
As :func:`butterworth_lowpass`: a record too short for ``filtfilt`` raises
from SciPy.
"""
values = np.asarray(data, dtype=float)
nyquist = 0.5 * float(fs)
if not 0.0 < float(low) < float(high) < nyquist:
raise ValueError(
f"band edges must satisfy 0 < low < high < {nyquist:g} Hz for fs={fs:g}; "
f"got ({low!r}, {high!r})"
)
b, a = scipy_signal.butter(
int(order), [float(low) / nyquist, float(high) / nyquist], btype="band"
)
if zero_phase:
return scipy_signal.filtfilt(b, a, values, axis=-1)
return scipy_signal.lfilter(b, a, values, axis=-1)Butterworth band-pass filter along the last axis.
| Name | Type | Unit | Description |
|---|---|---|---|
data | array_like | any | The waveform or a stack of them; the time axis is the last. |
low | float | Hz | Lower band edge. |
high | float | Hz | Upper band edge, above |
fs | float | Hz | Sample rate. |
order | int, optional | - | Filter order. |
zero_phase | bool, optional | - |
|
| Returns | |||
| — | np.ndarray | any | The filtered waveform, same shape as |
Convention
Zero-phase by default; see butterworth_lowpass for the causal
alternative and when each is right.
Raises
ValueErrorThe band edges do not satisfy0 < low < high < fs / 2.
Defaults · Applicability · Limitations
Defaults
order = 2 is a validated-workflow default shared with
butterworth_lowpass. zero_phase = True is the opposite of the
low-pass default because band-passed signals feed spectral and
mode-number analysis, where phase across channels is the measurement.
Applicability
Machine-independent.
Limitations
As butterworth_lowpass: a record too short for filtfilt raises
from SciPy.
butterworth_lowpass
Machine-independentConvention-sensitive
source
butterworth_lowpass(data, cutoff, fs, order=2, *, zero_phase=False)
Show source vaft/process/signal_processing.py, lines 449–510
def butterworth_lowpass(data, cutoff: float, fs: float, order: int = 2,
*, zero_phase: bool = False) -> np.ndarray:
"""Butterworth low-pass filter along the last axis.
Parameters
----------
data : array_like
The waveform or a stack of them; the time axis is the last [any].
cutoff : float
Cutoff frequency, strictly between 0 and the Nyquist frequency [Hz].
fs : float
Sample rate [Hz].
order : int, optional
Filter order [-].
zero_phase : bool, optional
``True`` for forward-backward ``filtfilt``; ``False`` for a causal
``lfilter`` [-].
Returns
-------
np.ndarray
The filtered waveform, same shape as ``data`` [any].
Raises
------
ValueError
``cutoff`` is not inside ``(0, fs / 2)``.
Defaults
--------
``order = 2`` and ``zero_phase = False`` are validated-workflow defaults:
the values the VEST SXR viewer uses [SXR]_, kept so that
:mod:`vaft.process.soft_x_rays` reproduces it.
Convention
----------
**Causal by default.** With ``zero_phase=False`` every feature is delayed
by the filter's group delay, identically on every channel, which is what
ratio and reference arithmetic between SXR channels needs. Pass
``zero_phase=True`` wherever a time is going to be read off the result --
an onset, a peak -- because a causal filter moves it.
Applicability
-------------
Machine-independent. The defaults are VEST SXR practice but embed no VEST
constant.
Limitations
-----------
``filtfilt`` needs a record longer than three times the filter's padding
length and raises from SciPy otherwise; nothing here catches that.
"""
values = np.asarray(data, dtype=float)
nyquist = 0.5 * float(fs)
if not 0.0 < float(cutoff) < nyquist:
raise ValueError(
f"cutoff must lie in (0, {nyquist:g}) Hz for fs={fs:g}; got {cutoff!r}"
)
b, a = scipy_signal.butter(int(order), float(cutoff) / nyquist, btype="low")
if zero_phase:
return scipy_signal.filtfilt(b, a, values, axis=-1)
return scipy_signal.lfilter(b, a, values, axis=-1)Butterworth low-pass filter along the last axis.
| Name | Type | Unit | Description |
|---|---|---|---|
data | array_like | any | The waveform or a stack of them; the time axis is the last. |
cutoff | float | Hz | Cutoff frequency, strictly between 0 and the Nyquist frequency. |
fs | float | Hz | Sample rate. |
order | int, optional | - | Filter order. |
zero_phase | bool, optional | - |
|
| Returns | |||
| — | np.ndarray | any | The filtered waveform, same shape as |
Convention
Causal by default. With zero_phase=False every feature is delayed
by the filter’s group delay, identically on every channel, which is what
ratio and reference arithmetic between SXR channels needs. Pass
zero_phase=True wherever a time is going to be read off the result –
an onset, a peak – because a causal filter moves it.
Raises
ValueErrorcutoffis not inside(0, fs / 2).
Defaults · Applicability · Limitations
Defaults
order = 2 and zero_phase = False are validated-workflow defaults:
the values the VEST SXR viewer uses [SXR], kept so that
vaft.process.soft_x_rays reproduces it.
Applicability
Machine-independent. The defaults are VEST SXR practice but embed no VEST constant.
Limitations
filtfilt needs a record longer than three times the filter’s padding
length and raises from SciPy otherwise; nothing here catches that.
define_baseline
Machine-independent
source
define_baseline(time, onset_time, onset_window, offset_time=None, offset_window=None)
Show source vaft/process/signal_processing.py, lines 1706–1772
def define_baseline(time, onset_time, onset_window, offset_time=None, offset_window=None):
"""Select the baseline samples around a signal's onset and optional offset.
Converts ``onset_time`` (and ``offset_time``) to indices with
``np.searchsorted`` and returns the ``onset_window`` samples immediately
before the onset, plus the ``offset_window`` samples immediately after the
offset when both are given. The result is what :func:`subtract_baseline`
fits through.
Parameters
----------
time : np.ndarray
Sample times, sorted [s].
onset_time : float
Time at which the signal begins to deviate from baseline [s].
onset_window : int
Number of samples before the onset to include [-].
offset_time : float or None, optional
Time at which the signal returns to baseline [s].
offset_window : int or None, optional
Number of samples after the offset to include; both offset arguments
must be given for the offset region to be used [-].
Returns
-------
np.ndarray
Indices into ``time`` of the baseline samples, onset region first [-].
Assumptions
-----------
The signal is genuinely quiet in the selected windows. A window that
reaches back into a previous pulse, or forward into pickup from a coil
ramp, is fitted as if it were baseline.
Applicability
-------------
Machine-independent. The onset and offset times, and the window widths,
are the caller's; on VEST they come from the magnetics processing window
in ``vest.yaml`` via :mod:`vaft.process.magnetics`.
Limitations
-----------
Windows are clipped to the record silently, so a window wider than the
pre-onset record yields fewer samples than asked, with no warning.
Provenance
----------
.. [1] The legacy VEST EFIT magnetics chain [VFIT]_, where the baseline is
taken before the discharge and after it and removed by a fit.
"""
baseline_indices = []
# Convert onset_time -> onset_idx
onset_idx = np.searchsorted(time, onset_time)
# Add onset window
if onset_window > 0:
start_idx = max(0, onset_idx - onset_window)
baseline_indices.extend(range(start_idx, onset_idx))
# Convert offset_time -> offset_idx if provided
if offset_time is not None and offset_window is not None:
offset_idx = np.searchsorted(time, offset_time)
end_idx = min(len(time), offset_idx + offset_window)
baseline_indices.extend(range(offset_idx, end_idx))
return np.array(baseline_indices)Select the baseline samples around a signal’s onset and optional offset.
Converts onset_time (and offset_time) to indices with
np.searchsorted and returns the onset_window samples immediately
before the onset, plus the offset_window samples immediately after the
offset when both are given. The result is what subtract_baseline
fits through.
| Name | Type | Unit | Description |
|---|---|---|---|
time | np.ndarray | s | Sample times, sorted. |
onset_time | float | s | Time at which the signal begins to deviate from baseline. |
onset_window | int | - | Number of samples before the onset to include. |
offset_time | float or None, optional | s | Time at which the signal returns to baseline. |
offset_window | int or None, optional | - | Number of samples after the offset to include; both offset arguments must be given for the offset region to be used. |
| Returns | |||
| — | np.ndarray | - | Indices into |
Assumptions · Applicability · Limitations · Provenance
Assumptions
The signal is genuinely quiet in the selected windows. A window that reaches back into a previous pulse, or forward into pickup from a coil ramp, is fitted as if it were baseline.
Applicability
Machine-independent. The onset and offset times, and the window widths,
are the caller’s; on VEST they come from the magnetics processing window
in vest.yaml via vaft.process.magnetics.
Limitations
Windows are clipped to the record silently, so a window wider than the pre-onset record yields fewer samples than asked, with no warning.
Provenance
- The legacy VEST EFIT magnetics chain [VFIT], where the baseline is taken before the discharge and after it and removed by a fit.
describe_time_grid
Machine-independentConvention-sensitive
source
describe_time_grid(time, *, rtol=0.001)
Show source vaft/process/signal_processing.py, lines 1001–1060
def describe_time_grid(time, *, rtol: float = 1e-3) -> TimeGrid:
"""Measure a timebase's spacing, uniformity and monotonicity.
Parameters
----------
time : array_like
Sample times, flattened [s].
rtol : float, optional
Largest ``|dt_i - median(dt)| / median(dt)`` for which the grid is
still reported uniform [-].
Returns
-------
TimeGrid
``n``, ``t0``, the median ``dt``, ``sample_rate``, whether the grid is
``uniform`` within ``rtol`` and ``strictly_increasing``, and the
``max_relative_jitter`` actually measured [any].
Defaults
--------
``rtol = 1e-3`` is a numerical convenience sized to an acquisition-era
fact: VEST shots after 42190 store their timebase as ``linspace(0, span,
n)``, so a nominally 4e-6 s grid is really ``span / (n - 1)`` =
4.00016e-6 s. That is uniform for every purpose here and a tight
tolerance would only reject it.
Convention
----------
``dt`` is the *median* spacing, robust to the single ragged interval a
concatenated acquisition can leave behind. ``dt`` and ``sample_rate``
are ``nan`` for a single-sample grid, where neither is defined; an empty
grid reports ``n = 0`` and ``nan`` throughout.
Applicability
-------------
Machine-independent.
"""
values = np.asarray(time, dtype=float).reshape(-1)
n = int(values.size)
if n == 0:
return TimeGrid(0, float("nan"), float("nan"), float("nan"), False, False, float("nan"))
t0 = float(values[0])
if n == 1:
return TimeGrid(1, t0, float("nan"), float("nan"), True, True, float("nan"))
steps = np.diff(values)
dt = float(np.median(steps))
strictly_increasing = bool(np.all(steps > 0.0))
if dt == 0.0 or not np.isfinite(dt):
return TimeGrid(n, t0, dt, float("nan"), False, strictly_increasing, float("inf"))
jitter = float(np.max(np.abs(steps - dt)) / abs(dt))
return TimeGrid(
n=n,
t0=t0,
dt=dt,
sample_rate=1.0 / dt,
uniform=bool(jitter <= float(rtol)),
strictly_increasing=strictly_increasing,
max_relative_jitter=jitter,
)Measure a timebase’s spacing, uniformity and monotonicity.
| Name | Type | Unit | Description |
|---|---|---|---|
time | array_like | s | Sample times, flattened. |
rtol | float, optional | - | Largest |
| Returns | |||
| — | TimeGrid | any |
|
Convention
dt is the median spacing, robust to the single ragged interval a
concatenated acquisition can leave behind. dt and sample_rate
are nan for a single-sample grid, where neither is defined; an empty
grid reports n = 0 and nan throughout.
Defaults · Applicability
Defaults
rtol = 1e-3 is a numerical convenience sized to an acquisition-era
fact: VEST shots after 42190 store their timebase as linspace(0, span,
n), so a nominally 4e-6 s grid is really span / (n - 1) =
4.00016e-6 s. That is uniform for every purpose here and a tight
tolerance would only reject it.
Applicability
Machine-independent.
detect_active_window
Machine-independent
source
detect_active_window(time, signal, threshold=0.01)
Return the active time window that contains a signal’s main peak.
Finds the largest sample and walks outward from it in both directions
while the signal stays at or above threshold; the window is the time
of the first and last sample reached. If the peak itself is below
threshold the whole record is returned.
| Name | Type | Unit | Description |
|---|---|---|---|
time | array_like | s | Sample times, one-dimensional. |
signal | array_like | any | The waveform, one-dimensional, same length as |
threshold | float, optional | any | Level at or above which a sample counts as active, in the units of |
| Returns | |||
| — | tuple of (float, float) | s |
|
Raises
ValueErrorThe inputs are not one-dimensional, differ in length, or are empty.
Defaults · Assumptions · Applicability · Limitations · Provenance
Defaults
threshold = 0.01 is a legacy compatibility value from the donor
vfit_signal_startend, which was applied to an H-alpha trace
normalised to its own minimum. It is absolute, so a caller with a
differently scaled signal must pass its own.
Assumptions
The signal has one dominant positive excursion and is at or above
threshold throughout it. A dip below the threshold inside the pulse
ends the window early.
Applicability
Machine-independent. VEST source selection and calibration belong in
vaft.machine_mapping; the resulting physical signal is what is
passed here. For a plasma onset with evidence and a verdict, use
vaft.process.onset instead.
Limitations
The threshold is absolute, not relative to the peak, and the walk is not
robust to noise: a single sample below threshold stops it.
Provenance
vfit_signal_startend, the function nested inFunction/vest_Halpha_tstart_tend.mof the legacy VEST workflow [VFIT]; kept here under the deprecated namevfit_signal_start_end.
detect_clipped_samples
Machine-independent
source
detect_clipped_samples(data, *, clip_values, tolerance)
Show source vaft/process/signal_processing.py, lines 116–183
def detect_clipped_samples(data, *, clip_values, tolerance: float) -> np.ndarray:
"""Return a boolean mask of samples sitting at an acquisition limit.
``clip_values`` is a single signed level or a sequence of them, and the
mask is their union: a sample is saturated when it lies within
``tolerance`` of *any* supplied level. Real acquisition hardware rails on
both sides and rarely symmetrically -- VEST's diamagnetic Rogowski channel
is a signed 16-bit ADC over +/-5 V, so its rails are exactly ``-5.0`` and
``5 * 32767 / 32768`` [1]_.
Detecting every level in one pass is not a convenience. Where a waveform
oscillates hard enough to hit both rails within a few samples, repairing
one rail at a time would fit the reconstruction through samples still
pinned at the other one.
Parameters
----------
data : array_like
The waveform, raw or calibrated; the rails are in its units [any].
clip_values : float or sequence of float
The acquisition limit or limits, signed, in the units of ``data`` [any].
tolerance : float
Half-width of the band around each limit inside which a sample counts
as saturated [any].
Returns
-------
np.ndarray
Boolean mask, ``True`` where the sample is saturated; same shape as
``data`` [-].
Raises
------
SignalRepairError
``clip_values`` is empty or non-finite, or ``tolerance`` is not a
positive finite width.
Assumptions
-----------
Saturation shows as a sample *at* the rail, not as a distorted sample near
it. A digitizer that soft-clips will not be detected by a narrow
tolerance.
Applicability
-------------
Machine-independent. The caller supplies the rails and tolerance; this
function infers nothing about the hardware. The VEST values are read from
``vest.yaml`` by :mod:`vaft.machine_mapping`, never here.
Provenance
----------
.. [1] ``vest.yaml``, the ``diamagnetic_flux`` processing block, where the
asymmetric 16-bit rails are derived; issue #285.
"""
values = np.asarray(data, dtype=float)
levels = np.atleast_1d(np.asarray(clip_values, dtype=float)).reshape(-1)
if levels.size == 0:
raise SignalRepairError("`clip_values` must contain at least one acquisition limit.")
if not np.all(np.isfinite(levels)):
raise SignalRepairError(f"`clip_values` must be finite, got {clip_values!r}.")
width = float(tolerance)
if not np.isfinite(width) or width <= 0.0:
raise SignalRepairError(f"`tolerance` must be a positive finite width, got {tolerance!r}.")
saturated = np.zeros(values.shape, dtype=bool)
for level in levels:
saturated |= np.abs(values - level) < width
return saturatedReturn a boolean mask of samples sitting at an acquisition limit.
clip_values is a single signed level or a sequence of them, and the
mask is their union: a sample is saturated when it lies within
tolerance of any supplied level. Real acquisition hardware rails on
both sides and rarely symmetrically – VEST’s diamagnetic Rogowski channel
is a signed 16-bit ADC over +/-5 V, so its rails are exactly -5.0 and
5 * 32767 / 32768 [1].
Detecting every level in one pass is not a convenience. Where a waveform oscillates hard enough to hit both rails within a few samples, repairing one rail at a time would fit the reconstruction through samples still pinned at the other one.
| Name | Type | Unit | Description |
|---|---|---|---|
data | array_like | any | The waveform, raw or calibrated; the rails are in its units. |
clip_values | float or sequence of float | any | The acquisition limit or limits, signed, in the units of |
tolerance | float | any | Half-width of the band around each limit inside which a sample counts as saturated. |
| Returns | |||
| — | np.ndarray | - | Boolean mask, |
Raises
SignalRepairErrorclip_valuesis empty or non-finite, ortoleranceis not a positive finite width.
Assumptions · Applicability · Provenance
Assumptions
Saturation shows as a sample at the rail, not as a distorted sample near it. A digitizer that soft-clips will not be detected by a narrow tolerance.
Applicability
Machine-independent. The caller supplies the rails and tolerance; this
function infers nothing about the hardware. The VEST values are read from
vest.yaml by vaft.machine_mapping, never here.
Provenance
vest.yaml, thediamagnetic_fluxprocessing block, where the asymmetric 16-bit rails are derived; issue #285.
detrend_moving_average
Machine-independentConvention-sensitive
source
detrend_moving_average(data, window_samples)
Show source vaft/process/signal_processing.py, lines 744–794
def detrend_moving_average(data, window_samples: int) -> np.ndarray:
"""Subtract a centered moving-average trend along the last axis.
Parameters
----------
data : array_like
The waveform or a stack of them; the time axis is the last [any].
window_samples : int
Width of the trend window in samples; ``1`` or less removes nothing [-].
Returns
-------
np.ndarray
``data`` minus its rolling mean, same shape [any].
Convention
----------
The trend is a centred rolling mean with ``min_periods=1`` semantics: the
window at index ``i`` spans ``[i - w//2, i + (w-1)//2]`` clipped to the
record, so edge windows shrink rather than yield NaN and the output has the
input's length. This is exactly ``pandas.Series.rolling(window,
center=True, min_periods=1).mean()``, the convention of the validated VEST
SXR viewer [SXR]_, reimplemented with cumulative sums so that
:mod:`vaft.process.soft_x_rays` does not need pandas for it.
Applicability
-------------
Machine-independent.
Limitations
-----------
A centred window is zero-phase but the shrinking edge windows bias the
first and last ``w/2`` samples toward the local level. The trend is a
plain mean: one large excursion inside the window pulls the trend toward
it and appears, inverted, in the neighbours.
"""
values = np.asarray(data, dtype=float)
window = int(window_samples)
if window <= 1 or values.shape[-1] == 0:
return values - values # zero trend removal, preserving shape/dtype
length = values.shape[-1]
cumsum = np.zeros(values.shape[:-1] + (length + 1,), dtype=float)
np.cumsum(values, axis=-1, out=cumsum[..., 1:])
# A centered pandas window of size w at index i spans
# [i - w//2, i + (w-1)//2], clipped to the record.
index = np.arange(length)
start = np.clip(index - window // 2, 0, length)
stop = np.clip(index + (window - 1) // 2 + 1, 0, length)
trend = (cumsum[..., stop] - cumsum[..., start]) / (stop - start)
return values - trendSubtract a centered moving-average trend along the last axis.
| Name | Type | Unit | Description |
|---|---|---|---|
data | array_like | any | The waveform or a stack of them; the time axis is the last. |
window_samples | int | - | Width of the trend window in samples; |
| Returns | |||
| — | np.ndarray | any |
|
Convention
The trend is a centred rolling mean with min_periods=1 semantics: the
window at index i spans [i - w//2, i + (w-1)//2] clipped to the
record, so edge windows shrink rather than yield NaN and the output has the
input’s length. This is exactly pandas.Series.rolling(window,
center=True, min_periods=1).mean(), the convention of the validated VEST
SXR viewer [SXR], reimplemented with cumulative sums so that
vaft.process.soft_x_rays does not need pandas for it.
Applicability · Limitations
Applicability
Machine-independent.
Limitations
A centred window is zero-phase but the shrinking edge windows bias the
first and last w/2 samples toward the local level. The trend is a
plain mean: one large excursion inside the window pulls the trend toward
it and appears, inverted, in the neighbours.
exp_baseline
Machine-independent
source
exp_baseline(x, a, b, c)
Show source vaft/process/signal_processing.py, lines 1822–1845
def exp_baseline(x, a, b, c):
"""Exponential model for baseline fitting: ``y = a * exp(b * x) + c``.
Parameters
----------
x : array_like
Abscissa, normally time [any].
a : float
Amplitude [any].
b : float
Rate; the sign decides growth or decay [any].
c : float
Offset [any].
Returns
-------
np.ndarray
``a * exp(b * x) + c`` [any].
Applicability
-------------
Machine-independent. A fit model for :func:`subtract_baseline`.
"""
return a * np.exp(b * x) + cExponential model for baseline fitting: y = a * exp(b * x) + c.
| Name | Type | Unit | Description |
|---|---|---|---|
x | array_like | any | Abscissa, normally time. |
a | float | any | Amplitude. |
b | float | any | Rate; the sign decides growth or decay. |
c | float | any | Offset. |
| Returns | |||
| — | np.ndarray | any |
|
Applicability
Applicability
Machine-independent. A fit model for subtract_baseline.
fir_filter
Machine-independentConvention-sensitive
source
fir_filter(data, cutoff, fs, *, kind='lowpass', numtaps=None, window='hann', zero_phase=True)
Show source vaft/process/signal_processing.py, lines 668–741
def fir_filter(
data,
cutoff,
fs: float,
*,
kind: str = "lowpass",
numtaps: int | None = None,
window: str = "hann",
zero_phase: bool = True,
) -> np.ndarray:
"""Linear-phase FIR filter along the last axis.
Parameters
----------
data : array_like
The waveform or a stack of them; the time axis is the last [any].
cutoff : float or sequence of float
Band edge, or the pair of edges for ``bandpass``/``bandstop`` [Hz].
fs : float
Sample rate [Hz].
kind : str, optional
``lowpass``, ``highpass``, ``bandpass`` or ``bandstop`` [-].
numtaps : int or None, optional
Filter length; ``None`` derives it from ``fs``
(:func:`fir_filter_coefficients`) [-].
window : str, optional
Window passed to :func:`scipy.signal.firwin` [-].
zero_phase : bool, optional
``True`` for forward-backward ``filtfilt``; ``False`` for a causal
``lfilter`` that keeps the filter's own group delay [-].
Returns
-------
np.ndarray
The filtered waveform, same shape as ``data`` [any].
Raises
------
ValueError
As :func:`fir_filter_coefficients`.
Defaults
--------
``zero_phase = True`` is a validated-workflow default matching
:func:`butterworth_bandpass`: a filtered fluctuation signal feeds spectral
and mode-number analysis, where phase across channels is the measurement.
Convention
----------
An FIR filter of this design is already linear-phase, so a causal
application shifts every frequency by the same ``(numtaps - 1) / 2``
samples and the delay can be removed exactly -- unlike an IIR filter, whose
phase distortion is frequency-dependent and only ``filtfilt`` removes it.
That is the reason to reach for FIR here rather than
:func:`butterworth_bandpass`: the cost is a longer impulse response and
more arithmetic per sample.
Applicability
-------------
Machine-independent.
Limitations
-----------
``filtfilt`` needs a record longer than roughly three times the tap count
and raises from SciPy otherwise; a one-millisecond default response is long
at a high sample rate.
"""
values = np.asarray(data, dtype=float)
taps = fir_filter_coefficients(
cutoff, fs, kind=kind, numtaps=numtaps, window=window
)
if zero_phase:
return scipy_signal.filtfilt(taps, 1.0, values, axis=-1)
return scipy_signal.lfilter(taps, 1.0, values, axis=-1)Linear-phase FIR filter along the last axis.
| Name | Type | Unit | Description |
|---|---|---|---|
data | array_like | any | The waveform or a stack of them; the time axis is the last. |
cutoff | float or sequence of float | Hz | Band edge, or the pair of edges for |
fs | float | Hz | Sample rate. |
kind | str, optional | - |
|
numtaps | int or None, optional | - | Filter length; |
window | str, optional | - | Window passed to |
zero_phase | bool, optional | - |
|
| Returns | |||
| — | np.ndarray | any | The filtered waveform, same shape as |
Convention
An FIR filter of this design is already linear-phase, so a causal
application shifts every frequency by the same (numtaps - 1) / 2
samples and the delay can be removed exactly – unlike an IIR filter, whose
phase distortion is frequency-dependent and only filtfilt removes it.
That is the reason to reach for FIR here rather than
butterworth_bandpass: the cost is a longer impulse response and
more arithmetic per sample.
Raises
ValueErrorAsfir_filter_coefficients.
Defaults · Applicability · Limitations
Defaults
zero_phase = True is a validated-workflow default matching
butterworth_bandpass: a filtered fluctuation signal feeds spectral
and mode-number analysis, where phase across channels is the measurement.
Applicability
Machine-independent.
Limitations
filtfilt needs a record longer than roughly three times the tap count
and raises from SciPy otherwise; a one-millisecond default response is long
at a high sample rate.
fir_filter_coefficients
Machine-independentConvention-sensitive
source
fir_filter_coefficients(cutoff, fs, *, kind='lowpass', numtaps=None, window='hann')
Window-method FIR coefficients for a linear-phase filter.
| Name | Type | Unit | Description |
|---|---|---|---|
cutoff | float or sequence of float | Hz | Band edge, or the pair of edges for |
fs | float | Hz | Sample rate. |
kind | str, optional | - |
|
numtaps | int or None, optional | - | Filter length; |
window | str, optional | - | Window passed to |
| Returns | |||
| — | np.ndarray | - |
|
Convention
The length is always odd, so the filter is Type I: symmetric with an
integer group delay of (numtaps - 1) / 2 samples, and with a response
that is not forced to zero at the Nyquist frequency – which a Type II
(even-length) filter would be, making a high-pass or band-stop unusable.
Raises
ValueErrorkindis not one of the four supported responses, the number of band edges does not match it, or an edge is outside0 < f < fs / 2.
Defaults · Applicability · Limitations
Defaults
numtaps = None and window = "hann" are validated-workflow defaults,
taken from VEST’s fluctuation analysis. The length is an impulse response
one millisecond long, int(fs * 1e-3) forced odd, which ties the
transition width to the sample rate rather than to a fixed tap count.
Applicability
Machine-independent.
Limitations
The window method sets the transition width by the tap count alone; it does not meet a stated ripple specification the way an equiripple design would.
infer_signal_orientation
Machine-independentConvention-sensitive
source
infer_signal_orientation(signal, *, mask=None, active_fraction=0.1, min_samples=8, agreement=0.6)
Infer the dominant sign of signal from its active region.
The region is mask when given, otherwise the samples whose magnitude
reaches active_fraction of the signal’s peak magnitude – so the long
zero stretches before and after a discharge do not vote. The verdict is
the sign of the region’s median, resolved only when at least
agreement of the region shares it and the region holds at least
min_samples finite samples; otherwise it is unresolved and the
multiplier is +1 (canonical), never a guess. abs is never applied
to the data and no sample is flipped on its own.
| Name | Type | Unit | Description |
|---|---|---|---|
signal | array_like | any | The waveform, one-dimensional; non-finite samples are ignored. |
mask | array_like of bool or None, optional | - | The region to vote over, instead of the magnitude rule. |
active_fraction | float, optional | - | Fraction of the peak magnitude a sample must reach to be in the region |
min_samples | int, optional | - | Fewest finite samples the region may hold for a verdict. |
agreement | float, optional | - | Smallest fraction of the region that must share the median’s sign. |
| Returns | |||
| — | SignalOrientation | - |
|
Convention
The multiplier is meant to be applied to the whole signal once, for display or for a sign-convention check. It never rectifies samples.
Raises
ValueErrorsignalis not one-dimensional.
Defaults · Applicability · Limitations · Provenance
Defaults
active_fraction = 0.1, min_samples = 8 and agreement = 0.6 are
empirical estimates fixed when #307 was implemented; #307 sets the rule
(median of an active region, never abs, never per-sample) but records
no derivation for the three numbers.
Applicability
Machine-independent.
Limitations
A signal that is genuinely bipolar over its active region – a Mirnov fluctuation, a flux loop through a reversal – is correctly left unresolved, which the caller must handle rather than assume.
Provenance
- Issue #307, which established the rule and its defaults.
is_signal_active
Machine-independent
source
is_signal_active(data, var_ratio_thresh=0.01, change_ratio_thresh=0.01, verbose=False)
Show source vaft/process/signal_processing.py, lines 2034–2101
def is_signal_active(
data,
var_ratio_thresh=1e-2,
change_ratio_thresh=1e-2,
verbose=False,
):
"""Decide whether a trace carries a signal or is flat.
Both tests are relative to the trace's own level, so the verdict does not
depend on the signal's units or gain.
Parameters
----------
data : array_like
The waveform; fewer than two samples is never active [any].
var_ratio_thresh : float, optional
Threshold on the variance divided by the squared mean absolute level,
the squared coefficient of variation [-].
change_ratio_thresh : float, optional
Threshold on the mean absolute sample-to-sample change divided by the
mean absolute level [-].
verbose : bool, optional
Print both ratios [-].
Returns
-------
bool
``True`` unless *both* ratios fall below their thresholds [-].
Defaults
--------
Both thresholds at ``1e-2`` are legacy compatibility values; no derivation
is recorded.
Applicability
-------------
Machine-independent.
Limitations
-----------
A trace held at a non-zero level with noise far below that level is
inactive, whatever the level. The converse is the caveat: the test is
scale-free by design and has no absolute floor, so a zero-mean noise
trace scores near 1 on both ratios and is reported active at any
amplitude. A caller that knows a noise floor must apply it before
asking. An identically zero trace is inactive.
"""
data = np.asarray(data)
if data.size < 2:
return False
variance = np.var(data)
mean_abs_change = np.mean(np.abs(np.diff(data)))
level = np.mean(np.abs(data))
# Both ratios are relative to the magnitude of the trace (#463). An
# identically zero trace has no level to be relative to and is inactive.
var_ratio = variance / level**2 if level > 0.0 else 0.0
change_ratio = mean_abs_change / (level + 1e-12)
if verbose:
print(f"Variance ratio: {var_ratio:.3e} (thresh={var_ratio_thresh:.3e})")
print(f"Mean |Δx| ratio: {change_ratio:.3e} (thresh={change_ratio_thresh:.3e})")
if var_ratio < var_ratio_thresh and change_ratio < change_ratio_thresh:
return False
return TrueDecide whether a trace carries a signal or is flat.
Both tests are relative to the trace’s own level, so the verdict does not depend on the signal’s units or gain.
| Name | Type | Unit | Description |
|---|---|---|---|
data | array_like | any | The waveform; fewer than two samples is never active. |
var_ratio_thresh | float, optional | - | Threshold on the variance divided by the squared mean absolute level, the squared coefficient of variation. |
change_ratio_thresh | float, optional | - | Threshold on the mean absolute sample-to-sample change divided by the mean absolute level. |
verbose | bool, optional | - | Print both ratios. |
| Returns | |||
| — | bool | - |
|
Defaults · Applicability · Limitations
Defaults
Both thresholds at 1e-2 are legacy compatibility values; no derivation
is recorded.
Applicability
Machine-independent.
Limitations
A trace held at a non-zero level with noise far below that level is inactive, whatever the level. The converse is the caveat: the test is scale-free by design and has no absolute floor, so a zero-mean noise trace scores near 1 on both ratios and is reported active at any amplitude. A caller that knows a noise floor must apply it before asking. An identically zero trace is inactive.
line_average_density
Machine-independentConvention-sensitive
source
line_average_density(n_e_line, path_length_m)
Show source vaft/process/signal_processing.py, lines 334–368
def line_average_density(n_e_line, path_length_m: float) -> np.ndarray:
"""Return line-average electron density given an explicit chord length.
Parameters
----------
n_e_line : array_like
Line-integrated electron density along the diagnostic chord [m^-2].
path_length_m : float
Length of the chord through the plasma [m].
Returns
-------
np.ndarray
Line-average electron density, ``n_e_line / path_length_m`` [m^-3].
Raises
------
ValueError
``path_length_m`` is not positive.
Convention
----------
Line-*integrated* in, line-*averaged* out. An interferometer measures the
former; the latter is what confinement scalings and Greenwald fractions
want. No calibration or geometry is inferred here -- the chord length is
the caller's.
Applicability
-------------
Machine-independent. The chord length of a VEST interferometer channel
comes from :mod:`vaft.machine_mapping`.
"""
if path_length_m <= 0:
raise ValueError(f"path_length_m must be positive, got {path_length_m}")
return np.asarray(n_e_line, dtype=float) / float(path_length_m)Return line-average electron density given an explicit chord length.
| Name | Type | Unit | Description |
|---|---|---|---|
n_e_line | array_like | m^-2 | Line-integrated electron density along the diagnostic chord. |
path_length_m | float | m | Length of the chord through the plasma. |
| Returns | |||
| — | np.ndarray | m^-3 | Line-average electron density, |
Convention
Line-integrated in, line-averaged out. An interferometer measures the former; the latter is what confinement scalings and Greenwald fractions want. No calibration or geometry is inferred here – the chord length is the caller’s.
Raises
ValueErrorpath_length_mis not positive.
Applicability
Applicability
Machine-independent. The chord length of a VEST interferometer channel
comes from vaft.machine_mapping.
linear_baseline
Machine-independent
source
linear_baseline(x, a, b)
Show source vaft/process/signal_processing.py, lines 1774–1795
def linear_baseline(x, a, b):
"""Linear model for baseline fitting: ``y = a * x + b``.
Parameters
----------
x : array_like
Abscissa, normally time [any].
a : float
Slope [any].
b : float
Intercept [any].
Returns
-------
np.ndarray
``a * x + b`` [any].
Applicability
-------------
Machine-independent. A fit model for :func:`subtract_baseline`.
"""
return a * x + bLinear model for baseline fitting: y = a * x + b.
| Name | Type | Unit | Description |
|---|---|---|---|
x | array_like | any | Abscissa, normally time. |
a | float | any | Slope. |
b | float | any | Intercept. |
| Returns | |||
| — | np.ndarray | any |
|
Applicability
Applicability
Machine-independent. A fit model for subtract_baseline.
process_signal
VEST-specificConvention-sensitive
source
process_signal(time, data, options=None)
Legacy conditioning wrapper: crop, resample, then filter.
| Name | Type | Unit | Description |
|---|---|---|---|
time | array_like | s | Sample times, flattened. |
data | array_like | any | The waveform, flattened. |
options | dict or None, optional | - |
|
| Returns | |||
| — | tuple of (np.ndarray, np.ndarray) | any | The conditioned |
Convention
Zero-phase (filtfilt) for the shaping filter. The sample rate for
filter design is taken from the first two samples, not the median.
Raises
ValueErrorAfilter_paramscutoff outside(0, f_s / 2), a malformed band-pass cutoff, or an unknown filter type.
Processing steps · Defaults · Applicability · Limitations
Processing steps
- Crop to
time_range. - Resample onto
arange(t[0], t[-1], dt)throughresample_to_time, so a rate reduction anti-aliases on the input grid first. - Apply a zero-phase Butterworth from
filter_params. This is a shaping filter and itscutoffis interpreted against the output grid’s sample rate, not the input’s.
Defaults
dt = 4e-5 s is the VEST slow-DAQ spacing, a machine-specific setting.
cutoff = 1000 Hz and order = 4 are legacy compatibility values
whose origin is not recorded.
Applicability
VEST-specific. The defaults embed the VEST acquisition grid, and the
option dictionary is the legacy call convention kept for existing
workflows; new code should call resample_to_time and
butterworth_lowpass directly.
Limitations
Kept for compatibility; its future is decided by the module-ownership audit tracked in #263.
quadratic_baseline
Machine-independent
source
quadratic_baseline(x, a, b, c)
Show source vaft/process/signal_processing.py, lines 1797–1820
def quadratic_baseline(x, a, b, c):
"""Quadratic model for baseline fitting: ``y = a * x^2 + b * x + c``.
Parameters
----------
x : array_like
Abscissa, normally time [any].
a : float
Quadratic coefficient [any].
b : float
Linear coefficient [any].
c : float
Constant [any].
Returns
-------
np.ndarray
``a * x**2 + b * x + c`` [any].
Applicability
-------------
Machine-independent. A fit model for :func:`subtract_baseline`.
"""
return a * x**2 + b * x + cQuadratic model for baseline fitting: y = a * x^2 + b * x + c.
| Name | Type | Unit | Description |
|---|---|---|---|
x | array_like | any | Abscissa, normally time. |
a | float | any | Quadratic coefficient. |
b | float | any | Linear coefficient. |
c | float | any | Constant. |
| Returns | |||
| — | np.ndarray | any |
|
Applicability
Applicability
Machine-independent. A fit model for subtract_baseline.
repair_clipped_interval
Machine-independent
source
repair_clipped_interval(time, data, *, clip_value, tolerance, min_support=4, return_mask=False)
Reconstruct samples saturated at an acquisition limit by interpolation.
Samples within tolerance of clip_value are treated as saturated
and replaced by a cubic spline fitted to the remaining samples on the
physical time axis. Every unsaturated sample is preserved exactly.
clip_value may be a single signed level or a sequence of levels, in
which case saturation is their union (see detect_clipped_samples).
With return_mask=True the saturation mask is returned alongside the
repaired waveform, so a caller can report which samples it reconstructed
instead of handing downstream consumers an unmarked mixture.
| Name | Type | Unit | Description |
|---|---|---|---|
time | array_like | s | Sample times, one-dimensional, same length as |
data | array_like | any | The waveform with saturated samples, one-dimensional. |
clip_value | float or sequence of float | any | The acquisition limit or limits, signed, in the units of |
tolerance | float | any | Half-width of the band around each limit inside which a sample counts as saturated. |
min_support | int, optional | - | Fewest unsaturated samples the spline may be fitted through. |
return_mask | bool, optional | - | Also return the saturation mask. |
| Returns | |||
| — | np.ndarray or tuple of (np.ndarray, np.ndarray) | any | The repaired waveform, and with |
Raises
SignalRepairErrorThe inputs differ in shape or are not one-dimensional; any sample is non-finite; the whole waveform is saturated; fewer thanmin_supportunsaturated samples remain; or the saturated interval reaches either end of the record, which would require extrapolation.
Processing steps · Input semantics · Output semantics · Defaults · Assumptions · Applicability · Limitations · Provenance
Processing steps
- Mask the saturated samples with
detect_clipped_samples. - Refuse if the mask covers everything, leaves fewer than
min_supportsamples, or touches either end of the record. - Fit
scipy.interpolate.CubicSplinethrough the unsaturated samples againsttime. - Evaluate it at the saturated instants and write those samples back.
Input semantics
Raw or calibrated – the rails must be expressed in the same units as
data. Unrepaired: a waveform that has already been filtered has
smeared its rails and cannot be repaired here.
Output semantics
Repaired. The reconstructed samples are interpolated, not measured; the mask says which they are.
Defaults
min_support = 4 is a numerical convenience: a cubic spline needs four
points to be determined at all, and this is the floor, not a quality bar.
Assumptions
The waveform is smooth on the scale of the saturated interval, so a cubic through its neighbours is a defensible estimate of what was lost. A saturated interval that hides a genuine fast feature is reconstructed as if the feature were absent.
Applicability
Machine-independent. Callers supply the limit and tolerance. On VEST the
PF6 supply clips near -5000 A and the repair is enabled per acquisition
era in vest.yaml, because PF6’s gain flips sign at shot 38110 and
before that a -5000 A sample is ordinary data [1].
Limitations
Interpolation only: a saturated run at the start or end of the record is refused rather than extrapolated. Reconstructing more than a few samples in a row grows less defensible with every sample; there is no cap on that here, so the caller should look at the mask.
Provenance
vest.yaml,pf_activeprocessingsaturation_repairand the per-era policy comment; the donorvest_pf.mapplied the -5000 A repair unconditionally. Issues #195 and #285.
resample_to_time
Machine-independentConvention-sensitive
source
resample_to_time(source_time, values, target_time, *, anti_alias='auto', cutoff_fraction=0.8, cutoff_hz=None, numtaps=None, min_ratio=1.05, extrapolate='clamp', nan_policy='segment', on_unsorted='error', axis=-1, return_filter_mask=False)
Project a signal onto target_time, low-passing first if that is a rate cut.
Use this instead of a bare np.interp anywhere a diagnostic is written
onto a common time grid. Interpolation and downsampling look identical in
source but are not the same operation: evaluating a signal at new instants
is always fine, whereas reducing the sample rate discards the information
needed to distinguish a high frequency from its alias, so the content above
the new Nyquist has to be removed while it still can be.
| Name | Type | Unit | Description |
|---|---|---|---|
source_time | array_like | s | Sample times of |
values | array_like | any | The waveform or a stack of them; the time axis is |
target_time | array_like | s | Instants to evaluate at; need not be sorted. |
anti_alias | bool or "auto", optional | - |
|
cutoff_fraction | float, optional | - | Anti-alias cutoff as a fraction of the target Nyquist. |
cutoff_hz | float or None, optional | Hz | Explicit cutoff, overriding |
numtaps | int or None, optional | - | Passed to |
min_ratio | float, optional | - |
|
extrapolate | {"clamp", "nan", "error"}, optional | - | What a target sample outside the source span gets. |
nan_policy | str, optional | - | Passed to |
on_unsorted | {"error", "sort"}, optional | - | Refuse an unsorted source, or sort it and collapse duplicates. |
axis | int, optional | - | The time axis of |
return_filter_mask | bool, optional | - | Also return, on the source grid, which samples reached the target band-limited. |
| Returns | |||
| — | np.ndarray | any |
|
| — | np.ndarray of bool | - | Only with |
Convention
The cutoff is placed against the target grid’s spacing measured in time
order, even when target_time is unsorted; the shuffled array would
report a median dt tens of times too large and design a filter that
eats the signal. An explicit cutoff_hz above the target Nyquist is
honoured as a deliberate choice to keep more band and gets
anti_alias_filter’s own transition instead.
Raises
ResamplingErrorEmpty source; unsorted or duplicated source times under the default policy; a target outside the source span underextrapolate="error"; a forced filter whose cutoff exceeds the source Nyquist; or a filter requested on a non-uniform source grid.ValueErrorAn option outside its allowed values.
Processing steps · Output semantics · Defaults · Assumptions · Applicability · Limitations · Provenance
Processing steps
- Validate options; sort and de-duplicate the source if asked.
- Measure both grids with
describe_time_grid. - Decide whether to filter:
anti_aliasas given, or under"auto"whetherdt_target / dt_source > min_ratio. - If filtering, refuse a non-uniform source, place the cutoff at
cutoff_fractionof the target Nyquist (orcutoff_hz), put the stopband at the target Nyquist, and runanti_alias_filteron the source grid. np.interpontotarget_timewith theextrapolatepolicy.
Output semantics
Interpolated. When no filter runs, the result is bit-for-bit
np.interp – deliberately, so equal-rate call sites can adopt this
primitive without moving a stored value. When one does, the result is
band-limited to the target Nyquist and then interpolated.
Defaults
anti_alias = "auto" and min_ratio = 1.05 are validated-workflow
defaults: the safe thing by default, the unsafe thing by name.
cutoff_fraction = 0.8 is a numerical convenience – firwin sits at
-6 dB at its cutoff and needs a transition band, so a passband edge below
Nyquist puts everything that can fold into the stopband. extrapolate =
"clamp" matches np.interp. on_unsorted = "error" because
np.interp returns silent nonsense for an unsorted x.
Assumptions
values is a band-limited physical signal. A validity mask or any
other logical series is not, and must be resampled with
anti_alias=False.
Applicability
Machine-independent. The motivating case is VEST, which acquires at
FAST_DT = 4e-6 s and SLOW_DT = 4e-5 s while every processed grid
in vest.yaml is 4e-5 s, so a fast channel written onto a policy grid
is a 10x decimation [1].
Limitations
A single-sample target has no Nyquist: "auto" interpolates with a
warning and True raises unless cutoff_hz is given. A non-uniform
source grid cannot be anti-aliased by a single-rate FIR and is refused;
resample onto a uniform grid first, or pass anti_alias=False to accept
a bare interpolation knowingly.
Provenance
vaft.database.rawFAST_DT/SLOW_DT, and the processed time grids declared invaft/machine_mapping/vest.yaml.
signal_on_offset
Machine-independentConvention-sensitive
source
signal_on_offset(time, data, smooth_window=5, threshold=0.01, verbose=False)
Show source vaft/process/signal_processing.py, lines 1939–1995
def signal_on_offset(time, data, smooth_window=5, threshold=0.01, verbose=False):
"""Smooth a waveform, then return the window in which it is above a threshold.
Parameters
----------
time : array_like
Sample times, one-dimensional [s].
data : array_like
The waveform, one-dimensional, same length as ``time`` [any].
smooth_window : int, optional
Savitzky-Golay window length in samples; must be odd and larger than
the polynomial order, 3 [-].
threshold : float, optional
Level at or above which the smoothed signal counts as active, in the
units of ``data`` [any].
verbose : bool, optional
Print the threshold [-].
Returns
-------
tuple of (float, float)
``(onset, offset)`` times [s].
Processing steps
----------------
1. Smooth with ``scipy.signal.savgol_filter(data, smooth_window, 3)``.
2. Hand the smoothed waveform to :func:`detect_active_window`.
Defaults
--------
``smooth_window = 5`` and the cubic polynomial are legacy compatibility
values from the ported workflow, unrecorded beyond that. ``threshold =
0.01`` is inherited from :func:`detect_active_window`.
Convention
----------
Zero-phase: a Savitzky-Golay filter is a centred polynomial fit and moves
nothing in time, so the returned onset is not delayed by the smoothing.
Applicability
-------------
Machine-independent. Callers in :mod:`vaft.omas.general` and
:mod:`vaft.database` pass ``threshold=0.05`` for VEST H-alpha; the default
is not a VEST policy. For a plasma onset with evidence and a verdict, use
:mod:`vaft.process.onset`.
Limitations
-----------
Those of :func:`detect_active_window`: an absolute threshold and a walk
that stops at the first sample below it.
"""
if verbose:
print("threshold for signal detection:", threshold)
# Smooth the data
smoothed = scipy_signal.savgol_filter(data, smooth_window, 3)
return detect_active_window(time, smoothed, threshold=threshold)Smooth a waveform, then return the window in which it is above a threshold.
| Name | Type | Unit | Description |
|---|---|---|---|
time | array_like | s | Sample times, one-dimensional. |
data | array_like | any | The waveform, one-dimensional, same length as |
smooth_window | int, optional | - | Savitzky-Golay window length in samples; must be odd and larger than the polynomial order, 3. |
threshold | float, optional | any | Level at or above which the smoothed signal counts as active, in the units of |
verbose | bool, optional | - | Print the threshold. |
| Returns | |||
| — | tuple of (float, float) | s |
|
Convention
Zero-phase: a Savitzky-Golay filter is a centred polynomial fit and moves nothing in time, so the returned onset is not delayed by the smoothing.
Processing steps · Defaults · Applicability · Limitations
Processing steps
- Smooth with
scipy.signal.savgol_filter(data, smooth_window, 3). - Hand the smoothed waveform to
detect_active_window.
Defaults
smooth_window = 5 and the cubic polynomial are legacy compatibility
values from the ported workflow, unrecorded beyond that. threshold =
0.01 is inherited from detect_active_window.
Applicability
Machine-independent. Callers in vaft.omas.general and
vaft.database pass threshold=0.05 for VEST H-alpha; the default
is not a VEST policy. For a plasma onset with evidence and a verdict, use
vaft.process.onset.
Limitations
Those of detect_active_window: an absolute threshold and a walk
that stops at the first sample below it.
signal_onoffset
Deprecated
source
signal_onoffset(time, data, smooth_window=5, threshold=0.01, verbose=False)
Show source vaft/process/signal_processing.py, lines 2019–2032
def signal_onoffset(time, data, smooth_window=5, threshold=0.01, verbose=False):
"""Deprecated compatibility wrapper for :func:`signal_on_offset`."""
warnings.warn(
"signal_onoffset() is deprecated; use signal_on_offset().",
DeprecationWarning,
stacklevel=2,
)
return signal_on_offset(
time,
data,
smooth_window=smooth_window,
threshold=threshold,
verbose=verbose,
)Deprecated compatibility wrapper for signal_on_offset.
smooth
Machine-independentConvention-sensitive
source
smooth(array, span)
Show source vaft/process/signal_processing.py, lines 371–446
def smooth(array, span: int) -> np.ndarray:
"""Apply MATLAB-like moving-average smoothing with edge tapering.
A centred moving average of odd width ``span``. At each end the window
shrinks symmetrically -- widths 1, 3, 5, ... -- so the output has the
input's length and no sample is padded, which is what MATLAB's ``smooth``
does and what the ported VEST workflows expect.
Parameters
----------
array : array_like
The waveform, one-dimensional [any].
span : int
Window width in samples; an even value is reduced by one, and a value
wider than the record is clipped to it [-].
Returns
-------
np.ndarray
The smoothed waveform, same length as ``array`` [any].
Raises
------
ValueError
``array`` is not one-dimensional.
Convention
----------
Zero-phase: the window is centred, so features are not shifted in time.
Edge samples are averaged over progressively narrower windows rather than
over a padded record, which biases the ends less than reflection padding
would but leaves them noisier than the interior.
Applicability
-------------
Machine-independent.
Provenance
----------
.. [1] MATLAB ``smooth(y, span)`` with the default ``'moving'`` method;
ported from the legacy VEST workflow [VFIT]_ with its semantics
preserved so that results match the donor sample for sample.
"""
values = np.asarray(array, dtype=float)
if values.ndim != 1:
raise ValueError("`array` must be one-dimensional.")
if values.size == 0 or span <= 1:
return values.copy()
if span % 2 == 0:
span -= 1
span = min(span, values.size if values.size % 2 == 1 else values.size - 1)
if span < 1:
return values.copy()
count = values.size
out = np.zeros(count, dtype=float)
half = (span - 1) // 2
# O(N) vectorized implementation using cumulative sums.
cumsum = np.zeros(count + 1, dtype=float)
cumsum[1:] = np.cumsum(values)
# Interior: full-width windows of size `span`.
interior = np.arange(half, count - half)
if interior.size:
out[interior] = (cumsum[interior + half + 1] - cumsum[interior - half]) / span
# Edge tapering: left and right sides use progressively narrower windows.
if half > 0:
left_idx = np.arange(half)
widths = 2 * left_idx + 1
out[left_idx] = (cumsum[widths] - cumsum[0]) / widths
right_idx = count - 1 - left_idx
out[right_idx] = (cumsum[count] - cumsum[count - widths]) / widths
return outApply MATLAB-like moving-average smoothing with edge tapering.
A centred moving average of odd width span. At each end the window
shrinks symmetrically – widths 1, 3, 5, … – so the output has the
input’s length and no sample is padded, which is what MATLAB’s smooth
does and what the ported VEST workflows expect.
| Name | Type | Unit | Description |
|---|---|---|---|
array | array_like | any | The waveform, one-dimensional. |
span | int | - | Window width in samples; an even value is reduced by one, and a value wider than the record is clipped to it. |
| Returns | |||
| — | np.ndarray | any | The smoothed waveform, same length as |
Convention
Zero-phase: the window is centred, so features are not shifted in time. Edge samples are averaged over progressively narrower windows rather than over a padded record, which biases the ends less than reflection padding would but leaves them noisier than the interior.
Raises
ValueErrorarrayis not one-dimensional.
Applicability · Provenance
Applicability
Machine-independent.
Provenance
- MATLAB
smooth(y, span)with the default'moving'method; ported from the legacy VEST workflow [VFIT] with its semantics preserved so that results match the donor sample for sample.
subtract_baseline
Machine-independent
source
subtract_baseline(time, signal, baseline_indices, fitting_opt='linear')
Fit a baseline through the selected samples and subtract it everywhere.
| Name | Type | Unit | Description |
|---|---|---|---|
time | np.ndarray | s | Sample times. |
signal | np.ndarray | any | The waveform, same length as |
baseline_indices | np.ndarray | - | Indices of the samples the baseline is fitted through, normally from |
fitting_opt | {"linear", "quadratic", "spline", "exp"}, optional | - | The baseline model. |
| Returns | |||
corrected | np.ndarray | any |
|
baseline | np.ndarray | any | The fitted baseline evaluated on the whole |
Raises
ValueErrorUnknownfitting_opt.
Processing steps · Output semantics · Defaults · Assumptions · Applicability · Limitations · Provenance
Processing steps
- Take
timeandsignalatbaseline_indices. - Fit the chosen model through them:
scipy.optimize.curve_fitfor the parametric models,scipy.interpolate.UnivariateSpline(s=0)– an interpolating spline – for"spline". - Evaluate the model on the full
timeaxis and subtract.
Output semantics
Baseline-subtracted. The second return is the baseline itself, so the caller can plot or store what was removed.
Defaults
fitting_opt = "linear" is a legacy compatibility value: the donor
workflow removed a linear drift, which is the right model for an
integrator’s offset. The exponential fit uses maxfev = 10000, a
numerical convenience.
Assumptions
The samples at baseline_indices contain no signal. The parametric
models assume the drift has that form over the whole record, including
the pulse, where it is extrapolated rather than measured.
Applicability
Machine-independent.
Limitations
"spline" interpolates the baseline samples exactly, so noise in them
is reproduced in the baseline; with the onset and offset regions far
apart it is a straight line between them in practice. "exp" can fail
to converge and raises from SciPy. Nothing checks that
baseline_indices is non-empty.
Provenance
- The legacy VEST EFIT magnetics chain [VFIT], where an integrated probe signal’s drift is removed by a fit through the pre- and post-discharge windows.
vest_coil_current_noise_reduction
VEST-specificConvention-sensitive
source
vest_coil_current_noise_reduction(data)
Show source vaft/process/signal_processing.py, lines 797–863
def vest_coil_current_noise_reduction(data) -> np.ndarray:
"""Suppress point spikes in coil current traces.
Walks the record and, wherever a sample's magnitude exceeds the mean of
its two neighbours' magnitudes by more than a fixed margin, replaces it
with the previous sample. A port of the donor routine, kept sample for
sample so that the legacy PF-current preprocessing reproduces.
Parameters
----------
data : array_like
Coil current, one-dimensional; records shorter than three samples are
returned unchanged [A].
Returns
-------
np.ndarray
The de-spiked current, same length [A].
Processing steps
----------------
1. For each interior sample (the first two and the last are never
touched), compute ``|x[i]| - |(x[i+1] + x[i-1]) / 2|``.
2. Where that exceeds ``0.001``, overwrite ``x[i]`` with the already
processed ``x[i-1]``.
Defaults
--------
The ``0.001`` margin is a legacy compatibility value from the donor
``VEST_CoilCurrentNoiseReduction.m`` [1]_. It is absolute, in the units of
``data``, and its derivation is not recorded. On a trace in amperes it
flags any excursion above a milliamp, so in practice every spike; on a
trace in kiloamperes it would let real noise through.
Convention
----------
Causal: a spike is replaced by its predecessor, so a genuine step is
delayed by one sample at its leading edge.
Applicability
-------------
VEST-specific. Written for the VEST PF coil current traces as digitized by
the legacy DAQ; the margin has no meaning for another machine's units or
noise floor.
Limitations
-----------
Only a single-sample spike is caught: two consecutive bad samples pass,
because the second is compared against the first. Sign is ignored in the
test but preserved in the replacement.
Provenance
----------
.. [1] ``Function/VEST_CoilCurrentNoiseReduction.m`` in the legacy VEST
workflow [VFIT]_; loop bounds and margin preserved.
"""
values = np.asarray(data, dtype=float)
if values.size < 3:
return values.copy()
smoothed = values.copy()
for index_i in range(2, values.size - 1):
local_ref = abs((values[index_i + 1] + values[index_i - 1]) / 2.0)
diff = abs(values[index_i]) - local_ref
if diff > 0.001:
smoothed[index_i] = smoothed[index_i - 1]
return smoothedSuppress point spikes in coil current traces.
Walks the record and, wherever a sample’s magnitude exceeds the mean of its two neighbours’ magnitudes by more than a fixed margin, replaces it with the previous sample. A port of the donor routine, kept sample for sample so that the legacy PF-current preprocessing reproduces.
| Name | Type | Unit | Description |
|---|---|---|---|
data | array_like | A | Coil current, one-dimensional; records shorter than three samples are returned unchanged. |
| Returns | |||
| — | np.ndarray | A | The de-spiked current, same length. |
Convention
Causal: a spike is replaced by its predecessor, so a genuine step is delayed by one sample at its leading edge.
Processing steps · Defaults · Applicability · Limitations · Provenance
Processing steps
- For each interior sample (the first two and the last are never
touched), compute
|x[i]| - |(x[i+1] + x[i-1]) / 2|. - Where that exceeds
0.001, overwritex[i]with the already processedx[i-1].
Defaults
The 0.001 margin is a legacy compatibility value from the donor
VEST_CoilCurrentNoiseReduction.m [1]. It is absolute, in the units of
data, and its derivation is not recorded. On a trace in amperes it
flags any excursion above a milliamp, so in practice every spike; on a
trace in kiloamperes it would let real noise through.
Applicability
VEST-specific. Written for the VEST PF coil current traces as digitized by the legacy DAQ; the margin has no meaning for another machine’s units or noise floor.
Limitations
Only a single-sample spike is caught: two consecutive bad samples pass, because the second is compared against the first. Sign is ignored in the test but preserved in the replacement.
Provenance
Function/VEST_CoilCurrentNoiseReduction.min the legacy VEST workflow [VFIT]; loop bounds and margin preserved.
vfit_signal_start_end
Deprecated
source
vfit_signal_start_end(time, data, threshold=0.01)
Show source vaft/process/signal_processing.py, lines 950–957
def vfit_signal_start_end(time, data, threshold: float = 0.01) -> tuple[float, float]:
"""Deprecated compatibility wrapper for :func:`detect_active_window`."""
warnings.warn(
"vfit_signal_start_end() is deprecated; use detect_active_window().",
DeprecationWarning,
stacklevel=2,
)
return detect_active_window(time, data, threshold=threshold)Deprecated compatibility wrapper for detect_active_window.
vfit_signal_startend
Deprecated
source
vfit_signal_startend(time, data, threshold=0.01)
Show source vaft/process/signal_processing.py, lines 2009–2016
def vfit_signal_startend(time, data, threshold: float = 0.01):
"""Deprecated compatibility wrapper for :func:`detect_active_window`."""
warnings.warn(
"vfit_signal_startend() is deprecated; use detect_active_window().",
DeprecationWarning,
stacklevel=2,
)
return detect_active_window(time, data, threshold=threshold)Deprecated compatibility wrapper for detect_active_window.