Detection and Integration Algorithm

A technical description of how INTEGRATOR is implemented

Implementation

INTEGRATOR and its viewer are implemented as self-contained, lightweight applications for macOS (Apple silicon) and Windows 11 (x86-64) that require no installation. INTEGRATOR provides a terminal user interface, while its viewer (MRMhub-viz) comes with a graphical interface. The core is written in Rust and compiled to a single native executable per platform, which reads and decodes mzML data directly and parallelises the integration across the available CPU cores for speed.

Memory management and lazy loading

INTEGRATOR is designed to scale to large projects — on the order of 10,000 samples or more — without holding every raw file in memory at once. During the first step, each mzML file is decoded once and rewritten to the project folder as compact per-transition binary files: the extracted ion chromatogram of every transition is stored separately, so any single transition can later be read back without touching the rest of the file.

The subsequent steps then load these binaries lazily, reading only the transitions currently being processed and releasing them once done, rather than keeping the full dataset resident in memory. Combined with the survey-and-cache approach — peak features are surveyed across all samples once and their positions written to disk for reuse — this keeps the memory footprint roughly independent of the number of samples and the processing time manageable, making even very large datasets tractable on relatively modest hardware such as a standard laptop.

Sample and transition-list validation

The mzML file names in the sample-list file are checked against the actual files in the data directory provided by the user. Files absent from the directory are reported in a text file (missing_files.txt) so the user can check them. The input transition list is used to assign transition names and attributes to the chromatograms in each mzML file by matching precursor-ion m/z, product-ion m/z, and RT. Transitions that fail to match any chromatogram are skipped and reported in a text file (missing_compounds.txt). Chromatograms in mzML files without any assignment are also excluded from the analysis. The user may proceed with the analysis even if some files or compounds are missing.

A transition is therefore integrated only if it is present in every mzML file and listed in the transition input file: the processed set is the intersection of the transitions common to all files with those defined in the feature list. A transition absent from any single file is currently dropped for the whole dataset, not only for the files that lack it.

Where the same transition (same Q1/Q3 values) with identical or nearly identical MRM start and end windows appears two or more times in an mzML file, the user is alerted and can choose which transition/MRM index to map to which feature in the transition input file.

Retention-time shift correction

The algorithm begins with an initial RT-drift estimation. For each transition, it treats the ion trace in the user-designated reference sample(s) as the reference pattern and uses it to track potential RT drift in all samples. The reference samples are expected to be selected from an early part of the analysis sequence, and not dispersed throughout the entire sequence. The chromatograms in all samples are then aligned to those of the reference samples within individual MRM windows, yielding estimates of the RT shifts.

The user can set a maximum allowance of RT drift (in minutes) in the parameter file (param.txt) using the RT_shift parameter. The syntax is RT_shift = [x, y], where x is the lower bound (negative — drift earlier than expected) and y the upper bound (positive — drift later): the algorithm allows the RT to move up to |x| minutes earlier and up to y minutes later than the expected RT in the reference samples. In addition, the RT_shift_bound parameter restricts the RT shift between adjacent sample injections.

RT-shift correction can be disabled by setting RT_shift = [0, 0], which can help for datasets with strong variability of features and/or interferences in the chromatograms, where the cross-correlation approach may yield inaccurate results. In the absence of chromatographic peaks for blank samples, their RT shifts are determined by interpolation from the shifts observed in adjacent samples of the analytical sequence.

When the user chooses to process only a specific part of the chromatograms — by globally setting an RT window relative to the expected RT — RT-shift correction is performed only on the corresponding RT window.

In outline, the shift for each sample is estimated and then smoothed against its neighbours in the analytical sequence:

for each non-reference sample:
    for offset in RT_shift grid (RT_shift[0] .. RT_shift[1]):
        shift the sample trace by offset, interpolate the reference at those RTs
        score = similarity(shifted sample, reference)          # spectral-angle / cosine
    keep the offset with the best score (smallest |offset| when references tie)

for each sample:                                               # neighbour smoothing
    if neighbouring samples agree with its shift:  keep it
    elif it deviates beyond RT_shift_bound:        snap to the neighbour median
    elif blank (no peak):                          interpolate from adjacent samples

Feature detection

INTEGRATOR performs a first-round search of peak features in all files using continuous wavelet transform (CWT)–based pattern matching. For each MRM window, using the resulting survey of features, the algorithm determines the target number of features per transition as the median number of features detected in the initial survey, and treats it as the upper limit for the number of stable, consistent features to report in the second-round survey.

In the second round, the algorithm maps peak features across the samples so that every reported feature is stable (reliably detected across samples) and consistent. Rather than chaining each peak to the one in the next sample, it ranks the peaks in each sample by intensity, aligns them by rank, and takes the median retention time at each rank position across all samples — retaining a position only when a majority of samples show a peak there within a small RT tolerance. This yields a consensus set of feature retention times that is robust to individual missed or spurious peaks. The matching accounts for the RT drift estimated above. The user sets the RT_tol parameter to indicate the maximum deviation of RT from the value given in the method table; if no feature can be detected within this bound, the integrated area is reported as zero.

In outline, the consensus retention times are surveyed as follows:

peaks_per_sample = detect_peaks(sample) for each survey sample
N = min(median(peak count per sample), 12)                     # target features per transition
for k from N down to 1:
    keep the top-k peaks by intensity in each sample, sorted by RT
    for rank position i in 0 .. k:
        median_rt[i] = median RT at rank i across samples
        keep median_rt[i] only if >= half the samples have a peak within ~0.025 min
    stop at the largest self-consistent set
map each declared isomer to the nearest consensus RT, accepted only within RT_tol

Determination of peak integration borders

In the final step, the algorithm goes through the transitions to finalise the integration borders.

For the simple case of unimodal ion traces, the user can set the peak_width parameter quartet [w, x, y, z] to define the default start and end integration points. This can be customised for individual analytes with a peak width column in the transition-list table. The user can also set the uniform_width option (its own column in the transition table) so that the integration width remains the same between samples for a given transition, and can set fixed left/right integration bounds to fix the integration points with respect to the reference samples.

The expected RT and any fixed left/right integration bounds are interpreted in the retention-time frame of the reference sample(s) and drift-tracked from there into every sample.

Manually entered RT and borders are referenced to the reference sample

Values read from a non-reference sample that has shifted relative to the reference are offset by that shift in every sample, systematically misplacing the peak or its borders. Read the RT and any fixed borders from a sample designated reference in run_order.csv.

In complex cases — ion traces showing multiple peaks with or without baseline separation — the algorithm searches for valleys between features to cut the integration in the drift-corrected RT space, reconverts the border positions to the original RT space, and writes all RT borders to a CSV file (RT_matrix.csv) for user inspection and optional manual adjustment before peak areas are calculated.

Baseline determination

The baseline is determined on the x axis from the integration borders, while three modes are available for its position on the y axis.

The default mode sets a horizontal baseline at the 5th percentile of the ion-trace intensities, which provides a stable estimate of the chromatographic background that is insensitive to the height of the peak itself. The v_drop mode instead places a horizontal baseline at the lowest intensity of the integrated region, dropping the baseline to the trace minimum. The valley-to-valley mode fits a sloped baseline between the points where the left and right integration borders intersect the ion trace, which is appropriate where the background rises or falls across the peak.

Note

Where to set these parameters. RT_shift, RT_shift_bound, RT_tol, peak_width, uniform_width, and the fixed integration bounds are all configured in param.txt and the transition list — see Input Files for the syntax and defaults, and Tuning Peak Integration for choosing values. To see the algorithm in action across the four processing steps, see Processing Workflow.