Skip to contents

Design Decisions: Why This Architecture?

Source: vignettes/articles/manual-03-design-decisions.Rmd
manual-03-design-decisions.Rmd

Manual

Audience

This article is for:

  • Power users who want to understand why QUANT works the way it does
  • Contributors who want to extend the package
  • Developers building tools on top of MRMhub

To start processing data directly, see Your First Analysis instead.

Decision 1: S4 Class (MRMhubExperiment) vs Tidy Tibbles

Choice: A single S4 object holds all data, metadata, and processing state.

Alternatives considered
Alternative Why not
Pure tibble workflow (tidyverse) No built-in integrity checks; too easy to desync data and metadata
SummarizedExperiment (Bioconductor) Models a feature × sample matrix, not a processing pipeline: no typed home for the original data, the processing state, or the non-rectangular annotation tables. Supported as an export target instead
R6 class Less formal than S4; no validity checking; mutable-by-reference is error-prone for data analysis

Rationale:

  1. Traceability — The S4 object bundles original data, processed data, annotations, and processing flags in one place.

  2. Integrity checking — S4 validity methods ensure data and metadata stay in sync. Analyses cannot be accidentally deleted without updating annotations.

  3. Pipeline state — Flags like is_istd_normalized, var_drift_corrected prevent accidental double-processing.

  4. Not SummarizedExperiment — but exports to it. SE models a feature × sample matrix. It offers no typed home for dataset_orig, for the processing state, or for the annotation tables that are not one-row-per-feature (annot_responsecurves, annot_qcconcentrations, metrics_calibration) — they can only ride along as an untyped metadata() list. And while colData happily carries qc_type and batch_id, no Bioconductor class models them, nor calibration curves or internal-standard relationships: nothing downstream knows that a PBLK column is a blank, or that one feature normalizes another. Enforcing that is the work QUANT exists to do.

    The argument is about the internal representation only. Several parallel intensity variables are no obstacle — SE holds them as parallel assays, which is exactly what the exporter relies on. Once an experiment is processed, save_dataset_summarizedexperiment() hands it to limma, lipidr, POMA and the rest of the ecosystem.

# The object structure:
slotNames("MRMhubExperiment")
#>  [1] "title"                  "analysis_type"          "feature_intensity_var"
#>  [4] "conc_analyte_unit"      "dataset_orig"           "dataset"
#>  [7] "dataset_filtered"       "annot_analyses"         "annot_features"
#> [10] "annot_istds"            "annot_responsecurves"   "annot_qcconcentrations"
#> [13] "annot_studysamples"     "annot_batches"          "metrics_qc"
#> [16] "metrics_calibration"    "status_processing"      "is_istd_normalized"
#> [19] "is_quantitated"         "is_filtered"            "has_outliers_tech"
#> [22] "is_isotope_corr"        "analyses_excluded"      "features_excluded"
#> [25] "var_drift_corrected"    "var_batch_corrected"

Decision 2: Long-Format Data Exchange

Choice: The canonical data exchange format is a long CSV with one row per feature per sample.

Alternatives considered
Alternative Why not
Wide format (features as columns) Scales poorly with many features; ambiguous NA meaning; awkward for filtering
HDF5 / Parquet Adds binary dependency; not human-inspectable at handoff point
Database (SQLite) Overkill for typical study sizes; adds complexity
Long format Wide format
Handles variable feature counts naturally Requires NA padding
Easy to filter, group, join Awkward for row-wise operations
Natural for ggplot2 Requires pivot before plotting
Explicit about missing vs zero Ambiguous NA meaning 
# Expected long format:
# | analysis_id | feature_id | area   | rt   |
# |-------------|------------|--------|------|
# | Sample_001  | LPC_18:1   | 12345  | 2.31 |
# | Sample_001  | LPC_16:0   | 23456  | 1.98 |
# | Sample_002  | LPC_18:1   | 11234  | 2.30 |

Decision 3: Feature and analyte are distinct identifiers

Choice: feature_id identifies a distinct signal extracted from the MS data, while analyte_id identifies the compound that signal measures. The two are kept separate because they are not one-to-one.

Alternatives considered
Alternative Why not
A single ID equating feature and compound Cannot express one analyte measured by several features, nor one feature shared by isobaric analytes
Transition-level IDs only Loses the compound-level grouping needed for QC concentrations and reporting

Rationale: A feature is a distinct signal extracted from the MS data. Feature and analyte are not one-to-one: one analyte can give several features (isotopes, adducts, transitions), and one feature can carry several analytes when they are isobaric or isomeric (e.g. SM/PC). INTEGRATOR selects the quantifier signal for each feature, so by the time data reaches QUANT each feature_id is one value per sample, while analyte_id links the features that measure the same compound.

Decision 4: Immutable Original Data

Choice: dataset_orig is never modified after import. All processing operates on dataset.

Why this matters
  • Reproducibility: re-run any step without re-importing
  • Debugging: compare original vs processed to verify corrections
  • Auditability: the raw data is always accessible for review
# You can always reset:
mexp@dataset <- mexp@dataset_orig

# Or compare before/after:
original <- mexp@dataset_orig |> dplyr::filter(feature_id == "LPC_18:1")
processed <- mexp@dataset |> dplyr::filter(feature_id == "LPC_18:1")

Decision 5: QC-Centric Correction

Choice: Drift and batch correction are exclusively based on QC samples, never study samples.

Alternatives considered
Alternative Why not
Study sample–based correction (e.g., median centering on all) Removes real biological signal
External reference only Not always available; less responsive to within-run drift

Reference: Broadhurst et al. (2018). Metabolomics, 14, 72.

Implication: Sufficient QC samples (≥ 5 per batch, evenly distributed) are required for correction to work well.

Decision 6: Explicit Processing Order

Choice: Functions should be called in a specific order. The package communicates via status flags rather than enforcing with hard errors.

# Recommended order:
# 1. import
# 2. add_metadata
# 3. set_analysis_order
# 4. normalize_by_istd
# 5. correct_drift_*
# 6. correct_batch_*
# 7. quantify_by_*
# 8. calc_qc_metrics
# 9. filter_features_qc
Why not enforce strict order?
  • Some workflows skip steps (no ISTD, no drift correction)
  • Researchers need flexibility to experiment
  • Hard errors would frustrate exploratory analysis
  • Status flags provide guidance without blocking

Decision 7: Separation of INTEGRATOR and QUANT

Choice: Raw data processing (peak integration) and post-processing (normalization, QC) are separate tools in different languages.

INTEGRATOR QUANT (R)
Automated, batch-oriented Interactive, exploratory
Computationally intensive Statistically intensive
Run once per study Run iteratively
No user decisions needed Many user decisions

The interface is a CSV file. Either tool can be replaced independently, and data can be inspected at the handoff point.

Why QUANT is an R package

QUANT is implemented in R rather than Python because the bulk of its work is statistical and visual: smoothing, robust regression, multivariate QC, and ggplot2-driven inspection plots. R provides established, audit-friendly implementations of these primitives (loess, cubic splines, prcomp, kernel smoothers) and a large neighbouring ecosystem (tidyverse, Bioconductor, shiny) familiar to most academic mass spectrometry and bioinformatics groups. Literate .Rmd / .qmd notebooks make the final analysis self-documenting: the same file produces the figures, the report, and the audit trail.

Extending MRMhub

Guidelines for new functions

All functions should:

  1. Accept MRMhubExperiment as first argument
  2. Return MRMhubExperiment (for pipeline functions) or a result object
  3. Update relevant status flags
  4. Not modify dataset_orig
Extension type Pattern
New correction method R/correct-*.R — takes and returns MRMhubExperiment
New importer Add parse_* function, register in import_data_main()
New plot Follow R/plots-*.R — takes mexp, returns ggplot
New QC metric Extend calc_qc_metrics() or add a helper

References

Next steps