Analytical Data
Analytical data, such as preprocessed measurement data (e.g., peak
areas), can be imported from various software platforms.
MiDAR currently supports data import from MRMkit, Agilent MassHunter
Quantitative Analysis (CSV files), MRMkit, and generic CSV files.
When importing data files generated by these platforms, it is important not to manually edit the files before importing. Manual edits can lead to file corruption or accidental errors in the data. MiDAR emphasizes reproducible and automated workflows, so manual editing of data is discouraged.
When importing data files generated by the supported platforms, it is important not to manually edit these files before importing them into MiDAR. This because this can easily lead to corrupt files that cannot be imported, or to accidential errors in the data. MiDAR emphasis reproducible/automated workflows, thus manual editing of data would be against this approach.
Two examples of how to import data from MRMkit and a plain CSV table with peak areas:
library(midar)
# Add analysis data from file generated by MRMkit
mexp <- MidarExperiment()
mexp <- import_data_mrmkit(mexp,
path = "datasets/sPerfect_MRMkit.tsv",
import_metadata = TRUE)
# Add analysis data from a CSV file (replaces all previous data)
mexp2 <- MidarExperiment()
mexp2 <- import_data_csv(mexp,
path = "data/plain_wide_dataset.csv",
variable_name = "area",
import_metadata = TRUE)See Importing analytical data for more details on importing analysis data.
Metadata
Metadata, refering to analysis metadata, i.e., data that describe/annotates the analytical data at analysis/sample and feature level, are essential data in the MiDAR workflow.
Metadata can be retrieved from the imported analysis data file as far as available. More commonly, additional or all required metadata needs to be important from other sources. Which metadata tables are required depends on the intended processing workflow, see Data and Metadata in MiDAR for an overview of metadata categories.
Integrity of metadata and data is key for accurate and reproducible data processing. MiDAR therefore inspects imported data/metadata for completeness of data and consistency of IDs used across different metadata tables. A summary of identified error, warnings and notes concerning metadata is provided after import.
Preparing metadata
Metadata can be imported from Excel Sheets, CSV files, or R data frames. Furthermore, a specific metadata template file can be used to prepare metadata in a structured way.
Preparation of metadata is often a manual step, where the information is collected from various sources. To facilitate this process, MiDAR provides metadata templates in Excel format. These templates contain column headers and instructions on how on how to use. In the beginning, it is recommended to start from a metadata template and fill in the required information. Templates for all metadata types supported by MiDAR can be obtained via:
midar::save_metadata_templates()Importing metadata
Specific metadata can be imported table by table, from CSV files:
mexp <- import_metadata_analyses(mexp,
path = "datasets/analysis_metadata.csv",
excl_unmatched_analyses = TRUE,
ignore_warnings = TRUE)
mexp <- import_metadata_features(mexp,
path = "datasets/feature_metadata.csv",
ignore_warnings= TRUE )Metadata can also be obtained from Excel workbook sheets, which
allows to store all metadata in one file.
In this case below we add metadata on internal standard and response
curves to the MidarExperiment object
mexp <- import_metadata_istds(mexp,
path = "datasets/metadata_tables.xlsx",
sheet = "ISTDs",
ignore_warnings= TRUE)Lastly, metadata can be obtained directly from R
data.frame objects, allowing users to prepare metadata in R
or to obtain them from, e.g., databases or a LIMS. Below we import
metadata on quality control samples:
df_qcinfo <- readr::read_table(file = "datasets/qc_metadata.txt")
mexp <- import_metadata_qcconcentrations(mexp, table = df_qcinfo)Preparing and Importing metadata using the MSOrganiser file template
Another option to import metadata is via the ‘MiDAR MSOrganzier’ template, an macro-enabled Excel file. See Metadata import.