Recipes

CLI recipes and data loaders

Recipes are declarative pipelines — a fixed sequence of ingest, scatter, reduce, and (optionally) classify steps. CLI recipes are executable Python scripts installed as console scripts; agent recipes are MCP tool sequences.

vikshep-recipe calibrate

Fits a linear regression from aggregate features to a target scalar. Useful for detector calibration and response correction.

vikshep-recipe calibrate \
  --features <manifest.json>   # required: path to the ingest manifest
  --target   <column_name>     # required: aggregate scalar to predict
  --output   <report.json>     # optional: default = calibrate_report.json

Flag	Required	Description
--features	Yes	Path to manifest.json from vikshep-ingest
--target	Yes	Aggregate column name to regress against
--output	No	Output report path (default: calibrate_report.json)

vikshep-recipe tag

Trains a classifier on aggregate features with optional DisCo mass-decorrelation. The --protect flag enforces that the classifier score is statistically independent of the named variable, suppressing mass sculpting.

vikshep-recipe tag \
  --features <manifest.json>   # required
  --label    <column_name>     # required: binary target
  --protect  <column_name>     # optional: variable to decorrelate from
  --lambda   <float>           # optional: DisCo penalty (default 0.0)
  --output   <report.json>     # optional: default = tag_report.json

Flag	Required	Description
--features	Yes	Path to manifest.json
--label	Yes	Binary label column (0/1 or bool)
--protect	No	Column whose correlation with the score must be zero
--lambda	No	DisCo penalty weight (0 = off, 1 = balanced, >1 = strict)
--output	No	Output report path (default: tag_report.json)

Data loaders

Loaders are discovered via Python entry points (vikshep.loaders). Each loader writes POSIX shared memory and returns a 28-hex SHA3-256 OID. All loaders are in backend/ingest and installed with pip install -e backend/ingest.

Loader	Format	CLI entry point
g4	Geant4 CSV (event_id, layer, phi, theta, momentum[, energy])	vikshep-ingest g4
root-uproot	.root (Geant4, CMS Open Data)	vikshep-ingest root
hdf5	.h5, generic HDF5	vikshep-ingest hdf5
well	The Well HDF5 (15 TB physics simulations dataset)	vikshep-ingest well_slice

All loaders are CPU-only and require no account. The well loader is available in the Free tier; hosted well-scale ingest (> 1 TB) requires Lab or Enterprise.

Agent recipes

Agent recipes are declarative MCP tool sequences defined in agent/src/recipes/. They chain the same steps as the CLI recipes but run through the TypeScript/Bun MCP orchestrator, enabling async streaming, provenance logging, and the engine GPU paths.

Recipe	Pipeline
hep-tagging-disco	G4 ingest → aggregates → r₂ → DisCo classifier
bsm-anomaly	ingest → scatter → log-mean → HNSW → detect
general-feature	ingest → scatter (Dim, Group from request) → reduce

Manifest anatomy

The manifest is the control-plane payload produced by any loader. It contains OIDs and scalar metadata — no raw tensors.

{
  "n_events": 10,
  "schema": "komal_v1",
  "aggregate_names": ["layer1_n_hits", "layer1_e_mean", ...],  // 32 scalars/event
  "grid_oids": ["a3f1...", ...],  // 28-char SHA3-256 hex, one per event
  "pad_policy_per_axis": {
    "phi": "Circular",     // correct for azimuthal angle
    "theta": "ZeroPad"     // correct for polar angle
  }
}

The 28-char OIDs are the only handles that cross the AGPL boundary into the engine. Raw grid tensors live in POSIX shared memory and never reach TypeScript or the browser. The schema is frozen; changing it is a breaking change requiring a major version bump in both control plane and data plane.

Benchmark harness

Run the harness on a real analysis pipeline to sweep the DisCo lambda and record AUC-vs-dCorr² trade-offs:

python -m bench.run \
  --manifest manifest.json \
  --label    <signal_col> \
  --protect  <mass_col> \
  --lambdas  0,0.1,1,10

The harness outputs a sweep table and computes the Asimov proxy and Jensen-Shannon divergence for each lambda value. All free, all local.