Scientific ML

Frame the scientific decision before choosing a model

Start with the quantity that a scientist or engineer must decide, not with an architecture. Define the operating envelope, trusted evidence, acceptable error, cost of a wrong answer, and the fallback that remains available.

Write the contract in physical terms

• Inputs: name each state, parameter, geometry, forcing, and boundary condition with units and allowed ranges.
• Outputs: identify whether the model predicts a scalar, field, trajectory, distribution, or decision-relevant functional.
• Regime: bound Reynolds number, temperature, composition, mesh family, forecast horizon, or another domain-specific coordinate.
• Reference: state which experiment, analytical result, or converged solver defines credible evidence.
• Tolerance: set error and constraint limits per variable and regime, not one unqualified score.
• Fallback: preserve a trusted solver, conservative controller, human review, or explicit refusal outside the validated envelope.

Treat data as sampled physics, not interchangeable rows

Scientific datasets have spatial, temporal, geometric, and experimental dependence. A random row split can leak nearly identical states into train and test sets while leaving the model untested on the regime that matters.

Build evidence around regimes

• Split by complete trajectories, experiments, sites, geometries, material families, time windows, or parameter ranges.
• Record units, coordinate systems, sensor calibration, missingness, mesh and discretization, solver tolerances, and boundary conditions.
• Nondimensionalize when it exposes shared physical structure, but retain the transformation and reference scales in provenance.
• Sample boundaries, discontinuities, rare events, and transition regions deliberately; uniform sampling can underrepresent them.
• Keep a protected challenge set that reflects deployment decisions and is not repeatedly optimized during development.

Separate uncertainty sources

1. Measurement uncertainty comes from instruments, labeling, preprocessing, and observation geometry.
2. Numerical uncertainty comes from discretization, convergence tolerance, approximation order, and solver implementation.
3. Parameter uncertainty reflects unknown coefficients, initial states, or boundary values.
4. Model-form uncertainty reflects missing or simplified mechanisms in the scientific model.
5. Surrogate uncertainty reflects finite data, imperfect optimization, and limited model capacity.

Collapsing these into one error bar hides which intervention can improve the system.

Choose where scientific structure enters the model

Scientific knowledge can shape the data, architecture, objective, or output. Stronger enforcement is useful only when the encoded knowledge is correct for the operating regime.

Balance fit, constraints, and coverage without hiding failures

A common physics-informed objective combines observed data, governing-equation residuals, boundary or initial conditions, and regularization. The terms usually have different units, scales, sampling densities, and gradient magnitudes. The scalar total loss can improve while one scientifically critical term gets worse.

Use the workbench to change the scientific case, model strategy, observation coverage, and constraint emphasis. Watch in-regime fit, equation residual, boundary error, extrapolation risk, and training cost move independently.

Read every objective as a ledger

• Normalize residuals with physically meaningful reference scales before weighting them.
• Report every component in its own units or nondimensional form; never publish only the weighted sum.
• Inspect gradient conflict when improving one term consistently damages another.
• Resample collocation and boundary points where residual structure changes, not only where data is dense.
• Stop training against protected validation regimes; use them to test the claim, not tune it.

The executable example mirrors the lab's illustrative budget. It keeps each evidence channel visible instead of treating a weighted objective as proof of validity.

Physics-informed neural networks are one tool, not the definition

A physics-informed neural network, or PINN, represents a candidate solution with a neural network. Automatic differentiation evaluates derivatives of that candidate with respect to its coordinates, and training penalizes governing-equation and boundary residuals alongside observations.

1. 1

   Contract

   Specify the mathematical problem

   Define variables, units, domain, equations, known and inferred parameters, boundary conditions, and the quantity used for validation.

2. 2

   Evidence

   Sample distinct evidence

   Keep observations, interior collocation points, boundaries, interfaces, and initial conditions identifiable in the dataset and objective.

3. 3

   Train

   Optimize a visible ledger

   Track data, equation, boundary, and regularization terms separately, including their gradients and regime-level behavior.

4. 4

   Validate

   Challenge the solution

   Compare against analytical results, converged solvers, experiments, conservation checks, and held-out parameter regimes.

Know when a PINN can struggle

• Multi-scale or stiff dynamics can produce badly imbalanced gradients.
• Shocks and discontinuities are difficult for smooth network parameterizations and residual losses.
• A small residual at sampled points does not prove a globally accurate solution.
• Inverse problems can remain non-identifiable even when optimization converges.
• Training one network per problem instance can be more expensive than a mature numerical solver.

Use a PINN when its mesh-free differentiation, inverse formulation, or data-equation fusion serves the scientific contract. Use a classical solver, hybrid method, or operator learner when that produces stronger evidence at lower total cost.

Validate a surrogate at matched scientific utility

Surrogate validation compares more than wall-clock latency. The reference solver and learned model must be evaluated at a matched level of accuracy and on the same hardware, precision, domain, and output quantities.

Design splits that test the claim

• Interpolation split: withhold samples inside the represented parameter envelope.
• Regime split: withhold a connected range, geometry family, forcing pattern, or material class.
• Resolution split: change mesh or sensor resolution without confusing resolution with new physics.
• Temporal split: evaluate longer horizons and later operating periods to expose accumulated error and drift.
• Cross-method split: compare with an analytical result, independent codebase, alternate discretization, or physical experiment.

Measure the quantities that can change a decision

• Field norms can hide local extremes, so report boundary, peak, interface, and integral errors where relevant.
• One-step error can hide rollout instability, so test the full decision horizon.
• Average calibration can hide failure in a rare regime, so report coverage and sharpness by regime.
• A speedup without preprocessing, data generation, transfer, or verification cost can misstate operational value.

Make uncertainty trigger an operational response

Uncertainty is useful only when it changes what the system does. An ensemble spread, posterior interval, or conformal interval must be checked against empirical errors on representative calibration data. Outside that evidence, uncertainty is a warning signal rather than a guarantee.

Use the release lab to inspect a regime change, vary independent evidence and empirical interval coverage, control interval width, and commit to a release posture. The lab separates calibration from sharpness: an interval can cover often and still be too wide to support the decision.

Define the fallback before the alert

• Route unsupported inputs to a trusted solver or measurement process.
• Narrow the operating envelope when evidence is insufficient for one regime.
• Request a new simulation or experiment where expected information gain is high.
• Escalate high-consequence results for independent scientific review.
• Preserve inputs, model version, data version, uncertainty output, gate decision, and eventual observed outcome.

Operate the model as part of a scientific instrument

The production unit is the complete evidence path: data acquisition, transforms, scientific model, learned weights, uncertainty method, validation suite, and downstream decision. Version and monitor that whole path.

Record enough provenance to reproduce a result

• Raw data and calibration versions, units, coordinate conventions, and excluded observations.
• Simulator source, mesh, discretization, solver tolerances, seeds, and hardware precision.
• Feature transformations, nondimensional reference scales, architecture, objective terms, and checkpoints.
• Evaluation datasets, regime definitions, baseline settings, uncertainty calibration, and reviewer decisions.
• Runtime input envelope, extrapolation indicators, fallback events, and downstream scientific outcomes.

Watch for scientific drift

• Sensor replacement can alter noise, resolution, or missingness before a distribution metric moves.
• A solver upgrade can change labels and numerical uncertainty without any physical change.
• A new geometry or material family can invalidate an interpolation claim.
• A downstream optimization policy can concentrate requests near the surrogate's weakest region.
• Recalibration can improve aggregate coverage while degrading a safety-critical slice.

When any boundary changes, reopen the evidence contract. Retraining alone does not establish that the changed system is scientifically credible.

Use a release checklist that can fail closed

1. Specification: variables, units, equations, assumptions, regime, and decision tolerance are explicit.
2. Evidence: observations and numerical labels have provenance, uncertainty, and leakage-resistant splits.
3. Baselines: the learned model is compared with trusted numerical and statistical methods at matched utility.
4. Constraints: residuals, boundaries, conservation, symmetry, and stability are measured separately.
5. Uncertainty: empirical coverage and interval usefulness are tested by protected regime.
6. Containment: extrapolation is detected and a trusted fallback is tested under realistic dependencies.
7. Reproducibility: code, configuration, data versions, seeds, precision, and hardware are sufficient to repeat the result.
8. Review: domain experts and independent reviewers can inspect the evidence behind the release decision.

Primary references

• Physics-informed neural networks introduces residual-based neural formulations for forward and inverse problems governed by nonlinear partial differential equations.
• Fourier Neural Operator for Parametric Partial Differential Equations learns mappings between function spaces across families of PDE instances.
• Neural Operator: Learning Maps Between Function Spaces develops the operator-learning framework and surveys approximation and discretization considerations.
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles evaluates ensembles as a practical predictive-uncertainty method.
• On Calibration of Modern Neural Networks distinguishes predictive accuracy from calibrated confidence and evaluates post-hoc calibration.
• PDEBench provides diverse time-dependent PDE benchmarks and evaluation guidance for learned surrogates.