Design an AI training pipeline.

### Signal to interviewer

I can build training infrastructure that accelerates iteration while maintaining reproducibility, governance, and deployment confidence.

### Clarify

I would clarify data sources, retraining cadence, domain risk level, and expected model release frequency.

### Approach

Use a data-to-model conveyor with stages: ingestion and filtering, feature/build prep, training orchestration, evaluation gates, and promotion workflow.

### Metrics & instrumentation

Primary metric: model promotion lead time. Secondary metrics: pipeline failure recovery time, training resource efficiency, and experiment throughput. Guardrails: post-release regressions, lineage gaps, and policy non-compliance events.

### Tradeoffs

Tighter validation improves reliability but slows iteration. More frequent retraining improves freshness but increases operational complexity.

### Risks & mitigations

Risk: data leakage into training; mitigate with strict split policies. Risk: non-reproducible runs; mitigate with immutable artifacts. Risk: silent quality drift; mitigate with continuous benchmark backtesting.

### Example

In a fraud detection system, daily data snapshots feed automated retraining, but promotion requires both offline robustness checks and shadow-online stability before rollout.

### 90-second version

Design training as a gated conveyor with full lineage and reproducible artifacts. Optimize release speed, but never bypass evaluation and governance gates.