Concept to governed production.

1. The model is the small part. The system around it — data, evaluation, governance, observability, human oversight — is where production value and production risk actually live.
2. A confident wrong answer is the enemy. Every design choice is measured against whether it makes the system more honest about what it does and does not know.
3. Governance is a property you build in, not a bolt-on. If traceability, evaluation, and human oversight aren't in the reference architecture, they won't survive delivery pressure.
4. Optimize for reversibility. Standardize the expensive-to-change layers — governance, evaluation, data contracts — early; stay deliberately swappable on the fast-commoditizing layers, above all the model itself.
5. Depth over duration. Judge an approach by shipped systems and demonstrated failure modes survived, not by how long a tool has existed.

Separate what never changes from what flexes per engagement.

Invariant layers — fixed across projects:

• Feature / context layer with online–offline consistency. Train-serve skew is the most common silent production failure.
• Model (or artifact) registry — the system of record for what is deployed, its version, and its validation status. Nothing serves that isn't registered.
• Evaluation harness — every model, prompt, and retriever passes it before promotion.
• Observability + evidence layer — inputs, outputs, drift signals, and a reconstructable record of every consequential decision.

Substrate — flexes on data gravity, total cost of ownership, and governance surface:

• Data-resident / warehouse-native when the data gravity and governance already live there — minimizes egress and copy-governance.
• Cloud-hosted / containerized when custom training, multi-model serving, or low-latency patterns don't fit the warehouse.
• On-prem / local model serving when data residency or restriction demands it.

“Is the model good?” is the wrong question. “Is it good enough to advance to the next level of exposure?” is the right one.

1. Offline. Versioned, representative test set with disaggregated subgroup metrics — subgroup failure blocks promotion even if the aggregate passes.
2. Shadow. Score live traffic without acting; compare to the incumbent and to realized outcomes. Catches train-serve skew and distribution shift offline eval can't.
3. Canary / A–B. A pre-registered success metric and a guardrail metric — measure the lift, watch for the thing that could break.

For GenAI / RAG the ladder is identical; the metrics change:

Classical vs LLM. Structured prediction, ranking, anomaly detection → classical: more accurate, cheaper, SHAP-explainable, deterministic and auditable. Unstructured retrieval, natural-language interface, document grounding → LLM/RAG. Mature systems are hybrid; the skill is knowing which layer owns which job. You cannot put a stochastic text generator in front of an auditor asking “why was this scored high.”

Build vs buy the model. The model layer is commoditizing fastest — stay model-agnostic and invest in the platform underneath. Own the governance and evaluation; rent the frontier model.

Substrate choice. Follow data gravity and total cost of ownership, not tool preference. Document the decision as a reusable pattern so the next engagement doesn't relitigate it.

RAG design. Chunk with overlap and per-chunk metadata for filtering → embed → vector store → semantic retrieval → re-rank (raw similarity surfaces plausible-but-wrong) → generate strictly from context with per-claim citation → abstain when retrieval is empty.

Three frameworks answer different questions; use them together.

• SR 11-7 (model risk management): effective challenge via validation independent of development, conceptual soundness, ongoing monitoring, outcomes analysis, backed by a model inventory and documentation.
• NIST AI RMF: the operating structure — Govern, Map, Measure, Manage.
• EU AI Act: risk-tiered obligations — classify each system, then meet the high-risk duties: risk management, data governance, technical documentation, logging & traceability, human oversight, accuracy & robustness.

The AI supports an accountable human decision; it is never the decision.

• Review — surface outputs with their evidence; the reviewer sees what the model saw.
• Escalate — route by confidence × risk; senior human attention goes to hard, high-impact cases, not uniformly.
• Override — the human can always overturn, and the override is captured as labeled feedback.
• Feedback — overrides and corrections flow back into the eval set and retraining.

• Data drift — input distribution vs training (population-stability-style). The earliest warning, because inputs shift before performance visibly degrades.
• Concept drift — rolling performance on realized outcomes; the input can look stable while the relationship underneath changes.
• Operational — cost, latency, error; for LLM systems add retrieval quality and abstention rate over time.

• Notebook-to-prod. A model with no tests, container, API, or CI/CD is a prototype, not a system.
• Aggregate-metric blindness. A strong headline number hiding subgroup failure.
• Explanation-as-validation. SHAP tells you what the model uses, not whether it should — use it to find leakage and proxies, not to certify fairness.
• Leakage. A feature available at training but not at prediction time; audit top features that look “too good.”
• Silent partial failure. A pipeline that half-runs and ships wrong numbers is worse than one that crashes loudly.
• Governance as afterthought. Traceability retrofitted after a regulator asks is the most expensive way to build it.