AI Operating Leverage Case Notes.

Six weeks of expert review time captured pre-engagement across three senior reviewers. Median 3 hours per document; P90 4.2 hours. Time-of-week pattern: Monday-Tuesday peak, Friday low. Reviewer fatigue increased review time by ~12% in the second half of the week.

Identified second-order risks before engagement start: (1) regulator scrutiny if AI introduced into review without audit trail; (2) reviewer-displacement perception inside compliance org; (3) hallucination in retrieval against contract templates with edge-case clauses; (4) escalation drift if exception-routing logic decayed unobserved.

Retrieval-Augmented Generation (RAG) review system deployed in a secure private environment over proprietary documents. Documents pre-processed by AI agent with source citations and exception flagging. Senior reviewer validates exceptions and signs off. Workflow shipped Day 0 of engagement window with full handover documentation and Git history.

Private GPT-class LLM (no third-party data egress), pgvector embeddings, hybrid retrieval (semantic + keyword), structured output schema with JSON validation, audit-trail microservice (immutable log), CRO-defined escalation rules. From a practitioner governance: model registry, eval harness, weekly drift review.

Chief Compliance Officer named in engagement letter. Metric definition signed off pre-engagement: median document review time, expert hours redeployed, error rate against blind review sample.

12 weeks post-go-live. Same instrumentation as baseline. Time-of-week patterns aligned. Two reviewer changes during window documented as confounders (parental leave, promotion).

Internal audit function validated against blind review sample and documented baseline. Outcome was the audit-function-signed number, not the consultant's claim.

Twelve months of historical IoT sensor data captured: vibration spectra, motor temperature, output speed, line pressure across 47 critical assets. Pre-engagement maintenance posture was reactive break-fix; mean time between failure and mean time to repair logged for 12 months prior to define instrumentation.

Pre-engagement risks: (1) false positives triggering unnecessary maintenance — costs as bad as missed positives; (2) operator trust in alerts decaying after first false alarm cluster; (3) sensor drift not captured if model trained without anomaly-class sufficient data; (4) IT/OT interface failure modes if cloud integration introduced unmanaged dependencies.

Predictive ML models trained on historical IoT signals. Anomaly detection on multivariate sensor patterns preceding machine failure. Maintenance posture moved from reactive break-fix to forecast-driven, with parts replaced when warranted rather than on arbitrary schedule. Per-asset model registry with operator-validated alert thresholds.

Edge inference for low-latency anomaly scoring; cloud retraining pipeline; per-asset model versioning; alert escalation through CMMS integration; operator-side review tool for false-positive flagging that fed retraining.

VP Operations named as metric owner. Pre-engagement sign-off on metric definitions: maintenance cost (parts + labor + lost throughput), OEE measured to spec, mean time between failure across instrumented asset class.

16 weeks post go-live; matched against equivalent operating-condition window from prior year. Confounders: one major asset class added mid-window (logged as out-of-scope for measurement), two operator role changes documented.

Plant finance function validated cost result against ledger; OEE validated by ops engineering against MES instrumentation. Two reviewers; both signed off the result.

Eight weeks of pre-engagement support metrics: ticket volume, average resolution time, CSAT, deflection rate, escalation rate. Support team of 24 agents handling ~14,000 tickets / month, of which ~58% were Tier-1 (returns, shipping, order tracking).

Pre-engagement risks: (1) over-deflection — customers force-routed to bot get angrier than if escalated cleanly; (2) CRM context loss in handoff to human agent; (3) brand-voice drift in conversational AI; (4) customer-data exposure if AI agent had over-broad permissions; (5) emotional-tone failure on grief / complaint cases routed wrongly.

Conversational AI integrated directly into inventory and CRM systems — autonomously handling returns, shipping inquiries, and order tracking. Seamless escalation of emotionally complex cases to human agents with full context attached. Brand-voice fine-tuning anchored to existing knowledge base + macros.

LLM-powered intent classifier; CRM integration via existing API surface; inventory lookup against OMS; sentiment-aware escalation routing; agent-side context handoff UI; private memory layer for customer-recognized cases (consent-managed).

VP Customer Experience named as metric owner. Sign-off pre-engagement on metric definitions: Tier-1 deflection rate, average resolution time across all ticket types, repeat purchase rate at 90/180 days.

12 weeks post go-live, with seasonal adjustment against prior-year comparable window. Repeat purchase rate measured at 180 days against matched cohort.

CX analytics function validated deflection and resolution time. Finance function validated repeat purchase rate against ledger. Both signed off.