Supply Chain Analytics · 13 Sep 2026 · 2 min read
What to ask a supply chain AI vendor before signing
Demand forecasting demos are easy to win. They run on clean historical data, against a naive baseline, with hindsight. Production is none of those things. Five questions close most of the gap.
One: what baseline are you beating
An 18% accuracy improvement is meaningless until you know the comparison. Beating a naive last-period forecast is table stakes. Beating your current planners, who apply judgement the model cannot see, is the real bar. Insist the baseline is your incumbent process, not a textbook one.
Two: how does it handle a promotion it has never seen
Most forecasting failures in retail and CPG are promotional. Ask what the model does with a discount depth or mechanic absent from history, because that is exactly what marketing will run next quarter.
the five questions
- What baseline are you measuring against, and is it our current process?
- How does the model behave on a promotion type it has not seen?
- What happens on a new SKU with no history, and for how long?
- Who retrains it, on what trigger, and what does that cost us?
- What does a planner see when they disagree with the forecast?
Three: the cold-start problem
New products are where planners most want help and where models are weakest. Ask specifically how long a new SKU is handled by fallback logic, and what that logic is. If the answer is that the planner overrides manually for twelve weeks, that is fine, but it changes the business case.
Four and five: who owns it after go-live
Retraining cadence and override design are the two questions most commonly skipped, and the two that decide whether anyone still uses the system in a year. A forecast a planner cannot argue with gets ignored, and a model nobody retrains degrades quietly. Ask who holds both jobs on day 400, and get the answer in writing.