The pitch is hard to argue with. Point an agent at three years of sales history, give it a tool that queries your warehouse, and let it tell you how many units to order next month. No data science team, no model retraining, no six-month project. Just a prompt and a schedule.

Teams try it. It produces numbers. The numbers look reasonable. And then, about two quarters in, someone compares the agent’s forecasts against what actually sold and discovers the whole thing has been quietly worse than the spreadsheet it replaced.

The failure isn’t that agents are useless here. It’s that most implementations put the agent in the one role it’s worst suited to.

An LLM has no feedback loop unless you build one

A statistical forecaster gets better because it is retrained on what actually happened. Last month’s error becomes this month’s correction. That loop is the entire mechanism.

An agent prompted to forecast has no such loop. It reasons from the same context window every run. When November’s actuals come in 30% below its prediction, nothing in the system updates — the next run reasons from scratch, makes a similar inference from similar data, and produces a similar miss. You can paste the errors back into the prompt, and teams do, but a context window is not a training signal. It’s a hint the model may or may not weight correctly, and you have no control over which.

This is the difference that matters most and gets noticed last, because a forecast that’s wrong in a stable direction still looks plausible every single time you read it.

Confident seasonality that isn’t there

Give a language model eighteen months of weekly sales and ask what it sees. It will find a story. There’s a clear Q4 uplift. Demand softens in early spring. The dip in week 23 looks like a supply issue rather than a demand signal.

Some of that will be real. Some of it is noise wearing a narrative, and the model has no way to tell you which is which, because it isn’t estimating confidence — it’s generating the most plausible-sounding description of a series. Statistical methods produce prediction intervals precisely so you can see when the data doesn’t support a claim. An agent produces prose, and prose is uniformly confident.

Eighteen months is two observations of Q4. No method can establish annual seasonality from two cycles. A forecasting library will tell you that. An agent will describe the pattern anyway.

Forecast error costs money asymmetrically, and accuracy metrics hide it

This is the one that quietly destroys the business case.

Understocking a high-margin product that customers will simply buy elsewhere costs far more than overstocking it. For perishables the asymmetry runs the other way, and hard. For a component that halts a production line, a stockout costs multiples of the part’s value.

An agent told to be accurate optimises symmetric error. It treats being 100 units over and 100 units under as equally bad, because in every metric it has absorbed from its training data — MAPE, RMSE — they are. Serious forecasting systems don’t optimise accuracy at all. They optimise quantiles tied to a service level, deliberately biasing the forecast in whichever direction is cheaper to be wrong in.

You cannot prompt your way to that. It’s a loss function, not an instruction.

The pattern that works: agent as orchestrator, not predictor

The teams getting real value from agents in supply chain aren’t asking them to produce the number. They’re wrapping them around a model that does.

The numeric forecast comes from an actual forecasting method — gradient boosting on engineered features, a seasonal statistical model, a deep learning architecture if the data volume justifies it. That component is backtested, versioned, and retrained on a schedule. It’s boring, and boring is the point.

The agent handles everything around it, which turns out to be most of the work an analyst used to do by hand:

• Data preparation and anomaly flagging. Spotting that a SKU’s history contains a three-week outage, deciding whether to treat it as missing or as genuine zero demand, and documenting the choice.
• Scenario runs. “What happens to Q3 if the promotion slips two weeks?” — an agent can set up, execute, and compare a dozen of these in the time an analyst configures one.
• Explanation. Translating a quantile forecast into a sentence a category manager will act on, including what would have to be true for it to be wrong.
• Exception routing. Deciding which of four hundred SKU forecasts need a human to look at them, and why.

That division of labour plays to what each component is actually good at. The model handles numbers under uncertainty. The agent handles context, language, and orchestration. The approaches worth studying in AI demand forecasting consistently land on some version of this split, and the projects that skip it tend to be the ones being quietly rebuilt a year later.

Four things to check before you ship

Backtest on a rolling origin. A single holdout period tells you how the system performed in one window that may have been unusual. Roll the origin forward across many windows and you find out whether it holds up.

Pick an error metric tied to inventory cost. If your evaluation is MAPE and your business loses ten times more from stockouts than from overstock, you are measuring the wrong thing and will optimise toward it.

Beat the naive baseline first. Same period last year, adjusted for trend. A surprising number of sophisticated systems don’t, and nobody checks because nobody ran the baseline.

Keep the override path and log it. When a human overrules the forecast, capture what they changed and why. That log is the most valuable dataset you’ll build, and it’s the only honest record of whether the system is trusted.

The hard part is organisational

The split is easy to describe and awkward to staff. Forecasting and LLM engineering are different disciplines with different tooling and, more importantly, different habits. Forecasting people are suspicious of anything that can’t be backtested. Agent people are used to shipping on prompt iteration and fast feedback. Both instincts are correct in their own domain and actively harmful in the other, which is why a team with only one of them tends to build the wrong half well.

That mismatch, rather than the modelling, is where most of these projects stall. The pattern Netguru sees work most reliably is to put one person in charge of the evaluation harness before either component is built. Settle what “better” means while it’s still an argument about metrics, and the later argument about whose model to keep mostly resolves itself.

The alternative — build the agent, build the model, then discover the two teams were optimising different things — is how a system that everyone signed off on ends up quietly losing to a spreadsheet.

None of this makes agents the wrong tool

It makes them the wrong tool for the specific job of predicting a number from a time series, which is a solved problem with well-understood methods that agents don’t improve on. Point them at the ninety percent of forecasting work that was never really about the number, and the results hold up.