The model you shipped last year is making decisions on this year’s world. Nothing has crashed. That is exactly the problem.
When infrastructure fails, it fails loudly. Servers go down, pages stop loading, pipelines throw alerts, and someone gets paged at 3am. Failure is visible, so it gets fixed.
Models do something worse. They keep answering. A credit model keeps scoring applicants, a demand forecast keeps ordering stock, a chatbot keeps advising customers – each of them slightly more wrongly than last month, with every dashboard showing green. Your AI was right the day you launched it. Whether it’s still right today is a question most organisations can’t answer, because nothing they monitor would tell them.
Accuracy Is a Photograph, Not a Property
A model is a snapshot of the world at training time. Customer behaviour shifts, prices move, fraud adapts, language changes, and the snapshot stays exactly where it was. Researchers call this drift, and it’s the normal condition of a deployed model rather than an edge case: a 2022 study in Scientific Reports tested 32 model-dataset pairs over time and found temporal degradation in 91% of them.
Degradation is not a risk your AI might develop. It is the default behaviour of every model you have in production, starting the day it ships.
The industry has spent three years learning to build these systems and almost no time learning to keep them healthy. Gartner predicted in 2024 that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025. MIT’s Project NANDA put it more bluntly the following year: 95% of enterprise GenAI pilots were showing no measurable return. Some of that is bad use-case selection. A meaningful share is systems that worked at launch and were never looked at again.
Nobody Owns the Run
Here’s the pattern we keep meeting in mid-market organisations. An AI or data system was delivered 6 to 18 months ago – sometimes by a build partner, sometimes in-house. The people who built it have moved on to the next thing. The internal team can keep the servers up, patch the containers, and restart the pipeline when it sticks. Ask them whether the model is still accurate and you get an honest shrug.
That’s not a criticism of the team. Detecting and diagnosing drift is specialist work: it needs logged predictions, reference datasets, statistical monitoring at the model layer, and someone who can tell the difference between a data-quality incident and genuine concept drift. It’s also a part-time need. No mid-market organisation can justify a full-time ML engineer to watch three models, so the capability simply doesn’t exist, and the risk sits unseen until it surfaces as a complaint, a bad quarter, or a regulator’s question.
Meanwhile two quieter problems compound. The inference bill creeps upward, because nobody owns cost at the model layer as a discipline. And the evidence trail – lineage, eval history, model cards – doesn’t exist, which matters more every year.
The EU AI Act makes post-market monitoring an explicit obligation for high-risk systems, with duties landing from 2026. “We built it carefully” is no longer an answer; “here is the evidence it is still behaving” is the required one.
Your Dashboards Are Green. Is Your Model?
The uncomfortable test: if your most important model had lost five points of accuracy since January, which screen in your business would show it?
For most organisations the answer is none. Infrastructure monitoring watches CPU, memory, latency, and uptime – the health of the box the model runs in, which is precisely the layer that doesn’t degrade. The model layer needs different instruments: data and concept drift detection against reference sets, accuracy tracking where ground truth arrives, evaluation scoring and guardrail monitoring for LLM systems, and cost per prediction watched as deliberately as cloud spend.
None of this is exotic. The tooling is mature and mostly open source. What’s missing is ownership – someone accountable for the model’s continued accuracy, on an agreed cadence, producing evidence a risk function can read. Run properly, it’s boring. That’s the point.
Q&A: AI in Production
How do I know if my AI model is drifting?
You can’t tell from infrastructure dashboards – uptime and latency stay healthy while accuracy decays. Drift detection needs model-layer monitoring: logged inputs and predictions compared against reference data, accuracy tracked where outcomes arrive, and statistical alerts when input distributions shift. If your system doesn’t log predictions today, that’s the first gap to close.
Whose job is it to monitor AI in production?
Someone must own it by name, and in most organisations today nobody does. The builders have moved on, and DevOps or hosting providers watch the infrastructure layer, which is the part that doesn’t rot. The role is specialist and part-time, which is why it usually makes more sense to buy it as a service than to hire for it.
What should AI model monitoring actually cover?
Four things: accuracy and drift (data and concept), data quality feeding the model, cost per prediction and overall inference spend, and governance evidence – lineage, eval reports, and model cards produced on a regular cadence. LLM systems add continuous evaluation and guardrail monitoring on top.
Does monitoring AI systems cost money to run?
Yes, and pretending otherwise causes mid-contract surprises. Logging and storing predictions, running drift jobs, retraining compute, and eval inference for LLM systems all carry real cost – usually modest next to the cost of an undetected failure, and partly offset by the inference savings that disciplined cost monitoring recovers. Any honest engagement surfaces these numbers up front.
Working Through This With Vertex Agility
This problem is why our Managed ModelOps service exists: the recurring assurance layer that keeps AI and data systems accurate, governed, and cost-controlled once they are live. We built it, or we adopt what you’ve built – every engagement opens with a fixed-fee audit that maps your stack, deploys monitoring in your own environment, and shows you exactly where you stand before anything ongoing is signed.
If you’re running models in production and can’t yet prove, on demand, that they’re still behaving, start with the smallest possible step: a free AI Health Scan on one model. Two data extracts, no labels, a straight red, amber, or green in days.