The Model Hadn't Changed, but the Camera Had

The first message said the results had become a little odd. There was no dramatic failure and no red error page. A system that usually produced ordinary, usable outputs was sending more cases for review, and a few familiar images were getting unfamiliar answers.

My first thought was a software change. I checked the model version, then the preprocessing code. Both matched the previous week's records. I read the deployment notes twice because I expected to find the answer there and was reluctant to accept that I hadn't.

Eventually someone sent a phone photograph of the room. A shelf had been moved. The camera still pointed at the same work area, but its angle had changed slightly during the rearrangement. A new light above the shelf was reflected in part of the image.

I had been inspecting the model's history while the relevant change was visible in the corner of an ordinary photograph. By the time I understood that, my eighteen-month-old daughter had rearranged half the objects on my own desk. I was beginning to appreciate how much a place can change without anyone describing it as a technical event.

A version number that reassured me

The pipeline was a collection of ordinary parts: a fixed camera, image checks, a trained model and a small review interface. We had spent most of our development effort on the part with learned weights. Once it was working, that part also received most of the attention in our discussions.

A stable model identifier reassured me because it meant we hadn't accidentally replaced the central component. But the identifier said nothing about what the camera now saw. The system could run the same arithmetic perfectly on a different kind of input.

I compared recent images with older ones. The new reflections were obvious once I put the examples side by side. Looking at either group alone, I could have accepted them as normal. My memory of the scene was much less precise than the saved photographs.

There were smaller changes too. A batch of material had a different surface finish. Some labels had been printed with slightly different spacing. Neither change was outrageous from the perspective of the people doing the work. Both mattered to a model trained on a narrower set of appearances.

I asked when each change had happened. The dates were approximate because nobody had been keeping a modelling diary for the room. We reconstructed the sequence from maintenance notes and a few photographs, marking the uncertain parts instead of turning guesses into exact timestamps.

This was slower than opening a dashboard, but it gave the numbers a setting. The rise in review cases began near the rearrangement. A different type of error appeared later with the changed material. There wasn't one universal explanation waiting inside a single metric.

What the system could tell us

Our monitoring covered whether the service was running and how long a request took. It did a reasonable job of detecting a stopped process. It was much less useful for detecting a process that kept running while answering the wrong question about changing inputs.

We added basic checks on the images themselves. Dimensions and file format were easy to inspect. Brightness summaries and a few other simple measurements helped identify some shifts, though none could describe the whole visual situation.

A bright reflection can occupy a small area while leaving the average brightness almost unchanged. A new object can be important without dominating a global summary. I didn't want a page of input statistics to create a new version of the same false reassurance.

We kept a small sample of recent inputs for authorised review, subject to the project's retention rules. Someone needed to look at actual images, not only derived numbers. The sample was selected by a documented process rather than by whichever examples made the system look most reasonable.

I also wanted rejected inputs to remain visible in the counts. If a new condition caused many images to fail before inference, the model's reported accuracy on the surviving images might remain steady. That would be a poor summary of what users experienced.

The Google material on monitoring production machine-learning systems describes several of these concerns, including differences between training conditions and serving conditions. Reading such guidance is straightforward. Remembering to apply it when a small system seems stable is the part I have found easier to neglect.

We started with a few checks tied to known failures. I preferred a limited set that someone would investigate to a comprehensive-looking display that nobody owned.

The answer arrives later

For some tasks, we didn't immediately know whether an output was correct. A reviewer might resolve a case later. Other cases would receive no explicit correction unless someone noticed a problem. Treating silence as confirmation would have made the numbers look better than the evidence allowed.

I separated outputs with confirmed outcomes from outputs whose outcome remained unknown. I also recorded how long confirmation took. A sudden fall in recorded mistakes could mean improvement, or it could mean that the checking process had slowed down.

We looked at how review cases were selected. If people checked only low-confidence outputs, we would learn more about that subset than about confident mistakes. A small, permitted sample from other outputs gave us a way to inspect a broader part of the work.

This required time from someone who already had other responsibilities. I couldn't solve that constraint by writing an ambitious review plan. We agreed on an amount of checking that could actually happen, then made sure the report reflected its limited coverage.

If nobody has checked it yet, the outcome is still unknown.

That sentence went into the reporting notes because our language kept drifting towards certainty. A missing correction was convenient to count as success. It wasn't the same thing.

The late answers also complicated comparisons between versions. A recent version had had less time to accumulate confirmed mistakes than an older one. We compared equivalent windows where possible and kept the unresolved cases visible. The resulting figures were less immediately satisfying and much harder to misread.

At lunch I drew a little timeline on a paper receipt to explain this to a colleague. The receipt had a large coffee stain across one end. It was still the clearest explanation I had produced that morning.

Retraining was an option, not the first button

We could collect new examples and update the model. That might eventually be necessary. It wasn't the first thing I wanted to do while we still had a moved camera and an uncontrolled reflection.

We checked whether the physical arrangement could be restored to the intended specification. For part of the problem, it could. The new material was a different matter: it represented a legitimate condition the system needed to handle. Calling every unfamiliar input a setup mistake would have been as unhelpful as retraining after every loose screw.

We divided the cases by cause as far as the evidence allowed. Some required changes to capture. Some required clearer input validation. Others belonged in an expanded evaluation set before any model update was judged.

When we tried an updated model, we checked it against both recent examples and an older held-out set. I didn't want to fix the new surface finish by quietly making the original conditions worse. We kept a record of which examples influenced development so that our final check wouldn't become another rehearsal.

A simple comparison found a regression on a previously easy category. The new model's overall result looked acceptable because that category occupied a small share of the test. We paused the rollout and inspected the errors rather than averaging the problem away.

There was a temptation to add special-case rules until every known example passed. Some rules were legitimate expressions of the task. Others would merely memorise the current collection. We discussed what each rule meant outside the example that had suggested it.

The eventual update was smaller than my first proposal. We repaired the capture arrangement, added a check for one unsupported input type and made a limited model change after testing it. The work took several days. Most of it wouldn't make an exciting demonstration, but it left us with fewer unexplained assumptions.

Someone has to receive the warning

A warning with no owner is an interesting decoration. We had one of those. It appeared in a place that everyone could access and nobody checked consistently. I had mistaken availability for a working process.

We agreed who would look at a warning, what initial checks they could perform and when to involve someone else. The first steps were concrete: inspect a small sample, compare input counts and check the recent change log. They didn't require the reader to understand every detail of the model.

We also wrote down what would justify stopping automatic acceptance and using a safer manual path. That decision had an operational cost, so it needed to be discussed before a confusing afternoon rather than invented under pressure.

False warnings mattered. If a check complained every morning about an expected variation, people would learn to ignore it. We adjusted checks based on observed conditions and recorded why a threshold existed. A threshold copied from a different system would have looked precise without necessarily meaning anything here.

The change log expanded to include relevant physical changes. Moving a camera, replacing a light or altering a label layout could now be recorded beside software changes. I kept the entry short enough that someone might actually fill it in.

The most useful new record was sometimes a photograph of the setup. A paragraph saying the camera had been adjusted could leave room for interpretation. A dated image made several details visible at once. We stored only what the project permitted and avoided including unnecessary people or personal material.

By the end of the week, the monitoring arrangement was modest. It had a few meaningful checks, a small review routine and a person expected to respond. I trusted it more than the earlier page of green status indicators.

Back at the same desk

At home, my daughter had discovered that a spoon could travel surprisingly far from the table. I found one beside the bookcase and another inside a shoe. My wife said she had watched the second one being delivered there with considerable seriousness.

I thought about the room photograph again. The people using that work area had been making perfectly ordinary changes to get their jobs done. I had built a system whose assumptions were easier to disturb than I had explained. The mismatch belonged partly to the model and partly to my description of the environment it required.

I now ask about physical changes earlier when results move. I still check the version number. I just don't let an unchanged version end the investigation. Data has a route into a system, and that route contains objects and habits as well as files.

The new report includes a small panel showing recent example images beside the counts. It isn't a complete account of performance, and nobody pretends that it is. It gives us a place to start asking what has changed.

Before closing the laptop on Friday, I added the final setup photograph to the notes. Then I moved my mug farther back on the desk. A small hand had appeared at the edge, reaching for the handle, and this was one change in the environment I could respond to immediately.

— John