The Light Above the Camera

The defect had disappeared again. In the first photograph there was a thin pale line across the dark fabric. In the next one, taken less than a minute later, the same place looked almost clean. I enlarged both images until the weave became a grid of soft squares. A colleague stood behind my chair holding a paper cup that had stopped steaming.

We had spent the morning discussing training examples. I had a list of possible changes to the model and a longer list of reasons why the previous changes had been inconclusive. Then someone walked past the inspection table. Their shadow crossed the cloth, and the line returned on the screen.

I remember being annoyed with the shadow for interrupting us. That was an impressive misunderstanding of what had just happened.

The experiment was a small computer-vision project for inspecting fabric. A fixed camera looked down at material passing underneath. We wanted to flag a few visible faults for a person to check. I had treated the camera as the place where data arrived. It took me several afternoons to start treating the whole arrangement as part of the measurement.

A sample in my backpack

I brought two small pieces of fabric back to the office in a transparent document sleeve. One was supposed to be clean. The other had the pale line, marked by a safety pin near its edge. Without the pin, I sometimes had trouble finding the fault myself.

That should have made me more cautious about the task. Instead, I thought that a sufficiently capable model would identify a subtle pattern better than I could. Sometimes it can. But I was asking it to work from photographs in which the pattern was occasionally absent, while quietly assuming the label remained fully informative.

At home, my wife asked whether the piece of cloth on the dining table was something we needed to wash. I explained the project. She picked up the sample, tilted it towards the ceiling light and said she could see the line from one direction. She turned it and couldn't see it from the other.

I had shown her a classification problem. She had handled a piece of cloth.

The next morning I looked through our image folder with that movement in mind. Some examples were brighter on the left. A few had a band of reflection near the middle. Several dark samples had been made lighter by the camera's automatic exposure. The data wasn't merely varied. Parts of the capture process were changing without appearing anywhere in the records.

Our early notes contained model names and image sizes. They didn't say whether someone had opened the nearby door, which let daylight reach the table. I had recorded learning rates more carefully than the position of the lamp.

The clean sample was also less useful than I had thought. Under a sharp light, ordinary changes in the weave looked dramatic. Under soft light, the real fault became faint. Asking a colleague to label the pictures without the physical samples produced a small, uncomfortable disagreement.

I wrote the disagreement down rather than resolving it by choosing the label I preferred. We needed to know whether we were testing a vision system or testing how confidently a tired person could interpret a poor photograph.

Moving one thing

We returned to the table after lunch with tape, a ruler and a notebook. The first useful adjustment was embarrassingly simple: mark the camera position so that moving the stand didn't become an invisible change. We also marked a place for the sample and photographed the whole setup with a phone.

Then we moved the light. We changed one thing at a time, took a short sequence and wrote down what had changed. I wanted to adjust the exposure and the angle together because the first result looked better. My colleague asked how we would know which adjustment mattered. I put my hand back in my pocket.

The difference between two light positions was larger than the difference between our last two model runs. That didn't prove we had solved the inspection problem. It did show that we had been spending our effort downstream of a large uncontrolled source of variation.

We can train again after we can take the same photograph twice.

That was my colleague's suggestion. We couldn't literally make identical photographs of moving cloth, but the sentence gave us a sensible next task. We repeated a small set of samples and checked how much the image changed under an arrangement we believed was fixed.

A loose part of the stand introduced another surprise. Someone resting a hand on the table moved the camera slightly. The sample still occupied most of the picture, so a quick glance didn't reveal the movement. A narrow feature near the edge shifted enough to affect our crop.

We tightened the stand. I wrote down the distance from camera to table. We checked focus on the actual material rather than a printed sheet somebody had used earlier. None of this felt like the work I had imagined when I chose computer vision. It was, however, the work the images were asking us to do.

We kept the first photographs from the bad setup. Deleting them would have made the experiment look cleaner and taught us less. Later, side by side with the improved captures, they explained several mysterious errors more clearly than any chart in my notebook.

The temptation to brighten everything

Back at my desk I tried a few familiar image transformations. Converting to grayscale removed a source of variation we didn't need for one particular fault. Changes to contrast made some lines easier to see. They also made harmless texture look more alarming.

The OpenCV documentation on histogram equalization is useful reading when I catch myself treating contrast as a single brightness slider. Different transformations redistribute information in different ways. They cannot recover a feature that the capture process failed to distinguish in the first place.

I made a little comparison sheet rather than choosing the version that looked most impressive on my screen. Each row held the same sample under different processing. I included clean material as well as faulty material. Otherwise I would have rewarded whatever made the fault look strongest, even if it made every ordinary fold look like a fault too.

There was a period of about twenty minutes when I was pleased with a strong local contrast adjustment. Then I opened an untouched clean sample. It looked as if someone had taken a pencil to it. I reduced the adjustment and started again.

My own monitor created another trap. I was looking at enlarged, carefully selected crops in a quiet office. The eventual reviewer would see many images, often at a smaller size. A feature visible after a minute of concentration wasn't necessarily useful during an ordinary shift.

We printed a few examples and sat beside the person who knew the material best. He pointed out something we had described as a defect that was acceptable for that particular batch. I had been trying to improve a boundary before we had agreed where the boundary belonged.

That conversation changed the task description more than it changed the code. We separated obvious faults from uncertain cases. We stopped forcing every picture into a confident answer. A review queue was a practical outcome for the difficult examples, provided the queue stayed small enough for someone to use.

A test with the door open

Once the arrangement seemed better, we deliberately disturbed it. We opened the door. We shifted the sample a little. We took pictures at different times of day. I wanted a sense of the conditions we could control and the ones that would return as soon as we stopped watching.

The door mattered less after we changed the lighting arrangement, but it still mattered. We couldn't claim that a tidy afternoon experiment represented every operating condition. I added a note about shielding unwanted light and asked what the installation could realistically accommodate.

The people using the table had sensible concerns. A large enclosure might interfere with handling the material. A lamp in the wrong position might make cleaning harder. The camera couldn't occupy the exact spot I liked if it blocked access to something behind it.

Those constraints weren't objections to a good technical solution. They were part of deciding whether the solution was good. I had produced an arrangement that worked while two engineers stood still beside it. We needed one that would keep working when people were busy with their actual jobs.

Our next test used a fixed set of material pieces and a written capture procedure. We kept a separate group of samples for checking changes. I made sure photographs of the same piece didn't quietly appear on both sides of the comparison. Otherwise the evaluation could flatter us by rewarding familiarity with a particular weave.

We counted the uncertain cases as well as the correct ones. On a small set, one difficult sample can make a percentage jump dramatically. I wrote the raw counts beside the percentage so that nobody would confuse a tidy number with a large body of evidence.

The new arrangement reduced several recurring mistakes. It didn't eliminate every problem. One fault still became difficult at a particular orientation, and a folded edge continued to produce false alarms. Those were now specific failures we could reproduce. I preferred that to the earlier situation, when a new run sometimes improved the headline result for reasons I couldn't explain.

Before leaving, I took another photograph of the table. The tape was already curling at one corner. We replaced it with a clearer mark and added the photograph to the notes. I had finally learned to record the ordinary objects that could undo an afternoon's work.

What I put in the handover

The handover included model settings, but the first page showed the camera and the light. I labelled their positions and wrote what we had checked. I included an example of a bad capture beside an example of an acceptable one, with a short explanation of the difference.

I also wrote down what should happen after someone moved the stand. The answer couldn't be to trust that it looked approximately right. We needed to repeat a small check using known samples before treating the results as comparable again. A simple habit was more useful than an elaborate paragraph nobody would read.

I left space for new observations. People working with the material would notice conditions I hadn't encountered during a visit. If the notes sounded finished, those observations might be treated as exceptions to ignore. I wanted them to become evidence that could change the setup.

On the ride home, I kept thinking about my wife's hand turning the cloth towards the light. I had watched her do the most relevant experiment of the week without recognising it immediately. She hadn't needed to know the architecture of our model to ask whether the line was visible.

When I got home she was sorting laundry, and I briefly considered explaining the entire afternoon. Instead I showed her two photographs of the same sample. She asked which one the system would see in the real place. It was another inconveniently useful question.

I still like training models. There is pleasure in seeing a careful change improve a result, and there are cases where better representations make a real difference. But when an image-based system behaves strangely, I now spend longer looking at the image before looking at the code.

This week the document sleeve is still in my backpack. The safety pin has caught on a receipt, and one piece of fabric is developing a crease that wasn't part of the experiment. I'll need to replace it before we use it as a reference again. Even the sample I carry home has a history, whether or not I remember to write that history down.

— John