PillWatch: the gap between 99 percent and a missed dose

This summer I have been running a small research program for four high school students, my son among them. Faculty mentorship comes from Prof. Tinoosh Mohsenin’s lab at Johns Hopkins. The project is PillWatch: a camera on a Raspberry Pi that watches a weekly pill organizer and reports which cells still hold pills.

The organizer has 21 cells: seven days, with morning, noon, and night for each day. That number is the whole project.

The math that reframes the problem

A per-cell classifier that is 99 percent accurate sounds finished. Raise 0.99 to the 21st power and the whole box is read correctly about 81 percent of the time. One reading in five contains an error somewhere. Nobody checks their pills one cell at a time. The unit that matters is the box.

That is the research angle. Per-cell accuracy is the headline metric in most papers, and it quietly hides a product that fails weekly. The students’ actual contribution is engineering whole-box reliability on commodity hardware, and being honest about what that takes.

The two errors are not twins

A cell that reads full when it is empty produces a false alarm. Annoying, self-correcting, cheap.

A cell that reads empty when it still holds a pill is a different animal. The system believes the dose was taken. The person who relies on the system now misses a dose, and nothing anywhere disagrees. So the design centers the false-empty rate, and the decision thresholds are tuned asymmetrically to keep that specific failure rare, even at the cost of more false alarms.

One discipline rule came with this: thresholds are chosen on validation data and frozen before the test set is scored. Tuning the threshold on the numbers you report is the most common way student projects, and not only student projects, quietly cheat.

What the hardware and models say so far

The rig is a Raspberry Pi 5 with an Arducam camera, about $60 of hardware, running everything on device. The team compared four trained models against a classic difference-of-Gaussians baseline.

Two results stand out from the current held-out test split. A small reference-conditioned CNN gives the best overall balance: macro-F1 of 0.81 with only 80,000 parameters. And a transfer-learned YOLO classifier is by far the most robust to camouflaged pills, scoring 87.5 percent on a set of 144 deliberately hard images where the other models fall to roughly 48 to 58 percent. Translucent vitamin D capsules under glare beat everybody. That failure mode is now a named enemy.

Temporal voting, reading the box several times and voting, helps with transient errors like a hand passing through the frame. It does not help when the condition itself fools the model, because the same wrong answer just wins the vote. Testing that distinction explicitly turned out to be a finding of its own.

Why I run the program this way

The students work independently during the week and compare results at a Saturday sync. They are learning the part of machine learning that tutorials skip: the metric you report is a decision, not a given, and the honest version of your result is usually smaller and more interesting than the impressive version.

The dataset behind this, small, condition-stratified, and being prepared for public release, gets its own essay. The team is aiming the work at the MIT Undergraduate Research Technology Conference this fall.

Code and hardware notes live in the PillWatch repository.