Hands-Free Mouse Control With a $0 Hardware Budget

Commercial assistive technology that tracks eye movement or head position typically starts at several hundred dollars. Dedicated hardware. Proprietary calibration software. A device that only works if you can afford it.

I built a hands-free mouse controller that runs on any laptop with a built-in webcam. No external hardware, no subscription, nothing to buy. The project is called Head Tracking & Blink Detection, and building it taught me more about the gap between "it works in testing" and "it works for someone" than anything else I've shipped.

How it works

The system uses MediaPipe FaceMesh to identify 468 facial landmarks in real time at around 30 frames per second on a standard laptop CPU. I track the position of the nose tip relative to the screen center to drive cursor movement — move your head left, the cursor moves left. A deliberate blink registers as a click.

That description makes it sound solved. The actual engineering challenge was almost entirely in those last three words: a deliberate blink.

Measuring a blink: the EAR algorithm

The blink detection algorithm I used is called the Eye Aspect Ratio, first introduced in a 2016 academic paper by Soukupová and Čech. The core idea is elegant: measure the ratio of the vertical distance across the eye to the horizontal distance. When your eye is open, this ratio is relatively high. When your eye closes, the vertical distance collapses and the ratio drops sharply toward zero.

Set a threshold — if the ratio drops below, say, 0.21 — and you have blink detection. In theory. The problem is that 0.21 is right for one person under one lighting condition and wrong for almost everyone else.

My first version used a fixed threshold. It fired click events constantly — every sideways glance, every squint, every moment of tiredness that made the eyelids drop slightly. The system was completely unusable. I had built something that technically detected blinks but practically fired at random.

The false positive war

Getting false positives down to an acceptable rate took longer than the primary implementation. Three things together made the difference.

Duration gating. A deliberate blink closes the eye for at least 80–100 milliseconds — about 3 frames at 30fps. A reflex squint or a flicker in detection is faster. Requiring the EAR to stay below threshold for 3 consecutive frames eliminates most of the noise from brief landmark fluctuations.

The bilateral check. When you glance to the side, one eye's aspect ratio can drop while the other stays normal. Looking to the right pulls the left eyelid slightly lower due to the lateral movement of the facial muscles. Requiring both eyes to show a simultaneous drop eliminates nearly all gaze-shift false triggers. This single change halved the false positive rate.

A post-click cooldown. After a valid click event, the detector ignores blink signals for 800 milliseconds. This prevents a slow natural blink from registering as a double-click. It also gives the user a moment to recover from the intentional blink without accidentally triggering another one.

Together these three filters brought false positives to a frequency that felt intentional rather than random — close enough that users could predict what the system would do.

Smoothing head movement

Raw nose-tip coordinates from MediaPipe jitter by a few pixels frame to frame even when you hold completely still. Direct mapping to cursor position makes the cursor vibrate in place. Unusable.

Exponential moving average smoothing blends each new position with a weighted history of previous positions. The smoothing factor — called alpha — controls the trade-off between responsiveness and stability. Low alpha produces smooth movement but noticeable lag. High alpha follows head movement precisely but lets jitter through.

The value I settled on (0.12) was too slow for quick movements across the screen. So I added a velocity-aware adjustment: when the head moves fast, alpha temporarily raises to 0.25 for snappier tracking, then relaxes back to 0.12 when movement slows. This small addition made cursor movement feel significantly more natural — fast when you need it, steady when you don't.

What I learned about building accessibility tools

The biggest lesson was about the standard of reliability. A standard mouse occasionally misfires and users don't think about it — it's a background event. When a hands-free controller triggers an accidental click, it breaks trust. The user stops predicting what the system will do. After a few unexpected clicks, they stop using it.

That asymmetry means the false positive problem deserves disproportionate engineering attention. The EAR algorithm itself took about a day to implement correctly. Getting click precision to a trustworthy level took closer to two weeks.

The second lesson: per-user calibration is not optional. The current version runs a 30-second calibration mode on first launch that measures the user's natural EAR range across different eye positions and lighting conditions, then sets the threshold automatically. Without that, the system would only work reliably for the one person who tuned it — me.

Building for accessibility means designing for the worst moment, not the average one. The system will be used by people who have no easy fallback when it fails.

The project is open-source and linked from the project page. If you're building something similar, the calibration logic is the part most worth reading carefully.