32leaves.net

Pebble, your light shines (too often)

Pebble is this great smartwatch that, among other things, sports an accelerometer and a backlight. The stock firmware supports a feature called “motion backlight” which combines the two: flicking ones wrist, resulting in high acceleration, activates the backlight. During my first week of using the Pebble, I noticed that the backlight would turn on by when not intended pretty often; and that often it would not turn on when I wanted it to. In short, the “wrist flick” gesture detection seemed to not perform so well.
So the question I want to answer is: how well does the current motion backlight implementation perform?
Spoiler alert: It does not perform very well. Over the course of two days, only 10 out of 1331 backlight activations were correct.

Procedure

To answer this question, I wanted some data. So I wrote a watchface that would record every backlight activation and allow me to self-report correct and missed activations; thanks to the new Pebble SDK, that is now possible. So I first wrote a Pebble wachface to see what accelerometer tap event would activate the backlight (at some point their documentation said that Pebble uses these taps to activate the backlight). Turns out that every tap event turns on the backlight. Next, I’d log every tap and self-annotation event to my phone using the new datalogging API. Unfortunately, that crashed the Android Pebble app, so back to good old App Messages. On Android side, I’d have an app receiving the data sent from the Pebble data and storing it in a SQLite database.

Analysis. Once I had collected the data, I wrote an R script that removes one backlight activation for each self-reported correct activation (true positive, tp). All remaining backlight activations are then false positives, fp. The script plots the data over time and computes precision \frac{tp}{tp + fp} and recall \frac{tp}{tp + fn}.
Limitations. This procedure has two main disadvantages: first, we don’t get the true negative rate, tn; that is whenever the “wrist flick gesture classifier” dismisses a movement as non-gestural. If we were interested in this, we’d have to think about time segmentation to count discrete tn events in a continuous time stream. Second, we make the assumption that every tap event results in a backlight activation. Empirically this seems to hold for all activations more than three seconds apart. Unless someone from Pebble confirms the details of how the motion backlight feature works, there is no knowing, though.

Results

I ran the “study” for about two days, during which I went about my normal daily activities. These include getting changed, roughly 30 minutes of cycling, time at the gym and office work. During the recorded time, I deliberately used the “motion backlight” feature tp = 10 times and it failed to activate when I wanted it to once (fn = 1). The backlight was turned on incorrectly fp = 1321 times. Precision (probability of an activation to be correct) was 0.7\%, and recall (probability of a “wrist flick gesture” being correctly detected) was 90\%.
Activations
Plotting the backlight activations over time shows a few clear clusters, that seem to occur during times of increased activity (cycling, cardio, squash), marked with a black line. During sleep or desk work, false activations rarely happen.

What does that mean?

Technical issue. The high number of false-positives (fp = 1321) is probably due to the simple gesture detection mechanism, which not only needs to perform well, but also be energy-efficient. While no documentation of how the “tap” mechanism works is available, I’d reckon it’s implemented by thresholding the acceleration vector’s magnitude; something that is easy to implement in hardware. A more refined process or some post-tap logic for detecting the “wrist flick” gesture might bring the number of false activations down.
Motion backlight usage. Seems like I’m not using the motion backlight feature that much. At least that’s what looking at the graph and number of deliberate backlight activations (tp = 10) over the course of two days would indicate. After this experiment I’ve turned motion backlight off, hoping to extend my battery life a bit.
Increased activity. Pebble’s motion backlight feature wastes energy when doing sports or at times of increased activity. In fact, I’ve first noticed the false activations while cycling; looking at the graph confirms that. To some extend, this was to be expected. The tarmac around here is rough (to say the least), so false positives are inevitable while cycling. But also when changing clothes or during simple activities (such as soldering, or using a laser-cutter), the Pebble tends to unintentionally light up.
This is a problem. I don’t exactly know how much energy is consumed per backlight activation, but I can speculate. Given that the Pebble has three backlight LEDs, assuming 5 mA per LED, and 3 seconds per activation, each activation consumes 1.25 \mu\text{Ah}. Given the 1321 false activations, that amounts to a total of 16.5 \text{mAh}: more than 12% of the battery capacity (130 \text{mAh}) over the course of two days.

How could this be fixed?

Off the top of my head I can think of three solutions: a workaround, improved recognition and the users context.
The workaround. Turn of the motion backlight feature. As didn’t use it so often to begin with, I turned it off altogether. Of course this more a workaround, rather than a proper solution; the motion backlight feature comes in handy at times, and certainly is a neat trick.
Improved recognition. The way the “wrist flick” gesture detection is implemented right now is likely a combination of technical necessity and low hanging fruits. The accelerometer hardware built into the pebble can trigger the “tap” events and wakeup the CPU; like this, the expensive CPU doesn’t have to be woken up so often. However, using the accelerometer values before or after the tap event (e.g., by utilizing the accelerometers capability to store values), recognition performance could be improved.
User context. Using the users context, we could enable or disable the motion backlight feature. For example, as we now know (or at least have an indicator for) that during sport activities the motion backlight is often triggered, we could disable the feature during such times. E.g. if a user starts the RunKeeper app, this app could automatically disable motion backlight. For this, we need an API to disable the motion backlight feature.

Want to repeat this experiment?

I’ve uploaded all the source code, binaries, R scripts and my results on GitHub: https://github.com/32leaves/PebbleLightLogger – it’s all under a MIT License. In case anyone repeats this experiment, I’d be very interested to hear about it and get the results. Feel free to leave a comment below.

Fork me on GitHub