Head ’n Shoulder: Gesture-Driven Biking Through Capacitive Sensing Garments to Innovate Hands-Free Interaction
Daniel Geißler, Hymalai Bello, Esther Zahn, Emil Woop, Bo Zhou, Paul Lukowicz, and Jakob Karolus. 2024. Head 'n Shoulder: Gesture-Driven Biking Through Capacitive Sensing Garments to Innovate Hands-Free Interaction. Proc. ACM Hum.-Comput. Interact. 8, MHCI, Article 265 (September 2024), 20 pages. https://doi.org/10.1145/3676510
Distractions caused by digital devices are increasingly causing dangerous situations on the road, particularly for more vulnerable road users like cyclists. While researchers have been exploring ways to enable richer interaction scenarios on the bike, safety concerns are frequently neglected and compromised. In this work, we propose Head 'n Shoulder, a gesture-driven approach to bike interaction without affecting bike control, based on a wearable garment that allows hands- and eyes-free interaction with digital devices through integrated capacitive sensors. It achieves an average accuracy of 97% in the final iteration, evaluated on 14 participants. Head 'n Shoulder does not rely on direct pressure sensing, allowing users to wear their everyday garments on top or underneath, not affecting recognition accuracy. Our work introduces a promising research direction: easily deployable smart garments with a minimal set of gestures suited for most bike interaction scenarios, sustaining the rider's comfort and safety.
Transcript 1,794 words
Welcome to our deep dive today. We're going to be looking at the intersection of technology, safety, and cycling. Sounds good. We're going to be looking at this research project called Head and Shoulder. It's about controlling devices while you ride. Yeah. So Head and Shoulder, they're trying to address this problem. Okay. Where cyclists are more and more distracted by their devices. Right. Which obviously leads to some dangerous situations when you're out on the road.
Yeah. And you kind of need to engage with those devices these days. Exactly. For navigation, communication, even just tracking your fitness. Sure. So just ignoring them isn't really much of an option for most cyclists. Yeah. And people have tried to come up with some solutions in the past. You know, trying to make it safer to interact with those devices while you're riding.
So things like custom handlebar buttons. Right. Voice control, which can be a little iffy when it's noisy. Yeah. Even some forays into augmented reality. Sure. But I mean, all of those come with their own problems. Yeah, they do. Some inherent risks, maybe some bulky hardware. Yeah. Or just not being practical in every situation you might be in as a cyclist. That's very true.
So it sounds like this Head and Shoulder Project is trying to come at this from a different direction. Yeah, they are. What is their approach? So they're using a wearable garment with capacitive sensors in it. And these sensors detect subtle movements in your shoulders. So like a discrete shoulder covering or something? Yeah. With sensors woven into the fabric? Exactly. Exactly. Okay. So what makes that better than these other options?
So the gestures themselves are really subtle, just shoulder taps. So they don't really affect your bike control at all. Right. And you can do them without taking your eyes off the road. Okay. Which is the important part. And that makes sense. Yeah. It kind of reminds me of that research about how even small posture shifts can help with rider fatigue. Exactly. Yeah.
It builds on that research. Very cool. Making that interaction just part of how you're already moving on the bike. Yeah. Okay. So how did they actually turn this idea into something real? So they started with a prototype. Okay. Just using a normal sports jacket to experiment with sensor placement. Right. And how well they could recognize different gestures. Makes sense. So you need to make sure it's interpreting your movements correctly.
Exactly. And to do that, they used a convolutional neural network. Okay. To analyze the data from those sensors and classify the gestures. And how'd that work out? In the lab setting, they got up to 95% accuracy. Really? 95%? Even through a jacket? Yeah. I guess I just assumed it would have to be, you know, directly sensing your skin or something for it to be that accurate.
That makes sense. But they designed it to work through layers of clothing. That's smart. They knew that was important for cyclists. Yeah. Because you're going to be wearing different clothes depending on the weather. For sure. So they tested it with extra layers. Yeah. And the accuracy stayed really high. So it seems like they really thought that through. They did. But lab conditions aren't exactly the same as, you know, being out on the road.
Right. So did they test it in a more realistic setting? Yeah. They did. They actually created a streamlined shoulder cover based on what they learned in the lab. Oh, wow. And they tested that out in the real world. So no more stationary bikes or anything? Yep. What kind of results did they see when they actually took it out on the road? Even better than in the lab.
Okay. They got up to 97% accuracy in real world conditions. Wow. 97%. 97%. That's impressive. It's really promising. Yeah. It really seems like this could actually be a viable technology. Yeah. But that's just the technical side. Right. What about the user experience? Yeah. Do the cyclists feel like this new method of interacting with their devices was intuitive and safe? That's where things get a little more complicated.
So it seems like while the head and shoulder system worked really well technically when they got feedback from the cyclists using it, there was this interesting finding about their perception of safety. That's unexpected. Yeah. Given that it's supposed to be less distracting than these other methods. Right. And it's not like it was interfering with their ability to control the bike or anything.
Yeah. So what were the cyclists saying specifically? So as part of the study, they had some cyclists try the head and shoulder system. Okay. And then others used a more traditional setup with buttons on the handlebars. Makes sense. And the ones using the head and shoulder system actually reported feeling less safe. Really? Yeah. Just slightly less safe than those who had the handlebar buttons.
That's kind of counterintuitive though. It is. If it's not actually impacting their control of the bicycle, why would they feel less safe? That's the million dollar question, isn't it? I guess so. It seems like there's some kind of psychological barrier there for at least some of the users. Hmm. Like what? Well, it could be just the unfamiliarity of the gestures themselves. Right.
You know, people aren't used to controlling things with shoulder movements. Or maybe it's the idea of introducing any kind of head movement while cycling. Yeah. That makes sense. Even if it's subtle, it might feel inherently less stable. Right. Whatever the reason though. Yeah. It does suggest that if they want people to actually use this technology, they're going to need to address this perception of safety.
Even if it's not really any less safe objectively. Exactly. It's about how it makes people feel. So did the researchers have any ideas for how to tackle this? They did. Oh. One thing they looked at was incorporating haptic feedback into the design. Oh, so the system would give the cyclist some kind of physical response. Yeah. To let them know it's working. Exactly.
Like a little vibration or something. That's clever. So they'd get that confirmation without having to take their eyes off the road. I like that. Yeah. It could be a simple way to make people feel more comfortable using it. Because they'd know for sure that their gesture had registered. Right. I feel like we often underestimate how important that kind of feedback is. Especially when we're talking about something like cycling.
Mm-hmm. Where safety is such a big concern. It's all about building trust. Exactly. This head and shoulder project has given us a lot to think about. It really has. And not just in terms of cycling safety. No, not at all. I mean, this idea of using capacitive sensing for hands-free, eyes-free interaction. Yeah. That has applications way beyond just riding a bike. Oh, absolutely.
Think about situations where you need to be able to control something. Yeah. But you can't use your hands. Right. Like surgery or operating machinery in a dangerous environment. Exactly. The possibilities are huge. They are. But even if we just stick with cycling for now, the potential here is really exciting. It is. I mean, this technology is discreet. Yeah. It's adaptable to different clothes, different bikes.
Right. It could be a game changer for all sorts of cyclists. For sure. Especially commuters who are already navigating traffic and trying to stay connected on the go. Definitely. And what about cyclists who are training? Oh, yeah. Good point. They could use this to track their performance. What? Control their music. All without fumbling with their devices. Exactly. This could be seamlessly integrated into the clothes they're already wearing.
Exactly. Imagine having a cycling jacket. Yeah. That just lets you control everything effortlessly. It'd be amazing. The perfect blend of style and function. I love it. And who knows? Maybe we could even combine this with other safety features in the future. Like why? Well, we could have integrated turn signals. Oh, cool. Or maybe a heads-up display projected onto your helmet visor. Wow.
The sky's the limit, really. I know. It feels like head and shoulders just the beginning. Yeah. Like the first step towards a whole new world of possibilities. Exactly. And speaking of possibilities. Yeah. There's one more really thought-provoking aspect to this research that I think you'll find interesting. Okay. I'm all ears. Lay it on me. So we've been talking about this whole head and shoulder thing.
Yeah. From the perspective of individual interaction. Right. One cyclist using the system to control their own devices. Exactly. Yeah. But what if we zoom out a little? All right. Think about the implications for group cycling. Oh, I see where you're going with this. Yeah. That could be really interesting. Right. Instead of just one person using it. Yeah. You'd have a whole group of cyclists all equipped with this technology.
Exactly. And imagine how that could change the way they communicate with each other. Oh, wow. Yeah. They could coordinate their movements. Uh-huh. Signal hazards. Right. All without having to take their hands off the handlebars or shout over the wind. Be like they're having a silent conversation. Exactly. A symphony of subtle shoulder movements. I love it. No more missed hand signals or trying to hear each other over the noise.
Yeah. That could be a game changer for group safety. Definitely. Especially in challenging writing conditions. It's amazing to think that something as simple as this. Right. Could have such a big impact. I think it just goes to show that we often underestimate the broader societal effects of these technological advancements. Absolutely. We tend to focus on the immediate applications. Mm-hmm. But it's often the unforeseen connections and possibilities that lead to the most groundbreaking changes.
I think that's a perfect note to end on. Yeah. For anyone listening who's about to head out on a ride. Mm-hmm. Pay attention to those little movements you make. Yeah. Those shifts in your posture that help you balance and control the bike. It all starts with the micro movements. You might just be witnessing the very beginning of a hands-free future. One shoulder tap at a time.
That's all the time we have for today's Deep Dive. Thanks for joining us. Thanks for having me. Thanks for having me.
Also.
Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye.