Whirling Interface: Hand-based Motion Matching Selection for Small Target on XR Displays
J. Lee et al., "Whirling Interface: Hand-based Motion Matching Selection for Small Target on XR Displays," 2024 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Bellevue, WA, USA, 2024, pp. 319-328, doi: 10.1109/ISMAR62088.2024.00046.
We introduce “Whirling Interface,” a selection method for XR displays using bare-hand motion matching gestures as an input technique. We extend the motion matching input method, by introducing different input states to provide visual feedback and guidance to the users. Using the wrist joint as the primary input modality, our technique reduces user fatigue and improves performance while selecting small and distant targets. In a study with 16 participants, we compared the whirling interface with a standard ray casting method using hand gestures. The results demonstrate that the Whirling Interface consistently achieves high success rates, especially for distant targets, averaging 95.58% with a completion time of 5.58 seconds. Notably, it requires a smaller camera sensing field of view of only 21.45° horizontally and 24.7° vertically. Participants reported lower workloads on distant conditions and expressed a higher preference for the Whirling Interface in general. These findings suggest that the Whirling Interface could be a useful alternative input method for XR displays with a small camera sensing FOV or when interacting with small targets.
Transcript 3,250 words
Welcome to your personalized deep dive. Today we're exploring a fascinating new input method for XR devices. It's called the whirling interface. Ah yes. As XR headsets are becoming increasingly commonplace, we're all on this quest, aren't we? Searching for those truly intuitive ways to interact in virtual and augmented worlds. Precisely. The more immersive the experience, the more critical it becomes to bridge that gap between our physical actions and the digital response. Exactly. And that's
where this research from Keiss and Georgia Tech comes in. They've directly compared this whirling interface to a method we're all familiar with, raycasting. Right. Sort of the gold standard, at least for now. But as the research reveals, it's not without its limitations. Which is exactly why this comparison is so exciting. It gives us a real opportunity to see the whirling interface's potential. So maybe we should start with the WHY. What problem are we even trying to solve here? Well,
it all boils down to one of the fundamental challenges of XR. Selecting small objects at a distance in these sweetie spaces. Think about it. Existing input methods, your touchpads, voice commands, even controllers, they all have inherent limitations. Because fits law. Precisely. Fits law. The smaller and farther a target is, the more difficult it becomes to select accurately. And then you add the limitations
of current hand tracking technology. Yeah. The potential for user fatigue. And it becomes very clear that we need some innovative solutions in this area. And that brings us to raycasting, right? Can you unpack that a bit? Sure. Imagine you're using a laser pointer. That's essentially what raycasting is. You use a hand gesture to aim a ray. It highlights an object and then you pinch to select it. Like we've all done a
thousand times. In this research, they actually used the Microsoft HoloLens 2 and the mixed reality toolkit for their implementation. Yes. And while raycasting has served us well, it's, well, it's not perfect. Yeah. Especially when you're trying to interact with objects at a distance, it starts to, it falls apart a little bit. Accuracy can really suffer. Exactly. You get things like jitter, the ray jumping
around, or occlusion where other objects are blocking your target. It's frustrating. Plus, there's that odd sensation of the ray changing size as you aim. It's just, it never feels quite right. It's like- It breaks the immersion. Yeah. So the whirling interface is attempting to address these usability challenges. How though? What's the core concept here? It hinges on a concept called motion matching. Motion matching.
It involves synchronizing your movements with onscreen cues to select those objects. Now, motion matching has been explored with eye tracking. Right. But the unique aspect of this research is that they're focusing on using hand movements. Instead of your eyes. So how does that work? How do I motion match with my hand? Imagine an orbital target circling the object you want. Okay. You sync your wrist motion to
that target, almost like conducting an invisible orchestra. Interesting. And as you get closer to matching, you get visual and auditory feedback that guides you right to that selection. And critically, this approach avoids that arm fatigue you get from holding it outstretched, like you do with ray casting. That's a good point. No more tired arms. So did they actually, did they test this new method?
They did. They had 16 participants test both ray casting and the whirling interface. The participants had to select the small targets at two different distances, one meter and two meters. Okay. So a good, solid comparison. So what did they find? Was the whirling interface actually easier to use? The results were, well, they were quite compelling. Let's look at success rates first. The whirling interface
had success rates consistently above 95% for both distances. Wow. So people were almost always able to select the object. Yes. Ray casting on the other hand, particularly at that farther distance, the two meters, the success rate dropped down to 86%. But here's the truly remarkable part. Okay. The whirling interface actually had higher success rates at the farther distances. Now that is counterintuitive. You would
think syncing your movement with a target that's farther away would be harder, not easier. Precisely. But okay. So it's more accurate, especially at a distance. But what about speed? Was it faster than ray casting? Well, that's where it gets a bit more nuanced. Ray casting was actually slightly faster at close range, the one meter target. But for those farther targets, the whirling interface was demonstrably
faster. And perhaps even more importantly, it was consistently fast across all the targets. So you could rely on it to be fast, regardless of how far away the object is. Very cool. Okay. What about mental effort? Was it more mentally taxing for the user? Not at all. In fact, when participants rated the mental effort required for each method, the whirling interface had a significantly lower workload than ray casting, especially again,
for those farther targets. For the closer targets, they were about the same. Interesting. So, so far, we have higher success rates, faster and more consistent completion times, and lower perceived workload, particularly for those distant targets. Any other advantages the whirling interface has over ray casting? Yes, actually. There are a few other noteworthy distinctions. The research looked at what they call the sensing field of view. Basically, the range of hand movement required for each method.
And they found that the whirling interface needed a much smaller range of motion compared to ray casting, especially vertically. So, less arm fatigue, and perhaps more comfortable to use in maybe more confined spaces. Yeah. That could be a real plus. Exactly. Okay. Fascinating. So, less arm fatigue, more comfortable, maybe even usable in tighter spaces. Yeah. What else? Anything else stand out from
this research? Well, while it wasn't statistically significant, the researchers also looked at what they call social acceptability. How comfortable people felt using each method in different social situations. More participants said they preferred the whirling interface. And it was rated a bit higher for use in, well, different contexts. That does make sense. If you're out in public, I think making these small, precise hand movements to select things is going to feel a lot more natural and discreet than pointing
a beam of light at everything. Precisely. And it's important to note some of the qualitative observations from the study, too. A lot of the participants went in thinking the whirling interface was going to be difficult, but they actually found it more intuitive and, believe it or not, enjoyable compared to ray casting. It does sound kind of fun. It does. And that's what really gets me excited about this,
it's not just about performance metrics. It's about how these new interfaces make us feel. That's a good point. Okay, let's move on to some of the broader implications of this. What could the whirling interface mean for the future of XR? I think the truly captivating aspect of this research is that it demonstrates the power of motion matching as a viable and, in some cases, potentially superior input method for XR. And it raises the question,
what else can we do with this? What other intuitive motions could we use to interact with the digital world around us? That's a great question. I mean, imagine a world where instead of awkwardly aiming a ray at a museum exhibit to get more information, you could just, you know, use a subtle wrist motion to select what you want to learn about. Precisely. The interface kind of melts away.
Yeah. It's like you're seamlessly weaving together the digital and physical worlds. And because the whirling interface relies on these smaller, less obtrusive movements, it has the potential to be even more socially acceptable than existing methods. Okay, so this feels like a big step toward making XR interactions truly intuitive, effortless, almost invisible. What are your thoughts on the potential for this to be adapted to future sensing technologies?
Well, the researchers suggested that because the whirling interface works with these basic wrist movements detected by a camera, it could be adapted to all sorts of future sensing technologies. As those technologies get more advanced, we could use even more subtle movements, or even incorporate things like biosignals to trigger interactions. Wow. I mean, the possibilities are kind of endless, aren't they? They really are. And the study also found that some participants naturally started using smaller gestures
as they got more comfortable with the whirling interface. So there's even more room for refinement and exploration in terms of the kinds of movements we could use. It's like we're just scratching the surface. Exactly. And another thing that the researchers highlighted is that using the wrist as the primary input makes the whirling interface more robust to tracking errors and self-occlusions, which are major hurdles with current hand tracking systems.
So it's more resistant to the limitations of the technology as it exists right now. Exactly. And as hand tracking improves, as XR devices get smaller and lighter, I think the whirling interface will only become even more effective and practical. It's almost like they designed this with the future of XR in mind. It's incredibly forward thinking. It really is. And it begs the question, what other intuitive motions could we use to interact with the digital world?
We're on the cusp of a paradigm shift in how we interact with technology. And as researchers and developers keep exploring, we can expect to see even more intuitive and seamless interfaces emerge. It's a very exciting time to be following this space. This research and the whirling interface really feels like it's opened a door to a whole new world of possibilities. It has. And the key takeaway is to pay attention to these emerging input methods.
They have the potential to completely change how we experience and interact with our increasingly digital world. I love that. And I think it's a good place to wrap up this part of our deep dive. We've covered the problem, the proposed solution, the experiment, the findings. We've even touched on some potential applications. Before we go, any final thoughts for our listeners? I think the main takeaway is this. This deep dive has given you a glimpse into the potential of the whirling interface to change the way we interact with XR.
It's a great example of how research is tackling the challenge of creating more natural and intuitive interfaces. But keep in mind that this is just one piece of a much larger puzzle. It's part of a larger movement toward more human-centered technology design. It's a powerful reminder that as technology continues to evolve, so too should the ways we interact with it. The whirling interface shows us that we can move beyond the limitations of those traditional input methods.
And that's an exciting prospect. The research we've discussed today highlights the idea that the best interfaces are often the ones that get out of the way and let us focus on the task or experience at hand. So, as you continue to explore XR, as well as other emerging technologies, keep an eye out for those innovations in this space. The future of interaction is being shaped right now.
And maybe, just maybe, the whirling interface will inspire you to think differently about how we interact with technology, period. After all, our imagination is the only limit. I certainly hope this deep dive has sparked some curiosity about the possibilities that lie ahead. Well said. Until next time, keep exploring, keep learning, and keep diving deep. It does, doesn't it? And that's really what excites me about this research.
It's not just about, you know, those raw performance metrics. It's about how these new interfaces make us feel. There's a certain, I don't know, elegance to it. There is. Okay, so we've talked about how well it performs, but let's zoom out a bit. What could the whirling interface mean for the future of XR? Where could this lead? I think the most captivating aspect of this research is that it demonstrates the power of motion matching.
It's not just a viable input method for XR. It's potentially superior to what we're using now. And that opens up so many possibilities. Right. Because if this works, what else could we do? What other intuitive motions could we use? It's like it really gets the imagination going. It does. Imagine instead of having to awkwardly aim a ray at a museum exhibit to get more information,
you could just use a subtle wrist motion to select what you want to learn about. It's almost like magic. The interface just disappears. It's like you're seamlessly weaving together the digital and the physical. Exactly. And because the whirling interface relies on these smaller, subtler movements, it has the potential to be even more socially acceptable. Because it's not as, I don't know, intrusive.
Right. It's less disruptive, more natural. And that's going to be key for wider adoption, especially in social or professional settings where people might be hesitant to use something that feels awkward or out of place. I can definitely see that. So to kind of summarize, this feels like a major step towards making XR interactions more intuitive, effortless, almost invisible. Do you see any potential challenges in actually getting there?
Oh, of course. There are always challenges. One of the biggest ones is going to be discoverability, making sure people understand how to use these new interfaces. Right. Because they're so different from what we're used to. Exactly. We'll need really intuitive tutorials, clear visual cues within XR environments, things like that. And of course, there will be a need for ongoing research and development to refine these interfaces,
work out any kinks. Because as with any new technology, you never know what unexpected usability issues might pop up. That makes sense. So while the whirling interface is incredibly promising, there's still work to be done to make sure it's widely adopted and can be seamlessly integrated into different applications. Right. What about its adaptability? Could this interface work with future sensing technologies? I mean,
things are evolving so rapidly in that space. The authors of the study seem to think so. They pointed out that because the whirling interface works with these basic wrist movements that are detected by a camera, it could be adapted to a whole range of future sensing technologies. As those technologies advance, we could see even more sophisticated and precise tracking. Maybe we'll be able to use even subtler gestures or even incorporate
biosignals to trigger interactions. Wow. That's mind-blowing. It is. The idea of controlling our digital surroundings with the same ease and fluidity as we interact with the physical world. It's transformative. It is. And this research might just be the catalyst for a whole wave of innovation in that area. Did the researchers uncover any other areas that they thought were particularly ripe for exploration?
Any other threads they wanted to pull on? One observation that stood out to me was that as the participants got more comfortable with the whirling interface, they naturally started using smaller gestures. Oh, that's interesting. It suggests that there's potential for even more subtle and refined movement-based interactions. Maybe we could use micro gestures, just finger movements, to control things. It could make the interfaces even more intuitive and less obtrusive.
And even more immersive, I imagine. Precisely. It could really blur the lines between the digital and the physical even further. The researchers also pointed out that using the wrist as the primary input method makes the whirling interface more resistant to tracking errors and self-occlusions. Those are some of the biggest challenges with hand-tracking systems right now. So that seems like a pretty significant advantage over other hand-gester-based interfaces.
It is. It means that the whirling interface is well-suited to the current limitations of the technology. But as hand-tracking gets better, as XR devices become smaller and lighter, I think we'll see the whirling interface become even more effective and practical. It has the potential to be a real game-changer. It really does feel like they designed this with the future of XR in mind.
It's so forward-thinking. It is. And it raises some really big questions, like, what other intuitive motions could we use to interact with the digital world around us? If we can break free from the constraints of controllers and handheld devices, what could we achieve? It's almost like we're standing on the edge of a whole new era of human-computer interaction. We are. We're on the verge of a paradigm shift.
And as researchers and developers continue to push the boundaries, we can expect to see even more intuitive, more seamless interfaces. This is a very exciting time to be following this space. This research on the whirling interface really feels like it's opened up a whole new realm of possibilities. What an incredible conversation. It has. And the key takeaway, I think, for everyone listening is to pay attention to these emerging input methods.
They have the potential to completely transform how we experience and interact with our, well, increasingly digital world, don't they? Absolutely. And on that note, I think it's a good place to wrap up this part of our deep dives. We've talked about the problem, the proposed solution, the experiment, those fascinating findings. And we even touched on some potential applications. But before we go, any final thoughts for our listeners out there?
Anything you want to leave them with? I think, you know, the biggest takeaway from this whole deep dive is that you've had a glimpse into what the whirling interface could mean for the future of how we interact with XR. The really concrete example of how research is addressing this, this challenge of creating interfaces that are more natural and intuitive. But, you know, we always have to remember, this is just one piece.
One piece of a much bigger puzzle. It's part of a larger movement towards designing technology that's more human-centered. It's a good reminder that as technology keeps evolving, the ways we interact with it need to evolve too. The whirling interface is a great example of how we can start to move beyond the limitations of those, you know, more traditional input methods. The research we talked about today really highlights that the best interfaces are often the ones that
just kind of fade into the background. The ones that let us focus on what we're actually doing, the task at hand. So, you know, as you continue to explore XR, as well as other emerging technologies, just keep your eyes open for these kinds of innovations. The future of interaction is literally being designed right now. And just maybe. The whirling interface will inspire you to think differently.
Not just about XR, but about how we interact with all technology. I mean, after all, it's our imagination, right? That's the only limit. Well said. I hope our conversation today has sparked some curiosity about what might be possible, about what's coming next. I think it has. Until next time, everyone keep exploring, keep learning, and keep diving deep.