atmoSphere: designing cross-modal music experiences using spatial audio with haptic feedback
We use cross-Modal correspondence -the interaction between two or more sensory modalities- to create an engaging user experience. We present atmoSphere, a system that provides users immersive music experiences using spatial audio and haptic feedback. We focused on cross-modality of auditory and haptic sensations to augment the sound environment. The atmoSphere consists of a spatialized music and a sphere shaped device which provides haptic feedback. It provides users imagination of large sound environment although they feel haptic sensation in their hands. First user feedback is very encouraging. According to participants, atmoSphere creates an engaging experience.
https://dl.acm.org/doi/10.1145/3084822.3084845
Transcript 1,120 words
You know, we always talk about getting lost in our favorite music. But what if we could, like, actually feel the music all around us? Not just through our headphones. Well, it's a wild thought, right? Like, actually experiencing the soundscape, you know? Totally. And that's actually what we're going to be diving into today, this whole idea of a project called Atmosphere. It's really trying to blend audio and touch.
Yeah, it's using, it's a really cool example of something called cross-modal effects. That's basically how our brains can combine, you know, like what we're hearing and what we're feeling at the same time. Oh, okay. I'm already kind of hooked. So how does it, how does Atmosphere actually, like, work? Okay, so imagine, imagine holding, like, you know, those kind of grapefruit-sized spheres. So you're holding one of those and it's got these, like, eight different sections.
And each of those has this, what they call a tactile sound transducer in it. Okay. And basically these transducers, they make vibrations, right? So when you pair that with, like, spatial audio in your headphones, it's like the sphere becomes part of the music. So are these, like, just random vibrations or is it, like, matched up to what we're hearing? It's totally matched up.
Yeah, it's, that's the cool part, right? The audio, like, what you're hearing and then those vibrations, the haptics, they're, like, totally in sync. So let's say there's a sound moving around you in the headphones. Yeah. You'd feel that motion, but, like, in your hand on the sphere. Whoa, okay. So, like, if there's a song and it sounds like someone's walking around you.
Exactly. You wouldn't just hear the footsteps. You would actually feel them moving around your hand as that sound goes all around you. That's, that's crazy. Yeah. I can totally see how that would be, like, super immersive. Totally. And, honestly, that's, that's just, like, the beginning. What's really cool is that they're looking at using atmosphere for way more than music. Okay, now you're speaking my language.
What else, like, what are we talking, what else can it do? So one of the, like, the big areas they're looking at is mindfulness. Like, imagine using this to actually, you know, help you meditate. Oh, wow. Okay, so, like, feeling the music to relax. Yeah. I like it. How, how would that work? So they're, they're testing it out with, you know, those smart glasses.
They can kind of track, like, how relaxed you are. Oh, really? Yeah, by, like, your eye movements and stuff. So it's, like, the glasses are saying, yeah, this person's totally zen. Or, like, nope, they're still stressed. That's so cool. And so they take that information and they use it to, to change how strong the vibrations are, you know, on the sphere. Yeah.
And the sounds in your headphones. So, like, the calmer I get, the more subtle everything gets. Exactly. Wow. It just, it eases you into it, you know? Yeah. So, like, imagine there's this, like, gentle rain soundscape playing. Okay, yeah. And you can feel that, like, on the sphere. And then as you relax, those sensations, they just get, like, softer and softer. That sounds amazing.
Have they, have they actually tried this on, like, real people? Yeah. So they've done some early testing and people are loving it. What are they, what are they saying? They're saying they feel, like, more present, like, really in the moment. That makes sense. It's got to be, like, super immersive. But do you have any, like, specific examples? Yeah, actually. One person, they said it felt like they were, you know, actually outside in the rain.
Like, they were really surprised by how real it felt. Wow. Even though they knew it was, like, coming from this, you know, sphere thing. Totally. That's, that's pretty wild. And that's just, like, with the basic sounds and vibrations, right? Yeah. Like, imagine what it'll be like when they really start fine-tuning it, you know? It really makes you think about all the things they could do with this.
Like, not even just for music. Exactly. There's so much potential. Like, for therapy, even. Or, you know, maybe even learning. So, what's next for this atmosphere thing? Like, where do they go from here? So, they really want to do some, like, more in-depth studies. Yeah. To really measure, you know, how it affects people. Not just, like, what they say, but, like, really measure it.
Yeah. Like, how, you know, how immersed you feel. It's like they're making a whole new way for us to, like, experience sound, you know? It's almost like a new language or something. Totally. And it kind of makes you wonder, like, could they use this to create, like, completely new experiences? Not just, like, recreate stuff that we already know. Okay. Now we're talking.
Like, what? Give me an example. Like, imagine feeling the texture of a sound. Okay. Not just where it's coming from. Yeah. But, like, if it's rough or smooth. How would that even work? So, think about, like, I don't know, a rock slide. Okay. You could feel how jagged and rough that sounds. Yeah. But then, like, a violin, that would feel smooth. Yeah.
So, we're talking about, like, actually feeling the difference. Exactly. Yeah. And get this. What if you could feel the emotions in the music? Okay. Now you've lost me. How's that even possible? So, instead of just hearing the joy in, like, a symphony or the sadness in a blues song. Yeah. You'd actually feel it. Yeah. You know? Yeah. Yeah. Like, in your hand.
Through the vibrations. It's like you're adding a whole other layer to how we experience music. Totally. And think about the possibilities, right? Like, for gaming. Hmm. Or even, like, virtual reality. Yeah. Like, feeling what it's like to actually be in the game. Or another world. Exactly. It's like we're just scratching the surface of what our senses can really do, you know? It's pretty mind-blowing stuff.
So, for our listeners who are, you know, totally geeking out about this like I am, what's, like, the one thing we should all remember. I think the biggest thing is just to, like, really appreciate how incredible our senses are. Yeah. Like, this atmosphere thing. It's just one example. Who knows what else we'll be able to experience in the future, right? I can't wait to find out.
Well, that's it for this deep dive. It's definitely got me thinking about all the amazing ways we can experience music and sound. Until next time.