Embodied Cognition: Making Gamers Cry?
After my research into the fundamentals of virtual reality and virtual reality filmmaking, I realised that sound would play a more significant part in my work than I expected.
I realised that sound could not be an afterthought, so I wanted to look deeper into how sound affects us on a psychological, physiological and emotional level. I stumbled on to a paper by Karen Collins from the Audio Mostly 2011 conference entitled "Making Gamers Cry: Mirror Neurons and Embodied Interactions with Sound". The paper focuses on what the author refers to as 'the holy grail' of game design. This post is to summarise the paper in my own words, just to document my understanding.
Mirror neurones are the key to generating empathy. If we see someone in pain (they trip for instance), we feel the pain (and the shame) of hitting the floor. We empathise because we understand how pain and shame feel from our own lived experiences of tripping over and hitting the floor.
As a species, we have a tendency to anthropomorphise. Computers provide instant feedback, so we're more likely to anthropomorphise game characters. This is based on the intentionality hypothesis. We attribute animacy to shapes when they react with other shapes in animation we anthropomorphise them and consider those shapes as beings.
In gaming, adding physical activity (controllers/keyboards/HMD) mirror neurone activation and adding sound adds to the reality. Game creators and designers use knowledge of visualising causality and intentionality of sounds to express a character's emotional state, so the sound is more likely to impact an emotional state in the player.
Game designers are leveraging kinaesonics, that is, the physicalisation of sound to bodily movements. Through sound in games when you're playing a shoot-em-up the sound of the trigger being pulled and the explosion of the round from the chamber makes you feel like you had shot the gun yourself. Regarding dialogue, our bodies mentally mimic vocal sounds. When given a multiple choice of responses to an NPC (non-playable character), gamers often feel uncomfortable giving rude or less polite responses, even though the character they are talking to isn't real.
Finally, there is something that Collins refers to in her paper that I want to examine further. Embodied cognition. I'm going to paste a quote here, because to be honest, I'm still getting my head around it. But I think the idea of embodied condition is something that I should be mindful of when creating my virtual reality film. I want to leverage how our brains interpret visual, audio and potentially physical stimuli to create this emotive work.
"Embodied cognition holds that our understanding of the world is shaped by our ability to physically interact with it. In this way, embodied cognition is complementary to phenomenology, but whereas phenomenology is grounded in philosophy, embodied cognition is grounded in psychological theory and cognitive science. According to embodied cognition theory, our cognitive processes use re-activations of sensorimotor states from our past experiences to understand the world. In other words, our knowledge is tied to the original neural state that occurred in our brain when we initially acquired the information. Our cognition is, therefore “embodied” because it is inextricably tied to our sensorimotor experience; our perception is always coupled with a mental re-enactment of our physical, embodied experience."