Role of the Virtual in the AR Environment
Allowance for the Real
The use of the game engine Unity, and visual programming languages like PureData / Max MSP have been invaluable in the development of the three outlined sound ARt experiences. Thinking about how they integrate with the real environment of your participant is how AR stays distinct from VR on an interaction level - after all, there must be some reason why as an artist or musician, we decide to work in tandem with, rather than shut off, the real world from our participants! Consider the following:
- What are the sensory boundaries implicit in both the real and virtual space I’m using?
- What different sensory affordances are provided by both the real and virtual environments?
- How might the development of augmented material be influenced by the above factors?
- What real objects are present in the space, is this intentional?
Choosing Experience Size and Complexity
It may be helpful to distinguish between different types of AR “experience sizes” when first starting out developing sound ARt. For doing this, I developed the following three categorisations. Implying or explicitly stating these boundaries (if it is a public installation) is necessary for building trust and ensuring safety. Intentionally setting boundaries may help in the creative process too.
Snippets describe a small-scale clip-like AR Experiences that occur in the approximate interaction space of 30cm3, e.g. between a users hands. The Snippet itself does not supply a full sensorial experience, instead providing two human-to-sense interactions through its AR subforms.
Scenes describe medium-scale AR Experiences that occur on and around the body, an approximate interaction space of 200cm3. They can be formed from existing Snippets, or created from scratch. They ideally feature more (and higher complexity) human-to-sense interactions, and therefore potentially more interactive relationships between real and virtual elements will be formed.
Spaces describe large-scale AR Experiences, involving multiple participants in a variety of differently sized interaction spaces in a room. For example, augmented hand / body interaction with the environment and other users, and multiple of zones of interaction in different sections of the space. Spaces provide fully multisensory immersive experiences, by making use of a combination of different sensory modalities and AR subforms.