Consideration of the AR Instrument

When describing the means through which a participant or performer meaningfully interacts or engages in any kind of information transfer in the AR system, it is through what could be termed the AR Instrument. The following categorisation – building on the work of (Lindeman & Noma, 2007) – may be helpful in considering the plethora of different options available for artists and musicians.

Citations from table to be added.

Wearable

Wearable AR Instruments include forms that are worn on the body, including output via head- mounted visual, audio, olfactory and gustatory feedback devices or ‘displays’, and body-mounted proprioceptive feedback devices

Tangible

Tangible AR Instruments include forms that can explored by holding or touching, such as devices that use conductive fabrics and textiles to track input, and then providing sensory feedback, e.g. vibrotactile stimulation (somatosensory). They can also be any object that can be granted instrumentality by a device that can track it and provide contextually aware (i.e. corresponding) sensory feedback via another device. For example, a wooden cube could be transformed into a Tangible Instrument through real-time image recognition, and specific interactions with it could provide auditory feedback. In this example, the auditory feedback would likely be delivered via a Wearable Instrument that was also processing the real-time image recognition such as an HMD with bone-conduction headphones.

Situated

Situated AR Instruments include forms that are anchored in a real world environment and therefore provide location-specific experiences. Activation is gauged by user enaction, or user presence via infrared camera tracking or proximity of a worn device. Examples of Situated AR Instruments could include an interactive projection mapping with wavefield synthesis providing auditory feedback, and anchored scent emitters providing olfactory feedback.

Works Cited

1. Lindeman, R. W., & Noma, H. (2007). A Classification Scheme for Multi-Sensory Augmented Reality. Proceedings of the 2007 ACM Symposium on Virtual Reality Software and Technology - VRST ’07, 175. https://doi.org/10.1145/1315184.1315216

---