Interactive, non-linear and adaptive sound

Because a program it can accept input it's possible to modify the sound output during playback. This is interactive audio. When we play a CD or DVD no further action is needed until the disc has finished. The order of the data is fixed and the procession from one value to the next is automatic. Something that starts at one point and moves at a constant rate to another is deemed linear. Alternatively, interactive[27] audio uses further input to start new sounds or change existing ones, and the value of the input affects the quality of sound produced. A MIDI piano is a fine example of simple interactive audio. To hear a note you must press a key and the loudness and timbre of the note is determined by the key velocity, how hard you hit it. Clearly a piano consists of more than one sound, taking pitch as a qualifier there are up to 88 different sounds, one per key. Something that can jump between a number of values in any order (discontinuous) or move at different rates or in different directions is non-linear. A turntable for playing vinyl is a potentially non-linear device when we do some funky scratching on it. Non-linear devices or programs are capable of being performed, they are instruments of a kind. At present all game sounds can be considered as interactive audio (except the music tracks in some cases which are linear). Each sound you hear depends upon a player action or world event. The order and timing of these sounds is not pre-determined as with recorded audio. Interactive applications like games can involve elaborate relationships between user input and audio output, but the common principle which makes the audio interactive is the need for user input. In a video game, certain situations arise which we call states. In an attempt to pull the mood of the player along with these states, perhaps representing emotional qualities such as fear in the presence of a monster or triumph at completing a level, music or sound effects are changed. We call this adaptive audio. It is a form of interactive sound where a complex function or state machine lies between the players actions and the audible response. Unlike a piano we may not get an immediate response for an input, the result may take some time or further input to yield effects. Normally audio in games is only given as a response to action based on visual play elements, but interactive audio only games for handheld consoles have been demonstrated[4]

Andy Farnell
http://obiwannabe.co.uk/