Factors against procedural audio

Several problems seem to stand against procedural game audio. Dynamically generated and synthetic sound is not a panacea, indeed there are areas where it fails and will never replace recorded sound. It still faces some practical software engineering issues, such as how to manage phasing in procedural methods alongside existing ones. Some of these to consider briefly are realism, cost, training, conflict with established methods and impediments to research and development.

Aesthetic

For sound effects realism is an interesting issue because it seems to fall into the political rather than technical category. Extremely high realism is possible but it cannot be pursued until the political obstacles to further development are removed. Once games did not have super 3D graphics, early titles like Wolfenstein and Quake were basically box walled mazes covered in low resolution textures. Synthetic sound is stuck at an equivalent stage of development, mainly because it has been excluded and neglected for 15 years. However these early games competed in the same market as one of the most visually interesting releases ever made. Myst presented a series of 2D and layered 2D images that were hyper-realistic fantasy landscapes created in Photoshop and other art packages. It was a visually stunning achievement although the images were essentially photographic. Nobody ever said to the developers of 3D games in 1995, "look guys, these 3D graphics you're producing aren't very realistic, why don't you do it like Myst?". Having seen photo-realistic scenery they were not spoiled by it and understood the benefits of real-time interactive content even if it wasn't yet realistic. For sound effects there is a "last mile" problem. In synthetic sound there are many naysayers who complain that it "isn't very realistic". They have been spoiled by sampled sound (photographs) and unable to take the view that these photographs are limited, that absolute realism isn't the point and that if synthetic procedural sound had support, budgets and time to develop it would improve in exactly the same way that 3D graphics have over the years since there are no fundamental obstacles.

When it comes to the aesthetics of procedural music scores the issue is far more cloudy. It seems there are fundamental obstacles, hard AI problems which may never be solved. The procedural approach has no hard and fast metrics by which to judge its output[19]. Composer Pedro Camacho reduces the test to some simple old wisdom, "There are only two types of music, good and bad.", with procedural music falling into the latter. Bruce Jacob, a long time researcher and algorithmic composer offers a critique[15]in which the role of procedural music is considered as a tool for inspiration, or as a kind of desktop calculator for the composer, but not a substitute. As I understand it the problem is essentially that expert systems, neural networks and Markov machines can codify the knowledge of a real composer but they cannot exhibit what makes a real composer, creativity. Composer and producer Will Roget points out that some of the goals of composition cannot be codified at all, to develop a concept in ways that are "memorable and maybe even clever". We all recognise good music by these characters, even if we are unable to articulate why a piece is memorable or clever.

Variable cost

This was mentioned as an advantage but is a double edged sword, it's also a disadvantage. Like variable interest rates, whether it's a friend or foe depends on what's happening. Because the cost of producing a synthetic sound can be hard to predict prior to execution we don't know how to allocate resources. This problem is common with other dynamic content production methods and it requires that we can either guess the cost of an operation in advance and carefully schedule resources, or produce methods which gracefully degrade as they run out of resources rather than suddenly breaking. Another solution is to restructure programs to include a pre-computation stage, a short wait that replaces what are currently load times with data model media.

Skills and tools

In fact this resistance is probably closely connected with the final point, from where much institutional and personal resistance to procedural audio comes. Even sound designers who are comfortable with progressive technology feel threatened by the need to adapt their skills and learn new tools. Moving from data to procedural models will never be a quick or complete process. Some sounds, most obviously voice artists, will never be replaced by procedural content. However there is an enormous investment of skills and knowledge in using existing tools, and at the time of writing this, late 2007, there is a shortage of sound designers and programmers in the industry even without a disruptive new technology. One of the greatest challenges in moving forwards to procedural audio will be developing tool chains and training new sound designers to use them.

Industrial inertia

Of course there is an enormous industry built upon the data model in the form of sample libraries and procedural audio is extremely disruptive to this status quo. A single procedure, for example a specialised FM or physical model for metal structures, can replace sounds for bells, iron gates, doors, weapon barrels, spanners, and thousands of other sounds with less than 1k of code. While procedural audio seems threatening to the traditional sound designers role it is not. The important place this technology has is in automatic generation of sounds for enormous new projects. Procedural sound can exist alongside sample data, indeed it must do so for a long time. While sample libraries and traditional field recordings may no longer be used for all sounds they will certainly not become extinct.

Software Patents

Software patents are an odious concept that has emerged in the United States during the last few years. Rejected by the rest of the world in recognition of their anti-progressive and stifling effect on technology and economies, shunned by respectable computer scientists, and even the major corporations who understand their damaging effects, they are very unlikely to prosper. But in the current political climate this has not stopped a software patent arms race running completely out of control in the United States. We cannot ignore that the USA is the largest market for games in the world. While software patents on well established methods and fundamental mathematical processes are absurd and unwelcome the consequent legal grey area leaves many developers cautious about pushing forward with progressive technologies for the US market. Until the US legal system is reformed the market for procedural audio, and many other new technologies is sadly limited to non-US countries.

Summary of developmental factors

Summary of pros and cons for Procedural Audio
For	Against
Rapid growth of product size	Shortage of audio programmers
Demand for more interactivity	Established production methods
Increased processing power	Lack of development tool-chains
Deferred aesthetic settings	Outsourced content producers
Huge body of research potential	US market closed by patent trolls
Automatic physical parametrisation	Physics engines designed for graphics
Simpler asset management	No effective open DSP engines