Imagine that 3D rendering engines had never been invented and all games still worked like Mystby presenting a series of flat photographs. That is the state of sound in current games technology in 2006. Sound designers create a set of static .wav files, one for each event or scene in a game and these play over and over. Every time you pickup an object or perform an action a preset file is played to the sound system. A little realtime parameterisation occurs on some of the better engines, perhaps a bit of reverb linked to the local geometry, some panning or surround processing, but that's it.
A typical game can contain 100MB of samples which must be edited, looped, packaged and named properly to provide hooks for the level designer or runtime engine. All these sounds have to be collected and preprocessed for use. Although a good sound designer can work miracles with transformations and editing existing sounds the problem remains that the sounds remain static, at best the code can produce a graded selection or a random selection from a set of static files.
Native digital synthesis is the future for multimedia and games. Instead of recording a bunch of sounds like photographs the synthesist creates code to be executed at runtime. These are the sounds, expressed as procedures or formulas which will be run by the client hardware. Synthesis is a direct analogy to the 3D rendering engine used for most modern games, only difference is that it's a sound rendering process.
There are a number of distinct advantages to the native synthetic approach. The size of code required to generate many hours of sound can be compact, occupying only a few kilobytes. While the space efficiency is a staggering improvement the real power of synthesis is far more subtle. The most exciting thing about synthesis is the way that objects, actors and events coincide with the 3D object aspects of game development. In other words, code can be built right into the objects in a game to handle their sounds at the same level as physics and other attributes. This means that sonic behavior for almost infinite permutations of object-object interactions may be implicit in a design, something no sample library however large could handle. Furthermore these features may be modified at runtime depending on context. It is possible to easily reuse algorithms by deriving subtlety new instances, scaling or combination. During this exploration we will see many analogies to the 3D development process with direct counterparts to the modeling, texturing, animation and lighting processes familiar to most games developers. Later we examine the reasoning at the root of these analogies and consider the idea that there is in fact a schism in games development which ignores massive optimisations possible by realising that the visual properties of objects and their sonic properties are deeply linked.
One reasonadoption is slow is because few people understand it. Here we intend to remedy that problem to some degree. While there are many producers on the block who work in multimedia development most come from a music background and employ old school techniques from the recording industry which are taught on many courses. These legacy roots are unhelpful since synthesis is as different to traditional sound design as 3D design is to photography. Synthesis is a technical art, based partly in mathematical theories, psychology, physics and engineering it requires a good all round scientific and artistic discipline, focus, imagination and patience for development to proceed. Another reason games sound is quite stuck in the 1990s is the persistence of a status quo in the traditional and established process of games development which favors visual aspects above all other considerations. Sound is often an afterthought. In metrics evaluations CPU cycles are allocated first to the visual procedures. Most games engines treat sound as an event based faculty instead of building sound right into the core as a realtime multi-parameter process along with physics and lighting where it belongs. Finally it seems to have acquired some bad cultural connotations. People think of cheesy 1980s music when they hear the word synthesis, so much that I sometimes feel embarrassed to use the word when talking about my work. But the colossal damage done to the image and reputation of synthesis by Stock, Aiken and Watermans abominable "Hit factory" music will not last forever, to call the 1980s the decade of the synthesiser is like calling the 1800's the age of photography. This bad reputation hasn't been helped by hardware manufacturers like Roland and Yamaha developing terribly crippled "preset" devices and computer cards which then infest cheaply produced pop music and fail to deliver on a general purpose platform. Such devices have very impoverished user interfaces which create the reputation that synthesis is hard to program. To put it mildly, we haven't even begun to scratch the surface of sound synthesis, so if you have any prejudices about what a synthesiser is or does please erase them from your brain now using an extra strength mind rubber.
This "Practical guide to synthetic sound design" is unlike any other tutorials I've seen or taught. Most students of synthesis come from a good maths background and proper academic courses in music technology and electroacoustic composition tend to focus on the theory first and leave the applications to the creativity of the designer during their practical career. Most of this gets forgotten in the same way that mathematics, so badly taught in our schools, is forgotten the moment students leave education, a lack of meaningful context is the cause. It is the ambitious aim of this programme to teach principles of synthesis in a refreshing way with as little recourse to maths or computer science as possible. This is made possible by the remarkable software written by Miller Puckette and others called PureData, a 21st century visual programming language for DSP. The author of this guide Andy Farnell is a computer scientist with a thorough background in sciences and digital media. As far as possible written descriptions are given and equations and graphs are not used unless absolutely necessary. The practical approach relies on asking important questions about the underlying physical nature of sound and psychological questions about the use and context of sound in multimedia.
During the exercises we will touch on physics, perceptual psychology, music and many other disciplines, but you don't have to be a programmer, mathematician, psychologist, games developer, musician, or scientist to start understanding and creating sound through synthesis, although hopefully you will learn a little bit about these subjects along the journey. Some enthusiasm for the material, patience to experiment and a little creative insight are all you need to go a long way. A basic understanding of sound and some familiarity with the software application are assumed. Ideally this course should be studied as a complement to reading a few essential textbooks given in the reference section, or as a practical counterpart to Puckettes deeper theoretical treatment of PureData synthesis.
Of course you aren't going to be able to apply the full power of this knowledge until games engine developers see the light and start to integrate synthetic sound DSP capabilities into the main code. In the meantime you can use synthesis to prepare ordinary samples in a structured way.
Miller Puckette said "We need to undo the MIDI revolution". I think we need to undo the sampling revolution too. Summary of points