Using real time audio synthesis on CELL CHIP for video games
Hi there, I'm glad to see that there are actually a few people out there at this IBM blog, that are interested in reading some "informal" but somewhat technical writing. I really need to rant about how much I love the idea of audio synthesis for CELL chip, and in turn the PS3. For a long time now, audio producers and composers have been re-sampling synthesizers into HUGE chucks of data, even 24Bit 96khz rates. Audio is one of the biggest storage factors on blockbuster style video games, even with wavelet based compressions, which do destroy the sound to some degree., in a noticeable way. This resampled data, is now both HUGE, and STATIC, both of these things are very counter productive to creative expansion in video games, as well as very horrible for the system, which relies heavily on DVD streaming, and low latency access. On a cell chip, this whole process can change a LOT, pretty much for free, thanks to the nature of signal generator DSP, and superfast, but small memory area, available to an SPU. Almost no memory is required for a LOT of modular synthesis, granular is a bit of an exception, but physical models, once again are extremely low memory, and furthermore sound more "interesting" that a lot of granular synthesis, because of the parameterization capabilities, and the generalness of the physical model, synthesis method, but FURTHERMORE, a general purpose modular system like a NORD modular, is capable of a LOT of physical modeling algorithms. This is a bit confusing, but the synthesizer industry, is very full blown, and very full of convolution. So the answer to all of this is a lightweight version of nice a modular synthesizer, like Reaktor, or Nord.
So what does this mean for games?
This means great things for LOTS of games, and less for others. I honestly feel that part of the using real time audio synthesis, will be stemmed back to the design of the game in the first place, due to it's dynamic nature, as well at it's "cheap digital distribution" style game possibilities. If the game is supposed to rely very heavily on "real world sounds" than, real time audio synthesis only comes in handy on sound tracks, BUT sound tracks often are completely synthetic, or use physically modeled instruments, like strings, which can sound better than samples now. Most often it is a combination, studio setup of synthesizers and effects, all of which can be accomplished via modular synthesis and sampling. With the right "DAW" on a SPU (3Ghz DAW's existed for a LONG TIME), you could no doubt have the entire soundtrack generated at run time. After all, the entire soundtrack of a video game was generated at RUN TIME, for a long long time. I honestly feel that time is back, especially on cell chip, because near zero cache penalty, and the how LITTLE it can memory it can take to store a modular program, plus the ability to "tweak" the game design, to need almost NO SAMPLING, and very minimal DMA accesses. You now have fully dynamic parameterizable "pre-designed" sfx, and a full dynamic soundtrack, with modern wavetable synthesizers, modular effects and physical models, and some sampling.
So does this mean new things?? Otherwise IMPOSSIBLE??
Pretty much,. you can have crazy evolving physical models, and sound generator sequences, that will last several hours, that can be computed at ultra high resolution in 5.1 that need almost never repeat, generated "free" on a cell chip. Storing this type of audio would take up HUGE amounts of space, especially when it's "multritrack" dynamic, it becomes very very very stupid. Audio synthesis is much more acceptable than image synthesis, so it seems. A lot of the audio even in a blockbuster game, really is a DAW that's not using "that much ram" if you take out the samples, and the DMA controller can handle that pretty good. Physical models + Modular/Wavetables + effects,. pretty much gives you almost every soundtrack, huge samplers are "that important" anymore. That's my opinion.; Not a lot of video game studios are using "John Williams" and the philharmonic. Physically modeled explosions are very cool, the creative possibilities of endless in terms of sound fx for sci-fi sylte games, or just plain old "retro ish" games, SID / chip tunes, even remains popular in many target demographics.
If I ever own a video game studio,. I'll be using the force on this one
... more to come!