Andrea Pessino on writing music for the interactive medium, part 1


This article was first published in the January 2001 issue of Sibelius’s web magazine, Siblings, which is no longer available on the Sibelius web site. It was written by Andrea Pessino, at the time a software engineer and composer working with one of the world leading developers of computer games, Blizzard Entertainment, makers of such blockbuster titles as StarCraft, the Diablo series and the WarCraft series. A native of Italy, Andrea has lived and worked in the US since 1989.

In this, the first of a three-part series, Andrea describes the particular demands placed on a composer when writing for the interactive medium of computer games. Part two will be republished tomorrow.

Musical scores for computer and video games are generally separated into two categories: in-game music (sometimes also called game play music) and cinematic music.

By in-game music we mean to describe sound material which plays more or less in the background as the player is actively interacting with the game world. Music of this type is very difficult to write, as it must blend perfectly into the game’s “texture” while rarely grabbing the spotlight; it must complement the action, not overwhelm it.

To get an idea of how hard it is to create effective in game music, try composing a piece which is only a few minutes long, clearly fits the atmosphere of the game and does not become intolerable after maybe hours of continued listening! Easier said than done, indeed.

For a successful example of in game music you might listen to the much-acclaimed music from our recently released title Diablo II. The in game score, written and realized by Matt Uelmen, is a perfect example of how to enhance the mood of a game and draw the player in through the music, without ever being intrusive. Another superb, if diametrically different, example of effective in game music was the score to Homeworld, a wonderful strategy game developed by Relic Entertainment and published by Sierra last year. Its score was barely musical, and I doubt I would ever listen to it on its own, but it fit the galactic war theme of the game to perfection and it did wonders to enhance the player’s experience.

On the more experimental side of things, some people have been exploring possibilities in interactive (or, should we say, “reactive”) types of composition, the goal of which would be to alter the music smoothly according to unscripted events occurring in the game world.

For example, the music might react to a player’s action by gradually modifying its harmonic or rhythmic pattern, its orchestral texture, or even its melodic line to reflect what the designers think the player should be experiencing at that moment. Ideally, this would cause the music to swell poetically when the player turns and a beautiful vista opens up, or a terrifying cue would smoothly emerge from a suspenseful theme as a horde of monsters approaches the unsuspecting hero.

In practice, all of the above is wishful thinking. Thus far, nearly all attempts at interactive in game music have been underwhelming at best. If the musical transitions are too smooth, one barely notices that a change is even happening. If, on the other hand, the score attempts to be more vigorous, then the fact that it’s “catching up” to the action (as opposed to being synchronized to it) gives the impression that the changes are “missing the moment.” Worse still, the whole concept places so many restrictions on the way the music is composed that the output ends up feeling mechanical and artificial.

A possible implementation of an interactive music system could be as follows: a number of different versions of the same piece are recorded, one for each “mood” (for example: frantic, dangerous, explorative and inspiring); all versions must be played with perfect synchronization and must be composed in such a way that any mixture of the tracks produces effective material (things could be setup in even grander detail by having additional, purely transitional tracks). The music would then be composite by streaming and mixing in real time a subset of the available tracks, while varying their proportions dynamically depending on the ever-changing state of the game world.

Notice that, in the above-mentioned system, all of the tracks would have to share the same fundamental harmonic and rhythmic structure, and the majority of changes would have to be conveyed through adding and removing of instrumental layers or melodic lines in counterpoint.

Without even considering its significant, but manageable, technical implications, the above scheme presents formidable challenges at a purely compositional level. Given that the underlying structure is unchanging, making the combined effect musically pleasing, as well as effective in delivering its intended emotional message, is a tremendously difficult task.

For practical purposes a better idea than the above is needed; one that allows for an entirely new compositional paradigm to be defined. It is conceivable that the only problem with interactive music is that no one has had such a breakthrough idea yet, and that when someone finally does its potential will unfold and its effectiveness will improve by leaps and bounds. Yet I am somewhat pessimistic as to how much promise the very concept shows, as lack of predictability makes writing situational music, a quintessentially linear endeavor, a near impossibility.

What we call cinematic music actually refers to any musical cue used to underscore non-interactive segments. Most high-profile games of today use computer animated shorts to introduce the player to the game world, at key times to advance the story (generally at each milestone the player crosses), and often to wrap up the whole deal at the end of the game. As always, things vary depending on genres and budgets.

Our company’s film department, for example, provides some of the highest quality animated cinematics (as they are called here) in our industry, and we have a large and incredibly talented group of people doing just that: they create computer animated movies to help immerse the player into the game story. Proper motivation is as essential in games as it is in any kind of fiction, so a great deal of effort and resources go into making these movies as effective and memorable as we can make them.

For musical purposes, the fundamental point here is that cinematic music does not happen during interactive segments, regardless of whether the “movie” is a computer generated short (as it has been the case with all of our previous titles), a live action film or it is a scripted segment played using the same visualization engine as the interactive portion of the game. The linear component characteristic of all non-interactive storytelling is restored, and scoring cinematic music for games becomes essentially the same as scoring for motion pictures or television.

Diablo II contains over twenty minutes of cinematics split into five shorts (an introduction and then a movie after each of the game’s four “chapters”), the longest of those being the opening segment which sets up the story before the player begins his or her journey. The score for the movies was created for the most part by Jason Hayes; Glenn Stafford (the director of our audio department, and a seasoned veteran of game music) scored some additional segments and, finally, I joined Jason in some “co-composing action” (as he would call it) and did orchestration work on various pieces.

Most relevantly to this discussion: I brought Sibelius into the picture (pun intended), and how that happened is the subject of next month’s article…

Leave a Comment

Your email address will not be published. Required fields are marked *