Andrea Pessino on writing music for the interactive medium, part 2


This article was first published in the February 2001 issue of Sibelius’s web magazine, Siblings, which is no longer available on the Sibelius web site. It was written by Andrea Pessino, at the time a software engineer and composer working with one of the world leading developers of computer games, Blizzard Entertainment, makers of such blockbuster titles as StarCraft, the Diablo series and the WarCraft series. A native of Italy, Andrea has lived and worked in the US since 1989.

In this, the second of a three-part series, Andrea describes how Sibelius has helped him sketch and compose more quickly and more effectively than ever before. Part one is here. Part three will be republished tomorrow.

Composing with Sibelius

It was over a year and a half ago that I first downloaded a demonstration version of Sibelius v1.0 for Windows. It was, at the time, a newly released product, and although I had heard of Sibelius 7 on the Acorn computer I had never had an opportunity to try it.

Within minutes of checking out the program I knew it was something special. I had used virtually all music notation programs up to that point, from the old Music Construction Set on the Apple II to Finale on Macs and PCs; none of those ever made me even consider giving up pen and paper.

One of the biggest problems with professional-level notation software before Sibelius was that the target user always seemed to be the copyist or the engraver, rather than the composer. The authors of Sibelius, when urged to add more features in the MIDI playback department of their program, always like to point out that “Sibelius is a notation program, not a sequencer!” While I do agree with their statement and their mission for the most part, I think they are underplaying one of the most revolutionary aspects of their creation: Sibelius is not just a notation package, it is also a compositional tool.

Every aspect of the program’s design seems directed at making composition a more focused experience. Once I became accustomed to the input method (I always enter music directly, never through MIDI) I felt so proficient with it that I could literally write music as quickly as I could think it. And, of course, with Sibelius I would end up with a professional looking score, ready to have parts extracted, transpositions automatically applied, playback recorded, and much more.

Sibelius never slows me down, there are no interruptions because the program has to reformat or flow the music; never have I seen a message of the kind “Computing… go get a cappuccino” while using Sibelius, and I have created some rather large and involved orchestral scores, at times with 80 or more staves (the number gets that high when I use multiple staves per instrument to layer sounds on playback).

Immediately after getting the program I decided to write a piece just to get familiar with it. There was this little waltz theme I kept going back to while improvising on the piano, so I opted to try my hand at developing and notating it simultaneously using Sibelius. I felt confident that doing so would give me a more balanced view of the program and get me familiar with its basics. About eight hours of work later the piece was more or less done (not much development, but hey, it was just a test!), and I was amazed. This was the very first bit of music I ever composed using Sibelius and the whole experience definitely changed my outlook on computer-assisted music composition.

[sibembed score=”” /]

Shortly afterwards, work on the score for the Diablo 2 game’s cinematics started, and I begun collaborating with Jason Hayes on some of the tasks. At this point I was already determined to use Sibelius for composition, and I did so while Jason was using Digital Performer and Pro Tools (these are the sequencing and recording tools that were used for production, respectively). Initially, we thought we were going to use Sibelius exclusively for composition, and then we would sequence the pieces from scratch. As it turns out, in certain instances where synchronization to picture wasn’t as crucial, we were able to use Sibelius instead of playing all the parts in.

For example, during the Diablo 2 opening movie there is a point where the action becomes much more intense as the monsters the mysterious wanderer has summoned attack the people in the tavern. The music cues in as the narration stops and a skeleton jumps out of a column of fire. As the skeleton’s bony feet touch the floor, the music starts with an ostinato figure in the cellos, soon followed by some elaborate and dissonant pizzicato chords in violas and second violins; a frantic scene of destruction follows as the music develops with horns and other brass instruments holding long notes in a dramatic crescendo.

The whole piece, up to the climatic chords, was played by Sibelius! The program’s playback data was output to a MIDI file, then imported into Digital Performer and, except for some minor tweaks and a few additional sounds, it was largely unchanged. This is not always possible to do, as scoring “to picture” requires great precision and Sibelius lacks the tools to synchronize itself with the images; yet in many cases you can get pretty close just by sketching things in Sibelius and then refining the timings with a specialized tool such as a professional sequencing package.

Also, in this scene the music had a very percussive, almost mechanical quality to it that was actually enhanced by using Sibelius rather than playing the parts in. Sibelius does a very good job of attempting to play back the score, including slight accents on primary and secondary beats, and unless you need a very lyrical, very expressive performance (which you often do) then its playback data can be useful as a starting point in a variety of situations.

Sketching for fun and profit using Sibelius

One of the aspects of working with Sibelius that intrigued me the most was the program’s ability to play back music immediately as I was composing it, including dynamics and articulations. For the first time, a notation program was making an attempt at generating playable MIDI data from nearly all standard music symbols, not just note data (other programs have copied this feature since Sibelius introduced it; with inferior results, in my opinion).

The immediacy of the transition was almost shocking at first; write a few measures, then hit P on the computer keyboard and a full orchestral score comes to life; a tremendous change in compositional dynamics, and another way in which the very nature of music writing is being reshaped by technological advances.

Of course, at the beginning the music was being rendered by a cheap and rather silly sounding card in my computer, and the result was not exactly lush or realistic. Yet the potential was there; ‘how much better could it sound?’ I soon wondered.

When live musicians are not an option (generally because of budgetary considerations) the only way to get reasonably realized music for a project is using a sequencer, lots of sample libraries, a combination of “playing the parts in” and manual labor, and a great deal of time and energy to make the whole thing sound almost like its real life counterpart.

This is the norm, these days, for the great majority of computer and video games, TV productions and for quite a few movies (even not-so-small budget ones). It’s only going to get worse (or better, depending on your opinion). One could probably call this type of scoring an art form in and of itself. There are some people out there who have become so good at imitating a real orchestra that the vast majority of those consuming their product could not tell the difference if they tried. Even more so when composers use a mix of sampled sounds and live instruments (many pieces in our previous games were done that way, for example), telling which is which can become nearly impossible.

The above is particularly true for many tried-and-true musical formulas for motion pictures: the “Hans Zimmer-like” action cue, the “John Williams-like” fanfare, the “trumpet-and-drum” hymn for the military movie, the “charming strings” for the romantic comedy, and so forth. Attempts at doing the same for less stereotypical musical styles, particularly for contemporary concert music, haven’t been nearly as successful; the reasons being that concert pieces tend to push the envelope when it comes to orchestral technique and quite often they venture into sonic spaces which are just not available in sample libraries, and also because the above mentioned musical styles are those where synthetic orchestras are most frequently needed; therefore the composer’s tools have been specializing in the direction of greatest commercial demand.

Some acoustic musicians feel very threatened by this revolution in music production; I think time will show they had nothing to fear. At the end, all of the technology we have only helps people make more music, particularly those that would have not had the means of expressing themselves orchestrally otherwise. Music played by great artists, especially when a number of them gets together for a symphonic or operatic performance, is an experience on an entirely different level. Live music will never be replaced by one or two people putting together a well-engineered sequence, yet it might be complemented or even augmented by emerging music technologies.

For one thing, I don’t believe there have ever been as many people interested in the orchestra as there are now. In order to imitate one must understand, hence, I believe, all of the new technological means are actually helping to consolidate the symphonic orchestra’s role as a pillar of our culture.

Now, suppose that you actually do have the means to hire a quality symphony orchestra to play the score for the game you are working on. If, like myself, you are used to writing music “the whole way” then you are not going to sketch out some music in a sequencer, hire an orchestrator, a copyist, a conductor and limo driver and let them run with it. No, you are a real composer and a control freak, so you are going to do the whole thing yourself, including driving the limo!

It’s a lovely plan, but the problem is that, in game development, you rarely write the score after the game is finished. Games are made up of a large number of components in areas such as design, visual arts, software technology, content management, and so on. Most parts have to be developed more or less in parallel, and then converge towards a whole in the last phases of production. The above is further complicated by the already mentioned non-linear nature of the medium. In practice, the degree of linearity depends greatly on the type of game, but the procedures tend to remain the same.

Therefore, most composers working on game soundtracks do so while the games themselves are being developed, always shooting at a moving target. Now, no matter how big your budget is, access to the orchestra is going to be extremely limited. You are going to get a few rehearsals and a couple of days of recording, if even that much, for a full hour of music in the game.

The recording sessions are likely to happen rather late in the development schedule, so that the game is as complete as possible before the music is committed to it. At this point the composer has a dilemma: the score must be created for the orchestra to play, and writing an hour or so of orchestral music takes a good deal of time even if you know what you are doing; yet sketches must be created as well, and not just for marketing necessities like demos or previews, but more importantly because it is vital to communicate concepts to other members of the team during development.

The producers and designers must get a good idea of how every piece will fit together in the finished work, and they must do so while the game is still being worked on. The music, being such an essential part of the of the player’s experience, must be evaluated to make sure not only that it is ‘good’, but also that it is compatible with the action.

This means there’s still the need to do sequencing in order to create sketches for the temporary score. A huge amount of time to spend on something that’s going to be tossed once the orchestra sessions are complete! Creating “bad” sketches is not really an option, as people tend to have difficulty making the extrapolation to what the real thing is supposed to sound like. The composer might hear this fantabulous sound in his mind’s ear, and he might emphatically proclaim that “…this will sound great when played by the orchestra!” but nobody will actually care. What they are hearing there and then is what they are going to judge, and in many ways they are right to do so.

So, how can the composer reconcile the need to provide decent enough sketches with the time consuming task of scoring for the orchestra? Some time ago I came to ask myself that very same question, and Sibelius turned out to be part of a possible answer.

Soon after realizing that I could use Sibelius to drive more sophisticated MIDI playback equipment than the typical sound card found in a PC, I began to dream of the possibilities offered by such a setup. Sibelius was a much more pleasant and productive environment to compose in than any sequencer, and I knew that it would be the right tool to write for the orchestra. If I could get the playback from the program to sound good enough to communicate my orchestral ideas, then I would have a sketch to use as a temporary track without any extra work!

I would be composing the music the way I liked best, in a composer-centric software environment which allowed me to write as I was accustomed to, and at the end I would have both my conductor score and the parts ready to print, as well as a sketch good enough to put in the game for the other team members to critique. The big unknown in the nice equation above was, of course, just how good could it all be made to sound.

As it turns out, a multitude of technical issues arose in the year following my initial encounter with Sibelius, most of them related to making a number of different pieces of sound-generating hardware, each with its own strengths and quirks, work well together while being driven exclusively by my Sibelius scores.

Getting into the details of how it all works would go much beyond the scope of this article; one could easily fill a good-sized book with issues and techniques relating to using sound modules, samplers and sample libraries to imitate a real orchestra, whether or not we even add Sibelius to the equation!

Hence, I will defer the above discussions to another time, and instead I will provide some examples of sketches I created during the past year.

Leave a Comment

Your email address will not be published. Required fields are marked *