An Introduction on Interactive Music in Games
Scoring music for games relies on many techniques inherited from film scoring, including harmonic, dynamic and rhythmic development, cadences and themes. The main function of music in media is to support the emotion, although, video game music differs significantly from music found in linear media such as film and television. The time it takes to play a game depends on many different factors, including length of the story, game variability, and most importantly the experience of the player.
In video games, many contemporary composers use various interactive, music techniques to adapt to the player in real-time and progression within the music is created with gameplay, resulting in an adaptive and interactive soundtrack. For example, Action and Ambient tracks, found in nearly every video game genre from puzzle games to shooters. Ambient tracks set the emotional atmosphere during lower energy gameplay in which the player is free to explore and engage in safe activities. Action tracks stimulate the excitement level of the player during periods of heightened activity. In this situation the game communicates the intense emotions of the experience. Phillips, 2014 describes ‘action and ambient tracks are designed to enhance two diametrically opposed states of gameplay’ considering this term, one could say that the state of the music depends on the type of gameplay the player in engaged in, and how the music transitions between these states is according to predefined interactive variables. Although, more layers mean greater diversity in the adaptive music allowing for more emotional range moment-to-moment than just two states (Sweet, 2016)
In his book ‘Writing Interactive Music for Video Games’ Micheal Sweet also speaks of these variables and how synchronisation in the music is achieved by following changes in emotional context. These changes then effect how the music might play, but since a composer cannot write a customised score for each individual player, he or she may instead write an adaptable score that takes the player skill level and pacing into account. There are many other factors in a game that interactive music can take into account, including the players health, proximity to enemies, AI character states, and the length of the music. In turn, the composer must score these multiple paths with several music cues that are able to transform from one cue to another seamlessly.
One way to produce an adaptive music effect is with Phrase Branching. ie: Moving from track to track with logical parameters. (If enemies are greater than 2, then play the combat music, if not then stay looping calm or ambient music). Intros,Transitions and Outros are very useful in this scenario.
Phrase Branching Advantages
- Most musical of all the horizontal re-sequencing techniques because it will never interrupt a musical phrase.
- Ability to change tempo, harmony, instrumentation or melody in the next phrase based on a game event.
Phrase Branching Disadvantages
- Non-immediate musical change because the music change will wait until the end of the current phrase which is dependent on the length of the phrases.
- Can be more disruptive to the player in terms of musical changes than vertical remixing.
Infamous Second Son. An Example of Adaptive Music
Matching a game’s score and soundtrack to the player’s actions is, as SCEA’s Senior Music Manager Jonathan Mayer describes, “one of the biggest challenges working in games.” He’s confident Sucker Punch pulled it off. Infamous: Second Son (2014) is an open-world action game, set in Seattle, featuring central character Delsin Rowe, who has “attitude”. These facts inform everything that follows. Open world means lots of time spent exploring around, so the soundtrack has to be large and varied. The location setting is also something that should be considered, in this case the setting is in Seattle which conjures a certain musical style which they didn’t want the music to sound like. However, they did want to incorporate that influence into the score.
Within the industry these interactive scores tend to require a team of many people working together to integrate these systems, rather than the composer working alone. The audio team at Sucker Punch, working on Infamous Second Son worked directly with the composers and programmers on the team to fine tune their adaptive music system so that they had granular control of the music tension based on enemies’ awareness states in regards to the player. To pull this off they really had to play the game as much as possible and really understand what their users are going to experience. ‘The end result was that having such an integrated music team made the music much more impactful throughout the game.’ Mayer
Below is an example of a phrase branching adaptive music approach based on contact with enemies. As the player explores the city a low intensity sparse guitar theme can be heard, until the player begins combat at 4:10:53 this theme is reintroduced in full force with more prominent guitar parts with melodic and rhythmic development, heavier drums and bass making the experience more intense. Phrase branching is another horizontal re-sequencing technique which waits for the current musical phrase to end before playing the next musical cue (Sweet, 2016)
An example of adaptive music reacting to player health can be heard in the example below. When the player reaches critical health the sound effects fade out and the music continues in the background, however this time an ambiguous sustaining vocal drone sound is incorporated into the music until the player can either escape or in this case, dies and a musical stinger is played to end the end the transition until the player re-spawns.
LA Noire, Interactive Music used for Ludic Functions
Ludic music, is music that is somehow part of a game’s rules as opposed to just its narrative. Ludic music in games is typically congruent with the action and can heighten feelings of mastery by providing emotional rewards for player achievements (Stevens and Raybould, 2014)
In Rockstars detective game LA Noire (2011) the music provides important feedback to the player based on gameplay. In the example below the investigation music theme plays throughout while the player searches a crime scene. When the player comes into contact with a clue a short piano motif plays. Upon collecting all of the clues another motif is heard, this time the notes rising in pitch which is providing positive feedback and reward for finding all clues. Once the music completely stops this is informing the player that there is no more clues left to find in the area.
Another example from LA Noire is the music providing feedback based on player right or wrong choices in the Interrogation scenes. A sustaining drone underscores the interrogations and when the Player is promoted to choose wether the suspect is lying, telling the truth or doubtful, upon choosing the correct or wrong choice a different musical phrase is heard.
Choosing the correct answer, triggers a major phrase associated with positive feedback
Choosing wrong answer: Minor 7th, associated with negative feedback
Play Experience and Repetition Problem
The play experience in games is significantly longer than the experience of other linear media and as result players don’t usually finish games in one setting. This has direct implications on the music. The length of play and enjoyment level of the player also determines it’s replayability. If a player plays through a game multiple times, how does that effect the impact and function of the music on a subconscious level? As the primary function of music in video games is to create tension, the resolution of that tension amplifies the players euphoria when finishing a goal. Hypothetically speaking, If a composer scores a scary cue for a Horror game to underscore the player engaging a new enemy. During the first encounter the player is tense and on edge, because their subconscious mind momentarily struggles to categorise this new uneasy, dissonant sound. If the music and the situation are both the same the second time around, the impact is diminished. Before long, the subconscious mind makes a connection between the music and that event and results in filtering out the music, because the information no longer carries meaning.
It may never be practical or desirable for a video game score to provide completely new music for every single moment in the game. Composers should remember this rule and work with developers to push the boundaries of technology to allow for music that feels less repetitive.
References
Campbell. C , Behind the music of Infamous Second Son, 2014, Polygon
GDC Vault, Adaptive music, The secret lies within Music itself GDC Vault
Infamous: Second Son, Sucker Punch, Sony ,2014
LA Noire, Rockstar, 2011
Martin. G, The Seattle Sound of Infamous: Second Son, 2014, Paste Magazine
Phillips.W, A composers guide to Game Music, 2014, MIT Press
Stevens and Raybould, Designing a Game for Music, Oxford Handbook of Interactive Audio, 2014
Sweet.M, Writing Interactive Music for Video Games, 2015, Pearson
Sweet.M, Top 6 Adaptive Music Techniques in Games, Designing Music Now,
Sarah Sherlock 
Leave a Reply