Victoria Smith is a 3D Artist living in Auckland, New Zealand. She recently completed her Bachelor of Creative Technologies (Game Art) at Media Design School, but she won’t let that stop her from looking at games through an academic lens. When she’s not writing or creating, she loves helping out with volunteer initiatives in her local game development community.

At their heart, adventure games are about exploration and discovery. Players must connect with the world around them to solve puzzles and progress forward, making player/game space interaction a key component of the genre. When looking at the design of these games, investigating them through the dual lenses of semiotics and affordances can help describe the subtle nuances of interactivity and game design, which can make or break player experience. According to Saussure (1983), semiotics is the study of signs, symbols and the meanings these have in communication, while Norman (2003) argues affordances describe human interaction with objects and spaces. By focusing these concepts on one game series, LucasArt’s Monkey Island, we can see how these ideas about spatial, visual and linguistic communication inform the design of these games and facilitate the player’s interactions as the series evolves. A large part of game design is effective communication with the player, and looking at well-established theories which deal with this can help explain why some design choices work and others do not. Adventure game players want to feel like their intuition can solve a puzzle, like their crazy ideas might just work. This means giving them the freedom to explore their environment and interact without restraint, and none of that is possible without clear communication.     

To fully appreciate these connections, a clear understanding of semiotic theory is required. Semiotics is the study of signs, and the contextually sensitive meanings given to them that reinforce the perceived reality of our surroundings. Signs can take the form of images, words, objects or actions – but it must be noted that “nothing is a sign unless it is interpreted as a sign” (Peirce, 1931).  Signs signify something other than themselves and have no intrinsic meaning, as they are simply a vehicle for concepts. These concepts are received by the observer and interpreted within the context of the situation. Signs can also convey cultural signifiers and universally agreed-upon meanings that may not require an immediate context or even a conscious interpretation. Signifiers can have multiple meanings depending on the context, the interpretation of the viewer, and any cultural bias or prior experiences the viewer may have (Saussure, 1983). One example of a basic sign is the word ‘Open’ on a shop door. The word itself is the signifier. The signified concept is the idea that the shop is open for business—that the observer of the sign can walk into the shop to purchase something (Chandler, 2007). Semiotics can only work if the relationship between the signifier and the signified is conventional, meaning the relationship is accepted and agreed-upon by the surrounding culture. In video games, this relationship works within the context of a specific game space; the same sign in one game may mean something completely different in another. Without proper context, or any cultural clues, feedback loops between the signifier and the signified can break down and the concept can be misinterpreted or lost.  For example, Frictional Game’s SOMA (2015) has many adventure game elements, but lacks a true inventory. All of your actions are context-sensitive and the player rarely needs to transport an object further than one or two rooms, so there is no need for a traditional grid structure of usable items that the player can access. However, the tutorial text prompts the player to press the Tab key to show an inventory, which shows a static image of the single item you have at the bottom of the screen. The player cannot click on the item or interact with it, and the inventory is never referenced or needed for the rest of the game. Although the concept of an inventory in an adventure game is a cultural reference the player will understand, the absence of feedback creates a frustrating lack of context for the player. It is likely they will never look at the inventory again, or overcomplicate what is an otherwise simple interaction system.

Affordances deal with the spatial relations between an object and its user, describing the most obvious interaction for a human to have with an object. Each object we encounter leads us to generate questions and hypotheses about its use, and affordances should provide visual cues and feedback to facilitate the correct interpretation (Norman, 2013). If we encounter a ball in a game, there is a reasonable expectation that we can roll it, due to the curved design of the object. Similarly, cubes or crates are flat-sided, so we expect to push them or jump onto them. In psychology, affordances are often defined as “action possibilities.” Action possibilities are all the actions that are available for a user to make in an environment, measurable by the individual’s ability to recognise and interpret them (Gibson, 2014). Using this as a framework, Norman identifies major design decisions that communicate affordances. The possible actions must be well displayed and allow the user to reach a conclusion on what can be done, and feedback must also be provided for the user’s eventual choice of action (Norman, 2013). Cardona-Rivera and Young (2013) expand this idea further, specifying the real affordances of what is possible in a video game, the perceived affordances of what the player thinks is possible, and the feedback loop that facilitates both of these and guides the player. Real affordances are defined as actions that are supported by the game’s underlying framework and are explicitly controlled by the game’s developers. These are sometimes at odds with the player’s perceived affordances, which will be shaped by any previous experience with games of the same genre and what actions they’ve already done in the current game. Bridging the gap requires a well-designed feedback loop, one which encourages the player to explore and experiment while gently guiding them.

Although the principles of semiotics and affordances can be applied to any adventure game and produce interesting insights, the Monkey Island series shows an explicit connection between the design of the game world, the user interface, and the abstract principles of semiotics and affordances. The first Monkey Island game, The Secret of Monkey Island (Ron Gilbert, 1990) used a user interface method called SCUMM (LucasArts, 1987). The player has a list of verbs they can use to interact with the 2D game space. They are simple words such as ‘Give’, ‘Push’, ‘Pull’ and ‘Talk to’. These words are real affordances; their actions are possible within the game and are explicitly coded into the world. They are not vague enough so that one word can be used for everything, but not defined to the point where one word will only work for one incident in the game. The words are signifiers of actions the player can take within the game. For example, the word ‘Open’ is the signifier, but the signified concept could change depending on the game space. It could signify opening a door, or a chest or a book. The signifiers remain constant throughout the game, creating an effective feedback loop that informs the player’s perceived affordances. By using words as signs, the developers are ensuring the player will understand the possible concepts they represent. As this is the main way the user will interact with the game space, it needs to be clear what actions they may need to take while playing the game.  

Figure 1. The Secret of Monkey Island verb array (BitTech, 2009).

Although this original method is robust, it can lead to a closed-off world with limited freedom for the player. With only a handful of verbs to use, any exploration the player may have done organically is limited by the actions available to them. In the field of semiotics, the Sapir-Whorf hypothesis is a linguistic theory proposing that our thoughts and actions are determined by language (Chandler, n.d). We do not live in an objective world, “but are very much at the mercy of the particular language which has become the medium of expression for [our] society” (Sapir, 1958). The SCUMM system was developed by LucasArts in the late 1980’s during the early days of adventure games. Early versions had lots of verbs, but with the release of The Secret of Monkey Island, it had been pared down to twelve, and then down to nine in later versions. If we apply this linguistic determination to the game space and imagine it as a world, then the player’s actions are determined by the signifiers they can interpret. In The Secret of Monkey Island’s case, the player’s expression within the world is literally limited by its language. The player cannot eat because there is no word for it and no framework within the world for that action to happen. The language available to the player not only limits the way the world can be interacted with, but their interpretation of the world. The player will view the world through the lens of the words available to them; a user might not consider a ladder to be a viable affordance if they do not have the word ‘Climb’. The game space is vital to the player’s understanding of game signifiers, but equally important is how the availability of signifiers shapes the play experience. This is a fascinating way to evaluate the use of semiotics and affordances in game design, and clearly shows the relationship between these concepts and the player’s experience of the game space.   

 

 

In the third game of the series, The Curse of Monkey Island (LucasArts, 1997), the user interface and player interaction with the game was changed significantly. Instead of a verb list, the player has a “verb coin.” When the player clicks on an object in the 2D game space, an image of a coin appears. The coin has a picture of a hand, a skull and a parrot on it. The hand represents all the verbs relating to interacting with an object—‘Pick up’, ‘Use’, ‘Open’, ‘Push’ and so forth. The skull represents the player’s eyes, they can click it to inspect an object or survey the game space currently shown on the screen. For the majority of the game, the parrot icon signifies talking, but it is occasionally used to eat items. This is the key difference between the SCUMM interaction system and the “verb coin.” The signifiers on the verb coin stay the same, but the concepts they are signifying change each time. Using the hand icon on a chest might open it, but using the hand icon on a rock might pick it up. The concepts the signifiers represent change depending on the context they are applied to in the world. Interestingly, the constantly changing signifiers allow for easier interpretation by the player. Instead of the player having to match their interpretation of the signs to the affordances determined by the developer, the player can merely click the hand to communicate that they want to interact with the object in some way. Instead of trying to force the given verbs into the perceived affordances they’re interacting with, the clearer UI gives the player freedom to experiment with their interpretations of the signifiers. The “verb coin” acts as a shortcut between the concepts the player is trying to use in the game space and the way they are able to execute them. Although words are a highly naturalised form of semiotics, they become unwieldy when the signified concept is too vague or too defined. Other symbols are not as defined as words in their possible interpretation, but they let the player communicate their general motives much more effectively. Although a relatively simple change, the UI difference shows how integral semiotics are to understanding and interacting with the game space. Naturalized signifiers may not always be the best choice; the most effective set of signs will work with the context of the game world and be the quickest way to communicate the game’s concepts.

Figure 3. The verb coin in use (StackExchange, 2014)

The Secret of Monkey Island and The Curse of Monkey Island are both great examples of affordances and semiotics in adventure game design and how these elements influence how players interpret and interact with the game space. The massive shift between the SCUMM engine and the “verb coin” is an excellent way to see the impact of signifiers and how they manipulate the player’s experience of a virtual environment. Similarly, applying the Sapir-Worf hypothesis to game spaces helps us to understand the surprisingly large role signifiers can play. This analysis of the Monkey Island series is but one way to apply these two theoretical fields of study to the very literal field of game design and how it can result in a deeper look at the construction of player experience. However, it is not limited to the Monkey Island series or early adventure games as a whole; Broken Age (Double Fine Productions, 2014) and The Walking Dead (Telltale Games, 2012) are two newer adventure games with minimal user interface and a strong focus on the player experience. Using simple symbols, like an eye for ‘Look At,’ they use key communications principles to streamline gameplay, and add to the growing cache of culturally recognized semiotics and affordances in gaming. As video games continue to evolve, old systems are replaced and new ones are codified. Adventure games are continually refining how the player interacts with the game-space, shifting the focus from the UI to the puzzles and environment of the game itself.    

References

Bit Tech. (2009). The Secret of Monkey Island: SE Review. Retrieved from: http://www.bit-tech.net/gaming/pc/2009/07/15/the-secret-of-monkey-island-special-edn/1

Cardona-Rivera, R.E., Young, R.M. (2013). A Cognitivist Theory of Affordances for Games. DiGRA, 07 (n.a). Retrieved from: http://www.digra.org/wp-content/uploads/digital-library/paper_74.pdf

Chandler, D. (2007). Semiotics: The Basics. New York: Routledge.

Chandler, D. (n.d) The Sapir-Whorf Hypothesis. Retrieved from: http://visual-memory.co.uk/daniel/Documents/short/whorf.html

Double Fine Productions (2014). Broken Age [PC]. San Francisco, CA: Double Fine Productions.

Frictional Games. (2015). SOMA [PC, PS4]. Helsingborg, Sweden: Frictional Games.

FromSoftware. (2011). Dark Souls [PS3, Xbox 360]. Tokyo, Japan: FromSoftware

Gibson, J. J. (2014). The Ecological Approach to Visual Perception: Classic Edition. Psychology Press.

LucasArts. (1987). SCUMM [Game Engine]. San Francisco, CA: LucasArts.

LucasArts. (1997). The Curse of Monkey Island [Windows 9x]. San Francisco, CA: LucasArts.

Norman, D.A., (2013). The Design of Everyday Things: Revised and Expanded Edition. New York, NY: Basic Books.

Peacock, D. W. (2010). The Curse of Monkey Island. Retrieved from: https://davidwpeacockthefirst.wordpress.com/2010/08/31/the-curse-of-monkey-island/

Peirce, C. S., (1931-58). Collected Writings (8 Vols.). (Ed. Charles Hartshorne, Paul Weiss & Arthur W Burks). Cambridge, MA: Harvard University Press.

Ron Gilbert. (1990). The Secret of Monkey Island [Amiga, Atari ST, DOS, Sega CD, Mac OS]. San Francisco, CA: LucasArts

Sapir, E. (1929): ‘The Status of Linguistics as a Science’. In E. Sapir (1958): Culture, Language and Personality (ed. D. G. Mandelbaum). Berkeley, CA: University of California Press

Saussure, Ferdinand de ([1916] 1983): Course in General Linguistics (trans. Roy Harris). London: Duckworth

StackExchange. (2014). What was the point of each verb/action in Monkey Island?. Retrieved from: http://gaming.stackexchange.com/questions/188650/what-was-the-point-of-each-verb-action-in-monkey-island

Telltale Games (2012). The Walking Dead[PC, PS3, Xbox 360, iOS]. San Rafael, CA: Telltale Games.