Today on the League we have a guest post from Franklin Kenter. By day, Franklin is a math professor, having earned himself a PhD in Mathematics at UC San Diego. But by all other times, Franklin is an enthusiastic game designer, having signed several upcoming games. Franklin is an active member of the design community, helping to organize multiple events including Unpub San Jose. Despite being a literal doctor of math, today Franklin focuses on an effective playtesting strategy, which is especially useful since half of his playtests are performed by his design partner who lives a thousand miles away.

TAKING AUDIO FEEDBACK

When playtesting your game, there are number of ways to take and gather feedback. Most notable is to simply take notes on the game or ask players to fill out forms.

However, one very underutilized method is to take an audio recording. In this digital age, most of us have smart phones, and they are very capable of taking audio (or even video) recording which can prove very useful.

I came about taking audio feedback because for most of my games, my collaborator is in California while I am in Texas. It serves as a simple method for passing playtest feedback between us.

To this end, one of the main advantages of audio feedback is that it is raw and unedited. When my collaborator says a playtest went a certain way, whether good, bad or wonky, I can hear it for myself. Even though I trust my collaborator, it allows us to remove our own biases when evaluating feedback. I can hear the feedback directly from the players themselves. Additionally, it allows us to review the feedback on our ways to work when we otherwise may not be able to read emails or feedback forms.

DO’S

Mention that you will record the feedback and acquire consent.

This sounds odd, but if you use audio feedback, make sure to make asking permission standard practice. First and foremost, the laws about audio recording vary, and it’s best to be open and explicit. Second, it allows players to withhold their feedback until you get your recording up and going.

Identify the players, score, and overview of the game.

It may seem obvious, but it’s extremely useful. Before players give their feedback, state the names of the players and whether or not they have played before. This is more personable. Also, you should give a quick overview of the game. For example, “Jay played Sparta and won with 8 points using a military strategy, Christine played as Athens and scored 0 points focusing on a science strategy”. These items together will create better context for the comments to come.

Start and guide the conversation.

It is important to have good questions that start the conversation. Our two general questions are “What is one thing you liked?” and “What is one thing you did not like?” Even if the test went smoothly, players will answer the question by mentioning something they did not like.

Allow a conversation.

One of the additional benefits of using an audio recording over feedback forms is that it allows the players to converse together as a group. There are many ideas we have utilized based upon the combined feedback of players that most likely would never have materialized without a dynamic conversation. This is powerful. It allows players to contribute to something larger than a singular feedback form.

In this regard, it is important to encourage shy players to give their feedback. Not uncommonly, a hobby gamer can dominate the conversation over general players.

Ask the specific questions you want to ask.

One of the shortcomings of a form is that it may not hammer down to the question that you really want answered. By having a genuine conversation, you can ask the freeform questions that you want to focus on. Especially for new prototypes, some players will focus their feedback on one minor overpowered ability. In this case, you can ask “What did you think of the overall game engine? Did it make sense, was it easy to learn?” The players will let you know what they found cumbersome or confusing. In many cases, it is not something that you missed upon just observing.

Move the device around.

Smart devices are great. However, they will not compensate for a loud noisy game room. Once you start moving the device around, the players will start doing it for you.

For everyone’s sake, repeat/restate questions without pronouns.

As one can imagine, it is easy for a player to say “I liked this ability” without ever mentioning explicitly what that power is. If your collaborator were to listen later, he or she would have no idea what the reference is.

For example, if a player said “I like this ability,” I could ask “what did you like about the Cleric role?” This way, it is clear to everyone, including the players as well as my future self, what the topic is. Indeed, this also helps guide the conversation.

Save it with a meaningful title and upload it right away.

Save the recording immediately; it is easy to accidentally delete. Use a title that is simple yet descriptive.

DON’TS

Don’t defend yourself, your opinions or your game.

The most important thing to do is to make it clear that you want to listen. You will get bad feedback or funny suggestions, or even suggestions that you had previously implemented. Your best bet is to listen actively. If you start becoming defensive, the players will stop giving honest feedback. Having listened to hundreds of audio files, it is clear that testers give more feedback if you let them. Conversely, you can tell that when my collaborator or I am defensive, players give far less feedback. You want to make it known that you are listening as actively as possible. If another player asks you a question, try to push your answer off until other players have had a chance to chime in. Occasionally, this rule needs to be broken when the discussion turns; for example, if everyone is wondering what the previous versions were.

And there you have it… advice on audio feedback from a designer with a lot of experience using the format! I hope you found it as helpful as I did. In fact, I feel like much of this advice is useful when it comes to other forms of feedback as well.

What about you? Do you have any other suggestions about audio feedback? How does taking audio feedback differ from other formats, be they forms, free form responses, or the designer taking notes?