Context
I had the neat opportunity to work at Ubisoft in Montréal as a Game User Research analyst. On top of fostering one of the largest research team in the video game industry, this is also the place of birth of the For Honor brand, one of my favorite online video games. It was such a great experience to stand on the creator side of this 4-year old project! While most of my work is under NDA (with some not even released yet at the time of writing !), I can talk broadly about the tasks I undertook there. What was interesting about the job was that it encompassed several aspects and not just one, so I've been part of several sub-projects within the game. I was first onboarded on those and gradually took ownership of them. Here is a part of what I've been doing on the battlegrounds !
Role : Game User Researcher responsible for the For Honor UR effort, worked closely with Vincent Gagnon (Game Intelligence Analyst), Eric Landry (User Research Analyst), Audrey Laurent-André (Associate UX Director) & Stefan Jewinski (Lead Game Designer)
Supervised by Olivier De Maeyer, Kevin Monette & Lanie Dixon (User Research Lab Managers)
Methodology : Surveys, Text Mining, Remote Playtesting, Interviews
Tools : Tableau, Qualtrics, Wordstat, Parsec
Time : 12 months
Testing new hero and content
At every new hero, we want to test if players have difficulty understanding and performing the heroes' moves, in addition to their special attacks. It was crucial as the newest hero I've worked on introduced new mechanics, unseen so far in the game. We needed to assess if those specific mechanics were straightforward when playing AS and AGAINST the new character. It would have been indeed disastrous if playing this character gave you an advantage over the competition!
Using our panel of testers, I planned two testing sessions with eight players in each with the help of other analysts and moderators. The protocol was the same for both sessions :
• First, we went through the tutorial with the player in a think-aloud scenario. I then asked players their first impressions of the hero. In that step, using a think-aloud protocol is the perfect solution to understanding players' intentions in real-time. It is much easier to do so against an AI bot, in a relaxed and controlled environment that is not too costly cognitively speaking. It becomes practically impossible to do so when engaged in a real fight.
• After the tutorial phase, we put the players in 4-vs-4 scenarios to observe their reactions when playing AS and AGAINST the same new hero. We played several rounds, to try to understand which strategies players were using along with the perceived effectiveness/strength of the hero.
• We ended with an interview of their overall thoughts.
Even though we carried the tests remotely, the insights gained were quite helpful. They allowed the team to catch some potential readability issues on several attacks. The team could also notice players developing strategies on the fly with the new hero. As most of the play sessions up to this point have only been done internally, with the people working on the game, such fresh and unbiased feedback was quite welcome.
Unannounced feature testing !
To assess understanding of an unannounced feature (and no, there were no NFT involved!), I've conducted tests on a prototype, as the project was quite early. This proved to be a good point on my end, as we used Adobe XD for doing so, and their export features are so simple to use with remote testing! There were a total of two iterations, with 6 participants in each where I asked them to complete the main user flow and asked them questions along the way.
The first results were very encouraging in the way that users could complete the flow, but we quickly identified the need for shortcuts. Also, it was interesting to observe some confusion on the feature scope, which was even more critical as participants were all part of the Ubisoft workforce. They, therefore, knew the industry better than "regular users". Testing internally on such early stages can be truly time-saving, as such issues can be found and solved more quickly for the following iterations.
The second prototype was then quickly built to try to solve some of these observed problems. And with the other round of 6 new participants, I discovered the initial confusion wasn't present anymore!
The team then felt much more confident regarding the rest of the project and much more convinced of the benefits of early User Research even with prototypes (yay !)
Season monitoring
In conjunction with all the work mentioned before, we also hosted season monitorings to reflect the rhythm of content delivery for the game. Those surveys allowed us to gather the players' opinions and evaluate their perception of the latest seasonal update. We also used those surveys to evaluate other topics we were curious about regarding the player base. As part of the job, I've gradually taken ownership of this recurring project, which led me to finally :
The first results were very encouraging in the way that users could complete the flow, but we quickly identified the need for shortcuts. Also, it was interesting to observe some confusion on the feature scope, which was even more critical as participants were all part of the Ubisoft workforce. They, therefore, knew the industry better than "regular users". Testing internally on such early stages can be truly time-saving, as such issues can be found and solved more quickly for the following iterations.
• prepare surveys and clean data from more than 3000 participants for each survey
• analyze the qualitative feedback using Wordstat and the quantitative data with Tableau, and help of other analysts
• presented the report to the production teams, helping for the next steps of content release
On top of harvesting precious opinions and thoughts about live content, this exercise was also fruitful for other topics. For instance, we assessed the newest changes in narrative direction the team wanted to take or the opinion regarding some in-game payment mechanics to be implemented.
Wrapping up
Spending time working as Game User Researcher on one of the universes I enjoyed the most at Ubisoft was a dream come true. I had the opportunity to see the inside out of the video game industry, right in the heart of Montreal. I also met some of the most passionate people I've ever seen, which was truly inspiring. And to discover so many cool professions! Tackling the user research questions of such a large project gave me strong confidence for the following role I had to take, which I will probably mention here in a bit :)
Want to read more ? Here some research I've done about music 🎵