4 Researchers
Researcher
March - April 2022
Illustrator
Photoshop
Miro
Research
As part of a class project, my team and I conducted a usability testing on Pokémon Go, a mobile app that allows players to catch various Pokémon based on their GPS location. Through this project, we learned the importance of usability testing in improving user's overall interaction and experience with any products or services.
During the usability testing, our team focused on delivering the following:
Pokémon Go gain its popularity in 2016 but began to decline in 2021. As a result, our team wanted to find out if there was any usability issues that contributed to its downfall. For my role, I was responsible for conducting research and usability testing through usertesting.com as well as meeting deadlines. money.
Pokémon Go gain its popularity in 2016 but began to decline in 2021. As a result, our team wanted to find out if there was any usability issues that contributed to its downfall. For my role, I was responsible for conducting research and usability testing through usertesting.com as well as meeting deadlines. money.
To prepare for the usability testing phase, our team formulated a comprehensive proposal that entailed identifying and analyzing potential issues with the Pokémon Go app. The proposal served as a foundation to outline recommended solutions and set guidelines for the usability testing process.
In preparation for conducting usability tests on Pokémon Go, we developed a comprehensive UX proposal that established clear guidelines for user interactions with the game. Our primary goal was to analyze and categorize potential usability and design issues, with a focus on identifying common user errors and understanding the underlying reasons behind them. Through our research, we sought to gain a deeper understanding of our users' goals and needs in relation to the game, and to identify areas where the user experience could be improved.
By taking a systematic approach to our study, we may be able to uncover valuable insights into how users interact with Pokémon Go, and to develop actionable recommendations for improving the overall user experience.
Our findings not only will help us to identify areas for improvement, but also provided a framework for ongoing user research and development of the game. Ultimately, our UX proposal and subsequent usability testing may help to ensure that Pokémon Go was a user-friendly and enjoyable experience for all players.
In order to develop a comprehensive proposal, our team conducted a thorough review of the Pokémon Go app, which helped us identify a potential problem area. Based on our analysis, we crafted a clear problem statement that will serve as a valuable guide as we move forward with usability testing:
By articulating our problem statement upfront, we are better able to focus our efforts and ensure that our usability testing is targeted and effective. Our ultimate aim is to help improve the user experience of Pokémon Go, and to create a game that is enjoyable, user-friendly, and engaging for players.
The Pokémon Go app lacks a cohesive onboarding experience and clear game instructions for new users. There is almost no assistance in learning about the multitude of features in Pokémon Go—including but not limited to how to catch a Pokémon and spin a PokéStop. At the moment, there are too many features in the app that users do not know exist or where they are.
Once we had identified the key problem areas and developed a UX proposal, we proceeded to create a detailed usability test plan. Our objective in conducting these tests are:
To achieve these goals, we carefully selected research methods and tasks that would enable us to gather the data we needed.
It was crucial to determine the specific aspects that needed testing to ensure a focused and streamlined process. To achieve this, we developed a comprehensive plan for our usability tests that served as a guide to help us stay on track and conduct efficient testing
Our test plan included the selection of the following research methods: Heuristic evaluation, PURE Research Method, and Card Sorting. Each research method played a vital role in our usability testing, and I will elaborate on their individual importance and contribution to our findings.
To begin our usability testing, we will conduct the heuristic evaluation and PURE research method, leveraging the expertise of our UX team. By starting with these methods, we can gather valuable insights that will inform the testing scenarios and questions we present to our users, ensuring that we focus on the most relevant and impactful aspects of the user experience.
A heuristic evaluation involves using a set of guidelines to review a system and identify any usability issues that could impact the user experience. One commonly used set of guidelines is Nielsen's 10 Usability Heuristics, which includes the following principles:
In addition to the heuristic evaluation, we will also conduct the PURE (Practical/Pragmatic Usability Rating by Experts) research method to assess the difficulty of different tasks. Our team is uniquely qualified to conduct this assessment, as we have experts in usability as well as Pokémon players with varying levels of familiarity with Pokémon Go. By combining our expertise, we can conduct a comprehensive evaluation of Pokémon Go based on a wide range of criteria.
By articulating our problem statement upfront, we are better able to focus our efforts and ensure that our usability testing is targeted and effective. Our ultimate aim is to help improve the user experience of Pokémon Go, and to create a game that is enjoyable, user-friendly, and engaging for players.
Our team took a step-by-step approach to break down each task into smaller steps, and we assigned a color-coded rating between 1-3 to each step based on its level of difficulty. A rating of 1 signifies a low cognitive load, while 2 indicates a moderate level of cognitive load, and 3 represents the most difficult level that requires significant cognitive load, possibly leading to task abandonment or failure.
To demonstrate the tasks we rated, we created a document that will enable us to collect quantitative data to evaluate the overall usability of Pokémon Go. We will utilize this data to generate an overall score and visually represent the app's performance.
To ensure the safety of all participants amidst the COVID-19 pandemic, our usability tests for Niantic Labs' game, Pokémon Go, were conducted remotely. In light of budget constraints, we opted for conducting three tests with three responses each on usertesting.com. Within twelve hours, we received the results for analysis. I will elaborate on each of the usability tests. The tests conducted with our participants are as follows:
To gain insight into how users prefer information to be organized in the game's menu, we conducted a card sorting activity with three participants via optimalworkshop.com. The task involved organizing 19 game features into three columns, representing the Battle, Profile, and Research Fields sections of the menu.
During the card sort, we asked participants to speak their thoughts aloud using the think aloud protocol, which helped us gather qualitative information. The results of this activity will guide us in determining the most intuitive and user-friendly organization of game features.
We designed our second test specifically for new users of Pokémon Go. Our objective was to evaluate the effectiveness of the onboarding process in teaching users how to play the game. We created a series of tasks that focused on completing the tutorial, and asked participants to perform the following actions:
For our final test, we focused on experienced players and asked them to locate features in the game that we had identified as potentially difficult. By doing so, we aimed to evaluate the effectiveness of the information hierarchy in Pokémon Go. Below are a few examples of the tasks we assigned to our experienced participants:
Our participants utilize the Think Aloud Protocol (TAP) while performing their assigned tasks. This method enables us to gain valuable qualitative data on how users navigate through different task-based scenarios. By vocalizing their thought process as they interact with the game, we can gain insight into how users perceive the user interface, identify any issues or areas of confusion, and make recommendations for improvements.
Follow-up questions were given to our participants after they completed the tests on usertesting.com. Through this approach, we were able to obtain feedback on their experience with Pokémon Go, which helped us identify the game's areas that required improvement.
After conducting usability testing and using the PURE research method, along with a heuristic evaluation, our team was able to identify weaknesses within the menus, hierarchy, and features of Pokémon Go based on the data gathered.
We found that:
Based on the data gathered from our usability testing, heuristic evaluation, and PURE research method, our team will propose several solutions to address the identified weaknesses in Pokémon Go's menus, hierarchy, and features
During the usability testing, participants pointed out that the onboarding experience lacked crucial information. They struggled to understand the timing and purpose of throwing the Pokéball and interpreting the colored rings. Furthermore, many important features of the game were not introduced during the tutorial, resulting in users being unaware of their existence. As a result, we suggest enhancing the onboarding experience to provide users with a comprehensive understanding of the game mechanics, including all of its features, to enable them to maximize their enjoyment of Pokémon Go
During the testing, several participants were unsure about the categories used in the game, as they found them too general and sometimes too similar to each other. Consequently, they struggled to locate certain in-game features. For instance, some participants looked for battle-related functions under the "Profile" category. To address this issue, we suggest improving the category names, adding more categories, or providing additional in-game guidance to clarify the content under each category
During testing, participants expressed frustration with the lack of in-game instructions. Participants reported getting lost or confused while playing, but were unable to find any help within the game. Even experienced players were unaware of certain features, such as the ability to create a battle party before starting a battle. We suggest implementing contextual tips or pop-ups within the game to guide users and draw their attention to overlooked features. This would help to improve the overall user experience and make the game more user-friendly.
During testing, we observed that many users struggled with navigation and understanding certain features in Pokémon Go. However, we found that the in-game help section was difficult to locate and not very useful in remedying their problems. As a result, many participants gave up on testing tasks or skipped them altogether. To address this, we recommend implementing a more robust onboarding experience or adding an easily accessible in-game help feature when they need it.
As someone who doesn't typically play games, working on this project was a new experience for me. I learned how to apply various testing methods to Pokémon Go, which helped me gain a better understanding of the importance of usability testing. The experience was valuable and taught me to appreciate the different types of test plans and their significance in determining the appropriate method to apply in different situations.
Looking back, some of the improvements I would make are:
Let's Connect!
