Throughout the design phase, we sought feedback from users by conducting informal studies with Second Life Residents in-world. During the latter part of the summer, we tested our final design by conducting a focus group and a Think Aloud, both using the interactive prototype.
Focus Group
To ensure that Second Life Residents find our design useful, and to hit any blind spots that we, as designers and engineers, may have had, we conducted a focus group session with the Residents who were interested in helping to improve Second Life's user experience. More than 15 Residents attended the one-hour session.
We received positive feedback in general regarding the concept of the system. However, Residents were extremely concerned about gaming and exploitation of the system by malicious users. Though we have been aware of this concern, the magnitude of concern was greater than expected. We were driven by these concerns to make our design even more robust against gaming. Originally, a Recommendation showed the overall rating by all Residents. We added the ability to see Friends Ratings and Like You Ratings to provide extra touchpoints for the user's decision. The user will find these ratings more reliable than the overall rating because he or she is acquainted with the users who voted.
A desire to meet people of similar interests was also expressed in the focus group. This was also reflected by peers during critiques and users duing concept validations. It is difficult to actively introduce Residents to each other, because without a context, users will feel awkward and have no motivation for initiating conversation. However, with the newly introduced Like You Rating, users can find other users who've enjoyed the same Recommendations and have similar interests. By using their mutual interest in a place or event as a context, users are less awkward initiating conversations.
Think Aloud
While the focus group revealed broad themes we need to consider, we were also interested in the appropriateness of each specific interface within the design. We conducted 10 Think Alouds by users to perfom 5 main tasks (12 total steps) on the prototype. Four participants were active Second Life users (3 experts and 1 novice), two participants have used Second Life in the past, and four participants have never used Second Life. Because the focus of the system is to enhance the new user's experience within Second Life, the feedback from the people who have never used Second Life was just as valuable as people who use Second Life daily.
The Old List View (top) and the New List View (bottom)
Users encountered several problems with the List View. The navigation bar was not visible at all, and users did not understand the need or function of centering a Recommendation entry. To alleviate this, the List View was redesigned so that the navigation bar and options buttons are more visible. The difference between a centered recommendation and an uncentered recommendation was increased, and the centered Recommendation was more closely associated with the details button on the navigation bar.
Hover state for Total Rating
Users also expected to be able to interact with the Total Rating star icon, while in actuality it was not interactive. We decided that users should have access to the exact rating distribution, so hovering the mouse over any star icon would display the exact number of thumb ups and thumb downs for that rating. Other than the List View and the interactivity of the star, users were able to grasp and use the system to find a recommendation in which they were interested.