My name is Philip Walmsley, and I am a Senior Visual Designer on the Firefox UX team. I am also one of the people tasked with making addons.mozilla.org (or, “AMO”) a great place to list and find Firefox extensions and themes.
There are a lot of changes happening in the Firefox and Add-ons ecosystem this year (Quantum, Photon, Web Extensions, etc.), and one of them is a visual and functional redesign of AMO. This has been a long time coming! The internet has progressed in leaps and bounds since our little site was launched many years ago, and it’s time to give it some love. We’ve currently got a top-to-bottom redesign in the works, with the goal of making add-ons more accessible to more users.
I’m here to talk with you about one part of the add-ons experience: ratings and reviews. We have found a few issues with our existing approach:
Some users just want to leave a rating and not write a review. Sometimes this is referred to as “blank page syndrome,” sometimes a user is just in a time-crunch, sometimes a user might have accessibility issues. Forcing users to do both leads to glib, unhelpful, and vague reviews.
On that note, what if there was a better way to get reviews from users that may not speak your native tongue? What if instead of writing a review, a user had the option to select tags or qualities describing their experience with an add-on? This would greatly benefit devs (‘80% of the global community think my extension is “Easy to use”!’) and other users (‘80% of the global community believe this extension is “Easy to use”!’).
We don’t do a very good job of triaging users actual issues: A user might love an extension but have an (unbeknownst to them) easily-solved technical problem. Instead of leaving a negative 1-star review for this extension that keeps acting weird, can we guide that user to the developer or Mozilla support?
We also don’t do a great job of facilitating developer/user communication within AMO. Wouldn’t it be great if you could rectify a user’s issue from within the reviews section on your extension page, changing a negative rating to a positive one?
So, as you can see, we’ve got quite a few issues here. So let’s simplify and tackle these one-by-one: Experience, Tags, Triage.
The star rating has its place. It is very useful in systems where the rating you leave is relevant to you and you alone. Your music library, for example: you know why you rate one song two stars and another at four. It is a very personal but very arbitrary way of rating something. Unfortunately, this rating system doesn’t scale well when more than one person is reviewing the same thing: If I love something but rate it two stars because it lacks a particular feature, what does that mean to other users or the overall aggregated rating? It drags down the review of a great add-on, and as other users scan reviews and see 2-stars, they might leave and try to find something else. Not great.
What if instead of stars, we used emotions?
Some of you might have seen these in airports or restrooms. It is a straightforward and fast way for a group of people to indicate “Yep, this restroom is sparkling and well-stocked, great experience.” Or “Someone needs to get in here with a mop, PRONTO.” It changes throughout the day, and an attendant can address issues as they arise. Or, through regular maintenance, they can achieve a happy face rating all day.
What if we applied this method to add-ons? What if the first thing we asked a user once they had used an extension for a day or so was: “How are you enjoying this extension?” and presented them with three faces: Grinning, Meh, and Sad. At a very high level, this gives users and developers a clear, overall impression of how people feel about using this add-on (“90% grinning face for this extension? People must like it, let’s give it a try.”).
So! A user has contributed some useful rating data, which is awesome. At this point, they can leave the flow and continue on their merry way, or we can prompt them to quickly leave a few more bits of even MORE useful review data…
Writing a review is hard. Let me rephrase that: Writing a good review is hard. It’s easy to fire off something saying “This add-on is just ok.” It’s hard to write a review explaining in detail why the add-on is “just ok.” Some (read: most) users don’t want to write a detailed review, for many reasons: time, interest, accessibility, etc. What if we provided a way for these users to give feedback in a quick and straightforward way? What if, instead of staring down a blank text field, we displayed a series of tags or descriptors based on the emotion rating the user just gave?
For example, I just clicked a smiling face to review an extension I’m enjoying. Right after that, a grid of tags with associated icons pops up. Words like “fast”, “stable”, “easy to use”, well-designed”, fun”, etc. I liked the speed of this extension, so I click “fast” and “stable” and submit my options. And success: I have submitted two more pieces of data that are useful to devs and users. Developers can find out what users like about their add-on, and users can see what other users are thinking before committing to downloading. We can pop up different tags based on the emotion selected: if a user taps Meh or Sad, we can pop up tags to find out why the user selected that initially. The result is actionable review data that can is translated across all languages spoken by our users! Pretty cool.
Finally, we reach triage. Once a user submits tag review data, we can present them with a few more options. If a user is happy with this extension and wants to contribute even more, we can present them with an opportunity to write a review, or share it with friends, or contact the developer personally to give them kudos. If a user selected Meh, we could suggest reading some developer-provided documentation, contacting support, or writing a review. If the user selected Sad, we’d show them developer or Mozilla support, extension documentation, file a bug/issue, or write a review. That way we can make sure a user gets the help they need, and we can avoid unnecessary poor reviews. All of these options will also be available on the add-on page as well, so a user always has access to these different actions. If a user leaves a review expressing frustration with an add-on, devs will be able to reply to the review in-line, so other users can see issues being addressed. Once a dev has responded, we will ask the user if this has solved their problem and if they’d like to update their review.
We’ve covered a lot here! Keep in mind that this is still in the early proposal stage and things will change. And that’s good; we want to change this for the better. Is there anything we’ve missed? Other ideas? What’s good about our current rating and review flow? What’s bad? We’d love constructive feedback from AMO users, extension developers, and theme artists.
Please visit this Discourse post to continue the discussion, and thanks for reading!
Philip (@pwalm) Senior Visual Designer, Firefox UX
Rollie started rapping many years ago, and since the beginning, Dollie was with him. In fact, when Rollie was born, Dollie was already a puppy. Dollie doesn’t talk, but with her expressions she can support Rollie before his concerts. Rollie already recorded many songs and videos, in which Dollie always appear. Dollie is also a theme in many of Rollie’s songs. She is the only dog Rollie ever had and besides of being close like siblings, Dollie is an inspiration for Rollie. Rollie wrote songs in which he talks of Dollie as his sister and he is going to sing it in his next concert. Now Rollie and Dollie doesn’t know if it’s a good idea to give milk to their dog.
Right now, Project Prox is taking shape as a mobile-first application on iOS. Starting with your location, we look for interesting places and events near you. That’s where our focus is at the moment — the here and now.
Yes, there are also other parts to this story too. But we’ve decided that focusing on this part of the “Traveller’s” journey right now makes the most sense. This was also something we touched on during our Design Sprint. If you’d like to know more (and haven’t already), you can read more about that process here.
But I don’t have an iPhone… why is it only on iOS?
Believe it or not, “only make this for iOS” wasn’t one of our goals. But with all our constraints, we felt that the iOS platform was the best place to start. We definitely want to expand to other platforms in the future. This came down to a matter of resourcing, time constraints, and the usual like.
It’s important to call out our goal here: to solve real user/people problems. We want to understand the problem space and provide user value through iterative research and design.
Is this something Mozilla should be doing?
When you think about Mozilla, it’s pretty much synonymous with Firefox. But surely that’s not the only way we can carry out our mission, is it?
Our mission is to ensure the Internet is a global public resource, open and accessible to all.
How do we define “the Internet”? It takes so many different forms nowadays. We might know what it looks like today, but what about the future? How will technology influence the way we interact with “the Internet” then?
Not too long ago, Mobile was the future. Now, the web platform itself is growing. Chrome’s Accelerated Mobile Pages, and Progressive Web Apps give us the ability to create new, better experiences for the mobile Web.
If we want to ensure that the mobile Web is also “open and accessible to all”, we need to grow the influence of the mobile Web too. We need to be open to new technology and consider all the different ways people are (and will be) accessing this information.
Ok, so what does Project Prox look like?
As soon as you open the app, we want to provide immediate value. Our goal is to enable easier decisions about what to do or where to go next. So, without even knowing what to start searching for, we want to leave you with a satisfying day.
This should work regardless of whom you’re connected to, because your social network shouldn’t be a prerequisite for getting the most from the web.
For those looking for a less immediate perspective, we also included a “Map View”. This was actually not something we tested in our Sprint. But the intention here was to offer an alternative view for users to give them more context about their surroundings. The goal here was to give users a better sense of spatial awareness.
Although we never finished it in time for Kona, we still got a lot of helpful feedback that is going to help us iterate and test this more for the next version.
Last but not least, notifications! If you’re nearby an event, we’ll notify you so that you can decide if you’re interested or not. The nuances of notifications makes this helluva lot trickier. But more than anything, we want to be useful without being annoying.
Keep in mind, all of this is still a work-in-progress. We’re going to continue iterating with the help of our User Research team until we have something we’re happy with.
All in all, it’s just too early to tell. Kona was our first announcement but we’ve yet to actually launch the product. But we do have a clearer picture of what success might look like for us.
We’re calling this a “V1” — so there’s lots of work to be done still. We’re going to address some of the feedback and bring the product to a level we’re comfortable with calling an MVP before releasing to a larger population. And obviously, it works better in some cities versus others.
We need to get more data and gather more feedback. Improving the experience and releasing the updated product in a larger, denser city will be our next challenge. What we learn there will help us determine the appropriate next steps.
If you’re interested in giving feedback, please reach out!
A recently landed feature in the experimental Firefox Nightly build integrates proactive tracking protection for Firefox Nightly users coupled with Do Not Track (DNT). With tracking protection active, ads and tracking technology are blocked before they reach the user’s browser. Subsequently, tracking ads and tracking pixels are not loaded by the browser and are not visible to the user. Further, by not loading the ads, tracking protection speeds up page load times.
In January 2015, the Firefox user research team conducted a three-day diary study with twelve demographically-diverse Firefox users in the US to understand their experience with Firefox Nightly’s implementation of Polaris tracking protection. For the diary study, participants were asked to download and install Firefox Nightly. Participants were instructed to turn on Tracking Protection in Firefox Nightly use it as their primary browser for the duration of the study. Participants responded to cues and direct questions while keeping notes of their daily online activities. After the diary study was completed, we conducted hour-long follow-up interviews with four of the participants via Vidyo.
We wanted to answer the following questions:
What does “tracking protection” mean to participants? How do they define it? How do they believe it works?
How does the implementation of tracking protection in Firefox Nightly match participants’ mental models of what tracker blocking means?
Does tracking protection disrupt participants’ current experience of browsing the web?
Findings and Observations
Definitions of Tracking and Tracking Protection Most participants understood that tracking involved a third party using a tool or technology to identify unique browser users and to collect data about those unique users’ activity online. Participants provided different descriptions of how they believe tracking works and what information being tracked. Most participants used more general vocabulary to describe the more tangible aspects of tracking, such as tracking technologies track “my likes based on my website visits” (P15) or “what I might be interested in.” (P10) A small number of participants described the technical details of how online tracking works with a high level of accuracy.
Who Do Participants Believe is Tracking Them? Overall, the majority of participants were able to describe at least one or many employers of tracking technology. While most participants did not describe the ecosystems of tracking and the relationships among companies involved in tracking, they did mention the piece of tracking technology that is most visible to them: targeted ads.
We also asked all the participants: “who do you believe is tracking you online?” Most participants mentioned Google, Facebook, or “marketing companies.” Two participants said the US government was engaged in tracking online activities
Benefits about Tracking Protection Feeling more secure and private. The most commonly cited benefit among participants was the feeling of greater security and greater privacy. Although some participants who said this did not articulate specifically how tracking protection worked or made them feel more private and secure. Some participants clearly believe the tracking protection feature’s labeling and/or its visible effects (e.g., blocking the display of some ads).
Fewer ads, less clutter. Among the participants who noted the absence of targeted ads with tracking protection on generally viewed the absence of targeted ads as positive benefit. Most of these participants viewed ads as “clutter” (P2) on a site or as a “distraction” (P4) from their intended browsing activities. A few of these participants said that they have sometimes found targeted ads helpful, but also said they did not miss the ads when they were absent.
Faster page loading. Some participants believed tracking protection speeded up their browsing experience by blocking tracking technologies that took extra time to load on a page. This extra speed was seen as a benefit. In our follow-up interviews, two participants believed that tracking protection slowed down their browsing because “the browser has to do extra work.” (P2). The other two participants described how tracking protection improved the speed of page loading.
Brings attention to the issue of tracking ads. A few participants noted how their relationship to and understanding of tracking changed over the course of the study. As P2 wrote, “It was very thought provoking because even though I did not use the internet much on my phone or computer I started to think about what a difference tracking can make…”
Relevant for mobile. Further, when asked about tracking protection on mobile, many participants said they hadn’t considered online tracking on mobile before. As P10 wrote, “None of my other devices give me the option of turning tracking on and off. And Firefox Nightly has definitely changed my perception of tracking because being that it gives me the option to turn it on and off it also makes me curious what the other browsers are using…It’s led me to various questions.”
Frustrations with Tracking Protection Confusion when targeted ads disappeared. Some participants found the blank areas of missing content where targeted ads were blocked as visually jarring and confusing. As P8 said, “I did notice unusual behavior with page loading or missing content. My reaction to the missing content were usually were Ads would appear…I eventually closed the page because it was odd.” Eventually, participants who mentioned the blank content grew to like having targeted ads blocked.
Options/Preferences Labels unclear. Some participants did not understand the functionality of the opt-in selections in the Options/Preferences menu. The two options sound very similar: “Do Not Track” is a technical framework to send a signal to ad servers requesting to opt out of tracking and “Prevent sites from tracking me” refers to the Firefox Nightly tracking protection feature. For these users, these two selections are a distinction without a difference.
Limited discoverability of interface and alerts. Participants were only alerted to the presence of the shield icon in day two of the study. Only one participant (P15) said he noticed the icon before it was pointed out in the instructions. Once they discovered it, participants did express an accurate understanding of the shield’s functionality to turn tracking protection on and off for the site.
Perceived video playback issues. Video playback, page loading issues, and crashing were issues participants attributed to tracking protection. In our subsequent tests, we demonstrated it is unlikely that tracking protection was the actual cause of instability or video playback issues. Regardless, likely because their attention was focused on the tracking protection feature as the subject of the study, tracking protection was perceived by some participants as the cause of these problems.
A feature for more than a privacy-centric niche. We sometimes think that privacy protection tools are valuable to only a small portion of privacy-centric users (3–8% depending on the study) or general users at specific moments (for example, using the private browsing window). It is noteworthy how many of the participants found value in tracking protection.
Most participants in this study were not already focused on tracking and privacy-related issues. Tracking protection in Firefox Nightly exposed them to a tool that offered them a good deal more privacy than they were accustomed. Some participants responded positively and wanted to continue to have tracking protection as an option. A few other participants said they would like to use tracking protection after they perceived it becoming more stable. As P2 said, “I do plan to use tracking protection beyond this study because I liked the way things loaded better and I feel safer for some reason. I will not seek out additional tools because I don’t think I use the internet enough that I would need something like that.”
Invisible and visible. Participants focused almost exclusively on the visible manifestations of privacy such as targeted ads. During the follow-up interviews, we asked participants about the aspects of trackers that they could not see. P4 and P12 described ad networks and Google but did not fully convey the complex web of relationships among advertisers and tracking technology. Instead, participants only described the targeted ads they saw or did not see with tracking protection active. This more limited understanding makes a good deal of sense given the abstract nature of privacy concerns and the behind-the-scenes nature of tracking tools.
In order to improve the user experience of the tracking protection feature, we recommended at the time a few changes to the design:
Better labeling in the interface, using more common language to distinguish between “Do Not Track,” “Tracking,” and “Privacy.”
Show the invisible aspects of tracking to users to demonstrate visually (via a counter and a list of blocked trackers) to users what is being block on the sites they visit. Making the invisible elements more visible would allow us to show more fully the impact of tracking protection. As some participants indicated in their diary study, these details are valuable to them and enlightening about tracking in general.
Many participants did not mind targeted ads disappearing, but some did find the missing ads affected the design of sites they visited. Whether users support ads or not, many sites are designed with space for ads. In order to retain the intended design of a site, we recommended filing in these empty spaces with wireframes of the ads being removed. Wireframes would also benefit new tracking protection users who were uncomfortable with sites appearing empty. [This proposal may be technically challenging to implement with accuracy.]
Tracking and privacy are complex and sometimes abstract issues. Introducing the tool with on-boarding and a visual tutorial is essential for adoption and education.
The Firefox User Research team will now be posting results from more of our smaller research studies. Our aim is to share our results and provide some insight into our research and product development process.
Imagine you just came home with a framed picture. You need to find a nearby hardware store so you can buy nails to hang your picture up on the wall. You open your web browser and navigate to a well-known hardware chain’s web site to find a nearby location. When you find the page with store locations, the hardware chains’ web site asks you to share your current location. Maybe you’re in a hurry…sharing your location in this case means you don’t have to type in your current address.
With Firefox, we believe that personal information should never automatically be handed over to a website without the user’s permission. After all, a web browser should be more than an application that blindly accesses web sites. Firefox mediates between you and those web sites to protect your privacy and security. When we talk about this role at Mozilla, we talk about how Firefox is the user’s agent on the web. Among our goals for Firefox is the mission to protect our users from privacy and security risks and to help them make more informed decisions.
A Better Privacy Permissions Panel
From previous internal research, the Firefox product team was aware that Firefox’s current privacy panel UI is ambiguous to users. For a geolocation request in Firefox’s current implementation, users are presented with a drop-down box displaying a catalog of options:
Share location with the current site (just this once)
To always share their location with this specific site,
Defer sharing their location
Never share their location
Further confounding the decision process for users is the “X” widget in the upper corner of the panel that dismisses the dialogue. What does dismissing the dialogue mean? How do users interpret the “X”? Do they believe that “X” means “don’t allow” location sharing ? Or do users believe they are deferring the choice and simply dismissing the dialogue to move on with their current task?
The Firefox privacy team wanted to improve the labeling and also the interactions in order to help Firefox users make a more informed decision. The team iterated on several designs that attempted to resolve these questions. The goals of the redesign were:
Communicate more clearly to users the choice they being asked to make
Capture the expression of user intent with more certainty
Present actions with more understandable outcomes
The team settled on a simpler design with better labeling and fewer interactions. However, questions remained about the possible usefulness of the “X” for users. What if users did not want to make a choice at that moment? What if they just wanted to make the dialogue go away and move on with their browsing? Would users be annoyed at being forced to make a choice? The team was unsure.
To evaluate the new design, the Firefox UX research team conducted two unmoderated, remote studies with 25 participants in the US, Canada, and the UK who selected Firefox as their primary web browser. Sessions were run between September 23, 2016 and November 1, 2016. For these initial studies, we focused on Firefox Desktop, not Firefox for Android or iOS. After downloading a special build of Firefox Nightly with the new privacy panel designs, participants followed a talk-aloud protocol to complete a series of tasks that triggered different privacy permissions panels. Participants’ test sessions were recorded and reviewed.
In each study, we divided participants into two groups. One group of participants used a special build of Firefox Nightly in which the new privacy permissions panel had the “X” widget; the other group had a version without the “X” widget.
In the first study, participants were told to use Firefox Nightly to complete three primary tasks:
Navigate to Starbucks.com and find a store nearest them (triggering panel requesting geolocation).
Revisit the Starbucks.com site to observe participants’ reactions to being prompted again or not (depending on their selection of “Remember my decision”).
Navigate to talky.io and follow the instructions to start a web-based video chat with an unspecified friend.
Here’s what we learned:
All participants appeared to understand the basic request presented to them in the permissions panel: sharing their current location with a site and allowing access to their computer’s webcam and microphone for a web chat site.
“Allow” means allow the site access to the requested resource or information. “Don’t allow” means deny access.
None of the participants appeared to be visibly or audibly frustrated or annoyed by being forced to make a choice between “Allow…” and “Don’t Allow.”
Among the group of participants with the option to dismiss the permissions panel using the “X” only one did. When asked about her the meaning of clicking the dismissal widget, she stated that she believed clicking “X” was the same as “no.”
Only one participant selected the “Remember this decision” checkbox. It’s unclear if participants saw the box or not. However, several participants expected the browser to not prompt them again. As Participant WxP3 said on being shown the privacy panel on the second visit to Starbucks.com, “I did not expect to see [the privacy panel again] again. I expected it to remember my location. I guess it doesn’t.”
We wanted to test the new geolocation privacy panel, but with an additional layer of complexity. Unlike the Starbucks geolocation prompt which has an obvious transactional purpose (it’s on the Find a Store page only), how do participants react to the geolocation prompt when the purpose isn’t obvious?
Again, participants were divided into two groups with a special version of Firefox Nightly: one group with the “X” widget and one without the “X” widget.
In the second study, participants were told to use Firefox Nightly to complete three primary tasks:
Navigate to T-mobile.com. A panel is triggered on loading the site, but on the T-mobile.com page, the transactional meaning is less clear.
Navigate to Starbucks.com and find a store nearest them (triggering panel requesting geolocation).
We asked participants to locate their current settings for their privacy permissions.
Here’s what we learned:
We observed similar results with regard to participants’ understanding of the permissions panel: all participants understood the general request.
More participants in the “X” group selected “X” this time and all but one of them had the same interpretation of “X”: “No, I don’t want [the site] to access my location.” (Participant WxP7)
One participant clicked “X” and interpreted “X” as both minimize and as “No.” Participant WxP3 said, “I closed it down…I really just minimized it. There was a pop up that was in my way and it wanted to share my location and I either said “no” or just minimized it. I just kind of got it out of my way.” For this participant, making the dialogue go away and not wanting to share their location are the same thing.
The context of the request for geolocation is important in users’ decisions. Several participants who selected “Don’t Allow” for T-Mobile said they did not know why T-Mobile was asking for their geolocation. As Participants WOxP6 said, “I felt like T-mobile didn’t need to know my location because I was just looking for a phone.”
Very few participants were able to relocate the status of their decision about sharing geolocation in Firefox. (TIP: It’s currently located in the address bar when you visit the specific domain.)
In spite of making a quick decision, participants do understand in general terms what is being asked of them. A simple “Allow” or “Don’t Allow” interface appears to be adequate for almost all of our participants to make a clear decision.
Further, the “X” widget is redundant because for all participants it carried the meaning of “Don’t Allow sharing.” In expressing her desire to make the dialogue go away and say “no,” one participant provided an additional layer of meaning. Based on this observation and the lack of frustration expressed by participants, we feel confident that the “X” dismissal widget can be removed because fewer possible interactions make for an easier user experience.
In the decision process, the context about the relevance of geolocation provided by a website appears to be very important. Based on participants’ answers to follow-up questions, it’s clear why Starbucks is asking them for their location. Participants may not want to share their location with Starbucks for privacy reasons (in fact a small number of participants said regardless of value they never shared their geolocation for this reason), but the request is clear to them. By contrast, it’s not clear from the testimony of participants why T-Mobile is asking for their geolocation data.
Technologies like geolocation in the browser can be potentially useful to both sites and users. But these tools can be used as gimmicks or in a way that does not provide visible user value. Web sites and developers need to situate why they are asking for users’ location in a meaningful context. When websites aren’t providing context for requesting this information, they are adding an extraneous task for users to perform that increases frustration and potentially eroding trust in the web site’s purpose and intent.
For Further Study
Which UI elements do participants see? Why did so few participants notice or select the “Remember my decision” checkbox? How can we improve users’ understanding of how to make persistent decisions?
Since most participants could not locate with the permissions management interface, how can we improve it? Many participants chose the Options (on Windows) or Preferences (on OS X) to locate permissions management.
How can we better support users like the small number of participants in our two studies who never wish to share their location?
Revisit privacy permissions on Firefox for Android and iOS. Is the experience or choice different on a mobile browser? How can we improve the experience there?
Our new Context Graph effort is steering us into some new and uncharted territory. But I can’t help but feel a sense of adventure because this opens up so many opportunities for new products and experiences — exciting times!
I spent the early part of my career working with a bunch of startups. “Move Fast and Break Things” was often the motto. Sometimes it was the right approach. Hammer away, and don’t be afraid to pivot later on. But stopping to question what or whywe’re hammering is becoming more and more important.
Our team was tasked to create a New Mobile Experience. Here’s how we’ve been approaching it.
The Design Sprint
We started by reading The Sprint Book written by the folks at Google Ventures. “Sprint” writes about a five-day process for iterating on ideas and ultimately shipping products that solve real user problems. Additionally, the book has some great examples of other teams that have followed the same process.
To help us figure out what we should start hammering on, we decided to run our own sprint.
Since working remotely is fairly common at Mozilla, the first thing we wanted to do was get together in person. A prerequisite to running a sprint, but also just very helpful things to do for a new team. As luck would have it though, scheduling proved to be our first issue. Not a great start.
But there’s still a lot we needed to do before meeting. Our two main goals right now are:
Familiarize ourselves with the Sprint process
Share an understanding of what problems we’d like to solve
Here’s where things get more interesting. Before meeting, we decided to try and run a modified sprint. Specifically, to running one remotely. Scary, I know. But the hope is that it will help us achieve the two main goals I mentioned earlier — to work together, and to learn.
Gemma, our Lead User Researcher, headed up this effort for us and designed with our first remote sprint. She also played the role of facilitator. You can read more about the process in her upcoming blog post (link to be posted soon!) but I’ll give a quick recap below.
At a high level, they’re two weeks long, requires a lot of video meetings (everyday to be exact), and involves a large amount of trust. Most days break down to a team working component followed by an individual homework assignment (to be completed before the following day).
Day 1: Team — Select a high-level topic, Individual — Research and map out the problem space
Day 2: Team — Share research, choose a topic of focus, Individual — Sketch out ideas for solutions
Day 3: Individual — Cont’d sketching ideas for solutions (Crazy 8s style)
Day 4: Team — Quickly present own sketches, vote on the best ideas
Day 5: Team — Turn the “best idea” into a testable hypothesis, Smaller Team — Begin drafting a storyboard
Day 6: Team — Share storyboard draft, give feedback, Smaller Team — Address feedback, or start on a prototype
Day 7: Team — Share the progress on the prototype, Smaller Team — Create, pilot and launch a user test
Day 8: Team — Watch user testing videos, capture learnings, Individuals — Brainstorm next steps
Day 9: Team — Evaluate the process, iterate on the prototype
Day 10: Team — Postmortem/follow up sprints
How Might We
The second goal here is to have a shared understanding of the goals and the problem spaces. It’s important for us to be research and data driven here so we created a template to help us out.
Since we’re distributed, it’s really helpful to have these posted somewhere. The team can refer back to them at any time, and they can be used to settle tie breakers. It provides a “North star” that everyone can always look to.
Adapting this from the Activity Stream Team and The Sprint Book, here’s how we’re phrasing our ideas in the form of a hypothesis statement:
We know this is a problem worth solving because (user research/background research/other sources, attach if possible) We believe that (doing this/building this feature/creating this experience) for (these people) will achieve (this outcome). We will know this is true when we see (this user feedback/data). The primary thing we want to be able to answer after user testing is if users (used/understood/adopted, etc).
Gemma, our Lead User Researcher, will be publishing a post on designing and running remote sprints.
Ricardo, our Senior Interaction Designer, just facilitated our second remote sprint and has already been sharing his thoughts as a first time facilitator .
Next week, we’ll be meeting in person for a full Sprint Week.
This process has already inspired two new experiments we’ve started hacking on. The first, a clipboard application to help users gather links for later. The second, a notification manager to give users more control over disruptive notifications.
The way teams are distributed across timezones at Mozilla, I think it will always be helpful to find new ways of working. Perhaps some of our learnings here will be helpful for other teams too. We’ll continue to iterate on this process and look for ways to improve it. Each sprint, better than the last.
What We Learned from the Government Digital Service Design Team
Toward the end of Mozilla’s All-Hands Meeting in London last week, about a dozen members of the Firefox UX team paid a visit to the main office of the Government Digital Service (GDS). GDS is a department within the UK government whose mission is to “help government make digital services and information simpler, clearer and faster” and to “put users’ needs before the needs of government.” Over the course of an afternoon, we got to sit in on a GDS design team meeting, tour their office, and share some of Mozilla’s recent user research.
Escalating enroute to a visit with @gdsteam UX! #mozlondonpic.twitter.com/j3xcHac8xT
What we realized soon after arriving at GDS is that we have a lot in common with them. They, too, are a mission-driven organization. Moreover, they model many of the values and practices at the center of our work at Mozilla. There is much that we can learn from GDS, including:
Define “users” broadly and inclusively
GDS design team members work on a range of high-profile projects, like GOV.UK Verify. Verify is a way for any Internet user accessing a UK government service to confirm their identity online and therefore protect against fraud in transactions with government. GDS design team members are also working on numerous projects where government employees from other departments are the primary end-users. For instance, GDS is helping government employees access wi-fi and update the hardware they need to do their jobs more easily.
The variety of GDS projects is a helpful reminder that, in order to fulfill our mission, Mozilla must continue to think broadly and inclusively about who our users are. It can be tempting to think of our users as mostly people new to Firefox or as superusers of our products, but the group on which the future of Mozilla depends is much more diverse than that.
Create an organization of user research advocates
At GDS, user research is highly collaborative. Researchers are embedded in project teams. GDS non-researchers are active participants in research projects. It was telling that, as we listened to a room full of GDS design team members give quick updates on their work, nearly everyone mentioned some kind of recent user research activity, including journey mapping, lab testing, and research planning. To promote user research, GDS maintains numerous resources, which can be accessed from their User Research blog, including monthly research training days to prepare “research buddies.”
In many organizations, user research (if it exists at all) is collaborative far more in spirit than in practice. GDS’ active dedication to involving designers and non-designers alike in their research is a valuable reminder to continue to explore ways in which we can scale our user research efforts and create an organization of research advocates.
Share your work
Both GDS and Mozilla value openness and transparency in what we do, as the nature of our visit to the GDS office made clear. We also learned that many GDS team members dedicate time to sharing their work, in-person, outside of government, and even outside of the UK. For example, we heard about individuals presenting their work at conferences and other meetups. It was striking to hear these activities described with the same attention and priority as other work, rather than as a footnote or obligation tacked onto one’s busy week.
At Mozilla, because of the volume of projects underway, it can be easy to de-prioritize sharing research practices and findings beyond our community. Our visit to GDS was an inspiring reminder that dissemination is an important means of reflecting and improving upon our work.
Though on the surface GDS and Mozilla are working in different contexts, our two organizations share many values. We are both committed to the overarching goal of digital transformation. We believe in challenging the status quo to make vital resources–government and the internet–more accessible and useful to people. At GDS, we found a peer group as well as an exemplary team who practice everyday putting users first.