In February, the Test Pilot team at Mozilla Labs rolled out a test to explore usage of the Firefox menu bar . This menu item usage study aims to help guide the UX team as they create a fully optimized design by answering 3 questions.
* Which menu items are the most commonly used?
* Which menu items are the least commonly used?
* How long do users spend exploring the menu bar contents before selecting each particular menu item?
After we received the raw data, Blake Cutler and Christoper Jung from Mozilla Metrics team did some great work to understand what this data is telling us. Here are a couple of preliminary findings regarding the first 2 of these 3 questions. Take a look!
Initial Findings
The most obvious way to determine the most and least commonly used menu items is to simply aggregate the total number of menu item clicks for all users.
This graph shows just that, presenting each menu item’s relative use for all UI methods (both mouse and keyboard shortcuts). Even from this simple analysis, we can see some justification for a condensed toolbar as many of the items are used very infrequently compared to the other menu items. For example, the menu items from “Page Setup” to “Character Encoding/UTF-16″ each make up less than 0.01% of the total menu bar clicks.
While looking at the total number of item clicks can be informative, since menu bars are designed for mouse use, it is more relevant to look at item usage for just the mouse UI method (excluding keyboard shortcuts).
Examining the data in this way presents a slightly different picture: the top 5 most commonly used menu items are now “User Bookmark Item”, “Copy”, “Paste”, “Add-Ons”, and “Back”. In addition to “Add-Ons”, “Options” and “Bookmark This Page” are newly part of the top 10, replacing “Find”, “Open Location”, and “Find Again”.
Again these changes simply result from eliminating keyboard shortcut clicks and help us distinguish between mouse driven menu items and keyboard driven items. For example, by comparing the mouse UI chart (right) with the original all UI chart (left) we can clearly see that “New Tab” and “Close Tab” are predominately driven by keyboard shortcuts (as expected) and may not be the two most critical items to a mouse oriented toolbar (as suggested by the original chart).
Another interesting approach to these questions is to group the items by menu and visualize the data in this form (again, data is just for Mouse UI).
This visualization presents information on two levels: the area of the circles are proportional to the total number of clicks for the menu group as a whole, and the slices correspond to the share of clicks for each item within the menu group. Bookmarks and Edit are by far the most utilized menus, representing over 70% of total clicks.
The high use of the bookmarks menu is somewhat surprising; an obvious problem of looking at aggregated data like this is the potential for outliers to skew the data. It will be interesting to delve into this issue more in depth and determine if the Bookmark menu (and other menus and menu items) is genuinely an important menu group for all users, or if the high usage is driven by a set of relatively few users who interact with the Bookmark menu extremely frequently.
Get Involved
These are some initial findings we learned from the study. We understand that we cannot draw simple design conclusions by just looking at these numbers, that’s why we really hope to keep the discussion with you in our forum! If you have any questions or new ways of interpreting them, please let us know!
Also, if you are interested to run some numbers by yourself, here is the place to download data samples for this menu study . Don’t forget to share your findings with us by submitting your analysis to the Test Pilot website!
By the way, if you haven’t joined the Test Pilot team, now it’s the time, – we are about to upgrade the Test Pilot extension to version 1.0 beta very soon!
Why?
Interesting study. I think it might be slightly skewed by how web developers/designers use websites though as View Source came about 20th, and I don’t think many regular users will be viewing source. I was also very surprised by the number of people who Zoom In (real users or designers wanting a closer look? Are we making sites with text that is too small?), and the number of people setting their browser full screen. I didn’t expect so many people to use either.
Perhaps it would be useful if Test Pilot asked if the user was a developer or not. I’ve a feeling most of the people interested in this extension are also interested in making sites. Saying all that, it’s still very useful and valuable and I am very positive that Mozilla does this testing.
We have some demographic data about pilots background, check them here:
https://testpilot.mozillalabs.com/testcases/demoinfo-menustudy.html
True. I am the user like others.
The most likely reason for high number of full screen users are the netbooks – where you try to use every bit of display to get the information.
I use practically all applications on the netbook in a maximized/full screen state.
Thank you.
New Tabs are awesome!
Amazing.
If you remove items from the menu that are primarily keyboard driven, how will new users know the keyboard shortcuts?
The whole point of having a GUI is that a user who doesn’t know what they’re doing can click through what they see to explore options. Menus have largely been the single largest source of education for teaching “ctrl+n” because they remind you every time you click that the action could be performed faster with a keyboard shortcut.
I’m all for simplifying the UI. There really are a lot of menu items that are hardly used. I’m just curious as to what will happen to keyboard usage over time with new users not having visual reminders of keyboard shortcuts. Is there another way to present this information to the user in a non-menu driven GUI that is also not intrusive?
So you compared keyboard SCs with mouse clicks in the standard drop down menu (at the top. However did you also take into account straight mouse clicks (e.g. right click/save as) or click close tab on the tab itself? It is what I use most of the time.
Hi,
I personally never use the Edit menu. I can do these things from the context menu, or with keyboard shortcuts. And the Bookmarks menu… I also rarely use it. I’m now using Minefield’s latest nightly build; have the File, View, Tools and Help menus in Personal Menu’s menu button, Bookmarks and History menus are in separate buttons. Until Menu Editor will allow to rename menu items, I can’t make a menu like in the mockups.