A Device Blind Users Will Love

The Internet is a global public resource that must remain open and accessible.

— Mozilla manifesto

Mozilla invests in accessibility, because it’s the right thing to do.

We have staff, a team of engineers, who exclusively focus on accessibility in our products and play a positive influence in the general accessibility of the web. This has paid off well, Firefox is well regarded as a leader in screen reader support on the desktop and on Android. We have the best HTML5 accessibility support in our browser, and we are close to having a fully functional screen reader in Firefox OS.

Mozilla accessibility logo

I say “close”, because we are not yet there. Most websites are fairly accessible with little to no effort from the site developers. The document model of the web is relatively simple and is malleable enough that blind users are able to access them through screen readers. Advanced web applications are a whole other story, developers are required to be much more mindful about how they are authored and account for users with disabilities when designing them. The most recognized standard for making accessible rich internet application is called ARIA (accessible rich internet applications), and it allows augmenting markup with attributes that will help assistive technologies (such as screen readers) have a good understanding of the state of the app, and relay it to the user.

In Firefox OS we have a suite of core apps called Gaia that is the foundation for Firefox OS’s user interface. It is really one giant web app, perhaps one of the biggest out there. Since our mission dictates that we make our products accessible, we have embarked on that journey, we created a screen reader for Firefox OS, and we got to work in making Gaia screen-reader friendly. It has been a long and sisyphean process, where we would arrive at one module in gaia, learn the code, fix some issues, and move on to the next module. It feels something like this:

helicopter dumps water on a grass fire
A California Department of Forestry helicopter dumps water on a grass fire in Benicia. (Robinson Kuntz/Daily Republic)

Firefox OS has grown tremendously in a couple of years. Things never slowed down, and we were always revamping one app or another, trying out something new, and evolving rapidly. This means that accessibility was always one step behind. If we got an app accessible in version n, n+1 was around the corner with a whole new everything. Besides working on Gaia, we have always been looping back to our screen reader, making it more robust and adding features. We have consistently been straddling the gap:

The gap between Firefox OS and the screen reader

Firefox OS has achieved some amazing milestones in its short life. Early in the project, there was still a hushed uncertainty. Did we over promise? Could we turn a proof of concept into a mass-market device? There were so many moving parts for a version one release. Accessibility was not a product priority.

The return on investment

When I think about making our products accessible for the people that can’t see or to help a kid with autism, I don’t think about a bloody ROI.

— An angry Tim Cook

Take 5 seconds, and let that sink in. Apple is not a charity, they are one of the most profitable companies on the planet. Still, they understand the social value of making their products accessible.

Yet, I will argue that there is a bloody return on investment in accessibility.

Mobile is changing our social perception on disability and blurring the line between permanent and temporary barriers. The prevailing assumption used to be that your user will sit in front of a 14″ monitor with a keyboard, mouse and an undivided attention. But today there can be no assumptions, an app needs to be usable in many situations that impair the user in comparison to a desktop setup:

  • A user will browse the web on a small, 3.5″ device with no keyboard, and only their inaccurate fat fingers as a pointing device for activating links.
  • A driver will need to keep their eyes on the road and cannot interact with complex interfaces.
  • A cyclist on a cold winter day will have gloves and will want to look up where they are going on a map.
  • A pedestrian will look up a nearby restaurant on a sunny day with plenty of glare making it hard to read their phone’s screen.
A driver texting in traffic
This shouldn’t happen.

The edge case of permanently impaired users is eclipsed by the common mobile use case which needs to appeal to users with all sorts of temporal impairments: motor, visual and cognitive. Apple understands that with Siri, and Google does too with Google Now. In Firefox OS, sooner or later we will need a good voice input/output story.

I made a case for accessibility, and I could probably stop here. But I won’t. Because the real benefit of an accessible device is priceless.

Graph showing impact on blind users in contrast to other users
While blind smart phone users are a small fraction of the general population, the impact on their lives is so much greater.

We all benefit from that smart phone in our pocket. The first iPhone was a real revolution. It allows us to check mail on the go, share our lives on social networks, ignore our family, and pretend like we we are doing something important in awkward parties. But for blind users, smart phones have increased their quality of life in profound and amazing ways. Blind smart phone owners are more independent, less isolated. and they can participate in online life like never before. Prior to smart phones, blind folks depended on very expensive gadgets for mobile computing. Today, a smart phone with a few handy apps could easily replace a $10,000 specialty device.

Smart phones in the hands of blind users is a very big deal.

Three blind iphone owners

What we need to do

To make this happen, every decision by our product team, every design from UX, and every line of code from developers needs to account for the blind user experience. This isn’t as big a deal as it sounds, screen readers support is just another thing to account for, like localization. We know today that designing and developing UI for right-to-left languages take some consideration. Especially if you live in a left-to-right world.

What we need is project-wide consciousness around accessibility. It is great that we have an accessibility team, and I think Mozilla benefits from it. But this does not let anyone else off the hook from understanding accessibility, embedding it in our products, and embracing it as a value.

I fear that this post will disappoint because I won’t get into how blind users use smart phones, and how should developers account for the screen reader. I have written in the past about this, and Yura has some good posts on that as well. And yes, we need to step up our game, document and communicate more.

But for now, here are two things you could do to get a better picture:

  1. If you own an Android device or iPhone, turn on the screen reader, close your eyes and learn to use it. Challenge yourself to complete all sorts of tasks with your screen reader on. Test the screen readers limits.
  2. With your Firefox OS device, turn on the screen reader. It works in the same fashion as the iOS or Android one does. Check your latest creation, and see what is broken and missing.

2015 is going to be a great year for Firefox OS. I have already heard all sorts of product ideas that have the potential of greatness. We are destined to ship something amazing. But for bind users, it could be life changing.

A Device Blind Users Will Love

Firefox OS App Accessibility Workshop Part 2: Analog Clock

This is a second post in a series about making Firefox OS apps accessible to blind users. You could read the intro here, and a post about labels here.

The Gaia Clock App has a pretty analog view, with ticking hands. You can stare at it and wonder how generations past were able to divine the time of day from a few muted lines. The analog clock should serve the same purpose for blind users as it does to sighted users: It should tell time. In this post I’ll give an overview of the steps I needed to take to make the analog clock view useful.

Analog View in Clock App

If you are a blind user,the screen reader won’t even bother telling you about the analog view. The view is composed from a collection of empty div elements. Since they don’t contain anything besides some special styling, the screen reader skips them and does not bother the user with some arbitrary nested divs. For a good reason, the entire Internet is a collection of redundant nested divs!

To make the screen reader display the view to the user, we need to give the view an appropriate role. This will tell the screen reader’s heuristics that the view is an item of interest. I chose to use the img role, since this is a graphic that describes time. There are other roles that would also be appropriate, such as marquee. But I chose img.

I applied this role on the clock’s container:

...
         <div role="tabpanel" id="alarm-panel" class="active panel">
           <div id="clock-view">
             <div id="analog-clock">
-              <div id="analog-clock-container">
+              <div id="analog-clock-container" role="img">
                 <div id="analog-clock-face">
                   <div id="analog-clock-hands">
                     <div class="analog-clock-hand" id="secondhand"></div>
...

When I tried the screen reader with this change, I found that I was able to land on the analog view. The screen reader would say “graphic”, which is not very useful, but it is something. Now the user knows what is taking up all that space on the screen.

The next step is to label the graphic with the current time. This was done in Javascript. Every time the clock hands are updated, we update the ARIA label as well:

...
   updateAnalogClock: function cv_updateAnalogClock(opts) {
     opts = opts || {};

     if (opts.needsResize) {
       this.resizeAnalogClock();
     }
     var now = new Date();
     var sec, min, hour;
     sec = now.getSeconds();
     min = now.getMinutes();
     // hours progress gradually
     hour = (now.getHours() % 12) + min / 60;
     this.setTransform('second', sec);
     this.setTransform('minute', min);
     this.setTransform('hour', hour);
+
+    // Update aria label for analog view.
+    var time = Utils.getLocaleTime(now);
+    this.container.setAttribute('aria-label',
+                                time.t + (time.p ? ' ' : '') + time.p);
+
     // update again in one second
     this.timeouts.analog = setTimeout(
       this.updateAnalogClock.bind(this), 1000 - now.getMilliseconds()
     );
   },
...

Another round with the screen reader, it now says “11:20 AM, graphic”. That is a lot better!

Screenshot of Screen Reader Emulator showing the analog view correctly.

Firefox OS App Accessibility Workshop Part 2: Analog Clock

Followup On Unlabeled Widgets

After my previous post, about labels, @ted_drake commentedIs there a reason why you didn’t simply add text to the button?” which is a good question! The short answer is, we don’t want to render the text in the button. Instead we want to keep it a graphical icon.

But this question demands an emphasis:

WAI-ARIA is a last resort, it is designed to be a stopgap that augments conventional HTML markup. As HTML evolved, many of the problems ARIA solves have been eliminated. Often, if you are using ARIA you are making a mistake. As Ted continues to say: “…the first rule of ARIA is to not use it when there is a standard alternative.

So back to the issue of labeling controls and elements; aria-label is a last resort. The best option is rendered text, so the visual display is consistent with the accessible naming, for example <button>Press me</button>. After that there are two attributes that can be used, title and alt.

An image may have an alternative textual description, that is what the alt attribute is for. Almost any element may have a title attribute, which will provide a human readable name for the given object.

In fact, I was wrong in the earlier post. Instead of using aria-label, title would have sufficed. The only times aria-label should be used is when you don’t want a tooltip with the text to be visible on desktop, and for naming more complex composite widgets.

I just learned something new, I hope you did too!

Followup On Unlabeled Widgets

An Introduction To Making Firefox OS Apps Accessible

Hello! This is a first post in a series that will outline the challenges and remedies for making an accessible app in Firefox OS.

First you may ask, accessible to whom? Good question. Accessibility is a very broad term. There are many reasons why Firefox OS would not be available to an individual. In this series we will focus on making apps that are accessible to blind users.

Secondly you may ask, how does a blind person use a Firefox OS device? They can’t. At least not yet. The accessibility team at Mozilla is focusing on bringing together the right tools and integrating them into Firefox OS to make that happen. Namely, we are working on a screen reader.

Thirdly you may ask, how does a blind user use a touch screen smartphone? Very efficiently. When the iPhone 3GS was introduced, it came with a screen reader. Today, iOS devices are very popular among blind users. With a few simple gestures, the entire feature-set of a smartphone becomes available to visually impaired people. This video highlights how that is done with the iPhone:

In Firefox OS we are working on a screen reader with a very similar interaction model to both iOS and Android’s solutions. The main design principal is to allow the user to explore the display with their touch without directly affecting the state of the underlying apps. To interact with an application, for example to activate a control, the user first finds the control by “feeling around” on the screen and then double-taps to activate it. In addition to exploring by touch, the user could flick left or right to advance through the UI in a linear fashion. That is the basic idea, there are many more features and gestures that a user could learn. But if they got the basics, they are good to go.

Fourthly you may ask, if there is no screen reader available on Firefox OS, how do I test my app to assure it works for blind users? There are two answers here.

Screen Reader Simulator Extension

If you work on Gaia with desktop Firefox, there is an extension that is automatically downloaded when you make with DEBUG=1. You can also use the extension, outside of Gaia, in a standalone fashion with Nightly. It is available here. This extension adds a devtools panel simply called “Screen Reader”. It has an “Enable” button. Once that button is toggled, the content view will have our screen reader applied to it. If you hold your mouse button and drag it along the screen, you will notice an orange rectangle highlighting different items. You will also notice that the main area of the devtools panel starts to populate with text. This is a log of what a blind user would hear as synthesized text. The extension has additional toolbar buttons, the “Navigation” buttons are there for you to easily navigate through the content, if you press the center button the highlighted item will be activated, for example a link would be clicked. The “Scroll” section allows you to scroll the currently active view. This is useful since the touchscreen gesture for scrolling is a two finger swipe, which is hard to do on most desktops. Here is a silent screencast of the extension in action.

Enabling The Screen Reader On The Device

You should only do this if you understand how the screen reader works. It could be hard to undo if you don’t, and you may end up needing to clobber your Gaia profile to get your device back. But if you are OK with all that, it is easy. You just need a build off of master (version 1.3), go to the Settings app, and drill down to the developer settings (Device Information->More information->Developer), under the Accessibility header is a checkbox for the screen reader. Enable that. That is it, you should have a talking device!

An Introduction To Making Firefox OS Apps Accessible