Normalizing Speech Rate

While working on the speech synthesis API in Firefox, I have been trying to figure out how to provide the most consistent experience across different desktop platforms. This is tricky, because each platform has its own speech API. Each API has slightly differing feature sets and idiosyncrasies.

A good way to foresee the difficulties others will encounter when writing a cross-platform speech app, is to actually write one. So when we started work on the Narrate feature, I needed to account for the fact that the Linux speech API is not capable of pausing speech mid-utterance. We managed to design the interface in a way that wouldn’t require pausing speech, but still give the user an intuitive way of stopping and starting the narration mid-way through the article.

When we landed Narrate, Ehsan noticed that the speech rate slider in the interface was useless at either extremes. Either silly fast, or glacially slow. Since I was developing this feature on Linux, I didn’t encounter the fast rates OSX users experienced. I first thought this was a bug in our OSX speech synthesis support, but later realized Mac was the exception, and this was an issue with the rate conversion in Linux and Windows.

Speech rate is subjective, in so many ways. What is a “normal” speech rate? I have found many numbers, ranging from 150 to 200 words per minute for US English speakers. Luckily, when it comes to the web API we don’t care what the “normal” rate is, we care about relative rate. But that doesn’t completely solve our conundrum.

The Web Speech API (and SSML) defines the rate as a multiplier of the normal speech rate for a given voice. Meaning, 1 is normal, 2 is twice the speed and 0.5 is half. The speech rate goes through a few conversions:

  1. Web page provides a rate to the browser
  2. The browser converts the rate to the platform-specific speech API.
  3. The speech API converts the rate to the speech engine API.
  4. In some cases, the voice converts the rate value the engine provides it.

In OSX, life of course is easiest. The rate is defined as words per minute, their docs say “normal” is between 180 to 220. All we need to do is multiply that, and we get a predictable rate. Things get a little hairier on Linux and Windows. In Linux, the Speech Dispatcher API docs says that rate should be an integer between -100 and 100 with 0 being default. In Windows, the situation is not much better: from a cursory glance at the docs for ISpVoice::SetRate, it just specifies that accepted rates are between -10 and 10.

Further digging brought up more information on Linux and Windows that made the rate parameter a bit more understandable:

  • In Linux, Speech Dispatcher’s eSpeak output module configuration file shows that the default rate is 160 wpm, the minimum is 80, and the max is 320. That’s pretty conservative, but at least it gives us an idea of what is going on.
  • Deeply buried in the Windows docs, I found a mention that the max is 3x normal, and the minimum is a third normal rate.

Now we have something to work with! Did I bore you? I am boring myself..

Next post, a speech rate benchmark tool, pretty graphs, and my attempt at “math”!

 

Normalizing Speech Rate

Narrate: A New Feature in Firefox Nightly

Reader View in Firefox makes reading articles, stories and blog posts enjoyable. It removes the noisy background ads and graphics, and gives you a clean single column optimized for your reading pleasure.

As of today’s Nightly build, you will find an extra button in the Reader View toolbar: the Narrate button. Press play in the popup, and you will have the page read out aloud.  You are now free to give your eyes a rest, knit, wash dishes, work out, play Candy Crush, whatever.

narrate

 

At Mozilla, we believe the web must remain open and accessible. Accessibility can mean many things. In our accessibility team, we work to make Firefox usable to users with disabilities.

Disability is not a binary, it is more nuanced than that. We define our users broadly, we don’t divide them into users with and without disabilities. There can be many reasons why you would choose to click play on that Narrate popup: eye fatigue, multi-tasking, dyslexia, or Angry Birds.

With features like Narrate, we want to make the web more accessible and convenient for everybody.

Narrate: A New Feature in Firefox Nightly

It’s MLK Day and It’s Not Too Late to Do Something About It

For the last three years I have had the opportunity to send out a reminder to Mozilla staff that Martin Luther King Jr. Day is coming up, and that U.S. employees get the day off. It has turned into my MLK Day eve ritual. I read his letters, listen to speeches, and then I compose a belabored paragraph about Dr. King with some choice quotes.

If you didn’t get a chance to celebrate Dr. King’s legacy and the movements he was a part of, you still have a chance:

It’s MLK Day and It’s Not Too Late to Do Something About It

Firefox for Android Accessibility Refresh

When we first made Firefox accessible for Android, the majority of our users were using Gingerbread. Accessibility in Android was in its infancy, and certain things we take for granted in mobile screen readers were not possible.

For example, swiping and explore by touch were not available in TalkBack, the Android screen reader. The primary mode of interaction was with the directional keys of the phone, and the only accessibility event was “focus”.

The bundled Android apps were hardly accessible, so it was a challenge to make a mainstream, full featured, web browser accessible to blind users. Firefox for Android at the time was undergoing a major overhaul, so it was a good time to put some work into our own screen reader support.

We were governed by two principals when we designed our Android accessibility solution:

  1. Integrate with the platform, even if it is imperfect. Don’t require the user to learn anything new. The screen reader was less than intuitive. If the user jumped through the hoops to learn how to use the directional pad, they endured enough. Don’t force them through additional steps. Don’t make them install addons or change additional settings.
  2. Introduce new interaction modes through progressive enhancements. As long as the user could use the d-pad, we could introduce other features that may make users even happier. There are a number of examples of where we did this:
    • As early as our gingerbread support, we had keyboard quick navigation support. Did your phone have a keyboard? If so, you could press “h” to jump to the next heading instead of arrowing all the way down.
    • When Ice Cream Sandwich introduced explore by touch, we added swipe left/right to get to previous or next items.
    • We also added 3 finger swipe gestures to do quick navigation between element types. This feature got mixed feedback: It is hard to swipe with 3 fingers horizontally on a 3.5″ phone.

It was a real source of pride to have the most accessible and blind-friendly browser on Android. Since our initial Android support our focus has gone elsewhere while we continued to maintain our offering. In the meantime, Google has upped its game. Android has gotten a lot more sophisticated on the accessibility front, and Chrome integrated much better with TalkBack (I would like to believe we inspired them).

Now that Android has good web accessibility support, it is time that we integrate with those features to offer our users the seamless experience they expect. In tomorrows Nightly, you will see a few improvements:

  1. TalkBack users could pinch-zoom the web content with three fingers, just like they could on Chrome (bug 1019432).
  2. The TalkBack local context menu has more options that users expect with web content, like section, list, and control navigation modes (bug 1182222). I am proud of our section quick nav mode, I think it will prove to be pretty useful.
  3. We integrate much better with the system cursor highlight rectangle (bug 1182214).
  4. The TalkBack scroll gesture works as expected. Also, range widgets can be adjusted with the same gesture (bug 1182208).
  5. Improved BrailleBack support (bug 1203697).

That’s it, for now! We hope to prove our commitment to accessibility as Firefox branches out to other platforms. Expect more to come.

Firefox for Android Accessibility Refresh

Using Pre-release Firefox on Linux

Every committed Mozillian and many enthusiastic end-users will use a pre-release version of Firefox.

In Mac and Windows this is pretty straightforward, you simply download the Firefox Nightly/Aurora/Beta dmg or setup tool, and get going. When it is installed it is a proper desktop application, you could make it your default browser, and life goes on.

In Linux, we rely much more on packagers to prepare an application for the distribution before we could use it. This usually works really well, but sometimes you really just want to use an upstream app without any gatekeepers.

The pre-release versions of Firefox for Linux comes in tarballs. You unpack them, and could run them out of the unpacked directory. But it doesn’t run well. You can’t set them as your default browser, the icon is a generic square, and opening links from other apps is a headache. In short, it’s a less than polished experience.

So here is a small script I wrote, it does a few things:

  1. It downloads the latest Firefox from the channel of your choosing.
  2. It unpacks it into a hidden directory in your $HOME
  3. It adds a symbolic link to the main executable in ~/.local/bin .
  4. It adds symbolic links for the icon’s various sizes into your icon theme in ~/.local/share/icons.
  5. It adds a desktop file to ~/.local/share/applications.

It doesn’t require root privileges, and is contained to your home directory so it won’t conflict with the system Firefox installation or touch the system libxul. Typically, you only need to run the script once per channel. After a channel is installed, they will get automatic updates through the actual app.

Nightly running in Fedora 22
See the nice icon?

So, here are some commands you could copy to your terminal and have pre-release Firefox installed:

Nightly

curl https://raw.githubusercontent.com/eeejay/foxlocal/master/foxlocal.py  | python - nightly

Aurora

curl https://raw.githubusercontent.com/eeejay/foxlocal/master/foxlocal.py  | python - aurora

Beta

curl https://raw.githubusercontent.com/eeejay/foxlocal/master/foxlocal.py  | python - beta

Release

curl https://raw.githubusercontent.com/eeejay/foxlocal/master/foxlocal.py  | python - release

Using Pre-release Firefox on Linux