Hacking Firefox Mobile, Not The Good Kind Of Hacking

October 17, 2011 //

Hi again. Remember my first post to Planet Mozilla? I hope not. Sorry for the male enhancement pills spam. Please stop contacting me for orders.

I am in Toronto right now, spending a week with Mobile folks. It has been about 6 weeks since I started working on a mobile extension, and the list of dirty tricks we employing to get things done is growing. Some of these hacks are there because there is no alternative, and some are there because I just didn’t figure out the right way to do it.

Here is a short list:

Gestures: I reinvented them, poorly. Our extension relies heavily on the ability to support a rich set of multi touch gestures. We have a hack, similar to how it was done in Touching is Good (I know!), where we swap out the callbacks in MouseModule and GestureModule for our own. This doesn’t seem right, for a few reasons. The main one being that this doesn’t seem like a real public API, and it will be broken sooner or later. The second thing is the fact that we are relying on shaky JS to re-interpret the platform-interpreted events. So maybe there is a better way to do things. The nsIDOMSimpleGestureEvent only serves us to a limited degree, I think there is room for a more low-level multi touch event API that is similar to the Android one. Or maybe a comprehensive high level API with more event types.
Speech: This should probably be implemented natively and have multi-platform support. Right now, in Android, we are doing Javascript <-> js-types <-> jni <-> Java. What could go wrong?! But seriously, it is kind of ugly right now, but I would like to make our JS solution prettier and useful to other extension writers who want to tap in to Android’s API and services.
Security: The Mozilla profile directory in Android is not world-readable, presumably for good reasons. Our extension’s media files need to be world-readable so that Android’s TextToSpeech service could pick them up and use them as earcons. This requires an install and uninstall hook that uses Android’s Context.getDir to get/create a subdirectory in Firefox’s top-level app directory where we could copy over those media files for system consumption. Maybe this is the best way to do it… But I can’t imagine anyone being happy about an extension writing to that directory. Sorry!
–enable-accessibility: Our extension depends on accessibility being enabled at build time. Since I am getting some face time with folks this week, it seems like a good thing to bring up.

I am sure there are other horrible horrible hacks in there. But the list above is really what I needed to confess to. Thanks for empathetic attention.

Categories Accessibility, Software

The Cool

October 13, 2011 //

Hello, strange new planet!

There have been countless Steve Jobs eulogies in the past week. Jobs is a complicated figure for me. He joins other historic American innovators such as Bell, Edison and Ford, who’s biographies celebrate the “land of opportunity” mythology, where anyone motivated could get ahead. On one hand they bettered society with affordable mass-produced technology, and on the other hand they employed aggressive business strategies, introduced bad labor practices, and ruthlessly quashed competition.

Dennis Ritchie passed away this Saturday. Eulogies are not competitions, obviously. But the contrasts between Jobs’s and Ritchie’s legacies are hard to ignore. Jobs introduced to the world iconic form factors, gadgets you could hold. But Ritchie and his co-inventors laid the foundation for modern software. That svelte iPad? Its operating system is 40 years old, Ritchie’s brainchild.

But along with the software, Ritchie and his friends introduced an entire philosophy. A philosophy that is just as seductive to an engineer as the latest Apple aluminium unibody product.

I grew up on UNIX. We had a machine at home with the hostname saris, Hebrew for eunuch. My dad taught me The Cool. Specifically the UNIX Cool of keeping it simple, less is more, and silence is success. It is this Cool that made me want to program, and it will outlast every fancy gadget.

My Dad. Still programming, still cool.

Categories Culture, Personal, Technology

Caribou Week Who Is Paying Attention Anyway – We Need a New Maintainer

August 7, 2011 //

After making a non-binding resolution to report my Caribou progress on a weekly basis, I flaked. Of course. But luckily Nohemi has picked up the slack and have kept you all up to date about the libcaribou powered GNOME Shell keyboard in her more binding GSoC reports. So no more architecture diagrams are needed, you all get the idea. But if you didn’t, let me make it clear: The goal of Caribou is to make it easy to implement new on screen keyboards where you would only need to provide the view, and libcaribou will be your model and controller.

It is better to admit now than later: I will not have the bandwidth to continue to work on Caribou as my time is slowly running out. So…

What we need:

A maintainer.
A GTK module, Nohemi is working on this, but we will need similar solutions for other toolkits and fallbacks (XIM?).
Unit tests for the library – libcaribou is gaining features and getting complex. We need some tests here.
More keyboard layouts/languages.
More function keys for the switch scanning keyboard.
Finer interaction modes:

Modifiers, use latching and traditional key gestures with multitouch.
Respect AccessX settings for sticky/slow/bounce keys.
A more hardware-like interaction where keys will auto-repeat when held down.
etc.
… and automatically choose the right mode for the keyboard without comfusing the user.

Revisit the “X adapter” and maybe use something more high level.

Anyway, plenty of exciting work. Are you at the summit, please find me if any of this interests you.

Tags caribou

Categories Accessibility, Software

I Am Speaking In Portland Next Week

June 15, 2011 //

I am getting really excited about the Open Source Bridge conference in Portland next week. It feels like most Open Source meetups I have attended lately have been on the other end of the Atlantic. It will be nice to be in my element, for once. It will also be nice to meet cool localish people. I am planning to train down there and bring my bike. I hope to see you there!

I’ll be talking about inclusive design.

Categories Accessibility, Personal, Software

Caribou Week Whatever

June 7, 2011 //

Hi again.

Since the last time I wrote to you, dear bloggy, I have been working a lot on Caribou, made a trip to welcome Jenny back from Haiti, and crossed the continent by rail.

Let’s talk about Caribou, so much has changed!

Antler

Antler is the Keyboard UI that is bundled with Caribou. It does not have the pretension of being ready for users any time soon, it is more a sample implementation of a libcaribou keyboard,and a place for me to try stuff out and see if it would work in our platform. You could follow Nohemi‘s progress to catch up on how libcaribou is being used to power the new GNOME Shell keyboard UI. With all that said, Antler is still kinda cool. Here is a crappy video of Antler’s touch keyboard in action:

Scanning Support

I redesigned Caribou’s switch support from the ground up with the goal of simple configuration. There is still plenty of more work to do, but after looking at commercial alternatives I feel like we could do a pretty good job. Here is me typing text solely with the right shift key:

Please excuse the green/red combo

Input Method Support (bye AT-SPI!)

This is an experimental tangent, that might or might not be worth the time I spend on it. Recently Caribou master received GTK2/GTK3 input modules that perform DBus calls to the Caribou keyboard, and have it show up when and where it is needed. This has proven to be pretty tricky. I will hopefully follow up with a post about this, and some interesting ~~hacks~~ innovations surrounding these methods. Future work includes writing similar modules for QT3 and QT4. And having the keyboard emit key activation signals that the modules could use for inserting text instead of using XTest which feels so hackish and wrong.