Optical Mouse Cam

OK, so I finally decided to get my hands on the Arduino platform and see what all the buzz is really about. And I must say, I'm impressed. This is by far the most developer-friendly development platform I've ever seen. It's main advantages :

• It's simple. The hardware is very simple and uses standard, commonly available components. The software IDE is basic but functional : editor window, status window, integrated serial terminal. That's it.
• It's reasonably powerful. The Arduino diecimilla that I used is based on the ATMega168 microcontroller clocked at 16MHz.
• It's based on open source hardware and software (If you omit the trademark controversy about the Arduino trademark). The board design is public. This has lead to the creation of several variants and clones (Freeduino) which contributed to the popularity of the platform.
• Power, firmware upload and serial communications through a single USB cable.
• It's expandable. The concept of "shields" that can be stacked on top of the main Arduino board is genius :) As a matter of fact, making a small prototyping shield was the first thing I did for this platform.

So, a while ago I came across this great optical mouse hack by Sprite. I decided it would be great to do as my first Arduino project (beyond the "Blinky LED" example, that is.)

First step, dissection of a wireless mouse from a cheap Typhoon wireless mouse / keyboard combo.

Unscrew it. Look for screws under the pads and stickers on the bottom.

My model came with two PCBs which are coupled using a 14 pins, two rows header :

The unusual looking chip in the middle is an ADNS-2051 optical sensor. This chip holds a 16x16, 64 levels of gray image sensor and a DSP which computes movements from the optical flow of the captured images :

Underneath the second PCB is the optical bloc with lens and LED collimator :

Better view of the optics. It was manufactured by Agilent technologies or its spinoff Avago technologies :

Agilent/Avago provides the optical block, motion sensor and LED assembly of most optical mice. Here's how everything stacks up :

OK, so going this far was not really necessary, I just like to see how things are made and to take macro shots of it :-) Going back to the ADNS-2051 chip and the Arduino, you need to make the following connections :

All the other pins should have the required circuitry already on board, so they need not be touched. In my case however I had to wire the LED, too, since it was not directly wired to VCC but switched on by the controller. This is because it was a wireless mouse build with a chip intended for a corded design, so they had to diverge a bit from the reference design. I was lucky, the sensor board had a female header which I could use to wire everything, so I did not have to solder anything. But you probably will.

Ignore the prototype board on top of the Arduino, you can wire everything directly to the Arduino headers.

You can download the Arduino script and the PC program below. The program is in java and requires the RXTX library to be installed. You need to get the "RXTX 2.1 for use withOUT Sun's CommAPI (namespace gnu.io)" version. No, as of 2008 java still does not natively support the serial port. It should work with Linux, Mac OS X and Windows, but I only tested it under Windows.

[Addendum: RXTX seems to have been discontinued and the website is now a blank page. I suggest using PureJavaComm instead, it works great and does not require to be installed directly in the JRE. The project may need to be modified a bit to adapt it to this library.]

• Arduino script
• Client program
• Client program source code

Once RXTX has been installed you can run the jar file simply by double-clicking on it or in the command line like this :

The window will show up. Simply select the serial port your Arduino board is connected to and click Connect. The sensor data will be drawn on screen in real time. You can increase or lower the number of lines to read from the sensor in each run. The frame dump routines are a bit slow so if you dump a full frame you'll get only about 5 fps. If you're moving the mouse at the same time, there will be severe shearing artifacts and the scan will not look pretty. In this case, reduce the capture window to 2 or 3 lines.

If you move the mouse while the application runs you can "scan" the surface below it. As you can see the quality is unmatched. You need to keep the mouse orientation exactly the same while scanning since there is no way to detect rotations, and this distorts the capture even more.

Here's how a white sheet of paper, the fabric from my pants and the small print on a 20 euro bill are seen by the sensor :

Alright, what else can we do with this very nice sensor ? Looking at the datasheet something struck my eye : the sensor is sensitive in a wide range of wavelenghts, from ultraviolet to infrared. So why not replace the standard red LED by an RGB LED ? :-)

I skip all the gory details, here's the not so impressive result :

I was attempting to scan a detail of a 20 euro bill. I've had several issues with the sensor when switching between diferent colors. The sensor would be nearly saturated or very dark on the border of the frames. I solved the problem by adding a 100ms delay after each color change so that the sensor could adapt. Another issue is that the sensor seems to use some kind of automatic aperture control, which makes it impossible to balance the colors. Below is what the same detail looks like with a "real" camera. We can kinda recognize the blue lines :-)

If you wish to reproduce this amazing result (who wouldn't ?), you need to do the following :

• Remove the red LED and replace it by the RGB LED.
• Connect the LED cathode to pin 14 of the ADNS-2051 (the LED needs to be common-cathode).
• Connect the RED anode to Digital pin 5 on the Arduino.
• Connect the GREEN anode to Digital pin 6 on the Arduino.
• Connect the BLUE anode to Digital pin 7 on the Arduino.
• Load this script on the Arduino.
• Run this horribly hacked version of the java program above to support colors.

Further information :

• ADNS-2051 Datasheet.
• Understanding Optical Mice. White paper from Avago technologies.
• Determinig Optical flow. Theoretical paper on optical flow by Berthold K.P. Horn and Brian G. Schunck.
• Optical navigation sensors product page on Avago website.
• Optical mouse cam hack on Sprites mods

OK, enough time wasted on this already ! :-)