Spray

created July 14, 2002
last updated July 26, 2002

Integrating NetworkedImageObserver

NetworkedImageObserver demonstrated that it is possible to get frame rates of 25 fps over the network from an AIBO doing basically nothing. But how does it do when AIBO is busy with a behavior? It turns out NetworkedImageObserver runs nicely with Ball-Sound-Track-Head (sound-track) at 25 fps with medium resolution.

The setup is really simple so I don't include it here. Instead I hope these results, presented as QuickTime movies, speak for themselves. Here is a 1 minute long movie demonstrating Ball-Sound-Track-Head running:
This movie is scaled 2x in this shot. Download instead of clicking on the movie links.
QuickTime (~660K)(frames dropped by compressor)

QuickTime (~2.5MB)(smooth)

FYI: the raw image data for this is 27MB.

The sound-tracking behavior can clearly be seen in the video.

(0:47) -- AIBO looses track of the ball as I raise the ball quickly
(0:50) -- AIBO performs visual search to the right
(0:51) -- AIBO stops turning its head right in response to me tapping the ball on the floor off to AIBO's left
(0:52) -- AIBO regains track of the ball

If motivated, one may correlate this with the output of the debugging log of the session.

Compression

Compression is one way to better balance the trade-offs between resolution and bandwidth. Here are a couple movies of two different compression algorithms in action.

Hi-Res Black-and-White

QuickTime (~660K)(frames dropped by compressor)

QuickTime (~5.6MB)(smooth)

FYI: the raw image data for this is over 114MB.

24:10 Color Compressor from AiboPet

QuickTime (~660K)(frames dropped by compressor)

QuickTime (~13.3MB)(a little choppy in the original too)

FYI: the raw image data for this is also over 114MB. I used the ImportExport sample QuickTime for Java (Windows) - (Mac) application to compress the movies for distribution.

Download

The Java code for producing these movies which should work on both Mac OS X and Windows is found on the Spray page.

Conclusions

Compression comes at a cost that sometimes makes sense. Pitching large amounts of data either by omitting color or dropping to a lower resolution are simple compromises to make for an already-busy CPU. I like max frames per second so I prefer 176x144Y at 25 fps. At 20 fps, AiboPet's 24:10 compressor is probably near ideal for color video.

We're getting video from Aibo at cognitive speeds as I'm starting to perceive something. This is good. High resolution movies (176x144 25fps lossless compression) show us a lot of shortcomings of video.

IR dot - with certain backgrounds you can see it pulse above and to the left of center
intra-frame shift - movies have wavy, otherwise vertical, lines as Aibo's head moves left and right
signal noise - the whole scene seems to shimmer white and scroll downward
color imbalance - pink ball doesn't look pink in very yellow light, as demonstrated with light from a gold foil lamp shade.

It might be interesting to try to detect and possibly filter some of these if a need arose.

BTW, anyone see the line of green and red indicators in the images. What is it?

The Dashboard

Tell me what you think it could be. Email