I thought I knew a thing or two about multi-touch.  I know that Jeff Han's multi-touch displays and Microsoft's Surface use cameras behind the screen to detect finger contact.  I know that Microsoft hasn't (publicly) been that successful in scaling a multi-touch screen into a really thin (or practical) form factor.  I know that Apple bought FingerWorks several years ago.  I know that Apple has a ton of IP related to multi-touch.  But one thing I didn't know is that Dell was going anywhere near the technology.  Now out of the blue this video drops of Dell's new Latitude XT Tablet sporting 5 finger multi-touch.  With all of Apple's multi-touch IP I was sure it would be first with a multi-touch tablet, but it looks like Dell might beat them.  Unless of course the long rumored ultra-portable MacBook Pro is going to be a multi-touch tablet and beats Dell to the punch.  Which now seems pretty likely.

When I first came across the HP TouchSmart, I thought it was just a cool all-in-one with a touchscreen display.  But what makes the TouchSmart really unique is that it uses cameras to detect finger touches, as described in this great NY Times article.  The only other touchscreens (I know of) that use cameras are big ticket multi-touch systems like Microsoft's Surface, and Perceptive Pixel's huge interactive wall, which use cameras mounted behind the screen to "see" touches.  The HP TouchSmart has two cameras on the outside of the screen in each of the upper corners.  This allows the TouchSmart to use a conventional LCD screen instead of rear projection and maintain its slim profile.  Since finding this out I've started to wonder if the HP TouchSmart may be able to detect multiple finger touches with some modification.

Apple announced lots of new iPods yesterday.  I think this is the first time that Apple has released new iPod while their annual student promotion is still active.  The biggest news is the iPod Touch, which is an iPod with a multi-touch screen and WIFI for web browsing.  Like a lot of people, I'm pretty psyched about the iPod Touch but also wish it offered more than 16GB capacity.  I'm also a bit baffled, by the iPod Classic; 80GB and 160GB iPod seems like overkill for most people.  And while that much storage is probably great for movies, the little 2.5" screen is not.  I fully expected that when Apple finally made a multi-touch iPod that it would be high capacity, and just be called "iPod."  However, it's very likely that an iPod with a big multi-touch screen AND a hard drive would AND WIFI would have suffered from really awful battery performance.  I can't wait to try out the iPod Touch when it shows up in stores, but I might just wait until they bump up the storage.

So after stepping back a bit, I think I may have to scale back my Apple multi-touch predictions.  I'm still believe that multi-touch will be the next big thing, but I think the transition may be a little more incremental than I originally thought.  After all, before Apple jumps into multi-touch with both feet they're going to need to be confident people really like it.  And that will probably mean a few baby steps along the way.  The news of a multi-touch mouse from Apple is a good example of such a step, even though I'm a little dubious that Apple will ever release it.

Shortly after the Apple Store opened back home, my mom and I were in the mall and we stopped in.  One of the things I showed her was the first generation iMac with LCD screen.  I adjusted the screen to be a comfortable angle for us both to look at it, and started to show her around the desktop.  While pointing to one of the icons on the Dock, my finger accidentally tapped the LCD screen which caused a little ripple around my finger as can happen with an LCD when you tap at little too hard.  My mom noticed the ripple too, an her eyes instantly lit up, "Can you just touch the screen?"  she said.  "No," I said, "not quite yet."  That simple experience was enough to convince me that touchscreens are the next big thing in modern computing; on the same order of the mouse and the graphical user interface.  On June 29th, Apple will release the iPhone, their first product with a touchscreen since the Newton.  So the question is, what will be Apple's next Multi-Touch product?

The other evening I started reading Wayne Westerman's doctoral thesis on "Hand Tracking, Finger Identification, and Chordic Manipulation on a Multi-touch Surface" which was written in 1999.  Dr. Westerman went on to found "FingerWorks" a company that manufactured and sold a multi-touch tablet called the iGesture and several other multi-touch products based on technology created by Westerman and Dr. John Elias.  In one section, Westerman describes different methods for detecting multiple finger touches on a surface, and I literally laughed out loud when I read this almost perfect description of Microsoft Surface:

Another approach is to place a camera under a translucent tabletop and image the shadow of the hands [81, 110]. Unfortunately the bulky optics under the table will limit portability and leg room...

p. 38 of the thesis, or p. 68 of the PDF document

Strictly speaking though, Microsoft Surface doesn't look for shadows of the hands, instead near infrared lights are mounted under the surface as well as cameras which pick up the reflected light from fingers in contact with the translucent top.  But still, his point about limiting portability and leg room are dead on.  Westerman doesn't go as far as to call it a big ass table, but he does describe more limitations of using cameras to detect finger touches.  And remember, this was written way back in 1999.  Also, many of you might be interested to know that a few years ago, FingerWorks was purchased by Apple, and I have no doubt that Fingerworks technology is in the iPhone and will probably be in a muliti-touch Apple tablet computer or multi-touch Cinema Display within a year or two.

Microsoft has just announced their new product Surface.  Surface is 30" multi-touch display on the surface of a table which can not only react to multiple fingers (and users), gestures, but even devices placed on the surface.  There's a  great video where the user places a wifi digital camera on the table and the photos appear to spill out of the camera.  The user then places a cell phone on the table and begins dragging photos onto the phone.  The demonstration videos are really incredibly impressive, and the technology holds incredible promise.  However, these devices are going to cost between $5,000 and $10,000 and are currently only being marketed for places like hotel lobbies and in-store displays.  The other significant downside is that, like the Microsoft TouchLight and Perceptive Pixel Displays, Surface uses cameras and rear projection to sense finger contact and display the image.  That means the space under the table top is used for the display components, not your legs.  This makes Surface look more like a conventional CRT TV turned on it's back than a table.  As always this is really great research from Microsoft that will help bring this new interface mainstream.  But until these multi-touch displays can be built around an LCD or even Plasma display, don't expect the technology to spread too quickly.  Also, given that Apple has lots of multi-touch related patents filed, I wouldn't at all be surprised if Apple releases a multi-touch LCD display or tablet computer at some point this year.

Microsoft Surface
Microsoft Surface: Behind-the-Scenes First Look (PopularMechanics.com)

In case you haven't figured it out I'm looking forward to all screens becoming touchscreens.  So I was totally shocked when the HP TouchSmart PC totally snuck under my radar.  The HP TouchSmart IQ770 is an all in one computer with a 19" touchscreen, built-in web cam, memory card reader, and wireless keyboard and mouse.  It even has a built-in TV tuner, DVR functionality and a remote control.  You can also you can dock an HP Photosmart printer on the TouchSmart for a real all-in-one experience.  The TouchSmart is being marketed as the perfect family computer for the kitchen or living room.  It has a cool customizable home screen that can deliver weather, traffic, notes, a family scheduler, and photo organizer, all of which can be driven with the touch of the finger or the included stylus.  The touchscreen interface is really perfect for a family computer to keep organizational drudgery easy and even fun.  Most people probably just accept using a mouse as part of the experience of using a computer, but it's easy to imagine how eliminating it could make using a computer more immediate and direct.  After all, with the the TouchSmart it would be much easier to use the computer while standing up or doing other things.  And the elimination of the mouse, reduces the amount of space needed for the device.  In fact, I'm a little surprised the keyboard doesn't also have built-in trackpad for the same reason.  I'm totally blown away by the TouchSmart, in many ways it's what the iMac should be by now, however I do wish it were a tad smaller and could be VESA mounted. Overall, Bravo HP!  You can be my PC anytime :)

HP TouchSmart IQ770 PC

A new video just surfaced of Jeff Han's amazing multi-touch interface. While the hardware driving this interface may be big, as it requires cameras behind the glass to see fingers touches and rear projection, the software is unbeatable. I wonder how much longer before he and his entire crew are snapped-up by Apple.

Thanks John!

Can't Touch This- FastCompany.com

Since the iPhone announcement at MacWorld some interest has developed in the similarity between Jeff Han’s multi-touch demonstration and Apple’s multi-touch. They both sense multiple fingers touches and they both demonstrated the “pinch” zoom gesture. However, there are some pretty fundamental differences in the hardware driving the two multi-touch interfaces. Read on for a more on this….

I think there's some kind of touchscreen fever going around, as a spate of touchscreen related entries has appeared on Gizmodo.com in last few weeks.  Below are some highlights:

Sony DSC-N2 Cyber-shot With 3-Inch Touch Screen

Digital camera interfaces are notoriously tricky to use, hopefully employing a touchscreen will make navigating those menus and changing picture mode settings just a little bit easier.

HiPDA Combines Tablet PC with TV and HSDPA

The title pretty much says it all, it's built around a 10.4" screen and unfortunately it doesn't look like it has a built-in DVD drive.  To me this makes for a great kitchen computer, as I could use it to look up a recipe and then go back to watching "Project Runway."  Wait did I say recipe and Project Runway? What I meant to say was porn and NASCAR, yes that is definitely what I meant...

Toshiba Dual-Screen UMPC PDA E-Book Doohickey

While I definitely agree that this will flop, the idea has potential.  Given how much hype the Optimus keyboard generated, this product proposes just doing away with 100+ OLED keys and just using a touchscreen.  Of course, if it can't do multi-touch don't ask me what they'll come up with to do Capitalization, Ctrl-C or any other keyboard chords ...

Philips In Touch Message Board Concept

Another concept touchscreen device from Philips Research, this device amounts to a very high tech message board which allows members of a household to record messages and send SMS messages to each other using this slick touchscreen interface.  It's an interesting concept, but seems like it might be better served as a computer application than an expensive single-purpose device.

Wow, big shout out to my buddy John, for showing me this.  Philips Research has developed a multipoint gaming touchscreen that can detect 45 touch points simultaneously that works with fingers as well as game pieces.  Perhaps the most impressive part is that it's all built around a 32 inch LCD screen, so the whole thing is only around 10cm (4 inches) tall.  Another interesting detail of the Entertaible is the technology used to determine touch locations, as it doesn't use any of the 3 most common methods.

The Philips Entertaible, however, is based on a series of infrared LEDs and photodiodes discretely mounted around the perimeter of an LCD screen.
Source: Philips Research Press Releases

Basically this thing detects finger placement like those obnoxious electronic door chimes they somtimes use in stores.  I wonder what kind of accuracy they can get out of that? 

One Minute IFA Preview: Philips Entertaible [techdigest.tv]
Philips Research:  Entertaible concept: combination of electronic gaming and traditional board games

Microsoft has their own multipoint touchscreen reseach research prototype, the TouchLight. Now don't get too attached to the name, before you know it they'll re-name it "Microsoft Windows Powered Smart Touch Screen Professional Edition". Anyways, the TouchLight uses multiple cameras to dertimine hand and touch location. By tracking hand movement they plan to be able to detect 3D gestures to control the image on the screen. The TouchLight is also able to digitally capture documents that are pressed against the screen by snapping a picture of it with one of the cameras. Microsoft has partnered with a company called EON and is saying we'll see this by the end of the year. Ok, sure! Still a neat prototype, and further evidence that in the future all screens will be touchscreens...

TouchLight Video (YouTube.com) via SlashGear.com
TouchLight Demo with Andy Wilson (channel9.msdn.com)
Microsoft's TouchLight makes 3D hands-on (CNET Videos)
EON Touchlight Video (YouTube.com)

Before there were even rumors of an Apple tablet, Microsoft had already unveiled their ill-fated Mira concept. Microsoft, in a stunning branding move, renamed the friendly sounding "Mira" to "Windows Powered Smart Display," good one guys. Anyways, a Windows Powered Smart display is a tablet display that wirelessly connects to a host Windows PC, allowing the user to access the computer from anywhere in the house, up to about 100 feet away. The idea seemed good enough, as ideally the Smart Display would be cheaper than a laptop, offer many of the same functionality without having to upkeep another computer. However, the execution was so abysmal that it never stood a chance.

Read on for more about why Mira failed.

About a week ago I wrote about Jeff Han’s awesome multipoint touchscreen, and questioned if Apple was working on anything like this. Well, it turns out the answer is a resounding “Yes.” Going as far back as 2004, Apple submitted patent applications for a multipoint touchscreen, gestures for a multipoint touchscreen (including the “pinch” gesture), and other important touchscreen related stuff. So, yeah…pretty sweet…

Read on for a summary of Apple’s more interesting patent application with lots of help from hrmph.com.

Given my current fascination with touchscreen displays and the idea that they may one day replace our current keyboard-mouse-monitor arrangement, I'm writing to announce that there will be a lot more posts about touchscreens, tablets and the like over the next week or so. Right now, I'm beginning to think that in the future we'll call multi-point, pressure sensitive, touchscreen displays, just plain old "displays."

For anyone who let out a resounding “Meh” after reading my Apple Table entry, check out this video. The video shows Jeff Han of New York University's Courant Institute of Mathematical Sciences demonstrating a large touch screen display that is pressure sensitive and can accept multiple touch points simultaneously. This enables multiple fingers (or people!) to interact with the display at once. While some of the examples are a bit silly, they still start to expose the power of the interface. I wonder if Apple has something like this on some workbench somewhere? Even if they do, they should still snatch Jeff up before someone else does, this thing is impressive.

Since playing with my friend’s Nintendo DS Lite, I’ve been totally convinced that touch screen displays will be the next major disruption to computer user interfaces. I was shocked not only by how fun and rewarding the games were but how the touch screen opened up so many possibilities for game play. I found myself instantly wanting a touch screen experience for my computer that offered the same level of creative thinking about the interface. Which led to one invariable conclusion, Apple needs to release a tablet based computer, and not one of these dinky UMPCs, I’m talking about a real computer.