Fantasy Device: Laptop with in-built projector and hand-tracking capabilities
For this assignment I began by designing a fantastic device inspired by my HP TouchPad (and my experiences with the iPad) called “An At-Least Somewhat Useful Tablet”. This device’s hardware requirements were realistic, but its software requirements and the apparent requirement for a redoubt of patents might push this device into the realm of fantasy. However, upon re-reading the assignment’s prompt I felt that the touchscreen/microphone/proximity sensor interface was not the fantastic physical design this project was seeking, so I will save that post for another time.
This fantasy device isn’t impossible; it could be implemented (maybe even at a reasonable cost) using consumer technology. I have seen patent applications and academic projects that implement aspects of this device, but these do not guarantee that such a device will ever be commercially realized.
My fantasy device is a laptop augmented with an LED or laser pico projector and a camera with software capable of tracking a user’s hands. The camera is positioned such that it can capture the projector’s image if it is thrown within a few feet of the laptop. The camera/projector apparatus should be an arm that extends from the laptop’s lid that can be directed at a desk or a wall behind the laptop, and should be able to face the projection surface at an angle to minimize parallax distortion. A depth-sensing camera like the Kinect should not be required. Note that in the (crude) mockup above the extended display is projected onto a sketchpad because my desk is dark. The sketchpad has no other relevance to this device.
The projector displays an extended desktop that the user can interface with using hand gestures. Tap-to-select, hold-to-right-click, pinch-to-zoom and flick-scroll in documents should be captured; “throwing” a window to the laptop’s display should move the window. Likewise, dragging a window from the laptop’s display off screen should move the window to the projected surface. Optimally, the mouse cursor could also move to the projected surface. The camera could track a user’s dragged finger on the projection surface for use as a Wacom-like drawing tool. This projected display would be useful for displaying relatively static, secondary content: instructions or example code for a program the user is writing on the main display, an email inbox, a chat window, etc. A virtual keyboard numberpad or a keyboard with custom characters/sequences could be projected to augment the laptop keyboard in ways specific to the application running on the main display(s). The user could run Dreamweaver’s code view on the laptop’s display, the page in a browser on the secondary display, and the file browser, an FTP client, or a copy/paste palette of HTML tags on the projected display.