Part of the art of writing patents is to protect concepts that might be used in future products without delineating them too clearly.

Case in point: Apple was awarded a patent yesterday: Gestures for controlling, manipulating, and editing of media files using touch sensitive devices. Here’s the abstract:

Embodiments of the invention are directed to a system, method, and software for implementing gestures with touch sensitive devices (such as a touch sensitive display) for managing and editing media files on a computing device or system. Specifically, gestural inputs of a human hand over a touch/proximity sensitive device can be used to control, edit, and manipulate files, such as media files including without limitation graphical files, photo files and video files.

Seems mainly about Apple getting a patent for gestures used to edit video on multi-touch devices. But I think the interesting phrase there is proximity sensitive device. That means we’ll be able to edit without touching a screen (or wearing special gloves).

Hidden in the middle of the patent are the following two sentences:

Finally, using a multi-touch display that is capable of proximity detection … gestures of a finger can also be used to invoke hovering action that can be the equivalent of hovering a mouse icon over an image object.

Ironically, one of the arguments against making Flash available on multi-touch devices is the fact that the majority of Flash implemented UI elements use the position of the mouse pointer without the mouse button being clicked as useful feedback to the user – a concept not possible using multi-touch. If devices included advanced proximity detection technology, then ‘mouseover’-equivalent events could be sent to Flash UIs – so they’d work they way they have since Shockwave and .fgd files.

Although granted yesterday, the patent was applied for in June 2007. In August 2007, I wrote about gestural edits that required the UI being able to detect fingertip position while not touching the screen.

I also wrote about Apple being granted a patent for using a camera mounted to a portable device to detect hand movement in three dimensions.

In which I remind you of Apple’s concept videos from the 80s and suggest it is time for a new one.

In 1987 and 1988 Apple were still facing an uphill battle with businesses when it came to convincing them that graphical user interfaces were better than MS-DOS command-line interfaces. Part of their campaign to show that the Mac way of doing things was the start of the future of computing was to create speculative videos of how computers might evolve in ensuing years.

The Knowledge Navigator video was set in the far off year of 2010. In 1987 John Sculley suggested that if Apple defined an idea future for the Mac, it would be more likely for that future to happen. Commentators have theorised that Moore’s Law, the prediction for the rate of improvement of the amount of computer power at a given price, has galvanized technologists to do all they can to do better than predicted.

The year isn’t stated in the video, but the figures presented only run up to 2009, so I’m guessing this is set in 2010.

It is interesting to see how close we are to this kind of interaction with our technology. In 2003 Jon Udell revisited this video and commented

Presence, attention management, and multimodal communication are woven into the piece in ways that we can clearly imagine if not yet achieve. “Contact Jill,” says Prof. Bradford at one point. Moments later the computer announces that Jill is available, and brings her onscreen. While they collaboratively create some data visualizations, other calls are held in the background and then announced when the call ends. I feel as if we ought to be further down this road than we are. A universal canvas on which we can blend data from different sources is going to require clever data preparation and serious transformation magic.

Last week Stephen Wolfram announced that his next project is an online system that can take your natural language questions and compute answers for you. That reminded me of Apple’s Knowledge Navigator. I imagine it will be able to answer questions like:

“Is there a link between the size of the Sahara and deforestation on the Amazon rainforest?” “What if we bring down the logging rate to 100,000 acres a year?”

It’ll be a while until we have foldable screens, but it seems that if WolframAlpha can be made to work, we might be closer to the Knowledge Navigator, or what computers should be doing for us anyway.

In 1988 Apple made another video, one that is less famous, but much more accurate in its predictions. Which is another way of saying, if we were to make a video today about 2020, this is what we’d be predicting right now.

A OK quality video can be found at http://www.mprove.de/uni/asi/futureshock.html

…or you can stay here and watch it encoded for YouTube:

You’ll see that some of the ideas are still be speculated today.

Microsoft especially likes the idea of real objects interacting with technology (as used in their Surface product). Microsoft has a video set 10 years in the future. It starts off with some impossible to implement stuff in a classroom, but continues with some good ideas:

(about that classroom, it’s all very well having augmented reality ideas (overlaying graphics onto the real world), but they only can work when there’s an audience of one – the display needs to take account of the position of the viewers eyes to line up the graphics in the right place. The kind of classroom telepresence shown at the start of the video would only work for one kid in each classroom at a time. For everyone else, the display would look odd and distorted. For more on this, see an older blog post.)

A much more realistic and specific Microsoft video was made in 2004, and set in 2010. You’ll see that their estimate of what we’ll be able to to in 20 months time:

On the subject of speculative videos, maybe we should start thinking of one for the creative industries. If collaboration is what makes TV and movies so satisfying, how will technology support media production in 2020? Or is the ultimate aim for 3D movies to spring out of people’s heads fully formed?