It's my humble opinion that Apple has been the clear victor in the battle to create an easy to use computer interface, however there are still elements that just don't make sense to everyone. Is it even possible to make a user interface that everyone can easily understand? Given that "It is impossible to make anything foolproof, because fools are so ingenious.", I doubt it. But I believe you're only saying it's impossible because no one has ever done it before. (major geek points to anyone who gets both references)
Let me start by admitting that I watched Discovery's "Curiosity: Did God Create The Universe" with Stephen Hawking last night. One of the things that Stephen drives home is that the laws of nature, or physics, are immutable and cannot be broken. They are also universally understood and obeyed. Keep that in your back pocket. I'm coming back to it.
Earlier today I logged into an OS X computer to assist a customer of mine and noticed something brief, but odd. The global menu bar at the top of the screen wasn't visible, yet its shadow was. Clearly this breaks a few fundamental laws of nature and would never happen in the physical world. So about 0.75 seconds later, I found that I had resolved that issue by tying Professor Hawking's cautionary notes about the laws of nature into a desktop environment. It was a desktop environment that embraced the laws of nature instead of poorly mimicking them or outright ignoring them.
The easiest way to define the desktop environment that I envisioned is to remind my imaginary development team that two objects having mass cannot occupy the same space at the same time. That and to force them into believing that computer graphics have mass. This is a law that even my fish understands; he knows that he can't go through the rocks at the bottom of his tank nor the glass that surrounds him. He might not understand exactly why, but it makes sense to him that he cannot pass through those things. He also understands how to move water around him, not through him, so that he can swim. Try to apply this rule to any part of the current desktop/windowing systems out there and I think you'll find that they break that law in almost every instance. Icons overlap each other on the desktop and neither one is clearly on top of another; two inactive windows are positioned behind a third window in such a way that it is unclear which one is in the middle and which one is behind the other two. Why have we complacantly put up with this for so long, realizing that our interactions with computers have no analog in the physical world.
The Missing Z Axis
Every windowing system attempts a half-hearted implementation of the z axis, as do some applications such as the browser your using right now (think z-index). The problem with the current system is that it doesn't provide a truly 3 dimensional experience for the user. Instead, it allows objects to be stacked in a particular order in 2 dimensions. But what if we introduced three extra attributes to those objects?
- Space: Everything we interact with is surrounded by space. We reach for things through space, we pry things apart by inserting a tool into a space too small for our fingers, we expect and understand space.
- Perspective Scaling: When things get further away from us, they appear to shrink. When they are getting closer, they appear to grow. Virtual objects should therefore also shrink and grow as is appropriate for their distance from us.
- Visual Clarity: This is one area where modern windowing systems are getting close, but still don't quite have it down. The blur effect is somewhat annoying in its current state, but that's because most implementations are boolean; blur on vs. blur off. But if we have 20/20 vision and something is 21 feet away from us, it starts to appear a little fuzzy. Objects that are 50 feet away are increasingly fuzzy. So let's take the blur effect and make it a little more intelligent. For a spiffier desktop environment, we could even introduce a lightweight particle system to simulate the particles that make things appear lighter (assuming a sunny day) the more distant they are.
But 3D Desktops Suck
Yep, they do and yes I realize that I'm describing one. I strongly believe the reason that they suck is the same reason 2D desktops suck: They are attempting to look interesting but don't actually remind us of anything in the physical world. Developers and engineers aren't to blame here. We simply don't have a holodeck yet, and until we do 3D is going to be a pretty tough sell for our senses.
Okay, so why am I proposing a 3D desktop if I think they suck? The answer is simply that I don't want a 3D desktop. What I want is an experience akin to a surgeon using a camera inside a patient to see where to cut. In other words, I want a 2 dimensional interface to a place that my mind can believe is 3 dimensional. No, it's not ideal, but it's a better use of current technology than what we're doing with it right now.
Interacting With a 3D World with a 2D Interface
We already know how to get from point A to point B on a cartesian plane; you just move your mouse and your cursor moves to that point. But how can we easily move along the z axis without special hardware? I'm sure there are plenty of ideas out there, but one of them might finally be a good use for "hot corners", or maybe even hot 'edges'. Say you want to zoom out to a window behind you, maybe you move the mouse to the bottom of the screen or to one of the bottom corners. Zooming in could be done by either moving your mouse to the top of the screen, or by simply clicking on a window that is in the distance. To make zooming a little more friendly, we can take a page from Apple's menu scrolling engine wherein scrolling starts slowly and immediately when the mouse enters the appropriate area of the menu, then increases in speed as the mouse is held in place.
Back to the Shadows
Remember way back at the beginning of this post how the whole idea started because I saw a shadow with no object? Let's wrap this up by getting back to that. Current window shadowing techniques generally only have 3 possible states: A bold shadow for the active window, a not-so-bold shadow for all of the inactive windows, and no shadow at all for those utility windows. You may have noticed as you peel your eyes away from the computer screen that this is yet another area where there is no real world analog. But hey, we've just described a 3D space for those windows which means we should be able to fix this problem as well.
The existence of a shadow implies three things; a light source, an object with mass in the foreground, and an object with mass in the background. Current shadowing techniques have, or at least imply, all three of these things. But where is that light source exactly? It's almost directly behind our heads. Every real-world scenario that I can come up with where any light source would be behind my head involves a face-plant, which I'm not terribly fond of.
We've got a 3D space for windows
and their shadows now, so let's move that light source to someplace a little more natural, someplace a little higher above us and maybe off to one side a bit. Heck, it could even move around a bit to correspond with the time of day. We can then project the shadows on the floor of our 3D space. In this instance, the window that is in focus wouldn't have a shadow at all while windows in the background would. Think of a scenario where you're reading a newspaper while waiting for the bus. The newspaper has no shadow that you care to acknowledge because its shadow is being cast outside of your field of vision. However, if you peer around your newspaper for just a moment and notice another person standing just like you, reading the same paper that you're reading, you
can see the shadow that their paper casts on the ground or perhaps on the other person's legs because both the paper and its shadow fit within your field of vision.
So let's move the shadows down to the ground where they belong, and lets let the things we're looking at feel like they are really right in front of us while everything else is far away. For a little per-widow depth, we could add subtle widget shadows. For example, I can see a shadow being cast in Safari right now that is meant to make the tabs appear to be beneath the address bar and slightly embedded. Let's keep that going, but make it adhere to the position of the light source so it makes sense.
Fin
So there we have it. A poorly described, but fairly thorough thought experiment into a 3D desktop with a 2D interface. You know what would really make this idea cool? Using it on a tablet that supports multitouch. I'd imagine you could zip around quite nicely within 3 dimensions on one of those things. If anyone decides to implement such a concept, I'd love to know about it and help test it. I'd also gladly accept royalties or shares in your wildly successful company. ;-)