what spatial interfaces can be

#
>

This is something that I will probably continue refining over time.

→
#
>

Overall, I want to talk here only about augmented reality. There is a whole world of possibilities in the fully immersive, VR space, but I am more optimistic about a world where our digital interfaces allow us to move and interact comfortably with our real world. This will allow us to ignore for now the problem of moving in these glasses. For that reason, I will not consider fully immersive VR in this current definition of “spatial interfaces”.

→
#
>

I am also not this version of augmented reality to rely on camera passthrough. I am imagining a version of these devices that feel no different than sunglasses that cover your whole peripheral vision.

→
#
>

Our minds are accustomed to three dimensions and object permanence

→
#
>

Mind palaces are a great example of how natural spatial reasoning is for most brains. We can easily memorize the act of mentally walking through a space. There is a natural, intuitive spatial sense humans have that spatial interfaces have the opportunity to take advantage of. These types of interfaces should be even more natural than

→
#
>

99% accurate occlusion will be the final step to this feeling like magic

→
#
>

we will be able to leave instances of apps open in rooms that we leave

→
#
>

We can take advantage of three dimensions for new types of communication

→
#
>

Video calls

→
#
>

3D data representation

→
#
>

Reliving moments

→
#
>

We can still compress information to two dimensions

→
#
>

We will be able to decide intentionally what should be represented in two dimensions vs in three dimensions

→
#
>

This will give us a new ability to blend between our Map and Territory. Both literally (overlaying map info over our view of the world), and figuratively ( blend information or representation with our senses of the world)

→
#
>

We will be able to build like never before

→
#
>

stabilizing an ethical platform for these interfaces will be hugely important

→
#
>

Shared experiences must feel like real life

→
#
>

relative perspective must be maintained

→
#
>

The problem of input and tactility

→
#
>

What new input methods will need to be built?

→
#
>

[Like you alluded, I think the most worrisome thing out of these reviews is the mention of the friction with using eye tracking as the main input method. I suspect our eyes happen to move more with our thoughts than with our actions, and we may have to align the thoughts and actions (and thus slow our thinking) to work with this interface. To be fair, this is kind of how the existing mouse interaction works, you can’t really click without checking that your pointer is where you expect it to be. But notably, proficient computer users try to stay on the keyboard as much as possible so they can have the comfort of separating actions from where their eyes are looking, which is more efficient. Perhaps the solution will lie in changing the interface (something other than buttons and text fields that’s not just voice), or in some new version of a mouse that allows us to move in 3D and use tactility to navigate the interface without having to look too concertedly.](graph/2024-02-01/66af8035-8883-453e-994f-00543684e7c9/66af8035-7913-4d4b-820e-5a28976697c9/66af8035-ca62-4cfd-b938-a011c86e2a62/65bd3221-af9a-4187-9c77-3dab3843f182)

→
#
>

We need to make sure this future is better

1
#
>

How do we solve the problem of taking off the glasses?

→
#
>

As mentioned in the vanity fair piece on Tim Cook and the Vision Pro, there is the risk of despair on stepping out of this digitally augmented world. Notably, this should not be considered the same as the difference between a fully immersive, VR world and the physical world, where they may have nothing in common. In this form of spatial computing, we are thinking about the physical world as is vs the physical world as augmented with digital objects.

→
#
>

One way to do this may be to limit how much you can “build” in your space. There may be design guidelines, or even hard restrictions on how much stimulation or deviation from the physical world are allowed in these digital spatial interfaces. We shouldn’t allow users to experience dopamine drop-off when “turning off” these interfaces. One way to get at this problem is with the thought: even when being worn, spatial interfaces should usually be off. This means that it should simply be a way to replace the phone pick up, not a way to simulate always having your phone up to your face.

→
#
>

Another way to do this is to not neglect our actual built environment.

→
#
>

All of the Experience Types must be supported

→