The UX of the future combines the digital and physical world [3 guidelines]


I see them everyday. People walking around on the street, with a smartphone in front of their face. Checking their Facebook while participating in traffic. More than once I almost bumped into one of them, because they were immersed in the digital world in their smartphone. Why do we spend so much time staring at a screen? Why are we so focused on the digital world, that we completely ignore the physical world?

Lees hier de Nederlandse versie.


The digital era

Since the rise of the computer and subsequently the smartphone, digital screens are all around us. They are sending information, everywhere you go. While going to work, riding a bike, or even laying in bed. The digital world has become accessible practically everywhere and at every time. However, the interaction with these screens resides in the digital world. Screens require our full attention, so that no attention can be paid to the physical world anymore. That is why people riding a bike while looking at their phone often forget the other traffic around them.


A calm world

In the physical world, people are pretty good at combining tasks. You can eat an ice cream while walking down the street, without bumping into anything. That is because these tasks only require a small part of our attention span. To create a truly seamless user experience, digital information has to work similarly. People have to be able to interact with the digital world, without losing track of the physical one. Like Weiser states with his ‘calm technology’, in the future technology should remain in our periphery, the boarders of our (visual) attention. Only when needed, information should move to the center of our attention.


Designing for calm interactions

Nowadays, interactions are primarily aimed at two-dimensional interactions on screens. Through learned patterns we try to mimic physical interaction to some extend. Instead of that, the digital and physical world will have to merge together into one interaction-cycle. We have to design interactions that do not require our full attention, but can remain in our periphery. To allow us to keep doing other things. New technologies offer a lot of possibilities to aid this process. A few things are important for this.


  1. Design for a glance

Information in your periphery needs to be easy to interpret. No complicated or detailed information, but manageable information that you can process at a glance. Only when you are interested, you should be able to view the information in more detail.

Augmented reality offers the possibility to show simple cues in the corner of your eye. You don’t view the information through a window, but directly in the physical world. Because of this, there is a clear link with a physical object or action. The cue elf needs less detail. For example, a red arrow pointing at your freezer can remind you to get your meat for tonight out.

Physical ‘connected’ objects can also give meaning to information. In his book ‘Enchanted objects’, David Rose gives the example of a pill container. When somebody does not take his medicine in time, the cap of the container lights up to remind him about it. Very subtle, but effective feedback.


  1. Make it human

We interact with the world around us in a ‘natural’ way. By talking and gesturing. Not by clicking buttons. By designing interactions in the digital world that resemble interactions in the physical world, we will no longer need our focused attention to interact.

Through the introduction of chatbots, we can already ask for help in a more natural way. Recently for example, Transavia and Mirabeau launched a chatbot that allows passengers to buy their tickets through Facebook Messenger. But when we are able to actually talk to technology around us, like is already the case with Siri and Google Home, it starts feeling even more natural. If you can tell your home to turn on the light, you can just keep reading your book.

Augmented Reality also offers ways to use natural gestures to interact with data. You can tap on something in the physical world to get more information about it. And perhaps you can even communicate through facial expressions in the future. No more Tinder on your phone, but simply blinking at a passing hotty and you’ve got a match!


  1. Use all senses

In the real world, we are accustomed to multi-modal interactions. These interactions are a lot richer than pure visual interactions. Think about pushing a button on your old stereo. You could feel the pushing of the button, and hear a confirming ‘click’, even before the song started playing.

Haptic feedback can offer more subtle information, without asking for your full focused attention. It is a form of kinesthetic tactile communication, or touch. Materials for example can adjust to a certain situation, or smart clothing can vibrate softly to point out something in the environment. Also, haptic information remains private more easily. Think about a debit card that reacts to your bank account. If you have little money, the debit card is soft and flabby.

Auditory feedback can also help to not require your full attention. A sound to confirm your action means you do not need to look at the interaction. By designing sounds correctly you can understand what is happening, without getting distracted.


So, what now?

What does this mean for us as designers? In the future there will not be just one interaction style. People interact with digital information in the whole world around them. More then ever, we need to look at their entire experience. What information does somebody need, and at what moment? And how can we deliver this information in the best way? Imagine, it’s morning and you are running out of the door in a hurry. An umbrella that lights up or makes a sound (example by David Rose) is more likely to point out to you that it is going to rain than a message on your phone, that is already in your pocket.


Bye bye screen!

Offering contextual information via calm interactions in your surroundings is the future, in my eyes. So, start thinking outside of the screen, and look at other possibilities for interacting. Only by combining all these technologies into one big user cycle, we can really create a more user-friendly (digital) world. So that we will stop looking at a screen all day.



Further reading:

David Rose (2014) Enchanted Objects. Simon and Schuster.

Weiser, M. (1991). The computer for the twenty-first century. Scientific America, 265, 94-104.

Weiser, M. & Brown, J.S. (1997). The coming age of calm technology. In: Denning, P.J. & Metcalfe, R.M. (eds). Beyond calculation: the next fifty years of computing. Springer, New York, 75-86.