Cubic.ai, a Smart Home application

Cubic Robotics was founded by the talented Yuri Burov, whom RBC included in the list of Russia’s best IT entrepreneurs.

Once I already did a project for Yura related to robotics — an interface for a telepresence robot. The project did not go into mass production, but its developments seem to be used by large companies in voice control systems.

Then the team thought that the equipment was too complicated, moved to California and decided to create an application to control a smart home. The project was named Cubic.ai, because the first prototypes of the hardware assistant were in the form of a cube.

When I was invited to develop the interface for the application, the team already had a minimal design ready and a visual concept was developed. The whole story revolved around the cube. The name was born from the form, the logo in the form of a cube in isometric projection appeared from the name:

When a designer has found such a coherent concept, half the work is already done. Even if the concept itself is controversial, consistently applying it to product design can achieve a much better result than drawing everything out of sync. A systematic approach to design works much like statistics in the stock market. The strategy can give a loss for a long time, but the total amount of deals will be positive.

The designer of the Cube got the logo right and developed the idea further, turning it into a virtual assistant button, like Siri.

Then the designer made a set of icons in the same style. The essence of the application in managing a smart home. Smart devices from different companies: colored light bulbs, thermostats, gas leak sensors — were combined into a single control panel. For each device, it was necessary to draw understandable icons that would have a status indication.

These are the icons the designer came up with for the thermostat. The shape of the icon is again taken from the logo, while the current temperature and, for example, air humidity put inside.

A system of rooms was made up for the app. It was possible to group devices by type: light bulbs, thermostats, and so on. But it is much better to create an association with the user with his home. Let there be such sections: living room, hallway, kitchen, bedroom. Then the control panel, which Cubic wanted to be, will feel like a real house.

For the names of the rooms, they came up with mockups and designed several icons in the same style as the devices — in the form of a cube:

It was somewhere at this stage when I joined the project.

Man sees the sound

Paradox: the idea of a voice gives unambiguous visual associations.

Voice is sound. Sound is vibrations of the particles of the medium. When we speak, air is expelled from the lungs, and the vocal cords contract and expand with the help of muscles. Due to this, the air molecules vibrate at different frequencies, and these vibrations reach the ears. There, air waves hit the eardrums, and our auditory receptors pick up these vibrations, and the nervous system converts them into a kind of electrical signal.

Hearing works in a completely different way than light. Light exists on its own, it is a stream of particles — photons. Sound is vibrations of anything: air, water, a piece of iron. Therefore, light can propagate in space, but sound cannot: there is no transmission medium.

A person does not intuitively understand this difference, therefore we represent both light and sound in the form of waves. In addition, our brains can confuse these two sources. A well-known fact: under the influence of LSD, a person hears light and sees sound. I checked. This is why it is so easy to represent sound waves as light waves. Animation of any wave that represents a sound recording has the same effect as LSD: we seem to see the sound.

Absolutely all voice assistants use this effect. There is not a single assistant where sound and light animation would not be used.

It’s not just about beauty. The point is feedback, which must be in the interface. If there is no feedback, then the person thinks that the interface is frozen.

Feedback is different for different levels of interaction. I think there is a rule here: the longer the interaction, the longer the feedback should be. Here, for example, everything is very simple for a button. We tap the screen for a fraction of a second, and the button becomes a slightly darker color for a fraction of a second. This is enough to indicate that the button has been pressed. We talk much longer. The commands that we give to the virtual assistant can last 10-20 seconds. And all the while, we need constant feedback. It will not be enough to highlight the voice assistant button with an inner shadow like it is pressed. The application may freeze after pressing the button without the user realizing it. So feedback for speech recognition needs to be continuous.

This is what animation in the form of waves is used for. It is desirable that this animation react to voice changes. The louder we speak, the larger the wave should be.

Therefore, even for a prototype of a voice assistant, animation is a must. And I proposed such an animation for Cubic. The animation of the Google Assistant came up as a basis. Cubic was released only for the Android platform, so such an animation would look organic, and it could be done quickly.

The same waves, only white, I placed in the icon of the cube. It was the simplest solution. The animation didn’t respond to voice changes, but the fact that it was animated was enough to give feedback.

Then the programmer intervened and said that although it was difficult to track voice changes, he could easily distinguish silence. Then the idea came up to add a second feedback when the person has already pressed the button, but has not yet begun to speak:

We got together with the team and thought: is there still a third state? It turned out that there is. When the command is given, the application needs up to 10 seconds to contact the server and recognize it. Therefore, one more indication is needed to show the user that his command has been accepted and is being processed:

This set of three indications is the bare minimum for any, even the simplest voice assistant prototype. Without this three-step feedback, you can’t release an app at all.

Thermostats

The feature of Cubic is to support different devices from different vendors in one app. Usually, each company releases a separate application for its devices, and it is inconvenient to control different devices.

Therefore, for Cubic it was necessary to come up with a bunch of different universal icons. The first version supported: Nest thermostats, smoke and carbon monoxide sensors, moisture and water leakage sensors, motion sensors, Philips Hue colored lights, door locks and smart sockets.

Let’s start with thermostats. Every American home has a thermostat, but it is manually controlled. New thermostats from Nest have been installed in homes recently, and they work through the application.

Thermostat regulates temperature, so the main indication in the icon is Fahrenheit degrees. The thermostat also has a target temperature that it must maintain in the house. Let’s say the apartment is 68 degrees Fahrenheit (20 degrees Celsius). We set the goal: to heat the air to 72 degrees, the thermostat starts working. The main problem: what temperature to display on the display?

Most cheap air heaters show the current temperature and therefore there is no feedback. How to understand that the thermostat has received the command to heat the air and is working? The target temperature must be shown. Not instead of the current one, but in addition to it, because otherwise we won’t know how many degrees are in the room now.

The display should show both temperatures: current and target. If they match (or there is no target temperature), then you need to display only the current temperature and highlight the thermostat, for example, in green.

By highlighting the thermostat in green, we intuitively introduced the concept of the norm. Let’s develop this idea. If there is a normal temperature, then there should be the concepts of “cold” and “hot”. Since we decided that the normal temperature comes when the current and target temperatures are the same, we can draw a logical conclusion: hot is when the current temperature is above the target, and cold is when the current temperature is below the target.

So there are two new colors: orange and blue.

There is another approach: to highlight the orange thermostat working for heating, and the blue one working for cooling. This approach works until we introduce the concept of overheating and undercooling.

Norm, heat and cold are relative concepts, but there are universal human limits. If the apartment is more than 95 Fahrenheit (35 Celsius), then this is overheating. Similarly, a temperature of 42 degrees (5 Celsius) is hardly acceptable in a residential building: this is hypothermia.

Overheating and undercooling should be reported to the user as a critical error requiring attention.

Indication of this level is attractive, so you need to expand the color coding. Let’s introduce red and blue colors as signs of a serious temperature deviation from normal, and in order to attract attention, we will fill the icon with color.

Here’s the approach with highlighting the heating/cooling mode instead of the hot/cold state and stops working. If the apartment is overheated and the thermostat is cooling, the icon will be highlighted in blue. If there is hypothermia in the apartment and the thermostat works for heating, the icon will be highlighted in orange. Thus, we’ll give false information to the user, as if there is no problem.

This is the biggest mistake! One of those that lead to accidents at nuclear power plants and plane crashes. Gotcha? But at the beginning it seemed that this was an insignificant nuance.

In addition to the temperature sensor, Nest thermostats have a built-in humidity detector. This functionality is rather additional, so the humidity can be displayed when tapping on the icon, with one exception. If the air is too dry or waterlogged, this is also quite dangerous, as it can damage sensitive equipment. Therefore, overdried and overmoistened air should be indicated in the same way as overheating and hypothermia.

The color coding is already taken, so let’s introduce an icon representing a drop of water, and we will display it large instead of the temperature. Let’s place the temperature under the icon.

If the apartment has problems with both temperature and humidity at the same time, it is better to give priority to temperature and display degrees, and place the drop icon down or remove it in the settings window. I didn’t elaborate this in the first version of Cubic.

Here is a set of all thermostat states:

Smoke detectors

Nest also makes smoke and carbon monoxide detectors. Both smoke and carbon monoxide are detected by the same sensor. There are 3 hazard levels for each leak: low, medium and high.

Color coding will be expanded. We’ll use red color for a high degree of danger, and display the low and medium degree in yellow and orange.

One sensor is responsible for 2 types of leakage. Therefore, we have to introduce additional coding using icons. The icons are not that complex, but we can’t just take one for smoke. It is usually depicted as stripes, which don’t convey a sense of danger at all.

Therefore, it is better to take the fire icon. This is not a direct association, but the fire is much better recognizable than three strange lines. And it looks more dangerous.

There were many icons depicting fire. I liked three. You can choose one of them, or you can take all and substitute according to the degree of danger:

There is no association at all for carbon monoxide. Carbon monoxide (chemical formula CO) is a colorless, odorless and tasteless gas, from which you can buy a farm and not notice. It’s probably better to take a scarier icon.

Trying icons:

Looks quite dangerous.

It may be that at the same time a small fire and strong carbon monoxide. In this case, the color of the icon should be made red, according to the highest degree of threat, and both icons should be shown.

Other icons

In addition to thermostats and smoke detectors, for the first version of Cubic, I prepared icons for light bulbs, a motion sensor, a water sensor, a door lock and an outlet.

Whole set of icons:

Devices are combined into rooms and placed on the main screen of the app:

Dashboard

Cubic is a free app, but with a paid option. For $10 a month, the user gets a dashboard — a system of notifications and tips. The dashboard will send notifications to your phone if something is wrong with the house, as well as give advice on what to do during a threat. For example, if there is a carbon monoxide leak, you need to open the windows.

Access to the dashboard is given from the main screen of the application. The Dashboard button is displayed on the menu bar.

In case of a threat, the dashboard call button should be highlighted.

The indication in the dashboard itself explains the statuses of all devices. In future versions, Cubic will support surveillance cameras, which will also be available in the dashboard.

Other sections

Light bulbs and thermostat control:

A built-in marketplace is one of the most popular monetization methods:

Screens for adding a new device (Philips Hue):

A concept of the voice assistant built into the system:

App’s fate

The app was released in 2017. Several American media have written about it. After a year of work, the project could not reach payback and was closed.