Voice input facilitates documentation - A hackathon topic

Cayas Software GmbH plans a team event every year. This year it was a hackathon that took place in the Thuringian Forest. Each team member presented an idea that could be worked on during the hackathon. The idea of simplifying the entry of food eaten and drunk in a fitness app was nominated for realisation. The background to this idea is that since the reach of e-health applications has visibly increased. Digital health and fitness solutions are reaching more and more people and supporting them with their health. Blood pressure, heart rate, oxygen saturation, steps, sleep patterns, calorie consumption, stress and other health parameters can be measured. But what is missing from all these control options? That's right, the documentation of the food we eat and drink every day. Because nutrition has a significant influence on our health and well-being. Just like sporting activities, a healthy diet promotes good health. The idea of having to type every food item into an app doesn't just sound annoying, it actually is. To simplify documentation, it should be possible to record this in the app via voice input. Everything that is eaten or drunk is spoken into the app, recognised by the system and saved.

These first words serve as background information on the idea. In this blog article, I would like to shed some light on the area of design and show what steps were taken to develop a user interface design for this task.

• Define usage situations
• Sketch out ideas
• Colour and font selection
• Result

Define usage situations

As a first step, it is important to list which elements are required to fulfil the task. Here it is helpful to imagine several scenes in which food recording by voice input could be used. The usage scenarios considered provide information on what needs to be taken into account in the design to ensure problem-free and appropriate use.

1 Scenario: Lunch is over and on the way back to the office you document what you have just eaten and drunk in the app.

The challenges that arise here are, on the one hand, that the input takes place while walking and therefore the mobile phone is not held still in the hand. Secondly, ambient noise will interfere with speaking and understanding. This means that the wrong food can be heard or nothing can be understood.

This usage scenario gives rise to the following requirements:

• The interaction elements must be large enough so that they can be interacted with while walking, i.e. while moving.
• The audio information must also be visually recognisable.

2 Scenario: The documentation of the food takes place during make-up in the bathroom.

The following requirement arises from this usage scenario:

• In this case, it is important that the voice input can be made without having to press the input button continuously. This leaves your hands free.

3rd Scenario: The entry is made directly after the meal in the restaurant

The following requirement arises from this usage scenario:

• In a restaurant, you don't want to speak aloud into an app what you have just eaten and drunk. In such a situation, it is convenient if the entries can also be made using a keyboard.

These are just three of many possible situations in which the documentation of the food consumed can take place. However, for a first draft with the basic functionalities, this is sufficient for now. To make it possible to chat about and list documented items, the screen needs to be able to be elegantly split into two parts. As soon as a food item has been entered and recognised, it is listed in the second part of the screen. Once the input is complete, all the foods entered are displayed in an overview. The different food categories of the food pyramid should be displayed in different colours.

Sketch out ideas

Scribbles are ideal for initial ideas on the way to a design. They are simple sketches that help to develop ideas and clarify the structure and functions of individual pages. The advantage is that they can be created quickly and therefore many different ideas can be developed. There are hardly any graphic specifications in a scribble. This means that form and colour do not play a major role at this stage. Due to the imperfect scribble, there are no inhibitions to make changes and tackle new ideas.

Colour and font selection

As the planned app is to be used in the area of food documentation, we looked for suitable colours in the food colour box. The following colours are available (from left to right)

Cucumber green, milk white, tomato red, lemon yellow, raspberry pink and chocolate brown With the background that the entered food categories are displayed in different colours, the app colour selection should be reduced in order to create calm and focus. The basic colour of the app will therefore be milk white and the action elements for inclusion will be cucumber green and tomato red. For the different food categories, additional graphics of foods for the different categories were created to support them even more visually.

As with the colours, the typography also needed to be clear and simple. The fonts Montserrat and Roboto (green, from left to right) were therefore chosen. Roboto is used for the chat messages, the listings and the overview of the recorded food. Montserrat is used for all other texts such as headings and other continuous text in the app.

Result

The app function that was the focus of the hackathon was to create a way of documenting food eaten and drunk by voice input. The design result of the hackathon comprises three screens, which contain the main functionality that was worked on.

The start screen can be seen on the left. If the record button is pressed or swiped upwards, the recording starts and the food eaten and drunk can be recorded. If a voice input is not recognised or requires a prompt, this is provided by the system. If the food is recognised, it is listed at the top of the app and the previous chat history is deleted to make room for the next entry. If there is an error, the listed food can be deleted directly from the list. To end the documentation, tap the stop button. All foods entered are now displayed in an overview. The basic colour of the app is not too dominant, so that on the one hand the main function of the recording is in the foreground and on the other hand the screen is not overloaded with colour as soon as the food categories are listed. By enlarging the curved chat area as soon as an entry is started, there is a clear separation between the active communication area and the passive documentation area. This app has potential for further development. There are other useful functions in the pipeline that could be integrated here.