Valentina: the virtual agent who assists you with banking operations

She has just appeared in Bper’s Innovation Room and everyone is already talking about her. Valentina is the virtual assistant we created for and with Bper, together with the support of other partners. Lately, you can read about Valentina in all the sector publications, such as “Affari&Finanza” of Repubblica or ItaliaOggi of Milano Finanza.

Valentina is a digital humanoid created as part of a larger project, the Innovation Room, which we have been running with Bper for several years. The Innovation Room is a physical and ideal space in which to experiment, create and prototype innovative solutions in the field of financial services. Valentina is the latest idea developed to offer customers a service totally focused on their expectations and needs, thus providing an effective customer experience. The virtual assistant guides the user through the branch, even outside opening hours, helping him to carry out his banking transactions.

In the following paragraphs, we will illustrate the Innovation Room 3.0 project, focusing on:

• The virtual assistant and the development of its capabilities
• The BGarden, an environment in which the agent enters the scene.
• Conversational scenarios developed

VIRTUAL ASSISTANT

The first step in the realisation of the virtual assistant was the purchase, through a specialised gaming company, of a basic model which we went on to detail and animate using the Unity programme. The starting point was to create as realistic a human figure as possible, taking into account aesthetics, animations and facial expressions.

The animations were recorded using the motion capture technique, which uses special sensors placed on the limbs and head to capture the movements of a real person and transfer them to the 3D model.
As well as this, Valentina’s face has been made even more lifelike thanks to facial animations, which are essential for conveying emotions and feelings.
Lastly, lip synchronisation, i.e. the correlation between the position of the lips and the sound emitted, or more specifically, the matching of visemes and phonemes, was taken into account.

Valentina was given two of her five senses: sight and hearing.
Thanks to a grid of eight directional microphones, which capture sounds throughout the Innovation Room, Valentina can receive sound stimulations and process them. The grid firmware is able, among other things, to estimate the position of the sound source. Valentina is then able to turn towards it.

In Valentina, the voice could not be missing: it was created through Amazon Polly, the voice synthesis service that uses advanced deep learning technologies. To reproduce a sound as close as possible to the human voice, we set the speed, pitch, timbre and phonetics.

BGARDEN

The “BGarden” is an environment attached to the classic branch, designed to allow customers to visit the bank at any time in total freedom and safety. Customer-only access is guaranteed through the use of the Smart Ring and biometric recognition.

The Smart Ring allows the user to log in and be recognised by the bank system by detecting the registered fingerprint. This allows the voice assistant to recognise the customer, access their profile and thus offer a complete personalisation of the service and a more empathetic approach.

The BGarden consists of two macro areas: Welcome Area and Self Area.In the first area, the client is welcomed by the agent who, projected on a polarised film about two metres high (HD projection), entertains the client by giving information about his role, the BGarden and proposes activities to entertain the user. The client’s presence and position are detected by Valentina using a Computer Vision system which is developed using four cameras positioned in the room. They allow Valentina to understand not only the position of the user, but also the direction of the speaker’s face.

This is essential to understand whether the user is talking to you or to other people in the room. The second part is the Self Area. The assistant appears half-length in a 3D holographic pyramid, projected in 4K.
In this area, the customer can carry out various operations, such as applying for a loan or withdrawing money. It can also confirm and validate operations via the Smart Ring. The holographic pyramid, the largest in Europe, was built and customised to our needs by an external partner.

CONVERSATIONAL SCENARIOS

Conversational design was a key element in designing a smooth and enjoyable user experience. This is why the dialogue creation required a major preliminary study of human interactions, as well as the evaluation of all possible ramifications that could be covered through questions and answers.

And that’s not all: the study and research into the phrases to be spoken by Valentina was combined with further research into the gestures and expressions that accompany speech, all to create a virtual assistant that is as human as possible. The discussion was carried out through a chatbot on which, over the months, important training was carried out by the CNR, the National Research Centre in Naples.

We decided to create three different conversational scenarios, each with a different purpose. The first was mainly demonstrative, the second was to allow testers to try it out, and the third and final was purely explanatory to highlight complexities and technologies.

VALENTINA IN OFFICES

The list of technologies and devices used to create Valentina is very long and spans several areas of digital innovation. In particular, reference is made to many of the so-called “invisible technologies”. Many tools and different technological solutions intertwine in the Innovation Room. The phases mentioned in this article are just some of the elements of this project, which will soon go beyond the experimental stage. As a matter of fact, the debut of the virtual assistant outside the Innovation Room is scheduled for 2021 at Bper Banca ATMs.

Learn more about Valentina.