Emotion Recognition for a Consumer Bank
Emotion recognition (computer vision)
4+ developers and QA engineers, 1 business analyst
Python 3, OpenCV, Tensorflow
Recognizing emotion is becoming an increasingly prominent factor in financial decision‐making. From shops to consumer banks, many retailers continue introducing this feature across channels to gather complete and unbiased responses.
Our client, a mid-sized retail bank, wanted to improve their reps and administrators’ communication in the bank’s brick-and-mortar offices across the country. Their target was to assess and improve their front office staff’s existing mechanisms and guidelines.
The traditional lists and surveys proved inefficient. It meant the bank was looking to go for computer vision (CV) tech to a) receive immediate feedback b) get its fast and unbiased evaluation. They planned to use the results to find the bottlenecks and improve the existing practices.
Solution and key capabilities
Our team offered to create a comprehensive CV system based on video footage and audio records, taken on customers’ consent. The solution should be capable of the following:
- Identifying a person’s facial expression
- Comparing it against the existing database
- Recognizing the specific emotion (happy, neutral, unhappy)
- Building and delivering reports highlighting mood shifts
Firstly, the development process started with identifying the customer’s essential requirements. The parameters included the emotions to recognize, the system’s throughput, reports generated, etc.
With the spec complete, we created a complex emotion recognition system. It was capable of fast processing the recorded video data, detecting faces, and analyzing their reactions. According to this data, our model determines the emotion. The software then sends this information to the data center, where it will be processed and converted to conclusions and statistics.
Based on our client’s needs, we classified emotions on people’s faces into one of the three categories, using deep convolutional neural networks. We trained the model based on the companies in-house dataset. It consisted of 11231 images of human faces conveying one of the three emotions: unhappiness, happiness, and indifference. To get access to that information, our developers created the CV reporting module. Lastly, they integrated it with the client’s existing infrastructure.
CV emotion recognition has become a valuable tool in our client’s sales and marketing activities. Firstly, cameras record movements in faces. Then, the footage is then analyzed to measure expressions and assess moods and immediate reactions. Finally, the solution also identifies the age and gender of the people researched.
The client employs the tech to analyze using new banking products and client feedback sessions, introducing powerful user response insights to new product rollouts. The solution now detects emotions during the product testing sessions with the client study groups, average the various reactions, and identify the prevailing types.
They also use emotion recognition for market research, including events organized for their prospective clients and existing client base. The CV emotion recognition instruments analyze how well the audience receives new ideas and banking products, improving offerings and customer journeys. It is beneficial for exploring mobile banking products’ feedback as most mobile devices today have built-in cameras, with no need for extra hardware.