The problem with emotion-detection technology

It didn’t matter that there was no physical evidence that Amanda Knox had killed her friend Meredith Kercher: as far as those investigating the murder case were concerned, Knox’s behaviour proved her guilt. Knox was calm and collected when she was expected to be grief-stricken. She also acted in a way investigators thought was inappropriately sexual, kissing her boyfriend outside the building where Kercher had died while the investigation unfolded inside. To the police, and much of the media, this wasn’t how a mourning friend would behave. Giacomo Silenzi, who had been dating Kercher at the time of her death, was widely quoted saying: “[Knox’s] eyes didn’t seem to show any sadness, and I remember wondering if she could have been involved.”

Knox’s conviction – which was acknowledged to be wrongful when she was acquitted in 2015 – demonstrates the profound role that demeanour can play in the courtroom. “People have stereotypes about how a situation should be appraised and what the person should feel,” said Phoebe Ellsworth, Professor of Law and Psychology at the University of Michigan, “and if the person doesn’t fit that stereotype, they are judged more harshly.” A study published in The Journal of the American Academy of Psychiatry and the Law found that, as a general rule, a defendant who gives a convincing display of remorse is more likely to get a lenient sentence. Meanwhile, rape survivors who are visibly distressed have a greater chance of being believed by the jury.

No matter how hard we might try to hide our emotions, our bodies can give us away

In these legal battles, a person’s nonverbal communication can sway the outcome of a case. Fortunately, many people will never experience the ordeal of defending themselves against a false accusation or fighting for custody of their child. Nevertheless, in day-to-day life, there are often scenarios where some kind of emotional evaluation is needed to make an important decision, ranging from mental health screenings to job applications. Technology has advanced to a point where organisations no longer have to make these decisions alone.

Today, the market for emotion-detection technology is worth roughly $21.6bn, and its value is predicted to more than double by 2024, reaching $56bn. Firms can purchase systems to help them vet job applicants, analyse advertisements for their emotional impact and test criminal suspects for signs of deception. One company, Oxygen Forensics, has suggested its emotion-detecting software could be used by the police to help detect anger, stress and anxiety.

But experts on the ethics of artificial intelligence are beginning to sound the alarm bells. In December 2019, the AI Now Institute, a research centre at New York University, called for new laws to restrict the use of emotion-detecting tech, saying the field was built on “markedly shaky foundations”.

Poker face
No matter how hard we might try to hide our emotions, our bodies can give us away. “During a stressful episode, your body usually experiences a chain of physiological changes such as pupil dilation, deeper respiratory breathing, intensified beating of the heart and increased muscle tension, among many others,” said Javier Hernandez, a research affiliate at the Affective Computing group of the MIT Media Lab. “These types of signals can be readily captured with cameras and wearable devices and analysed with artificial intelligence to detect stressful episodes.”

There are a range of business scenarios in which analysing a person’s stress response could come in handy. The artificial intelligence program Cogito, which emerged from the MIT Human Dynamics Lab in 2007, analyses a person’s voice for signs that they are in a state of heightened emotion. The program has been distributed to dozens of call centres in the US to help staff detect when customers become distressed over the phone. “In the call centre, customer service phone professionals must often handle more than 50 calls per day,” said Steve Kraus, Senior Vice President of Marketing and Finance at Cogito. “The repetition and continuous demand for compassion can be draining, leading to cognitive overload and emotional fatigue and, eventually, burnout.”

Emotion-detection technology could also help advertisers optimise their content. Realeyes, a London-based start-up that has so far raised $33.8m in funding, promises to enhance video content by analysing its emotional impact on viewers. “Our technology is trained on the world’s richest database of facial coding data and incorporates more than 620 million emotional labels across more than 3.8 million video sessions,” said Max Kalehoff, vice president of marketing at the company.

The technology is already highly sophisticated. Realeyes recently updated its predictive modelling for behaviours like view-through rate and responses such as interest and likability. As more data is collected on facial expressions, the accuracy of this technology can only get better. “Today, there are a select few companies that have access to large enough data sources to successfully train systems,” said Kraus. “As the data can become anonymised and shared more broadly, it can accelerate the development of new and more powerful technology.”

The truth of the matter
In its annual report last year, the AI Now Institute warned that we still lack a scientific consensus as to whether these technologies produce accurate results. One of the key problems that Professor Kate Crawford, co-founder of the research centre, identified was that some firms were basing their software on the work of Paul Ekman, a psychologist who led a pioneering study on nonverbal behaviour in the 1960s. He proposed that there are six basic emotions that are recognised around the world: happiness, sadness, fear, anger, surprise and disgust. “His cross-cultural research had a huge effect on emotion research,” said Ellsworth, “as it provided evidence that the recognition of emotions from facial expressions was similar across cultures, and thus that the link between facial expression and emotion might be universal.”

Given the huge influence Ekman’s work had on the science of emotion, his ideas are pervasive in emotion-detection technology, informing many algorithms, including those sold by Microsoft, IBM and Amazon. His emotion-detection research was also used to develop training programmes for the CIA, FBI and US Customs and Border Protection agency.

If there is so much room for error when it comes to reading a person’s facial expressions, we must question how a machine can ever be programmed to get it right

However, more recently, psychologists have begun to question the methodology of Ekman’s study. One of his most vocal critics, psychologist Lisa Feldman Barrett, took issue with the fact that Ekman had provided the study’s participants with preselected emotional labels to match with photographs. Feldman Barrett argues that this meant Ekman had nudged participants towards certain answers. When she conducted the same test without labels, the correlation between specific facial expressions and emotions fell dramatically.

It seems emotions aren’t as universal as we once thought. For the Trobriand people – a remote society in Papua New Guinea – the ‘gasp face’, associated with fear and surprise in many cultures, is seen as a threatening expression. More broadly speaking, different cultures have varying expectations for when it’s appropriate to use a certain expression and in what manner. In the US, for example, it’s common to smile at strangers. However, in other parts of the world, frequent and overeager smiling is not so well received; one Russian proverb roughly translates to “smiling for no reason is a sign of stupidity”.

Ekman’s study is problematic not just because it underplays the importance of cultural differences, but also because it assumes that there is a correlation between someone’s facial expression and their emotional state. Researchers have since found the exact opposite: a recent study by the Ohio State University suggested that facial expressions are often unreliable indicators of emotion.

This presents a problem for technology that relies on facial expressions to reach a conclusion. In certain scenarios, misreading a person’s emotions can have serious consequences. Last year, at the borders of Hungary, Greece and Latvia, the EU practised using a lie detector to process arrivals. A journalist for online publication the Intercept tested the lie detector – called iBorderCtrl – at the Serbian-Hungarian border and, despite answering truthfully, the machine suggested they had given four false-positive answers out of 16.

If there is so much room for error when it comes to reading a person’s facial expressions, we must question how a machine can ever be programmed to get it right. It’s on these grounds that the AI Now Institute makes its recommendation that regulators should ban the use of emotion-detection technology in “important decisions that impact people’s lives and access to opportunities”. Until such a ban is in place and regulations are strengthened, it concludes, AI companies should stop selling emotion-detecting technology.

Taken out of context
However, some affective computing experts – such as Jonathan Gratch, a research professor of computer science and psychology at the University of Southern California – think banning the technology goes too far. “I agree with the spirit of many aspects of the report,” he told The New Economy. “But I am actually concerned about some of the ways they’re representing the science and their particular recommendation – I feel – is overly broad.”

Gratch’s main complaint with the AI Now Institute’s report is its failure to outline the difference between emotion-detection technology that takes context into account, and that which doesn’t. For him, this difference is crucial: “It’s the main challenge with the technology right now. Many of these companies do context-ignorant emotion recognition. So they would just take an image or video of a person and attend to the facial movements, but the algorithms have no knowledge of what the context is.”

When trying to read a person’s emotions, context is everything. If someone cries at the birth of their child, it’s likely to be out of happiness, not because they’re upset. If a footballer puffs out their chest and bares their teeth after they score a goal, they’re celebrating a victory, not squaring up for a fight. “So it’s only valuable if an algorithm can say, ‘I see this person is in a trial and I know what’s going on in that situation’. Then you might be able to make valid inferences,” said Gratch. What’s more, for the most part, we are deliberately regulating the signals we send to others. This has repercussions for lie-detection technology in particular. If the system’s goal is to catch someone in a lie and that person knows they’re being watched, then the accuracy of that result will inevitably be affected.

To Gratch, banning emotion-detection technology outright is not the answer, because doing so would imply that it’s inherently problematic to make inferences about someone’s emotional state. But we already use a variety of techniques to do this in many aspects of society: personality tests are relatively common in work environments and are used for employee development and as part of the hiring process; no one can be clinically diagnosed with depression or PTSD without an assessment of their wellbeing; and many criminal cases involve an evaluation of the accused’s mental state. There’s little fundamental difference between such tests and emotion-detection software, with the exception that the latter is more technologically sophisticated. “[The AI Now Institute] report can be seen to propose a ban on these techniques as well,” said Gratch.

Technology might be even more accurate than some of these techniques. According to a recent study, one third of the psychological tests used in US court proceedings aren’t generally accepted by experts in the field. Tess Neal, author of the study in question and an assistant professor of psychology at Arizona State University’s School of Social and Behavioural Sciences, explained that many of these are projective tests: “Projective tests require the clinician to show some kind of stimulus to the person being assessed – like an inkblot – or ask the person to create something – like draw a picture.” However, tests like this risk placing too much significance on the clinician’s interpretation of the person’s behaviour. “[Projective tests] are, as a group, less scientific than so-called ‘objective’ measures that are based on algorithms and data from large groups of people,” said Neal.

Feel-good factor
This brings us to an aspect of emotion-detection technology that is often overlooked: its philanthropic applications. Researchers in affective computing hope that emotion-detection technology will soon be able to use the information it has gleaned about a person’s emotional state and – by also taking into account the context – react in a way that helps its users.

“Electronic devices like phones mostly ignore how we are feeling and they almost always behave the same way,” said Hernandez. “This is very different from the way we usually interact with other humans. If technology could read human emotions and adapt its behaviour accordingly, it could not only achieve more natural and less stressful ways of interaction, but [could] also help [us] cope with other sources of daily life stress. For instance, if a mobile device detects that we are having a stressful time, it could filter out negative news, recommend listening to positive songs or recommend talking to someone close to us to help find relief.”

Companies have hastened to develop emotion-detection technology, but some have done so on the back of outdated theories about nonverbal behaviour

At the MIT Media Lab, Hernandez and his team have explored how smart cars could lower aggression in drivers by modulating the temperature and changing the interior and exterior illumination depending on their mood. Researchers also believe that emotion-detection technology could be used in mental health screenings to assist people in building their interpersonal skills or to improve remote learning by helping teachers understand whether students are paying attention and when they need help.

We are still some way off realising this technology’s potential, though. “We are years away from creating a system that generally guides behaviour as a human would, and may never actually get there given all the nuances in human behaviour,” said Klaus. “That said, detecting and guiding emotional intelligence can be effective if it is done within a specific context.”

Companies have hastened to develop emotion-detection technology, but some have done so on the back of outdated theories about nonverbal behaviour. In particular, the technology’s use within the criminal justice system is a cause for concern, given the early stage the technology is at and the severe repercussions that can be felt as a result. However, that should not legitimise a blanket ban on the technology. “Any kind of inference is always probabilistic,” said Gratch. “What you’d hope is that you can make better decisions using that information than not, even if it is probabilistic.” The benefits of using the technology may continue to outweigh the risks, but it all comes down to how high the stakes are if the technology gets it wrong.