Face as the New Data Point: The Good, Bad and the Ugly of Facial Recognition Technologies

By: Riku Vassinen, Head of Digital Transformation, JWT Singapore

Riku Vassinen, Head of Digital Transformation, JWT Singapore

Although facial recognition technology has been around from 1960´s, it was again Apple who made this technology household name. After the launch of iPhone X, facial recognition opportunities have truly hit the mainstream. Besides locking your phone with glance, everyone is currently looking for possibilities to add face as the new data point.

"As marketing has become more data-driven and technology-driven, facial recognition is definitely an avenue that forward-looking brands should explore"

Facial recognition market is expected to garner $9.6 billion by 2022 growing over 20% annually. All the usual big technology players (Amazon, IBM, Google, Microsoft) are developing their facial recognition technologies. There is also a vibrant scene of companies specialized in Facial recognition (Kairos, Affectiva, Face++). Currently the highest-valued AI start-up is Chinese facial recognition technology company Sensetime. The company is working on facial and object recognition technology for Chinese government that accurately can spot people using security cameras. They are also collaborating with Honda on object recognition for its driverless car research.

It is clear that facial recognition provides effective tools for finance (payments with your face), law enforcement (criminal identification) and healthcare (identifying illnesses from patient´s features). As marketing has become more data-driven and technology-driven, facial recognition is definitely an avenue that forward-looking brands should explore. Facial recognition technologies are not without their threats both to consumers and companies. I have compiled some of the best use cases for facial recognition and also analyzed potential risks you would need to watch out:

Good: More effective targeting and personalized content

Using face as a data point enables marketers to target their audience more effectively. From 2013 Tesco has been piloting digital screens in gas stations that show ads based on customer´s gender and approximate age. Facebook has filed patents for technology that tailors ads based on user´s facial expressions.

Facial recognition does not only enable brands to do more relevant advertising. It also provides new additional data point to evaluate the effectiveness of your marketing. You could analyse how many people see your outdoor ad and what are their feelings. Or like Walmart, you could analyse how many people leave your store unhappy.

Facial recognition provides much more reliable data point for marketing research than traditional focus groups. By some estimates, over 90% of our communication is nonverbal. It is easier to lie that you like or dislike an ad compared to laughing or crying to ad. Facial recognition provides opportunity to understand your customers much more deeply.

Effective marketing is not only great targeting and developed measurement, it is also tapping in emotions. Some brands have already used facial recognition technologies in ways that are touching, funny and buzzworthy. For this year´s SXSW, Cheetos created Cheetosvision; an app which created portrait of you entirely composed of Cheetos cheese snacks. Nike demonstrated flexibility of their new Nike Free sole by campaign where you controlled the movement of the shoe by squirming your face. Plan, the global children´s charity, used gender recognition in their digital outdoor screen to highlight the inequality girls face in developing countries. And South African coffee company Douwe Egberts gave out free coffee for people they recognized yawning.

Bad

As many other machine learning algorithms, facial recognition algorithms are not free of bias. Algorithms are mainly build by white men (and to lesser extend Asians) and training sets usually consists of their equally white young colleagues or student friends. Not surprisingly facial recognition accuracy is really high for white young males. In MIT study testing face classification algorithms from IBM, Microsoft and Face++, all of these algorithms had significantly lower accuracy when evaluating dark female faces. Although many facial recognition systems boast accuracy rates near 90%, the accuracy falls significantly when they venture away from light-skinned faces.

For marketer the bias can potentially cause mis-targeting and potential backlash. Still failed facial recognition in marketing is annoying but relatively harmless for the customer. Bias can have more severe repercussions in other fields of facial recognition. There are currently estimated 117 million American adults in facial recognition systems in US law enforcement. In these cases accuracy error might get a person potentially to death row.

Ugly

Evolving machine learning will also mean that it will be easier to use your face in dubious purposes. This year there has been a surge of “deepfakes”, videos that have swapped faces. They are done with open-source program Fake app. You feed the program hundreds of headshots and it will create fake video. Not surprisingly the first videos were fake porn videos starring female celebrities such as Gal Gadot and Taylor Swift. Celebrities, politicians and other public figures are more prone to this kind of malicious content as there is vast amount of images of them available. The threat is not limited only to them. We are sharing more images of ourselves in Instagram, Facebook and other social media platforms. This makes creating a training set easier even for a non-celebrity. Potentially these fake videos could be used to blackmail and spread fake news about you. Currently deepfakes are relatively easy to recognize but as algorithms improve, they will become more and more authentic.

Facial recognition can make your life easier, whether it is walking to your work without wearing any badge or paying for your lunch only by smiling. At the same time your face will be more vulnerable to be used in ways that can harm you. Already 2 in 5 Americans have been victims of identity theft. In the near future we will have instances that your face identity has been stolen.

Face is an extremely valuable data point for marketers. With proper use of facial recognition technologies, marketers can provide more relevant and personalized content and improve their marketing efficiency. Face is also very intimate data point, so we marketers need to take privacy and data concerns seriously.