Percepta

Percepta is a company building ethical AI for retail shoplifting detection.

A Coming Dystopia?

The dangers of surveillance have long been feared, perhaps no better articulated than in George Orwell’s 1984. Add the power of artificial intelligence to the mix, and you’d have the beginnings of a modern dystopian novel or movie.

For many retailers, however, surveillance is an essential part of their business. In the US alone, shoplifting costs retailers nearly $18 billion in merchandise loss in 2017, not to mention the amount spent on measures to prevent shoplifting, such as installing security cameras and hiring loss prevention staff.

Retail surveillance is a perfect area to apply the power of AI. After all, there are volumes of training data from existing surveillance recordings, and plenty of money can be saved with even incrementally more accurate methods of detecting shoplifting.

However, the dystopian worries of both Orwell and many modern artificial intelligence experts are salient. How can consumers trust that a black-box surveillance algorithm protects their privacy? And for people of color who live in a world where racist stereotypes are ingrained into many aspects of society, how can they trust that surveillance algorithms will treat them fairly? Already, “smart policing” systems have been shown to disproportionately target black communities, and Google recently apologized after its Vision AI returned a “gun” label on an image of a black hand holding a thermometer but not of a white hand. These unanswered questions pose a major hurdle to many surveillance systems, as retailers may not wish to take such risks from both an ethical and legal standpoint.

An Ethical AI

Percepta’s founding story started in the classroom at Penn Engineering in August 2019. Founders Jonathan Mak (Eng UG ‘20), Neil Gramopadhye (College UG ‘21), and Philippe Sawaya (Eng UG ‘20), with a background in computer vision from research at Penn’s GRASP Lab, originally sought to apply AI to on-campus safety. However, as they spoke with security and retail experts, they came to better understand the need for a surveillance system without the privacy and bias issues of most algorithms. The problem became even clearer when they considered both their own and their friends' experiences with profiling.

“When we spoke with our friends who are people of color, they said, ‘Yeah, I get profiled all the time. It definitely damages my experiences as a consumer. It’s frustrating, it’s annoying.’” said Mak.

From there, Percepta was born. Since then, Mak, Gramopadhye, and Sawaya have been hard at work growing their company, now with a small full-time team alongside the founders. Together, they have built software that integrates with existing camera infrastructure to detect shoplifting while anonymizing personally identifiable information including skin tone, age, and gender. Having received guidance and funding from Techstars and security giant ADT, Mak says that the team is focused on improving their product and beginning sales efforts. Currently, they are deploying early versions of their product at several small grocery and retail stores and working closely with mentors to perfect their approach. “We’re honestly still pretty heads down with building our product out, everyone is putting in good work and working hard.” said Mak.

A Twofold Challenge

Mak saw a clear upside for retailers that adopt AI-driven loss prevention technology. He told me that their system can detect shoplifting at a rate three times higher than the average human, and that the race, gender, and age-blindness of their product mitigate the legal, ethical, and PR issues that often come with surveillance. The most challenging parts, according to Mak, have been building the product and breaking into the retail space.

The technical challenge is one at the heart of machine learning. The quality of an algorithm largely depends on the quality and size of its training data, but if Percepta anonymizes faces, skin tone, and other identifying features, they are giving the algorithm less data to learn from. So, how does Percepta still make useful predictions with less data?

“We found in our preliminary tests that actually, a lot of the additional data that the model is getting fed . . . just contains too much noise and, in layman’s terms, it confuses the AI on what to focus on.” said Mak. In fact, he states that there really isn’t a statistical difference between demographic groups when it comes to whether or not someone will shoplift, and by discarding that data and focusing on actions - eyeing the employees, not carrying a shopping basket - their algorithm gets better results without profiling.

Mak also acknowledged the difficulty of breaking into retail. He cites the thin margins, long sales cycles, and the need for numerous stakeholders to buy in as notable challenges, not to mention changes in shopping behavior caused by COVID-19. He is optimistic, though. Percepta is targeting initial sales to large retail chains with established security infrastructure, for whom Percepta can integrate their software on top of the existing workflow. Mak named integration with existing security systems as an important differentiator for the company, as “it's even conceivable that [retailers’] IT personnel could set this up with some very minimal guidance from us.”

When asked about the competition, Mak was confident that Percepta is uniquely positioned. While there are certainly other companies selling a shoplifting detection AI, he believes that Percepta is the only one with an explicit ethical focus. Vaak, a Japan based startup also focusing on shoplifting prevention, has already faced questions about the moral and legal implications of its technology, while companies like Amazon, Grabango, and Standard Cognition that are using similar technologies for cashierless checkout have also raised some privacy concerns. Percepta hopes to address these issues, and combined with a cutting-edge algorithm and mentors from the computer vision, security, and retail spaces, Mak is excited for the near future of the company, despite being relative newcomers to the space. “I personally live and die by cold email,” he said, smiling.

Looking Forward

While shoplifting is a pressing issue that can be solved with computer vision algorithms, Mak’s vision for the long-term future of Percepta goes beyond the five-finger discount. “Percepta is not a shoplifting detection company . . . we see Percepta as the ethical AI company.” With the massive amount of video data captured every day, Mak is excited about the potentially powerful insights that can be extracted from video, from better understanding customers in a retail store to detecting falls in senior care facilities. Privacy and bias, however, remains a major roadblock, and Mak sees the future of Percepta as a company that can remove that roadblock with a focus on ethical AI.

You can learn more about Percepta by visiting their website here.