October 22, 2020, CHAPEL HILL, N.C. – The Reese Innovation Lab at the University of North Carolina at Chapel Hill, with the support of Lenovo, is creating a safer live event experience during the COVID-19 pandemic through an innovative artificial intelligence (AI) technology solution. Student Fellows and staff at the lab designed and engineered the new Health Greeter Kiosks to encourage anyone passing the devices to wear masks and practice social distancing. The AI, specifically computer vision and machine learning, uses real time data from a depth-sensing camera to detect if someone is wearing a mask and whether there is proper spacing between individuals. As people walk by these strategically placed kiosks, a large display alerts them to correct or continue their behavior.
The fully anonymized data—with no images saved or transmitted—may help shape safety protocol and provide insight on how crowds behave during the ongoing pandemic.
The technology was first deployed and successfully tested at the UNC-Virginia Tech football game on Oct. 10. The Reese Innovation Lab team installed kiosks at key locations around Kenan Memorial Stadium, including bag check lines, ticket booths and entrances. The technology worked as intended, tracking and encouraging safe behavior. The kiosks will be deployed again during the Oct. 24 game against NC State.
“We needed real innovation to meet this unprecedented challenge, and pushing the limits of technology is at the core of our Lab’s mission,” says Steven King, the chief innovation officer of the Reese Innovation Lab and associate professor of Emerging Technologies at UNC’s Hussman School of Journalism and Media. “Engineering a technological response to COVID-19 and event attendance restarting is a real and rewarding challenge, I’m grateful for the support of UNC-Chapel Hill leadership, our exceptional and inventive students, and Lenovo.”
The lab had ten weeks to design the experience, which included developing and engineering on equipment provided by Lenovo, testing the technology, and targeting deployment before the fall semester began. The team executed most of these project phases while working from home and had to rapidly adapt to unpredictable coronavirus-related conditions. When the UNC campus closed, the team prepared to deploy the technology at the next in-person event deemed safe by the state government. The ideal opportunity turned out to be the attendance-limited resumption of the college football season.
“The kiosks accurately detected masks and distance, and I was happy to see people pass by and engage directly with the displays,” King says. “Some took photos of themselves with the screen confirming their masks, and many stopped to read about the devices and better understand the underlying technology.”
To develop the technology, King turned to Lenovo’s AI Innovation Center to help architect a powerful, reliable solution within a very compact space. Robert Daigle, Lenovo’s AI and business innovation leader, quickly assembled a team of data scientists, AI architects, engineers and partners to bring King’s concept to life. The team helped move the AI inferencing solution developed by Reese Innovation Lab students and staff on Lenovo ThinkStation PCs into devices barely larger than a smartphone: ThinkCentre Nano & Nano IoT. Combined with the Intel Distribution of OpenVINO toolkit, Intel Movidius Neural Compute Stick, and Intel RealSense D415 depth camera, the team created a scalable solution in just a few weeks of rapid, remote collaboration.
“The ongoing relationship with Lenovo and their first-class AI Innovation Center allowed us to innovate quickly by providing immediate access to the technology and expertise required to execute on our idea to help make public spaces safer,” says King.
These kiosks not only encourage compliance with UNC-Chapel Hill’s COVID policies, but they also provide valuable 3D spatial data that may be used with machine learning to inform better and safer crowd flow when students return to campus and public gatherings increase in size and scope.
“This is an exciting and important application of smarter technology and smarter IT infrastructure out of our AI Innovation Center, where we focus on engineering holistic solutions that go beyond any single device,” says Matthew Zielinski, president of Lenovo North America. “We are thrilled to collaborate with Steven and his team at UNC-Chapel Hill, sharing our technology and expertise with their ingenuity to help maintain guest safety at live events during the COVID-19 pandemic.”
“We see this as the starting point of wider deployment with opportunities to refine and customize the technology,” King says. “From campus hallways to outdoor events, these kiosks will help us better understand human behavior and encourage safe behavior, and I’m excited to see how we evolve and adapt this AI-powered solution.”
The development of this technology included computer vision engineer Max Hudnell (UNC-Chapel Hill MS Graduate and full-time Lab staff), Nautica Harvin (Undergraduate Innovation Fellow), and Husam Shaik (Master’s Innovation Fellow). Hudnell, like the others, needed to be quick and creative while working almost entirely from his apartment, and used mannequins to test the mask detection technology. The designs and gamification concepts involved undergraduate Innovation Fellows Natalie Huggins and Halynna Snyder.
All the data collected by the kiosks is immediately anonymized to protect individual identities, and the video images are deleted as soon as they are generated—30 times every second. The Reese Innovation Lab team developed the kiosk software and is following FERPA protocol on data security at UNC-Chapel Hill, and all staff who are handling the data have completed data security training to ensure human rights are protected.