Insights
In our society detection and tracking of individuals by video cameras are more and more used in many circumstances of everyday life. Main applications are for surveillance in critical environments where it is of primary relevance for security to understand who is doing what and check whether this is normal or abnormal behavior. However, less intrusive applications of profiling are increasingly used in other day life contexts to improve the quality of visiting (for example understanding the interests of a tourist) or to provide suggestions based on recent choices, activities and visited locations. While cameras and computer vision are widely used in conjunction with machine learning to perform detection and classification automatically in surveillance applications, in other day life contexts profiling is mostly based on collected user-generated information (f.e. via social networks) and extensive data mining supported by smart recommender algorithms. While these technological tools are powerful and widely used media, we recognize that computer vision has a complementary role in this task.
- it may support a visual characterization of an individual, his race, genre, age…., it can capture a privacy-respectful visual signature to perform re-identification inside a location, and it can, therefore, make it possible to track and understand his/her paths.
- it can detect and classify his/her dressing and therefore understand tastes and psychological dispositions.
- it can analyze his/her behavior, understanding of interests and feelings.
Recently, there has been growing interest to introduce such sensors in malls and shopping centers to understand visitor behaviors and derive suggestions to arrange products appropriately according to the interests and orientations of the mass. Advances in AI and computer vision have made it possible. The recently opened Amazon Go store is a first experience where advanced shopping technology is put in practice. Using the Amazon Go app the visitor enters the store, takes the products of interest and doesn’t have to make any checkout. Computer vision based on deep learning and sensor fusion automatically detect when products are taken from or returned to the shelves and keeps track of them in a virtual personal cart. In China, Tao Café, a pop-up convenience store from Alibaba, implemented cashier-free stores with facial recognition and other advanced technologies, that allow customers need only their smartphone with Alibaba’s Taobao app to enter and checkout. These are just examples of how AI and computer vision technology, eventually supported by IoT, can drastically transform our free-time shopping habits.
In this project, we aim to verify the extent to which Computer Vision and AI can support the derivation of individual profiles and provide appropriate personalized suggestions at digital signage terminals. We recognize that the shopping experience is strictly tied to the emotional state of the individual and therefore to personal momentary conditions. Therefore, the analysis of the individual behavior during the time of the visit is much more useful than personal attributes as derived by social network dialogues to understand the attitudes and interests of the visitor. For our research, we don’t aim at unattended shops like Amazon Go and Tao Café but have as a reference a less constrained context like a shopping center or a mall. Therefore, differently, from the cases above, we are not identifying people by managing their real identity and the customer does not disclose any sensitive personal information like a surname, credit card numbers, etc.. Following the current national legislation on privacy, Computer vision and AI cooperate to observe customers and infer their momentary interests by tracking them during the visit and analyzing their behavior and reactions to the suggestions provided by a digital signage terminal.
Customers’ identities are identified by progressive anonymized numbers and locations visited, interests, behaviors, and feelings are associated with such anonymized identities. They will be erased from the system as soon the person leaves the Mall. The project includes a preliminary analysis and interpretation of the legislation to derive strict guidelines in the design and implementation of the technological functions. A test at VERONAFIERE is planned.
More Info HERE