News Story

Meet HanS, the Health and Safety Robot Inspector

Friday 27 May 2022

The Intelligent Systems and Data Science (ISDS) robotics team has developed HanS, a Health and Safety Robot Inspector. HanS is an experiment in service robotics and has been designed to be able to patrol KMi to try and identify possible violations to Health and Safety rules. For instance, typical rules check that all fire extinguishers are in place and that emergency exits are free of obstacles and clearly labelled through the correct signs.

The HanS robotic prototype originated as a practical application of Agnese Chiatti’s PhD research, with the contribution of Dr. Gianluca Bardaro, and supervisors Prof. Enrico Motta and Dr. Enrico Daga. Indeed, the deployment of a mobile robot that can make sense of real-world, dynamic environments brings about several research challenges.

To tackle these challenges, the team developed a modular robotic platform, which relies on standard Robot Operating System (ROS) modules for robot navigation and localisation, while also introducing novel customised components for visual sensemaking and commonsense reasoning. Specifically, HanS was trained to recognise 60 object classes that are commonly found in KMi from only a few training examples. An Edge TPU accelerator is mounted on the robot, to ensure fast learning times. An important innovation developed for HanS is that the robot is also aware of the typical sizes and spatial relations between objects, as well as the common-sense facts about these objects. For example, it knows that paper is flammable and that flammable objects may cause a fire, if placed near electrical devices. These properties were autonomously extracted and summarised from a number of Knowledge Bases, including WordNet [1], ShapeNet [2], Visual Genome [3], Amazon product information, and Quasimodo [4]. These capabilities improve HanS’ sensemaking ability and also allow it to maintain a semantic map of the KMi space, where observations that refer to the same object are aggregated and updated over time, to reflect changes in the environment.

A demo that showcases HanS’ capabilities can be found at https://www.youtube.com/watch?v=h8sZgLt_KQw.

[1] Miller, G. A. (1995). WordNet: a lexical database for English. CACM, 38(11), 39-41.
[2] Savva, M., Chang, A. X., & Hanrahan, P. (2015). Semantically-enriched 3D models for common-sense knowledge. IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 24-31).
[3] Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., … & Fei-Fei, L. (2017). Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 123(1), 32-73.
[4] Romero, J., Razniewski, S., Pal, K., Z. Pan, J., Sakhadeo, A., & Weikum, G. (2019, November). Commonsense properties from query logs and question answering forums. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (pp. 1411-1420).