Insights
New paradigms of interaction and improvements in the realism of virtual environments offer novel opportunities to create learning tools for complex activities, especially in the medical field. Medical education requires trainers to teach and debrief well-defined procedures, as well as learners to acquire an high level of participation and control.
In the context of the RIMSI project, we developed a low-cost solution, which uses a Kinect sensor and 3D graphics, for two specific use cases: Basic Life Support Defibrillation (BLSD) and Surgical Safety Checklist (SSC) scenario, as a contribution to the development of innovative pedagogical models based on the goal to provide trainees with authentic activities which must then be reproduced in the real experience.
The proposed system exploits natural interaction paradigms and realistic 3D interfaces in order to train doctors and students to adhere to correct medical procedures. The virtual 3D simulation is visualised on a large-sized screen or projection and users can interact with the virtual simulation using body or hand gestures as well as voice commands. To detect interaction between users and the simulation the system use a Microsoft Kinect sensor.
The emergency medicine simulator is a single user application designed to train medical personnel in the procedure of Basic Life Support with Defibrillator. It is designed as free roam game, where the trainee can guide his virtual avatar in the 3D environment so to interact with virtual objects and the patient. The goal is to complete a BLSD procedure in an indoor situation with potential environmental risks for the operator and the patient (e.g. a gas leak). The user interacts with the simulation with body gestures, like walking, rotating or bending, or with hand gestures to select options in virtual menus.
The SSC simulator is a multiplayer game with turn and role taking dynamics designed to train users in the use and accomplishment of the Surgical Safety Checklist (SSC). The SSC has to be carried out before a surgical operation as defined by the World Health Organisation in order to minimise the risks endangering the lives of surgical patients.
The simulator associates users with a professional role on the basis of their position in the physical space (i.e. anesthesiologist, surgeon and nurse) and reproduce the scenario of a surgical operation. Every step of the checklist requires one of the three doctors to interact with others or to check medical devices status. Voice-based interactions occur when one of the professionals is expected to communicate with the patient or with another team members. Hand gestures are used are used to check the state of the medical equipment or to activate virtual menus.
The validity of the proposed system in terms of educational enhancement has been assessed through an evaluation with a group of 26 students attending the last year of the medicine school.
Students were divided in two groups, one of which was trained exclusively with the standard simulation approach, whilst the other performed a training session with the virtual interactive system before facing the standard simulation.
Although mannequin-based simulation allow to train a broader range of skills, results suggest that the use of the virtual simulation could effectively improve outcomes in the whole training process in term of time and correctness.
PUBLICATIONS:
- Bartoli, G., Del Bimbo, A., Faconti, M., Ferracani, A., Marini, V., Pezzatini, D., … & Zilleruelo, F. (2012, November). Emergency medicine training with gesture driven interactive 3D simulations. In Proceedings of the 2012 ACM workshop on User experience in e-learning and augmented technologies in education (pp. 25-30). ACM.
- Del Bimbo, A., Ferracani, A., Pezzatini, D., & Seidenari, L. (2013). A Natural Interface for the Training of Medical Personnel in an Immersive and Virtual Reality System. In Image Analysis and Processing–ICIAP 2013 (pp. 763-772). Springer Berlin Heidelberg.
- Ferracani, A., Pezzatini, D., Seidenari, L., & Del Bimbo, A. (2014). Natural and virtual environments for the training of emergency medicine personnel.Universal Access in the Information Society, 1-12.
- Ferracani, A., Pezzatini, D., & Del Bimbo, A. (2014, November). A Natural and Immersive Virtual Interface for the Surgical Safety Checklist Training. InProceedings of the 2014 ACM International Workshop on Serious Games (pp. 27-32). ACM.
VIDEO: https://vimeo.com/124400550
PARTNERS:
Gutenberg sicurezza in sanità srl (Arezzo, Italy), Dipartimento di Area Critica Medico-Chirurgica, University of Florence (Florence, Italy).
