Category Archives: Projects

Media Integration and Communication Centre projects

Continuous Recovery for real time PTZ localization and mapping

We propose a method for real time recovering from tracking failure in monocular localization and mapping with a Pan Tilt Zoom camera (PTZ). The method automatically detects and seamlessly recovers from tracking failure while preserving map integrity.

By extending recent advances in the PTZ localization and mapping, the system can quickly and continuously resume tracking failures by determining the best way to task two different localization modalities.

Continuous Recovery for Real Time Pan Tilt Zoom Localization and Mapping demo

The trade-off involved when choosing between the two modalities is captured by maximizing the information expected to be extracted from the scene map.

This is especially helpful in four main viewing condition: blurred frames, weak textured scene, not up to date map and occlusions due to sensor quantization or moving objects. Extensive tests show that the resulting system is able to recover from several different failures while zooming-in weak textured scene, all in real time.

Dataset: we provide four sequences (Festival, Exhibition, Lab, Backyard) used for testing the recovery module for our AVSS 2011 publication, including the map, nearest neighbour keyframe of the map, calibration results (focal length and image to world homography) and finally a total of 2,376 annotated frames. The annotations are ground-truth feet position and head location, used to decide if the calibration is correct or not. Annotations are in term of MATLAB workspace files. Data was recorded using a PTZ Axis Q6032-E and a Sony SNC-RZ30 with a resolution of 320 x 240 pixel and a frame-rate of about 10 FPS. Dataset download.

Details:

NN keyframe are described as a txt file where first number is the id of the frame and the next string is the id (filename of images in map dir) of the relative NN keyframe as #frame = keyframe id. Note that we store in the file only the frame number in which there is a keyframe switch.
Calibration is provided as a CSV file using the following notation [#frames, h11,h12,h13,…., h31,h32 ,h33, focal length], where hij are the i-th row and j-th colum of homography.
- A MATLAB script is provided to superimpose ground-plane in the current image(plotGrid.m).
- The homograhy h11..h33 is the world to image homography that maps pixel into meters.
Ground-Truth is under the name of “ground-truth.mat” and it consists of a cells where each item is the feet position and the head position.
In each sequence it is present a main script plotGrid.m MATLAB script that plots ground-truth annotations and superimposes the ground-plane on the image. ScaleView.m is the script that exploits calibration to predict head location.
Note that we have obfuscated most of the faces to keep anonymity.

ORUSSI. Optimal Road sUrveillance System based on Scalable video

1 Reply

The growing mobility of people and goods has a very high societal cost in terms of traffic congestion and of fatalities and injured people every year. The management of a road network needs efficient ways for assessment at minimal costs. Road monitoring is a relevant part of road management, especially for safety, optimal traffic flow and for investigating new sustainable transport patterns.

Road monitoring

On the road side, there are several technologies used for collecting detection and surveillance information: sophisticated automated systems such as in-roadway or over-roadway sensors, closed circuit television (CCTV) system for viewing real-time video images of the roadway or road weather information systems for monitoring pavement and weather.

Current monitoring systems based on video lack of optimal usage of networks and are difficult to be extended efficiently.

Our project focuses on road monitoring through a network of roadside sensors (mainly cameras) that can be dynamically deployed and added to the surveillance systems in an efficient way. The main objective of the project is to develop an optimized platform offering innovative real-time media (video and data) applications for road monitoring in real scenarios. The project will develop a novel platform based on the synergetic bundling of current research results in the field of semantic transcoding, the recently approved standard Scalable Video Coding standard (SVC), wireless communication and roadside equipment.

Dataset: thanks to the involvement of Comune di Prato (a local municipality), we were able to collect a very wide dataset of video sequences that turned out to be key for the project activities. The dataset is made of more than 250 hours of recording taken on a well-travelled county road, with different lighting and weather conditions. From these video sequences we have extracted an image dataset of about 1250 vehicle images. This data set, available here, can be used to train a vehicle classifier.

euTV: adaptive media channels

Leave a reply

The explosion of digital data in recent times, in its varied forms and formats (MPEG4 image, Flash video, WAV audio, etc.), has necessitated the creation of effective tools to organise, manage and link digital assets, in order to maximise accessibility and reduce cost issues for everyone concerned, from content managers to online content consumers.

euTV video annotation and transcription web component

On a larger scale, isolated information repositories developed by content owners and technology providers can be connected, unleashing opportunities for innovative user services and creating new business models, in the vein of on-demand, online, or mobile TV ventures.

The euTV project stems from above conditions and potentialities, to connect publicly available multimedia information streams under a unifying framework, which additionally allows publishers of audio-visual content to monetise their products and services. The backbone of euTV is a scalable audio-visual analysis and indexing system that allows detection and tracking of vast amounts of multimedia content based on Topics of Interest (TOI) corresponding to a user’s profile and employed search terms. The front-end is a portal that displays syndicated content, allowing users to perform searches, refine queries, and produce faceted presentation of results.

euTV logo

The three main content domains will be (a) news, (b) sports, and (c) documentaries. In the existing market of media monitoring and clipping, euTV distinguishes itself by simultaneously analysing multiple information streams (text, speech, audio, image, video) instead of a single one and tracking TOI in real time. This provides the user with a more robust identification of their TOI and greater insights into how the information is spread.

Joint laboratory MICC – Thales

Leave a reply

MICC, Media Integration and Communication Center of the University of Florence, and Thales Italy have established a partnership to create a joint laboratory between university and company in order to research and develop innovative solutions per safety, sensitive sites, critical infrastructure and transport.

MICC - Thales joint lab demo at Thales Technoday 2011

In particular the technology program is mainly focused (but not limited) on surveillance through video analysis, employing computer vision and pattern recognition technologies.

A current active filed of research, continued from 2009 to 2011 was that of studying how to increase the effectiveness of classic video surveillance systems using active sensors (Pan Tilt Zoom cameras) and obtain higher resolution images of tracked targets.

MICC - Thales joint lab projects

The collaboration allowed to start studying the inherent complexities of PTZ camera setting and algorithms for target tracking and was focused on the study and verification of a set of basic video analysis functionalities.

Thales

In 2011 the joint lab led to two important demos at two main events: Festival della Creatività, October 2010 in Florence (Italy) and Thales Technoday 2011 in January 2011 in Paris (France). In the latter the PTZ Tracker has been nominated as VIP Demo (Very ImPortant Demo).

Some videos about this events:

Onna: a natural interface system for virtual reconstruction

Leave a reply

A technology transfer project for an international exhibition about the story of Onna, an italian town near to L’Aquila, which was affected by the earthquake during 2009.

A natural interaction based system is designed and developed in order to present a large number of multimedia contents (videos, images and audio) collected and created by the curators of the exhibition.

Interactive system scenario in the Infobox at Onna

The project involves the study and development of an interactive system which adopts the paradigm of the natural interaction in order to allow users to access and consult multimedia contents related to different areas of the town of Onna, an italian town near to L’Aquila, which was affected by the earthquake during 2009.

The concept proposed for the user-interface is inspired by the educational game book published in the seventies about the devastation of the town of Pompei after the eruption of Vesuvius in 79 B.C. The pages of the book are composed of images of the destroyed Pompei which can be overlapped with images of the town before the eruption.

Our idea is to recreate a similar mode of interaction and use a background picture of the town of Onna after the earthquake so that the user can interact with some areas of the image and see them as they were before the earthquake. In addition for each area is possible to visualize multimedia contents about history, architecture and life before the earthquake.

The user-interface will be optimized in the environment of the exhibition in order to allow multiple users to interact independently with the system.

MAC-GEO: the effects of geothermal power in Tuscany

1 Reply

A technology transfer project for the Regione Toscana in order to provide a solution to predict the effects of geothermal power both in the same basin and the surrounding environment in some areas of Tuscany.

Geotherm viewer data visualization

A web application in a GIS environment is designed and developed in order to visualize and query simulation data and their scenarios stored in a geospatial database.

Geotherm Viewer

Project URL: http://macgeo.math.unifi.it/~macgeo/

The MAC-GEO project aims to give the Regione Toscana some simulation tools to predict the effects of geothermal power both in the same basin and the surrounding environment.

The project was funded by Regione Toscana CIPE funds, following the announcement Framework Programme Agreement Research and technology transfer for the production system – Supplementary Agreement III.

The project started in September 2008 and lasts two years.

MICC research staff designed and developed the Geotherm Viewer, a web application in a GIS environment for viewing and querying simulation data and their scenarios stored in geospatial database.

The system was designed for users who need to quickly view the results of the generated simulations in order to assess which groups of data for later analysis and deepen the effects of geothermal both in the same basin and the surrounding environment. In addition, the user-interface provides a set of interactive features for geographic data visualization, data from web map services and information generated by Google Maps.

The Geotherm Viewer has been developed with open source technologies.

Enrich

Leave a reply

Enrich is a targeted project funded under the eContentPlus programme. Its objective is to provide seamless access to distributed digital representations of old documentary heritage from various european cultural institutions in order to create a shared virtual research environment especially for study of manuscripts, but also incunabula, rare old printed books, and other historical documents. It builds on the Manuscriptorium Digital Library (http://www.manuscriptorium.eu) that has already managed to aggregate data from 46 collections from the Czech Republic and abroad.

Manuscripts found by Enrich system

The project groups together almost 85% currently digitized manuscripts in the national libraries in Europe. These collections will be enhanced by substantial amount of data from university libraries and other types of institutions. The consortium will make available more than five million digitized pages.

Manuscriptorium is a result of 15 years of work and development carried jointly by two important czech institutions: AIP Beroun LTD and the National Library of the Czech Republic. It is the richest digital manuscript resource in Europe putting at disposal more than one million digitized pages, having a safe digital archive, enjoying of state support for digitization, and talking in czech and english languages. Ca. 50% of its users come from abroad and it operates a special clone for support of teaching and learning in secondary schools. Its origins are in the Memory of the World programme of UNESCO; therefore, the National Library of the Czech Republic received the UNESCO world Jikji award in 2005. The Manuscriptorium-related digitization knowledge and know-how have been taught and shared in many countries of the world.

Manuscriptorium builds on a robust xml schema the most important part of which is the european master format for electronic description of manuscripts based on TEI.

The basic corpus of data is made available from the digital storage facilities operated by AIP Beroun LTD in the Czech Republic. The Enrich project will integrate even more data from remote foreign digital libraries. The metadata records for the central database will be preferably collected via the OAI protocol; they will contain links to images stored in remote image databanks. Necessary transformation routines will be created and tuned for each partner. Specialized on-line tools will be also developed to enable Manuscriptorium schema compatible metadata structuring and output validation for those partners that have digital data with no presentation tools and will like to make them available.

Enrich target user groups are content owners/holders, libraries, museums and archives, researchers and students, policy makers, and general interest users. The project will allow them to search and access documents which would otherwise be hardly accessible. Besides images, it will be also able to offer access to TEI-structured historical full texts, research resources, other types of illustrative data (audio and video files) or large images of historical maps. The Enrich consortium will closely cooperate with TEL (The European Library) and will become a component part of the European Digital Library when this becomes reality.

The users will get tools that will assist them to create their own documents and personal digital libraries in Manuscriptorium using any analytical objects of which the documents consist. Tools for usage of more languages will be offered for application in Manuscriptorium as well as multilingual ontologies enabling search in local languages and retrieval of data in source languages.

The Enrich consortium consists of 18 partners and the project is also supported by a number of other institutions among which there are many important content owners.

The project is coordinated by the National Library of the Czech Republic together with other two Czech partners: AIP Beroun LTD and Crossczech Prague Inc.

The interest to cooperate has been expressed, among others, also by the national libraries of Hungary, Kazakhstan, Moldova, Poland, Romania, Serbia, Sweden, and Turkey as well as by the university libraries in Bratislava, Bucharest and Heidelberg. The list of associated partners will grow during the project duration.

DanThe. Digital and Tuscan heritage

Leave a reply

DanThe is an online service, developed and implemented by Tuscany Region and MICC – Media Integration and Communication Center – University of Florence, to promote the resources related to digital cultural heritage of Tuscany. DanThe provides a direct access to collections, databases, regional museums, libraries and catalogues of cultural heritage.

Danthe. Digital and Tuscan Heritage

The “catalogue of collections” provides a selection of the Tuscan digital documents (texts, images, video, audio) concerning cultural heritage (artistic, historical, archaeological, ethno-anthropological, archival and bibliographical). These collections were previously catalogued by the Michael Project (Multilingual Inventory of Cultural Heritage in Europe). Within the universe of European digital resources Danthe identifies a Tuscan subset of collections and make it accessible to different users.

The “catalogue of cultural heritage” is designed to collect and make available descriptive and informative catalogue records – in accordance with local owners (municipalities, museums, institutions etc.) – and realized using the standards ICCD and the CART free software, produced by the Tuscany Region and carried out by the Scuola Normale Superiore in Pisa.

In accordance with the guidelines of Michael Project, Danthe selected digital collections regarding people, works and cultural heritage of Tuscany, promoted, financed or produced by Tuscan institutions and regional offices of national or foreign institutions. The collections presented in Danthe, as those ones published on the European portal of Michael, are available online and/or offline, are free, or available on the registration fee and may be completed, planned or in progress.

In future the system will enable an innovative technological regional application to provide services that manage digital contents regarding cultural heritage and it will be able for the integration with the Regional Cooperation Application of Tuscany – CART

In this perspective, the Regional Information System will be able to:

hold copies of data on cultural heritage (regional databases and third party) and their metadata in a centralized repository;
allow automatic updating of data through the application infrastructure for cooperation;
interoperate with Information systems of external structures (Michael, CulturItalia, Europeana) using appropriate protocols and server (OAI-PMH);
allow the use of such digital contents by the web interface unified Danthe.

Danthe. Digital and Tuscan heritage

DanThe is part of a national strategy (portal CulturItalia) and (Europeana) which aims to combine and enhance digital assets in Europe, to consolidate descriptive standards and to set minimum standards in the development of resources of various nature and non-traditional format, generated using innovative technologies and for heterogeneous users.

Because of the deep analysis of the existent carried on digital resources relating to cultural heritage, the European project Michael was the basis to fill and populate DanThe database. The close cooperation between national European project of Michael, the regional cataloguers and the editors of DanThe allowed to use the expertise and knowledge of the local census of digital collections and to personalize the presentation and the fruition of the digital resources themselves.

Even though DanThe borrowed from Michael all the entities structured and divided according to the descriptive standard of the project, the Tuscany recourses are presented in a singular and personal way. In particular, it were not censed catalogues of libraries when they gave no services other than paper, to consider as digital collections not only the resources derived from analogical source (books, art works and objects of art, etc..) but also those formed from digital material (videos, Word documents, audio files, etc..) and to give a direct access to the collections online providing, where possible, the specific URL of the resource rather than to the generic website or the catalogue containing it.

PointAt system at Palazzo Medici Riccardi

Leave a reply

Palazzo Medici Riccardi is one of the most important museums in Florence: in its small chapel, it hosts the famous fresco “La cavalcata dei magi” (“The Journey of the Magi”) by Benozzo Gozzoli (1421–1497).

The PointAt system’s goal is to stimulate the visitors to interact with a digital version of the fresco and, at the same time, make them interact in the same way they will in the chapel, reinforcing their real experience with the fresco. That is to use information technology to make teaching attractive and effective.

PointAt at Palazzo Medici Riccardi

Visitors are invited to stand in front of the screens and indicate with their hand the part of the painting that interests them. Two digital cameras analyse the visitors’ pointing action and a computer vision algorithm calculates the screen location where they’re pointing. The system then provides audio information about the subject.

In designing the system, we considered the following issues:

Easy and simple interaction. Visitors don’t need any instruction or have to wear any special device.
High-resolution display. The fresco is displayed on large screens so that visitors can appreciate even small particulars (almost invisible in the real chapel).
Interactivity for different categories of visitors. Interaction should be satisfactory for visitors who just want an idea about the fresco, for those who are attracted by particular characters and for those who want to have complete information on the whole fresco.
Not intrusive setting. The physical setting must host both active and passive visitors (for example, the relatives of the person who’s actually interacting with the system and those interested in listening but not in being active).
Pleasant look & feel. The interactive environment is integrated within the museum and it respects the visitors’ whole experience.

PointAt is considered to be a good vanguard experiment in the field of museum didactics, and has been functioning successfully since 2004.

TANGerINE Tales. Multi-role digital storymaking natural interface

1 Reply

TANGerINE Tales is a solution for multi-role digital storymaking based on the TANGerINE platform. The goal is to create a digital interactive system for children able to stimulate collaboration between users. The result concerns educational psychology in terms of respect of roles, development of literacy and of narrative skills.

Testing Tangerine Tales

TANGerINE Tales lets children create and tell stories combining landscapes and characters chosen by themselves. Initially, children select the elements that will be part of the game and explore the environment within which they will create their own story. After that they have the chance to record their voice and the dynamics of the game. Finally, they are able to replay the self-made story on the interactive table.

The interaction between the system and users is performed through the tangible interface TANGerINE, consisting of two smart cubes (one for each child) and an interactive table. Users interact with the system through the manipulation of cubes that send data to the computer via a Bluetooth connection.

The main assumption is that the interaction takes place through the collaboration between two children who have different roles: one of them will actively interact to control the actions of the main character of the story, while the other will control the environmental events in response to the movements and actions of the character.

The target user of TANGerINE Tales is made up of 7-8 year olds, attending the third year of elementary school. This choice was made following research studies on psychological methods for collaborative learning, on Human Computer Interaction and tangible interfaces; we exploited the guidelines for learning supported by technological tools (computers, cell phones, tablet PCs, etc..) and those extrapolated by projects of storytelling for children.

You can see pictures of the interface on MICC Flickr account!