A metric for multiple target tracking
A standard metric for evaluating the multiple target tracking algorithm is the CLEAR MOT. This metric is described in the paper [1] .
Details
We provide the code that implements the metric CLER-MOT has described by the authors in [1]. The function is implemented in MATLAB and has been tested on real data generated by a multiple-target tracker.
The tarball contains these MATLAB files:
- groundtruth.mat contains the labeled annotations of 3 objects. Objects are specificated by labeled bounding box as [ID tl.x tl.y br.x br.y]. The ID is the label, tl is the top-left point of the annotations and br is the bottom-right.
- result.mat contains the tracking results hypothesis as taken from the ground truth files. So, MOTA and MOTP results to 100% in this simple example.
- evaluateMOT.m is the function that performs the evaluation.
- main.m is the main file
- GreedyAssociation.m is the file that performs the association given the distance matrix. You can replace with other solvers like f.e. Hungarian algorithm.
These data files are only an example: if you want to use the script to evaluate your multiple target tracking result, you have to re-create the structures groundtruth.mat and result.mat with your own data.
Note that ID Switch (mismatch) should be carefully counted by visual inspection, considering that the automatic association is not able to manage situations where targets are very near each other.
Bur reporting is very appreciated, please contact me as soon as possible, leaving a comment below.
Download
The code is hosted on Github please download it here
Acknowledge
Please, note if you use the code, you have to cite our work:
@article{ masi:multimedia12, author = {Bagdanov, Andrew D. and Del Bimbo, Alberto and Dini,
Fabrizio and Lisanti, Giuseppe and Masi, Iacopo},
title = {Compact and efficient posterity logging of face imagery for video surveillance},
booktitle = {IEEE Multimedia},
year = {2012}, }
Changelog
– 2012/05/11 Initial release.
(c) Copyright 2011 - MICC - Media Integration and Communication Center, University of Florence. Iacopo Masi and Giuseppe Lisanti <masi,lisanti>@dsi.unifi.it
Demo Example
load groundtruth load result VOCscore = 0.5; dispON = true; ClearMOT = evaluateMOT(gt,result,VOCscore,dispON);
The result is the following:
------ ::RESULTS:: --------- rateFP = 0 (0%) rateTP = 1 (100%) rateFN = 0 (0%) ---------------------------- TP = 9 FN = 0 FP = 0 ID switch (MisMatch) = 0 ***NOTE***: ID switch should be carefully counted by visual inspection Sum of GrountTruth Obj = 9 Sum of FN+TP+IDSW = 9 ---------------------------- MOTP = 1 MOTA = 1 (100%) ----------------------------
References
- Keni Bernardin and Rainer Stiefelhagen. “Evaluating multiple objec tracking performance: the CLEAR MOT metrics” J. Image Video Process. 2008, Article 1 (January 2008), 10 pages.” DOI=10.1155/2008/246309 http://dx.doi.org/10.1155/2008/246309
