Ferrari, Vittorio

Keller, Frank

Papadopoulos, Dimitrios P.

Engineering and Physical Sciences Research Council (EPSRC)

2018-06-06T12:53:11Z

2018-06-06T12:53:11Z

2018-07-02

A central task in computer vision is detecting object classes such as cars and horses in complex scenes. Training an object class detector typically requires a large set of images labeled with tight bounding boxes around every object instance. Obtaining such data requires human annotation, which is very expensive and time consuming. Alternatively, researchers have tried to train models in a weakly supervised setting (i.e., given only image-level labels), which is much cheaper but leads to weaker detectors. In this thesis, we propose new and efficient human annotation schemes for training object class detectors that bypass the need for drawing bounding boxes and reduce the annotation cost while still obtaining high quality object detectors. First, we propose to train object class detectors from eye tracking data. Instead of drawing tight bounding boxes, the annotators only need to look at the image and find the target object. We track the eye movements of annotators while they perform this visual search task and we propose a technique for deriving object bounding boxes from these eye fixations. To validate our idea, we augment an existing object detection dataset with eye tracking data. Second, we propose a scheme for training object class detectors, which only requires annotators to verify bounding-boxes produced automatically by the learning algorithm. Our scheme introduces human verification as a new step into a standard weakly supervised framework which typically iterates between re-training object detectors and re-localizing objects in the training images. We use the verification signal to improve both re-training and re-localization. Third, we propose another scheme where annotators are asked to click on the center of an imaginary bounding box, which tightly encloses the object. We then incorporate these clicks into a weakly supervised object localization technique, to jointly localize object bounding boxes over all training images. Both our center-clicking and human verification schemes deliver detectors performing almost as well as those trained in a fully supervised setting. Finally, we propose extreme clicking. We ask the annotator to click on four physical points on the object: the top, bottom, left- and right-most points. This task is more natural than the traditional way of drawing boxes and these points are easy to find. Our experiments show that annotating objects with extreme clicking is 5 X faster than the traditional way of drawing boxes and it leads to boxes of the same quality as the original ground-truth drawn the traditional way. Moreover, we use the resulting extreme points to obtain more accurate segmentations than those derived from bounding boxes.

http://hdl.handle.net/1842/31088

en

The University of Edinburgh

D.P. Papadopoulos, A.D.F. Clarke, F. Keller, and V. Ferrari. Training object class detectors from eye tracking data. In Proceedings of the European Conference on Computer Vision (ECCV), 2014.

D.P. Papadopoulos, J.R.R. Uijlings, F. Keller, and V. Ferrari. We dont need no bounding-boxes: Training object class detectors using only human verification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

D.P. Papadopoulos, J.R.R. Uijlings, F. Keller, and V. Ferrari. Training object class detectors with click supervision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.

D.P. Papadopoulos, J.R.R. Uijlings, F. Keller, and V. Ferrari. Extreme clicking for efficient object annotation. In Proceedings of the International Conference on Computer Vision (ICCV), 2017.

computer vision

object classes

annotation schemes

eye tracking data

center-clicking

Efficient human annotation schemes for training object class detectors

Thesis or Dissertation

Doctoral

PhD Doctor of Philosophy