Deep discriminative tracking and segmentation of translucent objects

Translucent objects like glasses, cups, bottles, etc. are common in our everyday life. For the autonomous intelligent systems it is important being able to interact with the objects in the environment they are placed into. A crucial part for such autonomous interaction is robust tracking of translucent objects. The topic of translucent object tracking has thus huge potential for being used in practical applications as well as a large number of unanswered research questions. Tracking of general, well visible, or opaque objects is a widely researched area. On the other hand, much less attention has been paid to tracking of translucent objects. These types of objects have several specific properties, described in the following. Due to translucency the background is visible through the object, which significantly increases the appearance variation of the tracked object. Specific visual effects like light distortion or reflections are common for translucent objects and could be exploited for tracking. Translucent objects are frequently grouped, so that multiple visually similar objects are together, for example: a set of wine glasses, or glass cups in the laboratory environment. Due to these specifics, tracking algorithms should be designed differently compared to opaque object tracking algorithms. Recent benchmark highlighted the weaknesses of the existing opaque object tracking methods, when used for tracking translucent objects, while methods designed specifically for translucent objects (almost) do not exist. In this project, our goal is to develop methods for tracking translucent objects. All our methods will be built on deep neural networks, which show superior results on the majority of computer vision tasks, compared to handcrafted methods. The following research challenges will be addressed in this project: (i) a new training regime and dataset construction, (ii) handling of multiple similar objects, and (iii) high accuracy of the reported target position. A special attention will be put into integration of all developed modules into a single, end-to-end trainable deep network architecture. The project will be composed of four work packages. Discriminative tracking architecture robust to distractors (WP1) will be developed to address the challenging situations when multiple translucent objects with similar appearance are present in the same scene. A novel segmentation architecture will be developed (WP2) for segmenting translucent objects in a video. A new training dataset will be developed with a novel regime for training deep models. All models will be evaluated on translucent and opaque tracking benchmarks (WP3). The last work package (WP4) will cover dissemination of the results. A single post-doctoral researcher will work full-time on this project. The applicant is a member of Visual Cognitive Systems Laboratory (ViCoS) at Faculty of Computer and Information Science, University of Ljubljana. The project leader has a rich experience in visual object tracking research, thus a smooth realization and implementation of the project is guaranteed.