Deep Spatio-Temporal Modeling for Object-Level Gaze-Based Relevance Assessment


The current work investigates the problem of objectlevel relevance assessment prediction, taking into account the user's captured gaze signal (behaviour) and following the Deep Learning (DL) paradigm. Human gaze, as a sub-conscious response, is influenced from several factors related to the human mental activity. Several studies have so far proposed methodologies based on the use of gaze statistical modeling and naive classifiers for assessing images or image patches as relevant or not to the user's interests. Nevertheless, the outstanding majority of literature approaches only relied so far on the use of handcrafted features and relative simple classification schemes. On the contrary, the current work focuses on the use of DL schemes that will enable the modeling of complex patterns in the captured gaze signal and the subsequent derivation of corresponding discriminant features. Novel contributions of this study include: a) the introduction of a large-scale annotated gaze dataset, suitable for training DL models, b) a novel method for gaze modeling, capable of handling gaze sensor errors, and c) a DL based method, able to capture gaze patterns for assessing image objects as relevant or non-relevant, with respect to the user's preferences. Extensive experiments demonstrate the efficiency of the proposed method, taking also into consideration key factors related to the human gaze behaviour.

  • K. Stavridis, A. Psaltis, A. Dimou, G. T. Papadopoulos, P. Daras, "Deep Spatio-Temporal Modeling for Object-Level Gaze-Based Relevance Assessment", In Proceedings of the 27th European Signal Processing Conference (EUSIPCO), A Coruna, Spain, September 2-6, 2019. DOI:

  • Visual Computing Lab

    The focus of the Visual Computing Laboratory is to develop new algorithms and architectures for applications in the areas of 3D processing, image/video processing, computer vision, pattern recognition, bioinformatics and medical imaging.

    Download VCL Leaflet

    Contact Information

    Dr. Petros Daras, Research Director
    6th km Charilaou – Thermi Rd, 57001, Thessaloniki, Greece
    P.O.Box: 60361
    Tel.: +30 2310 464160 (ext. 156)
    Fax: +30 2310 464164
    Email: daras(at)iti(dot)gr