Search and Retrieval of Rich Media Objects Supporting Multiple Multimodal Queries


In this paper, a novel framework for rich-media object retrieval is described. The searchable items are media representations consisting of multiple modalities, such as 2-D images, 3-D objects and audio files, which share a common semantic concept. The proposed method utilizes the low-level descriptors of each separate modality to construct a new low-dimensional feature space, where all media objects can be mapped irrespective of their constituting modalities. While most of the existing state-of-the-art approaches support queries of one single modality at a time, the proposed one allows querying with multiple modalities simultaneously, through efficient multimodal query formulation, and retrieves multimodal results of any available type. Finally, a multimedia indexing scheme is adopted to tackle the problem of large scale media retrieval. The present framework proposes significant advances over existing methods and can be easily extended to involve as many heterogeneous modalities as possible. Experiments performed on two multimodal datasets demonstrate the effectiveness of the proposed method in multimodal search and retrieval.

  • P. Daras, S. Manolopoulou, A. Axenopoulos, "Search and Retrieval of Rich Media Objects Supporting Multiple Multimodal Queries", IEEE Transactions on Multimedia, Vol. 14, No. 3, Page(s): 734 - 746, June 2012

  • Full document available here.
    Contact Information

    Dr. Petros Daras, Principal Researcher Grade Α
    1st km Thermi – Panorama, 57001, Thessaloniki, Greece
    P.O.Box: 60361
    Tel.: +30 2310 464160 (ext. 156)
    Fax: +30 2310 464164