NOTE: The following materials are presented for timely
dissemination of
academic and technical work. Copyright and all other rights
therein are reserved by authors and/or other copyright
holders. Persoanl
use of the following materials is permitted and, however, people using
the materials or information
are expected to adhere to the terms and
constraints invoked by the related copyright.
Learning Contextualized Semantics from Co-occurring Terms via a Siamese Architecture
ABSTRACT
One of the biggest challenges in Multimedia information retrieval and understanding is to bridge the semantic gap
by properly modeling concept semantics in context. The presence of out of vocabulary (OOV) concepts exacerbates
this difficulty. To address the semantic gap issues, we formulate a problem on learning contextualized semantics
from descriptive terms and propose a novel Siamese architecture to model the contextualized semantics from
descriptive terms. By means of pattern aggregation and probabilistic topic models, our Siamese architecture
captures contextualized semantics from the co-occurring descriptive terms via unsupervised learning, which
leads to a concept embedding space of the terms in context. Furthermore, the co-occurring OOV concepts can be
easily represented in the learnt concept embedding space. The main properties of the concept embedding space are
demonstrated via visualization. Using various settings in semantic priming, we have carried out a thorough
evaluation by comparing our approach to a number of state-of-the-art methods on six annotation corpora in
different domains, i.e., MagTag5K, CAL500 and Million Song Dataset in the music domain as well as Corel5K,
LabelMe and SUNDatabase in the image domain. Experimental results on semantic priming suggest that our approach
outperforms those state-of-the-art methods considerably in various aspects.
Click
nn2016.pdf
for full text