Semantic entity-relationship model for large-scale multimedia news exploration and recommendation

Even though current news websites use large amount of multimedia materials including image, video and audio, the multimedia materials are used as supplementary to the traditional text-based framework. As users always prefer multimedia, the traditional text-based news exploration interface receives more and more criticisms from both journalists and general audiences. To resolve this problem, we propose a novel framework for multimedia news exploration and analysis. The proposed framework adopts our semantic entity-relationship model to model the multimedia semantics. The proposed semantic entity-relationship model has three nice properties. First, it is able to model multimedia semantics with visual, audio and text properties in a uniform framework. Second, it can be extracted via existing semantic analysis and machine learning algorithms. Third, it is easy to implement sophisticated information mining and visualization algorithms based on the model. Based on this model, we implemented a novel multimedia news exploration and analysis system by integrating visual analytics and information mining techniques. Our system not only provides higher efficiency on news exploration and retrieval but also reveals extra interesting information that is not available on traditional news exploration systems.