Multimedia Analysis and Retrieval Searching Services
An automated service for indexing, searching, and retrieving digital images and rich media.
Date Posted: February 28, 2005
|
|
 |
|
|
Update: August 30, 2006
Improved system efficiency; new image and video (MPEG1, MPEG2) processing; metadata and feature index options; simplified index and HTML generation using extracted index both for static and dynamic Marvel.
What is Multimedia Analysis and Retrieval Searching Service?
IBM Multimedia Analysis and Retrieval (MARVEL) Searching Service is an automated service for indexing, categorizing, and searching large volumes of images and rich media. Before IBM MARVEL, the process for categorizing and annotating multimedia data was extremely labor-intensive, requiring manual viewing and annotation of content. With the growing deluge of multimedia data, an automated process was needed. IBM MARVEL helps to automate the annotation process by using advanced content analysis techniques for labeling and indexing the data. The IBM MARVEL system was designed to support the emerging MPEG-7 multimedia content description standard by providing automated meta-tagging for all XML standard content, including images, video, audio, and text.
IBM MARVEL Searching Service, whose release coincides with the advent of Web 2.0, provides multimedia retrieval, search, and categorization of rich content on the Web, in data storage, and over mobile devices. IBM MARVEL Searching Service enables a company or individual to upload thousands of digital snapshots, video clips, or images. Once the images are uploaded, they can be labeled and retrieved as singles shots, groups, or other concepts defined by the user. IBM MARVEL Searching Service organizes images based on various included or user-defined definitions.
How does it work? Uploading of content to this secure service enables the user to quickly edit, tag, and find valuable information. IBM MARVEL Searching Service can save content in a similar manner to other photo- and file-sharing solutions that store pictures in albums. The user can browse content using keywords or objects that are "similar." There is no limit to the creative ways that the user can group material to be searched by himself or others.
IBM MARVEL Searching Service has pre-built and pre-defined searching and tagging capabilities, and it can provide bulk training or heuristic training on an image-by-image basis.
Some features are as follows:
- Text-based and image-based search retrieves pertinent media.
- Topics (concepts) can be browsed for discovery of data.
- A "drill-down" search can be refined further by the user's selecting an image or topic, which is used to issue the next query.
understand and further analyze the training requirements for IBM MARVEL – what is the sub-set of data required to train the system in terms of quantity of videos and quality of associated modality.
Results can be used for assessing the proportion of videos that must be used in training the system from the current set of digitized videos. We will not gain much advantage if we have to use numberous videos to manually train the system. [These 2 are not features.]
Key capabilities of IBM MARVEL Searching Service include the following:
- Visual feature extraction: extracts visual descriptors from video images for color, texture, edges, types, etc.
- Visual concept detection: automatically tags video content (label + confidence) based on trained models
- Visual clustering and cluster labeling: automatically clusters video images and tags them with descriptive labels
- Visualization: creates visual mosaic images summarizing extracted concepts and clusters
- Video search: searches the extracted feature, concept, or cluster information.
This service demonstrates the abilities of IBM MARVEL to browse and search digital images or video streams based on semantic concepts, visual feature descriptors, and metadata. IBM MARVEL Searching Service analyzes multiple different components within an image, group of images, or clips in order to define the taxonomy of, similarities among, and differences between the images.
|
|
|
| | About the technology author(s):
 John R. Smith is senior manager of the Intelligent Information Management Dept. at the IBM T. J. Watson Research Center, N.Y. He received his Ph.D. in electrical engineering from Columbia University in 1997. His research interests include multimedia databases, content analysis, compression, indexing, and retrieval. In 2003, Dr. Smith was co-recipient of the "Multimedia Prize" for best paper in IEEE Trans. Multimedia over a four-year period. Dr. Smith was co-editor of MPEG-7 Multimedia Description Schemes Metadata Standard and chairman of MPEG Multimedia Description Schemes (MDS) group from January 2001 to July 2004.
|
 Milind Naphade is a research staff member in the Intelligent Information Analysis Group at the IBM T. J. Watson Research Center. He received his M.S. and Ph.D. in electrical engineering from the University of Illinois at Urbana-Champaign in 1998 and 2001, respectively. His research interests include unstructured multimedia information analysis, retrieval, and semantic learning. Dr. Naphade was awarded the IEEE Circuits and Systems 2004 Outstanding Young Author Award for his seminal paper on modeling multimedia semantic context. He is author of over 50 research articles, publications, and book chapters, 12 patents (pending or granted), and he serves on several technical committees of the IEEE and ACM.
|
 Apostol (Paul) Natsev is a research staff member in the Intelligent Information Analysis Group at the IBM T. J. Watson Research Center. He received his M.S. and Ph.D. in computer science from Duke University in 1997 and 2001, respectively, and he was awarded an IBM Fellowship in 2000-2001. Dr. Natsev's research interests are in the areas of unstructured content analysis, semantic understanding, indexing, and retrieval. He has written more than 30 conference and journal papers, as well as eight U.S. patents (filed or granted).
|
 Jelena Tesic is a research staff member in the Intelligent Information Analysis Group at the IBM T. J. Watson Research Center. She received the Dipl. Ing. degree from the School of Electrical Engineering, University of Belgrade, in Serbia and Montenegro in 1998, and M.S. and Ph.D. degrees in electrical and computer engineering from the University of California, Santa Barbara, in 1999 and 2004, respectively. Her research interests include multimedia management, content analisys, scalable indexing, learning, and mining.
|
|
|
|
| |
