Image Retrieval: Ideas, Influences, and Trends of the New Age

Title: Image Retrieval: Ideas, Influences, and Trends of the New Age
Authors: RITENDRA DATTA, DHIRAJ JOSHI, JIA LI, and JAMES Z. WANG
Year of Publication: 2008
Publisher: ACM

Content-based image retrieval is an technology helps to organize digital picture archives by their visual content. There are two gaps, sensory gap and semantic gap, which define and motivate most of the related problems. The sensory gap is a gap between real object and the descriptive information, and the semantic gap is about the lack of coincidence between the information from the visual data and the interpretation from a user. Therefore, how to solve the gaps and satisfy users is the goal of CBIR.

The first important thing is to clarify user-system interaction. A user perspective involves what the user wants and what is the form used in query, and a system perspective is about how to interact with user. Simply, human-center based system is required for different kinds of user intent with several query types, including keywords, free-text, image, graphics, and composite. For example, it is possible for a composite query method to provide a system involving gestures and speech for querying, or help user refine the queries by hints. If a system can collect manual tags for pictures, not only facilitating text-based querying, but also building reliable training datasets. Moreover, how to design a retrieval system on portable devices which have many constraints, such as limited size and color depth of display, is one of the issues of visualization.

The two core problems of CBIR are (1) how to define a mathematical description or a signature of an image, (2) how to decide the similarity between a pair of images.

For region-based visual signatures, the first step is image segmentation. With k-means clustering or normalized cut criteria method, segmentation helps image understanding and extracts several types of features. A feature capture a specific property of an image, either globally for the entire image with higher speed for computation, or locally for a small group of pixels with more specific identification of important visual characteristics. Color features are usually summarized into histogram. Texture features are used to capture granularity and repetitive patterns. Shape is a key to specify regions. Spatial modeling and matching are regarding to local image entities. Interest points that can deal with significant affine transformation and illumination changes are based on local invariants. When constructing signatures from features, histograms is easy but tend to be sparse in multidimensional space. A region-based signature allow representative vector to adapt images and the region of color and texture is likely corresponding to an object in an image.

There are three types of signatures, feature vector, summary of local feature vectors , and region-based signature. Each of them has different appropriate similarity measures. Using the geodesic distances for a single vector may be better. Summaries of local feature vectors such as codebook and probability density functions are generated by vector quantization and KL distance separately. The region-based signature can form a histogram, and calculate the similarity from the pair-wise distances between individual vectors. More matching methods improve the basic idea from region weights, speed, or segmentation.

Due to faster retrieval, clustering and classification is practical and useful. Classification is treated as a preprocessing step and improve accuracy but require prior training data. Clustering helps visualization and retrieval efficiency but may not representative enough or accurate for visualization. Besides, in order to capture user’s precise needs, relevance feedback system which does iterative feedback and refinement is designed. That is, relevance feedback let users give feedback after querying, and the system learns case-specific query semantics dynamically according to the feedback.

There are some offshoots of CBIR. First, automated annotation attempts at automated concept discovery; what is more is that deciding on an appropriate picture set for a given story. Second, ranking or similarity of images is usually sorted by size, color depth, or shape; however, aesthetics may be another higher-level basis which involves the feelings or emotions of people. Moreover, CBIR may concern with possible security attack or image copy protection.

Finally, evaluation benchmark of CBIR must some key points: coverage, unbiasedness, and user focus. Ideally, it should be subjective, context-specific, and community-based.

How to give a good research talk

Reading : "How to give a good research talk," Jones et. al.

For a person who gives a 30-60 minutes talk, there are several suggestions to follow.

First, prepare appropriate content according to background knowledge of the audience. Omit unnecessary contents and remain the things that can convince the listeners of the primary topic. It may be more persuasive with precisely motivating examples. Second, use an overhead projector is effective. Put what is about to be explained on the slides, which not only saves the visual bandwidth but also gives the audience emphasis. Prepare the slides just one day before the talk may help us have fresh materials in our minds. Third, overcome nerves, show clear slides, and do not overrun. Try deep breathing or exercise may be effective to reduce nerves. Do not reveal a slide line by line or block people’s view. It is limited for a person to follow in a constant period of time, so take whether the listeners get the point into consideration before jump to the next section. Moreover, it is quite helpful to reorient the audience with a slide for each part.

In my opinion, prepare slides by typesetting with computer rather than writing by hand also has advantages for me. It saves time and does nice if someone is familiar with some software, such as Powerpoint. Besides, rehearsal several times before giving a talk is really helpful in my experience.

How to Read a Paper

Reading : "How to Read a Paper," Keshav, ACM SIGCOMM Computer Communication Review 2007

This writer shares an efficient method to read a paper. Because researchers may spent a lot of hours reading papers and it is terrible if wasting much effort in the process, he introduces the three-pass approach.

First of all, skim through the target paper quickly. Read only titles, abstract, introduction, and conclusion to get the main idea and contribution of that paper. After that, it is enough for us to decide whether we need to read further. Second, get the key points from figures, diagrams, and other illustrations. Graphs usually help us to know the thrust more clearly and show the results of the paper. After grasp more contents, it is time to skip the paper if abstruse or useless; otherwise, enter the third pass. In the third pass, do a deep reading to fully understand a paper. Pay attention to details, such as proves, assumptions, and particular techniques. After this pass, it should be able for us to reconstruct the overall idea and give practical comments. Besides, the writer suggests doing a literature survey iteratively. That is, after reading a paper, we may select relative papers to study according to citations and references.

The writer follows the discipline for many years. He is able to adjusts the depth of paper evaluation depending on his needs and how much time he has, and the three-pass approach really helps him to read a paper efficiently.