CiteULike: zeppe image_labelling

Using Multiple Segmentations to Discover Objects and their Extent in Image Collections

Bryan Russell — 2007-11-03T02:57:08-00:00

(2006), pp. 1605-1614.

Object Recognition via Local Patch Labelling

Christopher Bishop — 2007-06-06T11:21:55-00:00

Deterministic and Statistical Methods in Machine Learning (2005), pp. 1-21.

In recent years the problem of object recognition has received considerable attention from both the machine learning and computer vision communities. The key challenge of this problem is to be able to recognize any member of a category of objects in spite of wide variations in visual appearance due to variations in the form and colour of the object, occlusions, geometrical transformations (such as scaling and rotation), changes in illumination, and potentially non-rigid deformations of the object itself. In this paper we focus on the detection of objects within images by combining information from a large number of small regions, or ‘patches’, of the image. Since detailed hand-segmentation and labelling of images is very labour intensive, we make use of ‘weakly labelled’ data in which the training images are labelled only according to the presence or absence of each category of object. A major challenge presented by this problem is that the foreground object is accompanied by widely varying background clutter, and the system must learn to distinguish the foreground from the background without the aid of labelled data. In this paper we first show that patches which are highly relevant for the object discrimination problem can be selected automatically from a large dictionary of candidate patches during learning, and that this leads to improved classification compared to direct use of the full dictionary. We then explore alternative techniques which are able to provide labels for the individual patches, as well as for the image as a whole, so that each patch is identified as belonging to one of the object categories or to the background class. This provides a rough indication of the location of the object or objects within the image. Again these individual patch labels must be learned on the basis only of overall image class labels. We develop two such approaches, one discriminative and one generative, and compare their performance both in terms of patch labelling and image labelling. Our results show that good classification performance can be obtained on challenging data sets using only weak training labels, and they also highlight some of the relative merits of discriminative and generative approaches.

Incorporating Semantic Constraints into a Discriminative Categorization and Labelling Model.

A Quattoni — 2007-05-18T15:15:29-00:00

Computer Vision, 2005. Tenth IEEE International Conference on (2005), pp. 1877-1877.

This paper describes an approach to incorporate semantic knowledge sources within a discriminative learning framework. We consider a joint scene categorization and region labelling task and assume that some semantic knowledge is available. For example we might know what objects are allowed to appear in a given scene. Our goal is to use this knowledge to minimize the number of fully labelled examples (i.e. data for which each region in the image is labelled) required for learning. For each scene category the probability of a given labelling of image regions is modelled by a Conditional Random Field (CRF). Our model extends the CRF framework by incorporating hidden variables and combining class conditional CRFs into a joint framework for scene categorization and region labelling. We integrate semantic knowledge into the model by constraining the configurations that the latent region label variable can take, i.e. by constraining the possible region labelling for a given scene category. In a series of synthetic experiments, designed to illustrate the feasibility of the approach, adding semantic constraints about object entailment increased the region labelling accuracy given a fixed amount of fully labelled data.

Multiscale conditional random fields for image labeling

Xuming He — 2007-05-16T14:31:52-00:00

Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, Vol. 2 (2004), pp. II-695-II-702 Vol.2.

We propose an approach to include contextual features for labeling images, in which each pixel is assigned to one of a finite set of labels. The features are incorporated into a probabilistic framework, which combines the outputs of several components. Components differ in the information they encode. Some focus on the image-label mapping, while others focus solely on patterns within the label field. Components also differ in their scale, as some focus on fine-resolution patterns while others on coarser, more global structure. A supervised version of the contrastive divergence algorithm is applied to learn these features from labeled image data. We demonstrate performance on two real-world image databases and compare it to a classifier and a Markov random field.