Given the knowledge that the same or similar objects appear in a set of images, our goal is to segment that object from all of the images simultaneously. To solve this problem, known as the cosegmentation problem, we present a method based upon hierarchical clustering. Our framework first eliminates intra-class heterogeneity in a dataset by clustering similar images together into smaller groups. Then, our method utilizes multiple layers of segmentation from a single image in order to fully encapsulate the local information available. In order to relate the different hierarchical layers to one another, we introduce region (e.g. superpixel) intra-image constraints. Next, we take advantage of the information available from other images in our group. To accomplish this task, we design and present an efficient method that connects image regions from one image to all other images in a group. Given the intra & inter image connections, we perform a segmentation of the group of images into foreground and background regions. Finally, we compare our segmentation accuracy to several other state-of-the-art segmentation methods on standard datasets, and also demonstrate the robustness of our method on real world data.

Suppose you want to effectively search through millions of images, train an algorithm to perform image and video object recognition, or research the complex patterns and relationships that exist in our visual world. A common and essential component for any of these tasks is a large annotated image dataset. However, obtaining labeled image data is a complex and tedious task that requires methods for annotating and structuring content. Therefore, we developed a comprehensive online tool and data structure, Markup SVG, that simplifies the collection of annotated image data by leveraging state of the art image processing techniques. As the core data structure of our tool, we adopt Scalable Vector Graphics (SVG), an extensible and versatile language built upon XML. Given the extensibility of our framework, we are able to encode low level image features, high level semantics, and further define interactions with the data to assist the user with image annotation. We also demonstrate the ability to merge multiple online and offline datasets into our system in an effort to standardize image collection and its data representation. Lastly, we present our modular design; each component acts as a plug-in to our system. We developed several novel components and algorithms to highlight the possibilities of semi-supervised segmentation and automatic annotation within our proposed framework. Further, our modular design provides the necessary capabilities to incorporate future image features, methods, or algorithms. Our results show that our tool is able to greatly simplify the process of obtaining large annotated image collections in an online collaborative platform.

Cervical cancer is one of the leading causes of death for women world-wide. Early detection of cervical cancer is possible through regular screening; however, in developing countries, screening and treatment options are limited due to poor (or lack of) resources. Fortunately, low cost screening procedures utilizing visual inspection after the application of acetic acid in combination with low cost DNA tests to detect HPV infections have been shown to reduce cervical cancer by nearly 30%. To assist in this procedure, we developed an automatic, data centric system for cervigram (photographs of the cervix) image analysis. In the first step of our algorithm, our system utilizes nearly a thousand annotated cervigram images to automatically locate a cervix region of interest. Next, by utilizing both color and texture features extracted from the cervix region of interest on several thousand cervigrams, we show that our system is able to perform a binary classification on cervigram images with comparable accuracy to a trained expert. Finally, we analyze and report the effect that the color and texture features have on our end classification result.

We present a semantic based search tool, VIPs, i.e. Visual Image Persons Search, on the domain of VIPs, i.e. very important people. Our tool explores the possibilities of content based image search supported by ontological reasoning. Our framework integrates information from both image processing algorithms and semantic knowledge bases to perform interesting queries that would otherwise be impossible. We describe a novel property reasoner that is able to translate low level image features into semantically relevant object properties. Finally, we demonstrate interesting searches supported by our framework on the domain of people, the majority of whom are movie celebrities, using the properties translated by our system as well as existing ontologies available on the web.

In clinical decision processes, relevant scientific publications and their associated medical images can provide valuable and insightful information. However, effectively searching through both text and image data is a difficult and arduous task. More specifically in the area of image search, finding similar images (or regions within images) poses another significant hurdle for effective knowledge dissemination. Thus, we propose a method using local regions within images to perform and refine medical image retrieval. In our first example, we define and extract large, characteristic regions within an image, and then show how to use these regions to match a query image to similar content. In our second example, we enable the formulation of a mixed query based upon text, image, and region information, to better represent the end user's search intentions. Given our new framework for region-based queries, we present an improved set of similar search results.

We present a novel method for 3D brain tumor volume segmentation based on a parallel cellular automata framework. Our method incorporates prior label knowledge gathered from user seed information to influence the cellular automata decision rules. Our proposed method is able to segment brain tumor volumes quickly and accurately using any number of label classifications. Exploiting the inherent parallelism of our algorithm, we adopt this method to the Graphics Processing Unit (GPU). Additionally, we introduce the concept of individual label strength maps to visualize the improvements of our method. As we demonstrate in our quantitative and qualitative results, the key benefits of our system are accuracy, robustness to complex structures, and speed. We compute segmentations nearly 45x faster than conventional CPU methods, enabling user feedback at interactive rates.

As medical imaging rapidly expands, there is an increasing need to structure and organize image data for efficient analysis, storage and retrieval. In response, a large fraction of research in the areas of content-based image retrieval (CBIR) and picture archiving and communication systems (PACS) has focused on structuring information to bridge the semantic gap, a disparity between machine and human image understanding. An additional consideration in medical images is the organization and integration of clinical diagnostic information. As a step towards bridging the semantic gap, we design and implement a hierarchical image abstraction layer using an XML based language, Scalable Vector Graphics (SVG). Our method encodes features from the raw image and clinical information into an extensible layer that can be stored in a SVG document and efficiently searched. Any feature extracted from the raw image including, color, texture, orientation, size, neighbor information, etc., can be combined in our abstraction with high level descriptions or classifications. And our representation can natively characterize an image in a hierarchical tree structure to support multiple levels of segmentation. Furthermore, being a world wide web consortium (W3C) standard, SVG is able to be displayed by most web browsers, interacted with by ECMAScript (standardized scripting language, e.g. JavaScript, JScript), and indexed and retrieved by XML databases and XQuery. Using these open source technologies enables straightforward integration into existing systems. From our results, we show that the flexibility and extensibility of our abstraction facilitates effective storage and retrieval of medical images.

The accurate and automatic segmentation of tissue regions in cervigram images can aid in the identification and classification of precancerous regions. We implement and analyze four GPU (Graphics Processing Unit) based clustering algorithms: K-means, mean shift, deterministic annealing, and spatially coherent deterministic annealing. From our results, we propose a novel parallel algorithm using the CUDA programming language for digital cervigram segmentation and clustering. The first step of our fully automatic method is to compute the number of modes in the feature space of a color cervigram image using the mean shift clustering algorithm. Next, we use the number of modes in a novel spatially coherent deterministic annealing optimization technique to produce an approximate optimal solution for the clustering problem. Our GPU based methods perform approximately 38x (deterministic annealing), 134x (mean shift), and 276x (spatially coherent deterministic annealing) faster than an equivalent CPU solution. Our implementation decreases the computational time of an annealing method on a 1280x872 pixel image from 5 hours 3 minutes to 72.12 seconds, enabling the use of this optimization method in clinical settings and on large cervigram datasets.