LIRE（Lucene Image REtrieval）

zz from: http://hi.baidu.com/johnsoncr/blog/item/953d9f95f9aab9057af48078.html

 

LIRE（Lucene Image REtrieval）提供一种的简单方式来创建基于图像特性的Lucene索引。利用该索引就能够构建一个基于内容的图像检索(content- based image retrieval，CBIR)系统，来搜索相似的图像。LIRE使用的特性都取自MPEG-7标准： ScalableColor、ColorLayout、EdgeHistogram。此外该类库还提供一个搜索该索引的方法。

官方网址：http://www.semanticmetadata.net/lire/

文档 网址：www.semanticmetadata.net/wiki/doku.php

 

Creating an Index with Lire

Use the DocumentBuilderFactory to create a DocumentBuilder, for instance withDocumentBuilderFactory.getExtensiveDocumentBuilder(). Add images to an index using following steps:

• With this  DocumentBuilder Lucene documents can be created from images, for instance with builder.createDocument(FileInputStream, String).
• Eventually enrich the documents with your own data.
• Add the document to an index.

For a really big index use .getFastDocumentBuilder() method from DocumentBuilderFactory, otherwise the.getDefaultDocumentBuilder() or the .getExtensiveDocumentBuilder(). Following MPEG-7 descriptors are used by the different DocumentBuilders:

• Fast: ColorLayout
• Default: ColorLayout and EdgeHistogram
• Extensive: ColorLayout, EdgeHistogram and ScalableColor

For more information please consult the Java API Doc. Sample code is available in the sources, take a look at the JUnit test classes.

 

Sample Code

Note that the way of opening an index with the IndexReader and IndexWriter of Lucene has changed with Lucene 3.0. I assume to use LIRe v0.8, which already supports Lucene 3.0.1

/**
 * Simple index creation with Lire
 *
 * @author Mathias Lux, mathias@juggle.at
 */
public class CreateIndexTest extends TestCase {
    private String[] testFiles = new String[]{"img01.JPG", "img02.JPG", 
  "img03.JPG", "img04.JPG", "img05.JPG"};
    private String testFilesPath = "./src/test/resources/images/";
    private String indexPath = "test-index";
    private String testExtensive = "../Caliph/testdaten";
 
    public void testCreateIndex() throws IOException {
        // Create an appropriate DocumentBuilder
        DocumentBuilder builder = DocumentBuilderFactory.getExtensiveDocumentBuilder();
        // That's the way it is done with Lucene 3.0 - supported with LIRe v0.8
        IndexWriter iw = new IndexWriter(FSDirectory.open(new File(indexPath)), new SimpleAnalyzer(), true, IndexWriter.MaxFieldLength.UNLIMITED);
        for (String identifier : testFiles) {
            // Build the Lucene Documents
            Document doc = builder.createDocument(new FileInputStream(testFilesPath + 
    identifier), identifier);
            // Add the Documents to the index
            iw.addDocument(doc);
        }
        iw.optimize();
        iw.close();
    }
}
 

Searching with Lire

Use the ImageSearcherFactory for creating an ImageSearcher, which will retrieve the images from the index. This can be done by calling ImageSearcherFactory.createDefaultSearcher(). The ImageSearcher will query for an image, given by an InputStream or a BufferedImage, or a Lucene Document describing an image, for instance with the method search(BufferedImage, IndexReader) or search(Document, IndexReader).

Please note that the ImageSearcher uses a Lucene IndexReader and does the retrieval with a linear search in the index. The results are returned as ImageSearchHits object, which aims to simulate a Lucene Hits object.

Note also that the IndexSearcher only uses image features, which are available in the specific Document in the index. If documents only have been indexed with the fast DocumentBuilder there is no ColorHistogram or EdgeHistogram feature available in the indexed documents, only the ColorLayout feature.

 

Searching with Weights

Within the ImageSearcherFactory one can use the method createWeightedSearcher(int maximumHits, float colorHistogramWeight, float colorDistributionWeight, float textureWeight) to adjust the weights for searching. Note that the extensive DocumentBuilder has to be used to build the index to discover the full potential of weighting.

Also note that only weights in [0,1] are allowed and the sum of the weights has to be greater than 0. 

 

// [...] start snippet --------
// That's for Lucene v3.0+
IndexReader reader = IndexReader.open(FSDirectory.open(new File(indexPath)));
// three different possible versions ... for finding the 10 most relevant pictures
ImageSearcher searcher = ImageSearcherFactory.createWeightedSearcher(10, 0.2f, 0.8f, 1.0f);
// ImageSearcher searcher = ImageSearcherFactory.createWeightedSearcher(10, 0.8f, 0.0f, 1.0f);
// ImageSearcher searcher = ImageSearcherFactory.createWeightedSearcher(10, 0.0f, 1.0f, 0.0f);
FileInputStream imageStream = new FileInputStream(testFilesPath + testFiles[0]);
BufferedImage bimg = ImageIO.read(imageStream);
ImageSearchHits hits = null;
hits = searcher.search(bimg, reader);
for (int i = 0; i < 5; i++) {
      System.out.println(hits.score(i) + ": " 
                   + hits.doc(i).getField(DocumentBuilder.FIELD_NAME_IDENTIFIER).stringValue());
}
// [...] end snippet -------- 

 

Sample Code for a Simple Search Implementation

/**
 * Simple image retrieval with Lire
 * @author Mathias Lux, mathias <at> juggle <dot> at
 */
public class TestImageSearcher extends TestCase {
    private String[] testFiles = new String[]{"img01.JPG", "img02.JPG", 
  "img03.JPG", "img04.JPG", "img05.JPG"};
    private String testFilesPath = "./src/test/resources/images/";
    private String indexPath = "test-index";
 
    public void testSearch() throws IOException {
 // Opening an IndexReader (Lucene v3.0+)
        IndexReader reader = IndexReader.open(FSDirectory.open(new File(indexPath)));
 // Creating an ImageSearcher
        ImageSearcher searcher = ImageSearcherFactory.createDefaultSearcher();
 // Reading the sample image, which is our "query"
        FileInputStream imageStream = new FileInputStream(testFilesPath + testFiles[0]);
        BufferedImage bimg = ImageIO.read(imageStream);
 // Search for similar images
        ImageSearchHits hits = null;
 hits = searcher.search(bimg, reader);
 // print out results
        for (int i = 0; i < 4; i++) {
            System.out.println(hits.score(i) + ": " + 
    hits.doc(i).getField(DocumentBuilder.FIELD_NAME_IDENTIFIER).stringValue());
        }
  
 // Get a document from the results
        Document document = hits.doc(0);
 // Search for similar Documents based on the image features
 hits = searcher.search(document, reader);
        for (int i = 0; i < 4; i++) {
            System.out.println(hits.score(i) + ": " + 
    hits.doc(i).getField(DocumentBuilder.FIELD_NAME_IDENTIFIER).stringValue());
        }
    }