Re: [help-GIFT] GIFT Indexing

[David Squire]

Dayang Awg Iskandar wrote:
> Hi all,
>
> I installed GIFT without any errors and it worked fine with a small 
> image collection.
>
> I have indexed 169,685 images with GIFT and it took nearly two weeks 
> to complete it. I think this is the first ever attempt to index GIFT 
> with a large collection :) 

Nope. Others have done 50,000 images and more - if that's large :)

> My problem is that, the indexing process did complete, but there was 
> no *.ADI file generated and the inverted files:
> - InvertedFileFeatureDescription.db
> - InvertedFile.db
> - InvertedFileOffset.db
>
> did not contain anything. Below is the last part of the indexing 
> messages, and it seems that there was some failure:
>
> ********************************************************************************** 
>
> Processing File: 169685 CAccessorElement(169685 
> http://dayang.cs.rmit.edu.au/wikiCollection/wikipedia/99994.jpg 
> http://dayang.cs.rmit.edu.au/wikiCollection/wikipedia_thumbnail/99994_thumbnail_jpg.jpg 
> /home/dayang/public_html/wikiCollection//gift-indexing-data/Wikipedia//99994_jpg.fts)-- 
>
> ...
> h
> ...finished
> before mergesort
> STARTING mit MERGESIZE1
> MERGESORT MergeSize 12
> ../../libGIFTAcInvertedFile/include/merge_sort_streams.h:358 
> lToBeSorted false, after seekg(-2147483632)

This number is -2^31 (i.e. 1000 0000 0000 0000 0000 0000 0000 0000 in
two's complement notation, IIRC). It looks (as Henning suggests), as if
your OS and/or filesystem, like many, has an upper limit of 2Gb on file
size.

Regards,

David


--
Dr David McG. Squire, Senior Lecturer, on sabbatical in 2006
Caulfield School of Information Technology, Monash University, Australia
CRICOS Provider No. 00008C        http://www.csse.monash.edu.au/~davids/