[help-GIFT] Large collections: experiences?

[Roland Schweter]

Hi,

 

I would like to know if there are any installations of gift dealing with collections of a number greater than say 40 thousands images or so. Reason is: I have to deal with such a collection and ran into problems with value-ranges of integer-type while indexing :)

Last lines of gift-add-collection were:

 

MERGESORT MergeSize 805306368
endmerge
MERGESORT MergeSize 1610612736
endmerge
MERGESORT MergeSize -1073741824

 

PROGRESS: 99%

Copying /data02/gift/gift-config.mrml to /data02/gift/gift-config.mrml-old

PROGRESS: 100%

Gift was starting but didn't feel well afterwards and crashed. Well, I said, given that the number doubles every iteration the mergesize would hit the range of "signed int" the last time. What I did was changing the type in the sources to be a "unsigned long" which made the indexing succeed (libGIFTAcInvertedFile/include/merge_sort_streams.h).

 

Unfortunately I'm stuck to version 0.1.9, compiling more recent versions failed because of compiler-issues, so this might already be fixed in the current version. Is it?

 

The gift gives amazing results most of the time, sometimes the result are - ehhh... - a bit strange though. Now I think there might be other places where similar issues could come up with that collection-size, and I wonder if that's the reason for the rather esoteric results. Changing the build-environment is on the way but will take some time. Maybe there are other ways like reducing the number of extracted features or so?

 

So where are these large collections and how are the experiences?

 

Best regards,

Roland