help-gift
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[help-GIFT] Fwd: Big database on GIFT. gift-generate-inverted-file error


From: Wolfgang Müller
Subject: [help-GIFT] Fwd: Big database on GIFT. gift-generate-inverted-file error.
Date: Wed, 25 Feb 2004 08:41:35 +0100
User-agent: KMail/1.5

Forwarded message from Seventeen, to make discussion public.
-
Dear all,

I'm using gift-0.1.9 and has a image database which has 678929 images
(about 20G in size).

Few month ago, I devide these images into 5 directories and do
gift-add-collection.pl
This was ok to run without any error, just took a lots of time. (about
17 days on a P4 2.8G machine)

After I check the gift-add-collection.pl. I find that this perl script
mainly do 3 different things.

1. imagemagick convert to generated thumbnail and ppm files.
2. gift-extract-features and gift-write-feature-descs to get features
and creat .fts files.
3. gift-generate-inverted-file to generate InvertedFile.db.

I merge these 5 directories into single one directory and modify the
url2fts.xml.
The 1 & 2 stage dont need to be run again.
While I was doing gift-generate-inverted-file. It spends lots of time
(about 300 mins), and creat the gift-auxiliary-1 9.2G big. But finally
come out the following error message.

__FILE__:__LINE__: lToBeSorted false, after seekg(0)
__FILE__:__LINE__: lToBeSorted false, after seekg(-2147483648)
__FILE__:__LINE__: lToBeSorted false, after seekg(0)
__FILE__:__LINE__: lToBeSorted false, after seekg(-2147483648)

9.2G Feb 24 19:32 gift-auxiliary-1
672k Feb 24 11:46 InvertedFileFeatureDescription.db
173M Feb 24 11:46 url2fts.xml

Actually it's part of my master thesis work. I'm try to put GIFT with
big medial image database into a grid enviroment. I hope the grid
technology can give me a reasonable speed to run the GIFT with the large
medial image database. So I'd like to have a measurement of the speed of
1 nodes vs. 4, 8 16 or even 32 nodes.

I think the gift-add-collection.pl can speed up by put into distrubuted
cluster. And the idea to me is to seperate the big InvertedFile.db into
small one over lots of nodes. Then query these nodes (by
gift-mrml-client.pl) and get those result for finally processing.

So can you please tell me what's the error message mean and give me some
suggesstions.

Thanks in advance.

-- 
Seventeen Chen RHCE                                                        
Graduate Institute of Medical Informatics                                       
Center for Biomedical Informatics
Taipei Medical University
250 Wu Hsing Street                                                             
Taipei City 110                                                                 
TAIWAN                                                                          
TEL:+886-2-23776730 ext 107
Fax: +886-2-27279049                                                            
E-mail: address@hidden                      




reply via email to

[Prev in Thread] Current Thread [Next in Thread]