[help-GIFT] Introducing support for "active learning" queries

[Ralf Juengling]

Hi all,

I was thinking about how to introduce support for "active learning"
queries in MRML with minimal changes to the protocol.

To underline the problem first: What's different with user relevance 
in the "active learning" (AL) setting?
In both, the "classical" as well as the AL query-scheme, the CBIRS
prompts the user to "label" some images shown to him (i.e. to say 
which are relevant and which are not). In the classical setting,
the images shown to the user are the "the images to be labeled by
the user in order to make more precise, what she is looking for" 
and -- at the same time -- they are the "current best search result".
The idea in AL is to distinguish these two image sets.

So, what new MRML elements are reasonable to -- on the one hand -- 
distinguish these sets and -- on the other hand -- facilitate 
similar treatment of AL and classical CBIRS in a benchmark?

The first idea was to introduce a new query-step element as well
as a query-result element to clearly distinguish the two image sets.
However, this would prohibit similar treatment by a benchmark.

If you think about it, there's actually no need to introduce a new
query-result element: we simply allow the user label images of
both sets and those that got actually labeled will be send back
in the user-relevance-element-list by the client.

What's with the images in the query-result, which are sent from the
server to the client? Here one could distinguish the two image 
sets by use of two query-resultelement-lists, one of which bears
an attribute 'current-best-results="1"' and the other 
'current-best-results"0"' or so.
A client might display these two image sets within different panels;
a benchmark program would assess retrieval performance per query
step by merely considering the 'current-best-results="1"' images
but would label the images within the other query-resultelement-list
as well....

A different possibility is to distinguish the two image sets on
a per-image basis by a new "query-result-element"-attribute 
"relevance" (see below). Its possible values are

"known-relevant"
"known-irrelevant"
"guessed-relevant"
"guessed-irrelevant"
"unknown"

Omitting the relevance attribute would mean the same as
'relevance="unknown"'.

I think this approach has the benefit, that the client might use 
this information to support the user in the annoying labeling job.
The benchmark-programm again would assess retrieval performance
on the basis of the 'relevance=("guessed-relevant"|"known-relevant")'
images, but would label all images. 

<!ELEMENT query-result 
 ((query-resultelement-list*,query-result*)|error)>

<!ELEMENT query-result-element-list ((query-result-element|error)+)>
<!ELEMENT query-result-element (error?)>
<!ATTLIST query-result-element thumbnail-location      CDATA #REQUIRED
                               image-location          CDATA #REQUIRED
                               relevance               CDATA #IMPLIED
                               calculated-similarity   CDATA #IMPLIED
>


Please let me know what you think about this.

Cheers,
Ralf

-- 
-------------------------------------------------------------------------
Ralf Jüngling
Institut für Informatik - Lehrstuhl f. Mustererkennung &
Bildverarbeitung
Georges-Köhler-Allee
Gebäude 52                                       Tel:
+49-(0)761-203-8215
79110 Freiburg                                   Fax:
+49-(0)761-203-8262
-------------------------------------------------------------------------