Re: [Openexr-devel] [Openexr-user] Per-pixel Object Identifiers?

[Florian Kainz]

Hi Ger,

In the file, the objectID attribute will store the mapping from pixel values
to lists of strings in a compact form, without any indexing structure.  The
data will be expanded into a form that can be queried quickly during reading,
and packed tightly during writing.  The in-memory form of the attribute will
probably consist of two STL maps (which are usually implemented as RB trees
or AVL trees).

Regarding your "nitpicking": you are absolutely correct, the scheme is limited
by the number of unique per-pixel object sets, not by the number of pixels
in the image.  However, there cannot be more unique object sets than pixels,
so the scheme is guaranteed to work for any image with fewer than 2^32 pixels.

Florian

Ger Hobbelt wrote:
>     This will require storing per-pixel object identifiers in an OpenEXR
>     file.
>     In order to avoid re-inventing the wheel, I would like to find out
>     if any
>     OpenEXR user have done something like this already. If you have done it,
>     would you be willing to share how you did it?
>
>
> Did do something like that, but hadn't thought of using custom 
> attribute(s) to store the related data at the time, so I ended up 
> storing the (integer) references in the EXR and using an external store 
> for the mapping, using the integers (one per pixel) as record ids. Each 
> (externally stored) record could list one to many items.
>
> If I read correctly what you intend to do, then I suggest you provide an 
> option to page-align the attribute data: that way you can memorymap the 
> metadata and smartly employ any page-oriented tree storage algos. When 
> you think of the pixel to objectID collection mapping (N:M relation) as 
> a database relation/table, the memorymap/align suggestion becomes pretty 
> obvious: when you've got that data page-aligned you can just point your 
> b- or r-tree at the attribute data space and have a field day. :-) (fast 
> retrieval of arbitrary large sets of objects per pixel. When OpenEXR can 
> provide the handle and a pointer/reference at the (aligned) start of the 
> attribute space, anyone can do what they want (think of it as a 
> attribute acting as 'blob', the rest of this stuff can be done 
> application side. No need to load OpenEXR with the peculiars.)
>
>
>     Also, is there any interest in per-pixel object identifiers outside ILM?
>
>
> If you plan to do something like the above (which would be only an 
> augmentation of the attributes as far as OpenEXR itself is concerned, I 
> suppose -- I assume the knowledge how to relate the pixel int32 to 
> attribute X is kept at the application side), then yes, I'm interested!
>
>  
>
>        With 32-bit UINT pixel values this scheme could run out of object
>        identifiers for images with more than four Gigapixels, but in VFX
>        production images that large are rare.
>
>
> Nitpicking: wouldn't that rather imply a limit of 2^32 max 'references' 
> and no limit on the number of pixels themselves? (in your case: 2^32 
> sets of objects, each unique set having it own reference int32, so 
> multiple pixels can have the same reference int32 value when they are 
> covered by the same set of objects in your scene. Only when you don't 
> want to spend the effort of combining identical sets for multiple pixels 
> does this translate to a 2^32 pixel limit.)
> Or am I completely off track here?
>
>
> (2^32 is okay with me, unless you want to provide for hash-based indexes 
> and then I'd _hugely_ favor a 64-bit int per pixel, instead. But that's 
> just picking the right type of pixel channel, anyway. Could even map 
> float to attribute. I had just never considered storing such a load of 
> data in an attribute and that's the idea here if I read it correctly. 
> Cool!!    Now I hope AE/PS are already capable of ignoring such large 
> attributes; then I can keep on using them for visualization while 
> storing some other interesting tidbits in the EXR, as well.)
>
>  
> --
> Met vriendelijke groeten / Best regards,
>
> Ger Hobbelt
>
> --------------------------------------------------
> web:    http://www.hobbelt.com/
>         http://www.hebbut.net/
> mail:   address@hidden <mailto:address@hidden>
> mobile: +31-6-11 120 978
> --------------------------------------------------