Problem with find + AFS + acl="l"

[Daniel Richard G.]

I am using find(1) at an AFS site to produce file lists for indexing. This 
includes files in directories for which only the "l" (list) ACL permission 
is available. In these directories, you can only tell if an entry is a 
subdirectory or not; I am assuming that non-subdirectory entries are 
regular files (almost always true) and including them in the lists 
accordingly.

The current behavior of find(1) is problematic because in these restricted 
directories, readdir() returns a dirent->d_type of either 4 (DT_DIR, if the 
entry is a directory) or 0 (DT_UNKNOWN, if it is anything else). Though I 
am not using a -type test, find(1) calls lstat() on the DT_UNKNOWN entries, 
which invariably returns -1/EACCES. I've observed cases where each failed 
stat takes *two or three* seconds to return, due to AFS weirdness, and even 
when it returns immediately the sheer number of stats performed needlessly 
spikes network traffic and latency.

I've confirmed that find(1) works very nicely here if it does the readdir() 
and skips the stats. I experimentally hacked lib/savedirinfo.c to set 
d_type=DT_REG in cases where it was actually DT_UNKNOWN, and the resulting 
program tore through our entire cell much faster than I'd ever dare to 
guess.

I'd like to toss in a couple of ideas that would enable this, without any 
specificity to AFS:

        1. New find(1) option: -nostat_unknown. Basically, "don't stat a 
           path name if its type cannot be determined from reading its 
           parent directory."

        2. New argument for -type: "u" (i.e. unknown type). This would 
           match if the condition (d_type == DT_UNKNOWN && (nostat_unknown
           || stat() == -1)) is true.

This way, I could invoke

        find /afs/example.com/volume -nostat_unknown -type f -o -type u

or, perhaps,

        find /afs/example.com/volume -nostat_unknown ! -type d

to obtain my desired result.


Thoughts and comments will be greatly appreciated.


--Daniel


-- 
NAME   = Daniel Richard G.       ##  Remember, skunks       _\|/_  meef?
EMAIL1 = address@hidden        ##  don't smell bad---    (/o|o\) /
EMAIL2 = address@hidden      ##  it's the people who   < (^),>
WWW    = http://www.******.org/  ##  annoy them that do!    /   \
--
(****** = site not yet online)