Re: [lmi] Proposed workflow for proprietary repository

[Greg Chicares]

On 2016-03-20 19:50, Greg Chicares wrote:
[...]
> Here's how to create a bundle to share by email. Make any number of
> changes to the local repository, and commit them in whatever groups
> make sense:
> 
>   cd /opt/lmi/proprietary
>   git commit --all -m"One set of changes"
>   git commit --all -m"Another set of changes"
> 
> When it's ready to share, do this:
> 
>   cd /opt/lmi/proprietary
>   git bundle create YourBundleName ^origin/master HEAD --branches
> 
> (using a more descriptive name) and send the file as an email
> attachment.

Vadim--This has worked perfectly until about a month ago. Since then,
every time Kim prepares a bundle, permissions on many files get changed.
Please help us figure out how to prevent this.

Applying a bundle sent yesterday (possibly from a new machine configured
by corporate IT), I see:

data/NAME-REDACTED.database          |  4 ++--  [for each of six files]
src/my_db.cpp                        | 12 ++++++------

That's six files with four lines changed in each (24) plus one with 12,
for a total of 36, which is the same total as the summary line:

1030 files changed, 18 insertions(+), 18 deletions(-)

but there are just over a thousand lines like this:

data/sample.database                 |  0

and then another thousand or so like this:

mode change 100644 => 100755 data/sample.database

No binaries are stored in this proprietary repository, and no scripts
outside its hooks/ directory, so we really do want 644 for everything
except for these two hooks/ scripts that should use 755:

/opt/lmi/proprietary[0]$ls -o hooks/
total 8
-rwxr-xr-x 1 greg 1216 Aug 17 20:12 commit-msg
-rwxr-xr-x 1 greg 1662 Aug 17 20:12 pre-commit

Both those scripts are identical to the corresponding scripts in the
public savannah repository except that the pre-commit hook recursively
runs 'make check_concinnity' in all subdirectories of the proprietary
repository. But that makefile target diagnoses these mode changes as
errors and should have blocked the commit.

My guess is that the script wasn't run in this case because the msw
filesystem migration failed to preserve cygwin symlinks, and we must
do this:
  cd /opt/lmi/proprietary
  ln --symbolic --force --no-dereference ../hooks .git
manually each time we get a new machine. However, that technique only
_detects_ undesirable permission changes, yet I'd like to find a way
to _prevent_ such problems from arising. For example, several hundred
of these files are freshly regenerated by msw-native programs (rather
than being migrated from another machine), and it would be helpful to
intercept and reverse permission changes before a git bundle is
created.

I hesitate to suggest doing the following (for cygwin only):
  git config core.filemode false
because it seems like a kludge, and data files really shouldn't ever
have the executable bit set.

I'm thinking of adding appropriate commands at the beginning of the
pre-commit hook, something like [not quite correct]:
  chmod +R -x+X
in the hope that this will prove inexpensive and perfectly robust.
If that sounds like a good idea, could you help me figure out how
to write the command correctly?

[...here, as so often, my stream of consciousness leads me through
a thicket of ideas whose inutility becomes apparent only after they
have been written out, and I'm unwilling to delete them in case I've
dismissed good ideas for bad reasons, but it seems only polite to
suggest you read from the bottom up, starting with section [3] and
stopping at the bottommost line where the problem is well solved...]

[1] I wrote "-x+X" above to remove execute permission yet retain the
flag that makes directories work normally, but even if that's what
it really does, it's not right because some files really should be
executable. I've seen commands like these suggested, e.g. on
stackoverflow:

# First make subdirectories accessible:
  find . -type d -print0 | xargs -0 chmod 755
# Then remove execute permission on all files:
  find . -type f -print0 | xargs -0 chmod 644
# Then restore execute permission on selected files:
  find . -name '*.sh' -type f | xargs chmod +x

where they always lead to arguments: some commenters want to avoid
'find ... xargs', while others suggest slight syntax changes.

The rules I seem to want are:
  +X for directories
  -x for most files
  +x for '*.sh', '*.sed', 'hooks/*', and 'debian/rules'
but that feels wrong because it doesn't provide for new files that
should be executable...and now I see that I forgot to mention the
scripts under lmi's tabs/ subdirectory.

[2] Maybe I should invert the sense of this, recognizing that Kim
isn't going to add any executable file ever and that the only problem
is "+x" leaking through from her msw+cygwin setup, so perhaps I could
come up with a reliable list of files that should never have "+x":
  md5sums *.txt *.?pp *.cns *.gpt *.ill *.mec *.ini *.inix
  *.database *.funds *.policy *.rounding *.strata *.rates
Ewww--on second thought, that doesn't seem like a good approach;
and, oops, I forgot '*Log' and probably some others.

[3] For my third attempt, I observe that this rule in 'GNUMakefile'
has worked very well for years:

        @$(LS) --classify ./* \
          | $(SED) -e'/\*$$/!d' -e'/^\.\//!d' -e'/.sh\*$$/d' -e'/.sed\*$$/d' \
          | $(SED) -e's/^/Improperly executable: /'

for each directory in which we run 'make check_concinnity'
(which excludes hooks/ and debian/). And the pre-commit hook
for the proprietary repository already names every subdirectory
we care about (and would have to be modified anyway if we ever
add another subdirectory, but that might happen once a decade):

printf "checking "
printf "src..."
cd $toplevel/src    && check_concinnity
printf "data..."
cd $toplevel/data   && check_concinnity
printf "test..."
cd $toplevel/test   && check_concinnity
printf "tables..."
cd $toplevel/tables && check_concinnity

so maybe I should just copy that logic...or restructure it thus:
  for d in src data test tables; do \
    printf "$d..." && cd $toplevel/$d && MAGICAL_COMMAND \
  done
and then just ask your opinion on what MAGICAL_COMMAND should be.
Maybe use some 'find ... xargs' thing to exclude '*.sh' and '*.sed',
and apply 'chmod -x' to every file not excluded? E.g. [untested]:

find . -name '*.sh' -o -name '*.sed' -type f | xargs chmod -x

Would that work reliably in a directory with thousands of files?
(I think is should, and that's why we use 'find', but I don't
understand this stuff very well.)