Re: [lmi] Proposed workflow for proprietary repository

[Greg Chicares]

First let me ask whether you received the message that you replied
to here in normal email, because I did not, not even at my corporate
email account that is also subscribed to this mailing list.

On 2017-11-10 22:16, Vadim Zeitlin wrote:
> On Fri, 10 Nov 2017 17:59:09 +0000 Greg Chicares <address@hidden> wrote:
> 
> GC> On 2016-03-20 19:50, Greg Chicares wrote:
> GC> [...]
> GC> > Here's how to create a bundle to share by email. Make any number of
> GC> > changes to the local repository, and commit them in whatever groups
> GC> > make sense:
> GC> > 
> GC> >   cd /opt/lmi/proprietary
> GC> >   git commit --all -m"One set of changes"
> GC> >   git commit --all -m"Another set of changes"
> GC> > 
> GC> > When it's ready to share, do this:
> GC> > 
> GC> >   cd /opt/lmi/proprietary
> GC> >   git bundle create YourBundleName ^origin/master HEAD --branches
>                                          ^^^^^^^^^^^^^^^^^^^
>  This is off-topic, but I'd just like to note that now that you're aware of
> Git "A..B" notation, you could replace this part with "origin/master..HEAD"
> which is a bit a more understandable IMHO as closer to the standard
> interval notation (except in Git intervals are open-closed semi-intervals).

Thanks for pointing that out. I had forgotten what the '^' above means, but:
  https://git-scm.com/docs/gitrevisions
| ^r1 r2 means commits reachable from r2 but exclude the ones reachable from r1
That '^'-prefixed syntax does seem unnatural, so I've updated 'gwc/develop2.txt'
to use the more comprehensible ".." form.

> GC> > (using a more descriptive name) and send the file as an email
> GC> > attachment.
> GC> 
> GC> Vadim--This has worked perfectly until about a month ago. Since then,
> GC> every time Kim prepares a bundle, permissions on many files get changed.
> GC> Please help us figure out how to prevent this.
> 
>  Brief answer is to set core.filemode to false. When I started to use Git
> with Cygwin (admittedly, this was a long -- was it really 9 years?? -- time
> ago), I experimented with different settings/combinations of file
> modes/ACLs and this was the only possibility to make things work reliably.
> Maybe things have changed since then, but your experience seems to disprove
> this and in any case the one thing I'm sure about is that they still work
> fine with core.filemode set to false.

[...and it's twice as fast...]

What are the exact consequences, in a world where:
 - Kim uses msw, and I do not
 - Kim sets 'core.filemode' to false, and I do not
 - Kim adds or modifies nonexecutable files often, but executable ones never
? AFAICT, under those assumptions, any file she touches is mode 644, but all
the files she ever touches should be 644, so this really will just work.

> GC> Applying a bundle sent yesterday (possibly from a new machine configured
> GC> by corporate IT)

[...and they're going to re-reconfigure it soon, causing another thousand
headaches...]

> GC> mode change 100644 => 100755 data/sample.database
> 
>  If you really want to debug this, you need to look at MSW ACL for this
> file, using cacls or icacls as discussed just recently.

Thanks for explaining that. Those commands remind me of the way Dickens
named characters--every time I struggle with them, I feel like their
inventor is cackling in sick delight at the vexation he's causing me:
"I cackles at you, Mr. Greg! Stamps me feet and cackles, I does!"
Preferring brute force to being thrown into a "purple leptic fit" by
all that cackling, we just obliterated the offending files mentioned
in that recent message and recreated them.

>  Also, I'm not completely sure if the file is actually executable on the
> machine where "git bundle" was made, could you please confirm whether it is
> the case?

I'm pretty sure it does have the executable bit set, from my own
experience using cygwin. I can't say that I ever actually tried
to execute such a file, but I have seen the 'x' bit set.

> GC> [...] yet I'd like to find a way
> GC> to prevent such problems from arising. For example, several hundred
> GC> of these files are freshly regenerated by msw-native programs (rather
> GC> than being migrated from another machine), and it would be helpful to
> GC> intercept and reverse permission changes before a git bundle is
> GC> created.
> 
>  Unfortunately I don't think there are any hooks invoked during git bundle
> creation.

Oh. Right. I was hoping that if I ran 'chmod' early enough in the hook,
it would modify files before they got committed, but by the time the
hook begins to execute, it's too late.

> And, anyhow, detecting the problem at this moment could be too
> late to be helpful. So the only solution I see is to indeed ensure that
> commit hooks check for this -- and are executed, of course. Maybe you could
> print something from the commit hook even if all checks passed successfully
> (of course, I advise this, but I'd hate it for the main lmi repository...)
> and ask everybody committing to it to check that this message was given.

OTOH, looking for an anticipated "Okay!" message is not as good as full
automation; and if it is not observed, then manual cleanup is necessary.
Instead, because we really want a cut-and-pastable series of maximally
robust commands, I think I'll amend the instructions thus:

   # When everything is ready to share...
 
 cd /opt/lmi/proprietary
+printf "forcing correct permissions "
+ for d in src data test tables; do (\
+      printf "$d..." \
+   && find ./$d -maxdepth 1 -type f -not -name '*.sh' -not -name '*.sed' | 
xargs chmod -x \
+ )done
 git bundle create YourBundleName origin/master..HEAD --branches

I tested that, and it seems to do the right thing, quickly enough...

/opt/lmi/proprietary[0]$time (for d in src data test tables; do (\
      printf "$d..." \
   && find ./$d -maxdepth 1 -type f -not -name '*.sh' -not -name '*.sed' | 
xargs chmod -x \
 )done)
src...data...test...tables...( for d in src data test tables; do; ( printf 
"$d..." && find ./$d -maxdepth )  0.00s user 0.00s system 25% cpu 0.031 total

....that it's probably tolerable even on slow magnetic drives.

> GC> I hesitate to suggest doing the following (for cygwin only):
> GC>   git config core.filemode false
> GC> because it seems like a kludge, and data files really shouldn't ever
> GC> have the executable bit set.
> 
>  Honestly, if this solves the problem -- and I think it ought to -- I'd do
> it without a second thought. Yes, core.files is a kludge, but then the
> entire concept of Cygwin is a kludge anyhow, so I don't think it's worth
> fighting for immaculate purity when using it.

Okay, that logic seems compelling.

> GC> find . -name '*.sh' -o -name '*.sed' -type f | xargs chmod -x
[...]
> I guess it's just a typo, but the logic of the command above is reversed:

Yes, your message arrived just as I was about to correct that.

>  Other than that the command looks fine to me. Purists would argue that you
> should use "find -printf0" and "xargs -0", but as long as you avoid white
> space in your file names (which is an excellent idea anyhow), this is not
> necessary.

None of us is ever going to commit a file with whitespace in its name,
but perhaps I should add a test like
  find -type f -name "* *"
to 'make check_concinnity' anyway...someday, when I remove the
  $(LS) --classify ./* | sed ...
horror in 'GNUmakefile'.

>  And I think the idea of making non-executable all but whitelisted files is
> a better one than blacklisting some files because if you add another
> category of files that should be executable and their executable bit is
> reset by this script, it will get noticed much sooner than if you add some
> new type of non-executable files and forget to blacklist it.

That seems so obvious when you express it in five lines rather than
the hundred or so lines it took me to demonstrate its contrapositive.