Re: [lmi] Branching to replace XSL-FO

[Greg Chicares]

On 2017-10-28 00:56, Vadim Zeitlin wrote:
> On Sat, 28 Oct 2017 00:27:32 +0000 Greg Chicares <address@hidden> wrote:
> 
> GC> Suppose I now want to bring all the new '*.mst' files into my branch:
> GC> only them, and nothing more.
> 
>  Before explaining how to do it, let me say for one last time that I don't
> consider it to be a good idea to do it like this. IMO it doesn't make much
> sense to have a commit adding unused files, as you plan to do.

I'll explain my motivation below. It is true that this means adding files
today that won't be used until tomorrow, and that may not be to everyone's
taste, but neither does it do any harm.

> It also will
> complicate things later: it's not that difficult to bring just these files
> into your branch, but this means that you will need to combine multiple
> commits touching the C++ code too later too because you won't be able to
> use earlier commits, using old .mst file names, unchanged. So you will be
> making life more difficult for yourself in the (near) future. Is it really
> worth it?

I need to review the changes in your branch vs. master, i.e.:
  62 new files added
   8 old files deleted
  13 changed files
Reviewing has several aspects, e.g.: reading and understanding all those
changes; looking for possible defects; writing inline comments to explain
implicit assumptions or future directions; or making revisions such as
reformatting a block here or adding an assertion there. I don't want to
put code into production without first reviewing it this way.

This set of changes can be viewed either as about 80 files or as 154
commits. I want to review files rather than commits. Consider, e.g.:
  git log --follow nasd_supp.mst
This file is trivial. I can see at a glance that it's okay. I don't need
to analyze how it changed through the four commits that affected it.
I just want to accept it now.

I want to review all '*.mst' files together before looking at the C++
changes. It makes sense to review them as a separate group: they're
written in the same language; they often refer to one another;
together they constitute the presentation layer for PDF output; and,
being logic-less templates, they are unlikely to contain any logic
errors, but they might contain English errors inherited from XSL
templates. Furthermore, about half of the 80 or so files I need to
review are of this filetype, so if I can move them into the "accepted"
category, that halves the number of remaining files to review.

You could call this a divide-and-conquer strategy. Later, I can look at
the remaining set of files and subdivide them into batches that are most
readily reviewed together. For example, I might group the five deleted
XSL files together, and perhaps I'll read through them, comparing them
to the new '.mst' files, or perhaps I'll just ignore them...but I'll
have five fewer files to deal with. Eventually, only a handful will be
left, which will be the hardest to review (because I've accepted all the
easy ones), but reviewing them will have become a tractable problem,
because there will be only a handful.

How can I do this while preserving your commit history?

> GC> I can imagine various ways of doing that,
> GC> as follows; which is best?
> 
>  I don't know what is the criterion for being "best".

Best is a method that lets me work the way I want while preserving your
commit history.

>       $ git checkout vz-no-xslfo '*.mst'
>       $ git commit -m 'Create template files'
[...]
>  Of course, this loses all the history of the changes to these files,
> including their renaming (granted, this is not very important), but you
> probably don't care about this anyhow (in this branch, at least).

That seems like a fatal flaw. The roadmap we've established is for me to
create my own branch G that accepts and modifies the contents of your
branch V, and then IIRC to git-merge V into G and then G into M. If G
loses the history of V, then M won't have that history either.

> GC> (1) Find the latest SHA1 for all these files. To identify the commits,
> GC> I can check out your branch:
> GC>   git checkout vz-no-xslfo
> GC> and search for '\.mst' here:
> GC>   git log --stat
> GC> It appears that these two commits should cover all '*.mst' files:
> GC>   982c9f0 Remove consecutive blank lines from a Mustache template
> GC>   2fd6a11 Use shorter and more consistent names for external templates
[...]
>  So this approach is rather hopeless.

Okay, so that way won't work.

> GC> (2) Instead, maybe I should just import everything from your branch,
> GC> all at once, creating a complete copy; and then work on it as I would
> GC> normally work on 'master'. In effect, I'd still be doing my work
> GC> linearly as always, except on a different line.
> 
>  I don't see how does it really help solve the problem

It approaches the task in a different way. Idea (1), discarded above,
would do this:

> ("bring only *.mst
> files in your branch and only them")...

but that seems to be a dead end, so I'm looking for a different way.

I think the obstacle is that I want to review 80 files, but git cares
only about contents and commits, so these two views are incompatible.

Ideally, I'd like to bring over files in batches, reviewing and probably
modifying most of them as above. I start with a pile of 80 unreviewed
files. I move half of them into a "working" pile, studying, annotating,
and updating them, and then moving them into a "finished" pile. That
single cycle yields one pile of 40 accepted, finished files, an empty
"working" pile, and a leftover pile of 40 unreviewed files. Subsequent
cycles diminish the unreviewed pile and increase the finished pile.
Notionally, if not physically, that's how I'm going to do this work.

It is convenient if these "piles" correspond to the physical reality:
if a file doesn't exist yet, then it hasn't been reviewed; if it does
exist, then either I'm working on it, or I've finished working on it.
That's convenient, but there are other ways to track which file is in
which "pile": for instance, I can maintain a list of all files with
a code for each one's status.

Files must move from one git branch to another by means of some git
mechanism. Apparently git-cherry-pick is not that mechanism: it chooses
commits that normally affect multiple files, but I want to address only
individual files (or groups thereof). I want to preserve each file's
git history, but I don't want to review that history: I only want to
review its final state in your branch.

My hope is that some compromise such as this will preserve the commit
history while allowing me to work the way I want, the only cost being
some manageable loss of convenience for me. This leads to another idea,
which would be more convenient for me:

> GC> (3) Alternatively, suppose I just put your branch into a subdirectory
> GC> of master, and then gradually move files into the main directory? If I
> GC> did that with 'git mv', would git be smart enough to recognize them,
> GC> so that their (your) history would be preserved? This way, we'd never
> GC> even need to do a 'git merge'.
> 
>  No, no, no, we will definitely need to do "git merge", otherwise the
> history of my changes will be lost. We don't do the merge to merge the
> contents (we had decided that we're going to take your version of them),
> but to merge the two histories together. So unless you cherry-pick all
> commits of my branch in order (which you don't plan to do at all), the
> merge is required.
> 
>  "git mv" won't help you at all because when you "put my branch into a
> subdirectory", it presumably won't be under git control at all,

I was trying to ask if there's a way to do that under git's control.

> so it's
> just like a variant of (2).

I've failed to express my intention clearly. That's difficult, because
I don't have a deep enough understanding of git (otherwise I'd know how
to solve the problem). I certainly don't want to lose the history of
your changes.

In both (2) and (3), I'm suggesting that we "import" your entire branch
all at once, into a branch, a subdirectory, or whatever as long as it's
kept distinct from master. I don't know the git terminology for this, but
conceptually it gives me all of your changes as a single, indivisible blob.
That's not ideal from my POV, because I'd rather "import" it in pieces
(but my ideal pieces would be your final revision of each file, and that
seems to be unworkable), but I'm guessing that treating it as an atomic
blob ensures preservation of all the history.

I'm not sure what git command would do this. Perhaps it's git-merge.
Or perhaps it's just a matter of renaming your branch, or starting my
own branch at the tip of your branch. IOW, our plan had been:

    M0 ------------------------------> M1
    |\                                 ^
    | \                               /
    \  --> GWC1 -> ... -> GWCm ---> GWC*
     \                              ^
      \                            /
       ---> VZ1 -> ... -> VZn  --->

but now that seems unworkable, so instead I'm suggesting

    M0 -----------------------------------------------------------> M1
     \                                                              ^
      \                                                             /
       ---> VZ1 -> ... -> VZn  --->                                /
                                   \                              /
                                    \                            /
                                     ---> GWC1 -> ... -> GWCm -->


>  Anyhow, I think my method in the very beginning should satisfy your
> requirements -- but please let me know if it doesn't.

This method:
        $ git checkout vz-no-xslfo '*.mst'
        $ git commit -m 'Create template files'
"loses all the history of the changes", and that's not what we want.

(1) is hopeless. (2) and (3), at least as you originally interpreted my
poor explanation of them, are broken. Let's let (4) signify the ASCII
art above, which is what I was trying to say with (2), but we'll call
it (4) to sidestep the ambiguity. (4) cannot lose any history, right?
I was about to ask what git command would turn VZn into GWC1 in the
diagram above, but is any command needed? Doesn't it really amount to:

    M0 -----------------------------------------------------------> M1
     \                                                              ^
      \                                                             /
       ---> VZ1 -> ... -> VZn  --->  **** GWC1 -> ... -> GWCm -->  /

where '****' signifies that I start working on "your" branch? Would
I 'git-merge' your branch into mine, or would I just make a local
copy of it?

Ideally, I'd like to do something like what I was trying to describe
in (3), but let me restate it here and call it (5):

    M0 -----------------------------> M1/no-xslfo
     \                               <^>
      \                               /
       ---> VZ1 -> ... -> VZn  --->  /

where <^> signifies a hypothetical git operation that merges your
branch into master as a subdirectory, preserving all your history,
while leaving master otherwise intact. If this operation can be
performed within git, then I imagine that I can 'git mv' files from
that subdirectory in batches (under git's control, so that history
would be preserved)--so that, in reference to the discussion above,
master's main directory is my "working" or "finished" pile, and
master's no-xslfo/ subdirectory is my "unreviewed" pile. If git
can't do that directly, then is there a way to git-merge your
branch now, and then immediately move all its changes into a
subdirectory (under git's control), and then somehow (again, all
under git's control) reset the base directory to its state today?
I'll be surprised if you say git can do this, but it doesn't hurt
to ask.