Re: Understanding how \tag works in \relative pitched music

[David Wright]

On Mon 07 Aug 2017 at 22:33:12 (+0200), David Kastrup wrote:
> David Wright <address@hidden> writes:
> 
> > Neither do I. But the post I was commenting on was constructing an
> > Aunt Sally argument, "the unpredictability of \relative with \tag"
> > as a reason to make the change.
> 
> Maybe "unpredictability" was not the best expression.  It's clear that
> Kieren was referring not to non-deterministic but surprising behavior.
> 
> The truth is that \relative does not interact with \tag at all.  But
> that means that you get different results for
> 
>     \relative \keepWithTag ...
> 
> and
> 
>     \keepWithTag ... \relative
> 
> because then \relative gets to work with different material.

Whether that's a useful distinction, I wouldn't know. I can
see the point of having \tags mixed in with \relative sections
as in the OP's example. I can't see the point in mixing
\relative with \score and \staff constructions, particularly
when they contain commands such as \keepWithTag, except as
an exercise in regression testing or, possibly, obfuscation.

But in any case, if someone is surprised by the behaviour of a
particular construction, one that they are determined to use, then it
behoves them to find out in more depth how it behaves and/or examine
their expectations and whether they are justified.

The flexibility of LP's syntax certainly allows one to obscure
what the final interpretation will be. But it's hardly unique
in that respect.

> > Well, that's what I tried to do by removing the parts of the code
> > that had no bearing on \relative's behaviour but (a) were obscuring
> > the simple relation of the note pitches being input and (b) were
> > believed erroneously by the OP to have some influence on the pitches
> > of those notes.
> 
> At some point of time we have to acknowledge that people and humans are
> wired differently.  The difference might not be all that large when
> looking at me but then you would likely not want to reduce composers to
> that kind of crop.  So it's important to be able to cater to people with
> differing capability of in-depth understanding of mathematical and/or
> programming logic.
> 
> >> So he has
> >> all the necessary information for considering and/or trying and/or
> >> rejecting such a switch.
> >
> > Oh come on. There are contributors here whose words carry far more
> > weight than many of us, and with good reason. Their posts get pasted
> > into archives and consulted later, so their accuracy is all the more
> > important.
> 
> "Accuracy" is not the same as "agrees with David", actually for any
> value of David.

I'm not sure why you focus on "David"s.

> >> \fixed is also pretty good at minimizing fly-shit.  It becomes worse
> >> in that regard for stuff with widely navigated ranges,
> >
> > Agreed. And I was just expressing how it might have been better,
> > in my opinion. As it is, I gain nothing over \relative so I
> > haven't used it.
> 
> That's fine, and you are free to express this opinion just as Kieren is
> free to express his.

Thank you.

> And you have harped over it a lot longer than he
> has.

Both of us are good at splitting a paragraph and commenting on each
part, just as I've done here.

> > But I'm not trying to dissuade anyone from using \absolute and, now,
> > \fixed. In fact, I was contradicted when I expressed an opinion that
> > it would be difficult to _compose_ directly into anything other than
> > \absolute because of all the cutting and pasting involved.
> 
> It's a contrarian list and it doesn't help that I am an active
> participant.

Evidently.

> >> >> It doesn't do that impressively when communicating with other
> >> >> programs, so its main incentive is communication with humans.
> >> >> When humans are confused about what it does, it fails on one of
> >> >> the core tenets of its justification.
> >> >
> >> > That argument fails because not all humans are the same.
> 
> All humans are the same as themselves, and most LilyPond users are
> themselves the most relevant human reader of the LilyPond source they
> write.
> 
> Of course humans are capable of learning to navigate any amount of
> complexity with enough determination, like they can learn to play bad
> instruments well.  But if you find in the long run that fighting your
> instrument is becoming a major drain of your focus and thus is taking
> resources better invested elsewhere, improving or changing your
> instrument might be smarter than trying to become smarter.

I can't see a connection between this and the decision whether to
employ \relative, \fixed or \absolute to define some notes in LP.

> > I didn't say that. I said that you haven't shown it fails on one of
> > the core tenets of its justification merely because some humans have
> > difficulty learning/understanding/retaining the concept.
> 
> It fails for them.  Whether fixing that makes sense is up to them.
> 
> >> Now make-relative is a convenient but quite heavy-handed macro.
> >> There are situations where it fails, and then it does so
> >> spectacularly.
> >> 
> >> The reason is that it allows creating a more natural "user
> >> experience" by a totally unnatural and complex actual inner operation
> >> that has to be defined individually for each music function written
> >> by the programmer.
> >
> > Hey, programming's hard. Glad I'm retired.
> 
> It's not that programming is hard but that the concept of \relative does
> not map well to music functions or other forms of rearranging input
> since the user expectation with regard to \relative is focused on input
> order rather than music expression internal order.
> 
> In fact, the mess \relative creates was such that issue 2240 (changing
> the music expression internals, various followup issues) was necessary
> for fixing chord repetition in \relative mode, in order to have issue
> 2263 for fixing the long-standing issue 1110.
> 
> Issue 2240 and all of the followup work took several months of work, and
> \relative has intervened for additional work several times.  The "Hey,
> programming is hard.  Glad I'm retired." attitude coupled with the
> expectation that some sucker is going to do it because you won't is
> actually pretty offensive considering on how little money I had and have
> to get by for making such actually inherent problems mostly go away.
> What kind of retirement entitles you to have me and other programmers
> make LilyPond work according to your expectations?

I honestly hadn't realised that, when considering the design and
documentation of a language construction, one should take into
consideration the finances of a person who has decided to implement
it.

I don't think I've ever expressed any entitlement with respect to the
way that LilyPond behaves. About the only changes I have suggested
making are at the trivial level (I think), like treating a more
comprehensive collection of whitespace characters as white space
(seeing that LP now uses Unicode). That's the sort of thing one might
put on the back burner for a code tidy-up session, I would imagine.

I've served my time doing difficult programming for a quarter of a
century. I happened to work in mass spectrometry, and the consequences
of getting something wrong in my code were serious. People could lose
samples that cost a lot of time, and involved difficult, dangerous and
expensive work. The machines could waste a lot of time doing nothing
outside office hours. Any mathematical mistakes could lead to the
publishing of incorrect results in scientific journals. And while I
could do regression testing of the data processing aspects with fake
data, there's no way to test the overall machine control except by
running real samples; their behaviour is so much more complex than
running analytical standards.

During my time I saw the introduction of new types of hardware and new
isotopic systems for analysis, which on occasion necessitated major
design changes. That's just the way things are in scientific
research. But my users didn't have to persuade me into doing the work
that they profited by. I did the work because I enjoyed and gained
satisfaction from doing it.

Now I'm retired, I'm happy to be a user of other people's software,
and the programming I do is limited pretty much to scripting in
bash and python on linux, for convenience rather than necessity. The
pressure's off for me; what's wrong with that? I don't see why you
should try to make me feel any shame in being retired.

> > One way of reducing complexity for the users is to try and keep things
> > orthogonal, so that doing one thing doesn't lead to a side-effect
> > elsewhere. So if you believe a side-effect exists and try to exploit
> > it (like here where the OP thought that \tag could turn \relative on
> > and off), is it not best to disabuse the OP of the interaction rather
> > than leave them with, or even reinforce, the impression that \tag is
> > leading to unpredictability?
> 
> We have warnings in the manual exactly because users in general find the
> interaction of \relative and the \tag-related commands surprising.  If
> you lecture users on the same questions year after year after, at some
> point you ask yourself where the point in being right is when it is so
> much trouble.

So much trouble for whom? The people who post here are the ones who
have problems, not just with \relative but all sorts of constructions.
The people who don't have problems don't post. Why don't they have
problems? Perhaps they don't use a particular construction, or they
use it but heed the warnings, or try a simple test case to make sure
they understand it. I fall into all those categories, and there are
probably more.

> >> Now as people tend to work more and more with LilyPond, they tend to
> >> become bit by bit more of a programmer rather than a mere user, work
> >> less by trial and error, and cover more complex input sources with
> >> multiple voices.  As they do, dealing with the complexities of
> >> \relative and the resulting ripple effects of local source
> >> modifications and reorganizations become more of a nuisance and one
> >> becomes more aware of the _cost_ of fly-speck cleaning.
> >
> > So what you appear to be saying in these paragraphs is that \relative
> > doesn't play well with scheme programming.
> 
> It doesn't play well with logic.  It raises naive expectations that
> don't map well to music structures.
> 
> > Two things occur to me:
> >
> > LP should have a well-defined and predictable behaviour, and this
> > should be reflected in its documentation.
> >
> > If one can't understand the definition, behaviour, upsides and
> > downsides of \relative, is there much hope of advancing into
> > scheme programming and how that interacts with LP?
> 
> Issue 1110 was raised in 2010.  The first attempt of fixing the
> interaction of chord repeats with \relative was
> 
> commit 811cdc2809ee224dfbe79576c4ac6e1fe9057db5
> Author: Nicolas Sceaux <address@hidden>
> Date:   Thu Dec 3 11:44:10 2009 +0100
> 
>     Chord repetition: \relative mode, cautionary accidentals
>     
>     \relative mode: Introduce a new RepeatedChord music type, which element
>     property is the copied chord (as produced by the chord repetition 
> function).
>     Define a relative callback for repeated chords, which avoids
>     octaviation of repeated chords, by modifying the first note octave
>     before applying the octave relativization.
>     
>     Only memorize <..> chords.
>     
>     When copying a chord, unset cautionary and forced accidental (fix #925).
> 
> It didn't work reliably.  \relative also has consequences in the
> implementation of \repeat and its unfoldings.  Whenever a user does some
> programming, chances are that the results will not operate as expected
> within \relative.  Dozens of LSR examples were (and some likely still
> are) affected.
> 
> Soa power user deciding to forego \relative is likely to make their life
> easier.
> 
> \fixed is much less afflicted in that manner.  Yet.  It is somewhat
> equivalent to \transpose (not entirely so since \fixed leaves \absolute
> and \relative and other \fixed music alone while \transpose doesn't).
> 
> So
> 
>     \fixed c'' { c d e f g a b c' }
> 
> maps to
> 
>     \absolute { c'' d'' e'' f'' g'' a'' b'' c''' }
> 
> So what do you think
> 
>     \fixed a'' { c d e f g a b c' }
> 
> should map to?

Could, not should.

\absolute { c'' d''' e''' f''' g''' a'' b'' c''' }

> Maybe
> 
>     \absolute { c' d' e' f' g' a'' b'' c'' }
> 
> ?  So as the argument to \fixed rises, the result gets lower until the
> octave jumps and then it gets higher?  That's sort-of discontinuous.
> The continuous result would be
> 
>     \absolute { c'' d'' e''' f''' g''' a''' b''' c''' }
> 
> but you can't really make anybody cheer for that as a user interface
> even though it would be logical.  So you are likely rooting for the
> discontinuous approach.

Under a definition I described earlier, \fixed a'' { c d e f g a b c' }
wouldn't be an octave scale, because the point in a written scale
where a prime or comma needs to added or removed would be the
transition between g-a, not b-c. So a scale on c'' would be written

\fixed a'' { c d e f g a' b' c' }

> Now \relative is implemented through relativization hooks: expressions
> know themselves how they are supposed to deal with relativization.
> Unfortunately this knowledge does not map to your \fixed scheme.  The
> current definition of \fixed (hey, should use for-each instead of map)
> is:
> 
> fixed =
> #(define-music-function (pitch music)
>    (ly:pitch? ly:music?)
>    (_i "Use the octave of @var{pitch} as the default octave for @var{music}.")
>    (let ((octave-marks (1+ (ly:pitch-octave pitch))))
>      (cond ((not (= 0 octave-marks))
>             (ly:music-transpose music (ly:make-pitch octave-marks 0 0))
>             ;;In order to leave unchanged the notes in any enclosed
>             ;; \absolute or \fixed or \relative, make a cancelling shift
>             (map (lambda (m)
>                    (ly:music-transpose m (ly:make-pitch (- octave-marks) 0 
> 0)))
>                  (extract-named-music music 'RelativeOctaveMusic)))))
>    (make-music 'RelativeOctaveMusic 'element music))
> 
> 
> Programming is hard.  Good that you are retired.

Yes. Yes, thank you. (Respectively.)

--

(I've added the x here in case anyone were to read this paragraph later,
in isolation from the above discussion.)

\fixedx c'' { f, g, a, b, c   d   e   f   g   a   b   c'   d'   e'   f' }
\absolute {   f' g' a' b' c'' d'' e'' f'' g'' a'' b'' c''' d''' e''' f''' }
                          ↑

\fixedx a' { b, c, d, e, f, g, a  b  c   d   e   f   g   a'  b'  c'   d' }
\absolute {  b  c' d' e' f' g' a' b' c'' d'' e'' f'' g'' a'' b'' c''' d''' }
                               ↑

\fixedx fis' { b, c, d, e, f  g  a  b  c   d   e   f'  g'  a'  b'  c'   d' }
\absolute {    b  c' d' e' f' g' a' b' c'' d'' e'' f'' g'' a'' b'' c''' d''' }
                           ↑

IOW the argument immediately following \fixed would give the bottom
note (accidentals being ignored) of the chosen unadorned scale. Each
mark would 'add or ,subtract an octave as appropriate. This scheme
would allow you to choose exactly where the unadorned scale lies
(according to the frequency of occurrence of the following notes,
or any other criterion, eg the bottom tonic of a part).

Given a free rein, the argument could have been defined as the
_middle_ note of the scale so that the unadorned notes would be
symmetrically disposed around the argument. (Thus f'', d'' and b'
respectively, would fit that role in the examples above.)
However, here I have followed the pattern defined by LP such that
using c in \fixed c { … } behaves the same.

Cheers,
David.