Re: overloading of internal priority calculations in agenda

[Adam Spiers]

Hi again,

On Wed, Dec 02, 2020 at 02:20:53PM +0000, Adam Spiers wrote: 
> Hi all,
>
> I'm currently working on adding a feature to org-agenda which allows 
> manual ordering of entries in combination with the existing automatic 
> ordering (as dictated by `org-agenda-sorting-strategy'). 
>
> During my investigations I noticed that while `org-get-priority' converts 
> [#B] style cookies into a numeric priority which is a multiple of 
> 1000, further adjustments are made in functions like 
> `org-agenda-get-scheduled' before adding this numeric priority as a 
> text property on the entry: 
>
>    'priority (if habitp (org-habit-get-priority habitp)
>                (+ 99 diff (org-get-priority item)))
>
> In this case `diff' refers to the number of days between now and when 
> the item was scheduled.  A slightly different calculation is made in 
> `org-agenda-get-timestamps': 
>
>    (org-add-props item props
>      'priority (if habit?
>                    (org-habit-get-priority (org-habit-parse-todo))
>                  (org-get-priority item))
>
> I further noticed that this overloading of the internal priority by 
> including timestamp and habit data causes disruption to the behaviour 
> I imagine most users would expect from `org-agenda-sorting-strategy'. 
> For example, if you have `priority-down' as the first entry in the 
> `agenda' section and `category-keep' as the second, then differences 
> in the SCHEDULED timestamp are included in the priority calculation 
> and can therefore prevent sorting of two adjacent [#B] items by 
> category.  This seems like a bug to me, or at least breaks the 
> Principle of Least Surprise. 

[snipped]

> Given that `org-agenda-sorting-strategy' now supports all manner of 
> sorting criteria, many of which are time-sensitive, I would like to 
> know if there is any reason not to remove this overloading of the 
> priority calculation, i.e. decoupling it to depend purely on the 
> result of `org-get-priority' and `org-habit-get-priority'? 
>
> If fact, perhaps we could go one step further and add support for new 
> habit-priority-{up,down} sorters to `org-agenda-sorting-strategy', so 
> that the priority-{up,down} sorters sort purely by the priority cookie 
> and nothing else? 

Gently bumping this as I didn't get any replies yet.  I would like to 
continue working on a solution, but obviously don't want to waste time 
on something which would be rejected. 

If it is considered important to preserve the exact behaviour 
currently offered by `org-agenda-sorting-strategy' then I would 
propose the following: 

- Keep the existing priority-{up,down} which combine priority cookies
  with timestamp data and the result from `org-habit-get-priority',
  but probably also deprecate it and remove it from the default value.

- Introduce new priority-cookie-{up,down} sorters which operate purely
  on [#A] and [#1] style priority cookies and nothing else.

This would facilitate decoupling of the sortable criteria whilst 
remaining backwards compatible. 

Does this sound reasonable?  I am keen to proceed very soon (ideally 
over the Xmas break).  I have already written some new ert tests for 
`org-agenda-sorting-strategy' which would be included in any submitted 
patches. 

Thanks!
Adam