Re: [O] Showing Property in headline; generating table from properties; exporting table to CSV

[Pip Cet]

I hesitate to share this code (it's not very pretty right now), but I
have been working on two ideas that I think combine to solve problems
like this in a nicer (and lazier) fashion than the current approach:
headline properties and reverse inheritance.

Headline properties are simply about setting properties in the
headline: no properties drawer, potentially nicer syntax (I'm
currently using :property=value: to emphasize that properties behave
like tags, but I'm thinking of moving to [property: value] (I'm
worried about collisions with other org syntax, but since this would
be effective only at the end of a headline, I think the only collision
is with footnotes, and the solution to that is "don't have a property
called FN")), and easier transition from tags to properties and back.
The problem, of course, is what happens when you try to set more than
one short property in a headline.

This is where the second idea comes in: reverse inheritance. *if* all
of a headline's children agree on what a property's value should be,
it's as though the property were specified in the master headline. In
particular, if there's only one child headline that sets a property,
it automatically migrates up a level:

* John Smith
** :address@hidden:

would migrate up the "email" property to apply to the top-level
headline, so columns view would look something like this:

ITEM | EMAIL |
* John Smith | address@hidden | ....

(this example doesn't quite work yet in practice, with my current code).

If there are two conflicting values for a property, the property is
not set. This is so

* Students
** John Smith
*** email :address@hidden:
** Jane Miller
*** email :address@hidden:

doesn't set an email property for the "Students" line (if there's just
one student, with just one email address, there's probably no harm in
having that property set for the Students line. Also, in that case,
you have an attendance problem.)

In combination, the syntax I currently have in mind would allow you to
set properties one at a time in sub-headlines of your data item:

* Chemicals
** water
*** [smiles: O]
According to Wikipedia.
*** [molarmass: 18.01528]
Also according to Wikipedia.
*** [safety: GRAS]
I think.

(Notice how that gives you a well-defined place to put comments and
footnotes, which I imagine is useful if you need to have citable
references for your properties. I think you can currently place such
comments in the PROPERTIES drawer, but then they probably won't be
exported.)

What if the master headline already sets the property? There are two
cases here: in case it's set to an ordinary value, the child headlines
are ignored. But if it's set to a special value signifying that a
"compactor" function is to be used, that function is called to produce
the property value for the headline based on the child nodes. In the
simplest case, that means the property is set to something like
"mean(^)" (again, that syntax is likely to change to something
prettier), and the result of

* calculate this for me :v=mean(^):
** 3 :v=3:
** 4 :v=4:
** 6 :v=6:

is as though you'd written

* calculate this for me :v=4.3333:
and so on

(It's possible to do something very much like this with the columns
property; I'm not claiming it's a new feature.)

Ordinary inheritance, reverse inheritance, and compaction can be combined:

#+COLUMNS: %ITEM %8GRADE %TAGS
* Document
** defaults
*** default to compacting with the arithmetic mean function :grade=mean(^):
*** student grades
**** 1999
***** John Smith
****** test paper 1 :grade=3:
****** test paper 2 :grade=4:

will work, and with column mode produce the output attached as
students2.org.html. Note that "3.5" is John Smith's grade, the average
of all students' grades in 1999, and the average grade averaged over
all years. I've attached a more complete example as
students.org/students.org.html that demonstrates how the mean is
calculated.

I think people can imagine what the columns view would be with some of
the sub-headline levels hidden, but I'd like to point out both that it
does give you a really nice list of students with their grades and
that there's a potential problem with missing tests: there's no
default to a grade of 0 for them, so you have to be careful when
entering them to distinguish between a 0-graded missed test and a test
that's simply ignored for the average grade.

Currently the code for calculating the reverse inheritance is working,
but really slow (it doesn't cache calculations at all, so it's
probably something like O(n^3)); I don't have special code for
displaying the results yet, but columns mode followed by
org-htmlize-buffer produced the attached HTML files. At this point,
this is just to demonstrate the ideas; while I think the two ideas in
combination apply well to the student-grade problem set, I've actually
implemented them for use in a literate programming major mode.

I'd appreciate any comments (but I know the code isn't particularly
clean, fast, or anywhere near ready for inclusion right now),
particularly on the syntax questions:
1. what's a good syntax for setting properties in a headline?
 - ** headline [property=value]
   ** headline [property:value]
 - ** headline [[property: value]]
 - ** headline :property=value:
 - something else entirely
2. what's a good syntax for specifying that a property is to be
calculated from child properties?
 - ** headline [property:=mean(property)]
 - ** headline [property=mean(^)]
 - ** headline :property==mean(^):
 - something else entirely
3. is it useful to align properties to the right border like tags are
currently aligned? I must admit I don't particularly like the visuals
of that, but it would be handy for looking over your students'
grades...

Sorry that this post has gotten somewhat long and self-promoting, but
I really think the current PROPERTIES drawer syntax is very ugly and
unnecessary, so we should do _something_ about it.

Look at the code:
https://github.com/pipcet/org-mode/compare/master...pipcet:compact-properties?expand=1
Check out the code: git clone -b compact-properties
https://github.com/pipcet/org-mode.git

On Thu, Aug 6, 2015 at 2:19 PM, John Kitchin <address@hidden> wrote:
> All my gradebook related stuff is here:
> https://github.com/jkitchin/jmax/tree/master/techela, and there is a lot
> of it ;)
>
> I have a whole ecosystem built around using git through emacs and
> org-mode with my students, which at this point even I find complex!
>
> For grading, I grade directly in the student's turned in org-file using
> functions from here
> https://github.com/jkitchin/jmax/blob/master/techela/techela-grade.el. I
> define a minor-mode with a keymap to make grade entry, commenting,
> etc... relatively easy for me. The grades are weighted by different
> categories, and saved as a filetag (not a property). I tend to put one
> problem per file so the grade is just for the one problem, and assign
> 2-3 of these per week.
>
> For gradebook stuff, most of it is here:
> https://github.com/jkitchin/jmax/blob/master/techela/techela-gradebook.el
>
> The grades are stored in the student org-files which I return to them. I
> lookup all the assignments from the syllabus, and then map over the
> org-files for each student for each assignment getting the grades out.
>
> I will be using it again this fall, so /maybe/ the documentation will
> even improve ;)
>
> Matt Price writes:
>
>> On Thu, Aug 6, 2015 at 9:13 AM, John Kitchin <address@hidden>
>> wrote:
>>
>>> How do you enter your grade? I use a function, bound to a convenient key
>>> like s-s g, which sets the grade property. You could have that function
>>> change the heading TODO state to DONE so you know it is done, and maybe
>>> add a tag with the grade, or just append the grade on the end of the
>>> headline.
>>>
>>
>> I just set the grade with "C- C-x p GRADE", too slow!  I would love to see
>> your function though I can doubtless at least write THAT myself. Yeas, then
>> having that same function change the todo state (to "READY" in my case -- I
>> change to "DONE when I email out the comments), would make a lot of sense.
>>
>> Adding the grade as a tag doesn't seem quite right, as I often change
>> grades after a rewrite.  I'd need to get rid of the original tag.  Adding
>> to the headline might work, but I'd have to change some of my existing
>> functions which use the headline value as a proxy for the student name.
>> Could definitely be done.
>
> see previous email about using overlays
>
>>
>>>
>>> I also use a function that runs org-map-entries and constructs a
>>> temporary gradebook as an org-table in a new buffer.
>>>
>>
>> please please show us? Thank you!
>>
>>
>>
>>>
>>> Eric S Fraga writes:
>>>
>>> > On Thursday,  6 Aug 2015 at 07:24, Matt Price wrote:
>>> >
>>> > [...]
>>> >
>>> >> - I currently store my grades as properties of level-2 headlines.
>>> However,
>>> >> I would really like to be able to see the grades when the headline is
>>> >> folded, so I can have a quick visual sense of how many papers I've
>>> marked,
>>> >
>>> > I would like something like this as well.  The nearest I have found is
>>> > to use column view.
>>>
>>> --
>>> Professor John Kitchin
>>> Doherty Hall A207F
>>> Department of Chemical Engineering
>>> Carnegie Mellon University
>>> Pittsburgh, PA 15213
>>> 412-268-7803
>>> @johnkitchin
>>> http://kitchingroup.cheme.cmu.edu
>>>
>
> --
> Professor John Kitchin
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu
>

  
  
#+COLUMNS: %ITEM %8GRADE %TAGS
* Student grades  | 5.4166.. | :grade=mean(^): | 
** class of 1999  | 4.5      |                 | 
*** John Smith    | 4.5      |                 | 
**** test paper 1 | 4.0      | :grade=4:       | 
**** test paper 2 | 5.0      | :grade=5:       | 
** class of 2000  | 6.3333.. |                 | 
*** John Smith    | 7.5      |                 | 
**** test paper 1 | 8.0      | :grade=8:       | 
**** test paper 2 | 7.0      | :grade=7:       | 
*** Joan Miller   | 9.5      |                 | 
**** test paper 1 | 9.0      | :grade=9:       | 
**** test paper 2 | 10.0     | :grade=10:      | 
*** Steven Smith  | 2.0      |                 | 
**** test paper 1 | 2.0      | :grade=2:       | 

  




  
  
#+COLUMNS: %ITEM %8GRADE %TAGS
* Document                                                  |          |                 | 
** defaults                                                 |          |                 | 
*** default to compacting with the arithmetic mean function |          | :grade=mean(^): | 
** student grades                                           | 3.5      |                 | 
*** 1999                                                    | 3.5      |                 | 
**** John Smith                                             | 3.5      |                 | 
***** test paper 1                                          | 3.0      | :grade=3:       | 
***** test paper 2                                          | 4.0      | :grade=4:       |

students.org
Description: Binary data