Re: [O] proposal to have ignoreheading tags/properties

[Eric Schulte]

Hi,

Nicolas Goaziou <address@hidden> writes:

> Hello,
>
> Eric Schulte <address@hidden> writes:
>
>> Why TODO types rather than a tag?  IMO using a TODO type would conflate
>> task management and document structuring.  What do you think about the
>> attached patch which should add this functionality to the core.
>
> Thank you. Unfortunately, in many cases this code will make the parse
> tree invalid. Consider the example below:
>
>   * H1
>     Text1
>   ** H2 :inline:
>     Text2
>
> A simplified version of the parse tree is:
>
>   (headline
>    (section
>     (paragraph "Text1"))
>    (headline
>     (section
>      (paragraph "Text2"))))
>
> With your function, it becomes
>
>   (headline
>    (section
>     (paragraph "Text1"))
>    (section
>     (paragraph "Text2")))
>
> which is invalid, as a given headline is not expected to have more than
> one section.
>
> Of course, it is possible to add code in order to merge both sections
> and get
>
>   (headline
>    (section
>     (paragraph "Text1")
>     (paragraph "Text2")))
>
> which is exactly what you obtain in the first answer of the FAQ, along
> with its limitations (see the :noexport: example in the same question).
>
> Actually, the problem is deeper than that. This :inline: tag is just
> a convoluted way to ask for a positive answer to another FAQ: « Can
> I close an outline section without starting a new section? »
> (http://orgmode.org/worg/org-faq.html#closing-outline-sections). Indeed,
> allowing :include: tags is equivalent to allowing to close sections
> before the next one, at least at the export level:
>
>   * Section one
>
>   Some text
>
>   ** Subsection one
>
>   Some text
>
>   ** Subsection two
>
>   Some text
>
>   ** end Subsection Two                                                     
> :inline:
>
>   Continue text in section one.
>
> This is not possible and goes against so many assumptions in Org that it
> will always introduce problems, as your function does.
>

Thanks Nicolas.  Point clearly stated.  All of the structural issues
hinge upon the inclusion of the contents of an inlined (ignored)
headline, so the solution is to change the behavior s.t. the non-subtree
contents of ignored headlines are also removed from export.  This is
implemented below.

>
> Since it cannot work in the general case, I do not think it should go in
> core. Fortunately, a simple function in `org-export-before-parsing-hook'
> provides a decent solution already. Users requiring more sophistication
> can always implement their own function and add it to that hook.
>
> OTOH, the situation could be improved wrt :export: and :noexport: tags.
> We could allow nesting :export: tags within :noexport: tags with the
> following rule: the :export: headline with the lowest level within
> the :noexport: tree gets promoted to the root of the tree.
> Other :export: headlines have their level set relatively to this one.
> Thus:
>
>   Text before first headline
>   * H1
>   Body1
>   ** H2 :noexport:
>   Body2
>   *** H3
>   Body3
>   *** H4 :export:
>   Body4
>   **** H5
>   Body5
>
> will be seen as
>
>   Text before first headline
>   * H1
>   Body1
>   ** H4
>   Body4
>   *** H5
>   Body5
>

In my opinion the manual interleaving of "noexport" and "export" tags is
overly cumbersome and is non-obvious.  The obscure nature of this
solution is evidenced by multiple discussions and implementations of
filter functions to handle situations which could be covered by this
noexport/export pattern.

I think the attached patch should be applied to the core.  It includes
the following.

- a single new export function which removes the headlines and contents
  (the "section" element) of headlines tagged "ignoreexport", then
  retains and promotes all sub-headlines contained therein.

- a single new export tag named "ignoreexport" (ignore seems to be the
  crowd favorite, and adding "export" reduces possible conflicts with
  existing personal tags)

- some tests protecting this new functionality

- I'd like to add documentation of this tag to the manual to improve
  visibility beyond the existing FAQ entries (which have failed to
  prevent multiple on list discussions and re-implementations of this
  feature).  It is not clear where to document such a tag.  The "export"
  and "noexport" tags are only documented as a side effect of
  documentation for the SELECT_TAGS keyword.  I think it would be
  beneficial to add a "selective export" section or paragraph (which
  existed in the old manual) to the new manual.

>From the included tests, the effect of this patch is to convert a tree
like the following,

,----
| * H1
| Text1
| ** H2                                                          :ignoreexport:
| Text2
| *** H3
| Text3
| **** H4
| Text4
| *** H5
| Text5
| **** H6                                                        :ignoreexport:
| Text6
| ***** H7
| Text7
| ***** H8
| Text8
`----

on export to a tree like the following.

,----
| 1 H1
| ====
| 
|   Text1
| 
| 1.1 H3
| ~~~~~~
| 
|   Text3
| 
| 1.1.1 H4
| --------
| 
|   Text4
| 
| 1.2 H5
| ~~~~~~
| 
|   Text5
| 
| 1.2.1 H7
| --------
| 
|   Text7
| 
| 1.2.2 H8
| --------
| 
|   Text8
`----

I'm sympathetic to arguments about maintaining simplicity of the core,
and even more so to arguments about maintaining structural validity of
trees during export.  I believe that this revised patch fully maintains
valid tree structures, and I'd suggest that the increase in complexity
of a single keyword is justified by the demonstrated user demand for
this functionality.

Thanks,

>From 9d95ad617f890142fe0d31ae355d205ef40ce6f3 Mon Sep 17 00:00:00 2001
From: Eric Schulte <address@hidden>
Date: Sun, 15 Jun 2014 19:46:31 -0400
Subject: [PATCH 1/2] export remove ignore headline and promote children

* lisp/ox.el (org-export-ignore-headlines-retain-and-promoting-children):
  A new function.
  (org-export-as): Include
  `org-export-ignore-headlines-retain-and-promoting-children' into the
  export process.

* testing/lisp/test-ox.el (test-org-export/ignored-headlines-text):
  Example org-mode file for ignoreexport headline tests.
  (test-org-export/handle-ignored-headlines-system): System tests for
  org-export-ignore-headlines-retain-and-promoting-children.
  (test-org-export/handle-ignored-headlines-unit): Unit tests for
  org-export-ignore-headlines-retain-and-promoting-children.
---
 lisp/ox.el              | 31 ++++++++++++++++++++++++++++
 testing/lisp/test-ox.el | 54 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 85 insertions(+)

diff --git a/lisp/ox.el b/lisp/ox.el
index 4bfef52..ae3a11c 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -2320,6 +2320,34 @@ tree is modified by side effect and returned by the 
function."
                (plist-get info prop)
                info))))
 
+(defun org-export-ignore-headlines-retain-and-promoting-children (data info)
+  "Remove headlines tagged \"ignoreexport\" retaining sub-headlines.
+DATA is the parse tree.  INFO is a plist containing export
+options.  Each headline tagged \"ignoreexport\" will be removed
+removing its contents but retaining and promoting any children
+headlines by a single level."
+  (org-element-map data org-element-all-elements
+    (lambda (object)
+      (when (and (equal 'headline (org-element-type object))
+                 (member "ignoreexport" (org-element-property :tags object)))
+        (mapc (lambda (el)
+                ;; recursively promote all nested headlines
+                (org-element-map el 'headline
+                  (lambda (el)
+                    (when (equal 'headline (org-element-type el))
+                      (org-element-put-property el
+                        :level (1- (org-element-property :level el))))))
+                ;; insert back into parse tree
+                (org-element-insert-before el object))
+              ;; drop first three elements of headline
+              ;; 1. headline tag
+              ;; 2. properties
+              ;; 3. section
+              (cdddr object))
+        (org-element-extract-element object)))
+    info nil)
+  data)
+
 (defun org-export--remove-uninterpreted-data-1 (data info)
   "Change uninterpreted elements back into Org syntax.
 DATA is a parse tree or a secondary string.  INFO is a plist
@@ -3124,6 +3152,9 @@ Return code as a string."
         ;; Handle left-over uninterpreted elements or objects in
         ;; parse tree and communication channel.
         (org-export-remove-uninterpreted-data tree info)
+        ;; Remove headlines tagged "ignoreexport" and promote their
+        ;; children.
+        (org-export-ignore-headlines-retain-and-promoting-children tree info)
         ;; Call options filters and update export options.  We do not
         ;; use `org-export-filter-apply-functions' here since the
         ;; arity of such filters is different.
diff --git a/testing/lisp/test-ox.el b/testing/lisp/test-ox.el
index 234032e..2ed0af9 100644
--- a/testing/lisp/test-ox.el
+++ b/testing/lisp/test-ox.el
@@ -1782,6 +1782,60 @@ Paragraph[fn:1]"
                (org-export-as (org-test-default-backend)
                               nil nil nil '(:with-tasks nil))))))))
 
+(defvar test-org-export/ignored-headlines-text
+  "* H1
+Text1
+** H2                                                          :ignoreexport:
+Text2
+*** H3
+Text3
+**** H4
+Text4
+*** H5
+Text5
+**** H6                                                        :ignoreexport:
+Text6
+***** H7
+Text7
+***** H8
+Text8\n")
+
+(ert-deftest test-org-export/handle-ignored-headlines-system ()
+  "Test `org-export-ignore-headlines-retain-and-promoting-children'."
+  (let ((exported
+        (org-test-with-temp-text test-org-export/ignored-headlines-text
+          (org-export-as (org-test-default-backend) nil nil nil nil))))
+    ;; ensure ignored headlines and contents are not present
+    (mapc (lambda (s) (should-not (string-match (regexp-quote s) exported)))
+         (list "H2" "Text2" "H6" "Text6"))
+    ;; ensure all other headlines and contents are present
+    (mapc (lambda (s) (should (string-match (regexp-quote s) exported)))
+         ;; should not be included
+         (list "H1" "Text1" "H3" "Text3" "H4" "Text4"
+               "H5" "Text5" "H7" "Text7" "H8" "Text8"))))
+
+(ert-deftest test-org-export/handle-ignored-headlines-unit ()
+  "Test `org-export-ignore-headlines-retain-and-promoting-children'."
+  (let ((data (org-export-ignore-headlines-retain-and-promoting-children
+              (org-test-with-temp-text test-org-export/ignored-headlines-text
+                (org-element-parse-buffer))
+              nil)))
+    (flet ((level-of (name)
+                    (let (out)
+                      (org-element-map data 'headline
+                        (lambda (el)
+                          (when (string= name
+                                         (org-element-property :raw-value el))
+                            (setq out (org-element-property :level el)))))
+                      out)))
+      ;; ensure ignored headlines and contents are not present
+      (mapc (lambda (pair) (should (= (car pair) (level-of (cdr pair)))))
+           '((1 . "H1")
+             (2 . "H3")
+             (3 . "H4")
+             (2 . "H5")
+             (3 . "H7")
+             (3 . "H8"))))))
 
 
 ;;; Keywords
-- 
2.0.0

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D (see https://u.fsf.org/yw)