RE: attachment: link type export to HTML invalid attach dir

[Gustav Wikström]

Hi again,

> -----Original Message-----
> From: Nicolas Goaziou <address@hidden>
> Sent: den 20 januari 2020 02:25
> To: Gustav Wikström <address@hidden>
> Cc: address@hidden
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> Gustav Wikström <address@hidden> writes:
> 
> > Ok, noted. To my defense I was thinking more in the terms of subtyping
> > then hardcoding. Because we have multiple link types which try to
> > share an interface. But the interface isn't perfect for all different
> > kinds of links.
> 
> Then, I suggest to improve the interface.

That was kind of what I was trying to do, by allowing subtyping, so that the 
parser would be more knowledgeable about particular types of links, by allowing 
extended attributes. Maybe not implemented in the most extensible way though 
I'll admit.

My suggestion to the link parser would be to have the following properties as 
the catch-all properties for links:
- type :: Which type of link protocol applies. E.g. file, http, ftp, 
attachment, duckduckgo (ex. for a custom link abbreviation to search ddg)
- raw-path :: The path to use for the particular protocol. Would contain 
everything in the link following "protocol:"
- format :: Which format the link has. Plain, bracket or angular bracket
- description :: The description part of the link 
([[protocol:path][description]])
- begin, end, post-blank :: The default properties used for all elements. Not 
sure if contents-begin and contents-end are a part here as well.

In addition to that each protocol would be able to declare their own properties 
using a :parse function. All would probably implement a path property. Some 
would implement an options property, and some maybe would maybe declare their 
own properties.

As you've stated previously, from a performance perspective maybe it will be 
best to not expand the specific properties and instead let the expansion of 
those be handled in code by the link type owner (e.g. org-attach.el for 
attachment links). But not letting specific protocol parsers expand their own 
properties into a link makes it more difficult to implement support outside of 
emacs for the org specification, which in my opinion includes also a certain 
set of link protocols which do have their own specifications.

> > So it doesn't seem too farfetched that some of those link types would
> > benefit from additional custom properties, specific for those types.
> > =application= and =search-option= for example isn't universally
> > applicable to all link types.
> 
> As a side note, :application is on its way out, i.e., "file+emacs" stuff
> is in "org-compat.el".
> 
> Also, IIRC, "docview" links have "go to page" option, and they don't
> touch the parser.

Ah yes. Just briefly looking at the docview code. Docview doesn't seem to use 
the parser when extracting information about that "go to page" information from 
the link. Which means docview links doesn't really care how the link 
information is encoded in the parser anyways. Maybe if docview were allowed to 
encode custom docview information into the parsed link in the parser it and 
others could reused that encoded information later? Docview would be able to 
add a ":go-to-page" option, for example. Just a thought.

> > What I'm trying to argue for here is: Maybe it's not that crazy after
> > all to allow links to have additional properties in the parser based
> > on its type? (While certainly still trying to avoid it if possible!)
> 
> If new link types may not function correctly without touching the
> parser, how do you create new link types out of Org's core? This is not
> modular at all.

Most link types probably won't need custom link properties anyways. But could 
maybe let the parser know stuff by use of higher order functions? Or push for 
being important enough to be included into Org and allowed to be known by the 
parser.

> > (Off topic) I'm sorry to hear that you think I'm intentionally making
> > things fuzzy.
> 
> Not at all! My wording is terrible. When I wrote
> 
>     Also, you sometimes seem to blur, on purpose, the difference between
>     "attachment" and "file" links.
> 
> I meant something like:
> 
>    It seems to me that your intent is to have "attachment" links
>    transparently become "file" links to the user.
> 
> Hence my suggestion to use link abbreviations. There's nothing negative
> in it.

Ok, got it. Thanks for explaining. I'll admit it hasn't been totally clear to 
me what the best way forward is. After refreshing my understanding on links the 
conclusion I came to was that link abbreviations in its current form was not a 
fitting concept for attachment links. Attachments are pretty much similar to 
file links though, so even if my intention isn't to blur the meaning they will 
and should be treated very similar, since functionality is so similar.

> > One can indeed use the buffer position to derive the part of the path
> > that comes from the subtree. But leaving it at that puts more
> > requirements on the user of the parsed link in order to use it.
> 
> And higher order functions in "org-attach" could take care of it. Note
> that expanding a link in the parser means all attachment links are
> always expanded, even if they are not used. There's a cost to consider.

Higher order functions were used for exporting attachments previously. But the 
intention to keep functionality as close to file links as possible made me move 
the code from there into the separate exporters, to let all functionality be 
inherited by the way file exports were done, with the minor tweak needed to 
make it work for attachment links as well.

I've attached a patch that lets org-attach take care of expanding the link 
element into a path, instead of sub-typing inside org-element. While I'm still 
pro sub-typing links according to their type, this patch removes that. I.e. the 
performance cost of parsing is moved to the points where the information 
actually is used, with a sort of "informational cost" of not letting the org 
parser know about the peculiarities of certain link types.
 
> Besides, parser focus on the contents of the buffer. I think it is out
> of its scope to infer file names for abbreviation links. It's more the
> job of the parts consuming the parsed data.
> 
> > There is no :raw-path available in the properties for a parsed link.
> > If there were we surely wouldn't have this conversation and I'd be
> > using that already! :)
> 
> I meant :raw-link, sorry.
> 
> > I.e. I like this suggestion. I considered using the :raw-link
> > property, but the way it's currently used by the parser (i.e.
> > expanding abbreviations) stopped me.
> 
> Please take how link abbreviations are handled in the parser out of the
> equation. I already stated this is not a good way to handle them. We
> should probably expand them in a "temp-link" variable, and parse it,
> while keeping original link in :raw-link (barring white spaces fixes).
> That would not be perfect either, because we would still be inferring
> stuff.
> 
> That's another topic, really. Let's just ignore it for now.
> 
> > So, we're discussing regarding attachment as either: an abbreviation
> > or a proper separate link type? I'm not sure what other options we
> > have? I personally can't categorize it as an abbreviation since that
> > is handled by a separate piece of code with other specifications. I.e.
> > attachment wouldn't fit well inside org-link-abbrev-alist.
> 
> What makes you think it wouldn't fit well?

That list seems more like a feature to let end users expand links using 
find-replace with a tag in an existing path. Actually looking more into it now 
I see that it can also use expansion with functions. So maybe that could be 
used as a way of implementation. I'm not sure if there are other side effects 
of that though so it's nothing I'm pursuing now.

> > Attachments should function in the same way as file links, with search
> > options and all. But be limited to the current set of attached files.
> 
> This isn't incompatible with the above, AFAICT.

The path for attachment links needs to be complemented with the part from the 
attachment directory to make it work in the file system. That is the difference 
between the link types. 

> > That's basically the way it was before the patches to fix stardiviners
> > issues. Except not functioning well enough. It would require quite
> > some code in org-attach to replicate existing file links
> > functionality. Mostly code duplication I presume.
> 
> I didn't read stardiviner issues. Would you mind summarizing them? Or,
> at least could you explain what is not functioning well enough?

The expansions of image links didn't work correctly for attachment links. 
Simply because org-attach-export-link didn't support that.

> Anyhow, you may have missed the "let's spot the needs and improve these
> tools" part. If current tools lead to code duplication, we can fix that.

Speaking from the perspective of attachment links, if there were a file link 
type exporter function available that could be used by attachment links without 
code duplication. Another thing might be to make the higher order functions in 
the interface for adding new link types be able to use the parsed link instead 
of just certain properties of the link. Not sure if performance would be an 
issue then though.

> > The raw-path option sounds better to my ears.
> 
> Except I was meaning :raw-link :)
> 
> > One improvement I can think of (outside of the discussion above) is to
> > make it possible to pass the argument to org-link-open along to each
> > separate link type specialization.
> 
> Isn't that the job of :follow when defining a new link type?

Yes, and the function that implements that would also need the user argument as 
input. In addition to the link path.

> > That one has bugged me for some time now, when I've wanted to force
> > opening attachment links in emacs but couldn't.
> 
> IMO, forcing users to open stuff in a specific, non-configurable, way is
> a bad idea. Why would we know better than them?

I was not clear enough with what I meant. I meant not to force the user in the 
code to a specific choice, but to let the user force emacs to open the link. By 
use of arg as is specified in org-link-open.

Anyways, patch attached that makes org-element not know about attachments any 
longer, and moves most of the responsibility to org-attach.el. The exporters 
still needs to know to use the new function inside org-attach.el though.

Regards
Gustav

attachment-links.patch
Description: attachment-links.patch