Re: Remove "shell" as a supported Babel language within ob-shell.el

[Christopher M. Miles]

At first, thanks for this long parsing and explanation.

Matt <matt@excalamus.com> writes:

>  > Matt matt@excalamus.com> writes:
>  >
>  > > Is there a reason you're using "shell" instead of one of the shells 
> listed in `org-babel-shell-names'?
>
> I'm still curious why you're using "shell".  I want to know if it's something 
> you're using for a specific reason.  There's no wrong answer!
>
> I ask because I have an agenda: as far as I can tell, "shell" as a Babel 
> language is a historical accident.  
>
> #+begin_longwinded_explanation
> Originally, ob-shell.el was called ob-sh.el.  The main function was called 
> `org-babel-execute:sh' and only /usr/bin/env sh was supported.  Over time it 
> became clear that to support other shells, the "sh" name shouldn't be used 
> for the package or the main function.  That is, 'sh' refers to a specific 
> binary and, if other binaries such as bash, dash, csh, etc. were to be 
> supported, it would be misleading for the Babel language to refer to a 
> specific shell, 'sh'.  So, the terminology was changed to something more 
> general, "shell".  The package was renamed to "ob-shell.el", the "namespace" 
> updated to "shell" (for example, `org-babel-execute:shell'), and the package 
> load call changed from (sh . t) to (shell . t).  This officially happened 
> with Org 8.2 (ORG-NEWS noted the change in commit 
> 1a9a6666dcb6d34bcbdff45445c827ed99dbf0b8, Tue Jan 21 09:52:31 2014).  I think 
> this gave people the (understandable) impression that "shell" was a valid 
> Babel language, in addition to those listed in `org-babel-shell-names'.  
>
> And this is where the accident happened: "shell" as a Babel language only 
> **happens**to work.  The Babel framework looks for a function prototype like 
> "org-babel-execute:<language>".  When ob-sh.el was changed to ob-shell.el, 
> the function `org-babel-execute:sh' became `org-babel-execute:shell'.   A 
> call like follows is perfectly legal as far as the Babel framework is 
> concerned:
>
> #+begin_src shell
> echo "hello, world"
> #+end_src
>
> When such a block is run, Babel looks for a function called 
> `org-babel-execute:shell'. Running the
> block prior to Org 8.2 should have failed because no 
> `org-babel-execute:shell' function existed. The
> name change happened to source Fri Dec 13 09:52:05 2013 in commit
> 7a6c0e35415c4a173d101336029262f3a09abb91. After the name change, the function 
> existed and a block
> using "shell" would execute!

Yes, I originally use "sh" too. But at some time point, I saw an article
somewhere, then I switched to "shell". I forget the specific reason
already.

> The "shell" language specifier, as far as I can tell, was never really 
> intentionally supported.
> Instead, it just happened to work. It happened to work because, as far back 
> as the first
> org-babel-sh.el commit, the process buffer is created using the `shell' 
> function. I don't know the
> history of `shell', but presently the documentation says,
>
> Program used comes from variable ‘explicit-shell-file-name’,
>  or (if that is nil) from the ESHELL environment variable,
>  or (if that is nil) from ‘shell-file-name’.
>
> That is, the `shell' command falls back to `shell-file-name'. I assume that 
> `shell' has always had
> that, or a similar, fallback. The `shell-file-name' is a direct path to an 
> executable. This means
> that when "shell" is used for the language, `shell-file-name' is called and 
> **any** startup script,
> such as .bash_profile or .bashrc, is called. The prompt could be set to 
> **anything** and Emacs will
> never know, and can never know, what the prompt is without the user 
> explicitly informing Emacs.
>
> Aside from the code change which allowed "shell" to work, "official" support 
> of "shell" comes from
> Org manual commit 9d072ad67517875069f913315d762c9bb1e9c3ec, Sun Dec 17 
> 11:06:05 2017 (for example,
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/doc/org-manual.org?id=f7aa8c19f5170dbf09538686fb569f9b60acbd6c#n18410).
> This appears unconnected with the code change. The addition to the manual 
> happened 4 years after the
> code name change and none of the commit messages around the time of code 
> change suggest that "shell"
> was intended to work as a language. In fact, I found this email from Eric 
> Schulte (creator of Babel
> and maintainer at the time of the code change) which suggests that "shell" is 
> in fact not supported
> or intented as a language 
> (https://lists.gnu.org/archive/html/emacs-orgmode/2013-12/msg00294.html):
>
> In response to the statement,
>
> "a coworker used "#+BEGIN_SRC shell" where he should have written 
> "#+BEGIN_SRC sh"
>
> Eric says,
>
> "[The suggested work around] would protect against this particular error"
>
> #+end_longwinded_explanation
>
> Regardless of whether "shell" was intended to work as a Babel language, the 
> fact remains that it
> does work and that it's been advertised in the manual (at least) for 6 years. 
> What are the pros and
> cons of "shell"?
>
> What benefit does "shell" provide?
>
> - The "shell" language allows an arbitrary executable to be run. This means 
> that shells other than
> those given in `org-babel-shell-names' can be run. People using a 
> non-supported shell could still
> benefit from ob-shell.
>
> What downsides does "shell" bring?
>
> - "shell" falls back to `shell-file-name' which can be an arbitrary 
> executable. Whenever I hear
> "runs an arbitrary executable", my ears perk up and I start to sweat. There 
> may be security
> considerations.

This arbitrary executable fallback mechanism indeed is a con side.

> - If that executable is a shell, then the prompt gets set independently from 
> Emacs. For the prompt
> to be filtered from the output, users would need to provide Emacs with the 
> correct regexp. A recent
> thread discussed creating a header arg to address this:
> https://list.orgmode.org/87ttzgeg3w.fsf@localhost/
> - We would get bug reports about non-supported shells which kind of work, but 
> have issues because they're not supported
> - Maintence associated with supporting arbitrary (shell) executables
>
> As the current maintainer of ob-shell, I'm in favor of removing "shell" as a 
> Babel language. The
> cons appear to far outweigh the pros. However, I'm aware others may have good 
> use for it. It's been
> a part of Org for nearly a decade. I'm sure it's part of people's workflow, 
> especially since it's
> been in the manual for 6 years. Are there any pros, cons, use-cases, or 
> considerations I've
> overlooked?

If the "shell" language will be removed, I'm ok with that. I hope this
library can warn user "shell" is deprecated. Because I have a lot of
already written Org mode files using "shell" as source block language.
Replacing it with command-line tools like "sed" etc is ok. Like adding a
line warning code:

#+begin_src emacs-lisp
(warn "The 'shell' language is deprecated already, use 'sh' instead.")
#+end_src

-- 

[ stardiviner ]
I try to make every word tell the meaning that I want to express without 
misunderstanding.

Blog: https://stardiviner.github.io/
IRC(libera.chat, freenode): stardiviner, Matrix: stardiviner
GPG: F09F650D7D674819892591401B5DF1C95AE89AC3

signature.asc
Description: PGP signature