Setting HISTTIMEFORMAT is causing problems in >=bash-5.2

[Thomas Deutschmann]

Hello!

I would like to report an issue with bash version >=5.2.

For years, I have had the following line in my ~/.bashrc:

> export HISTTIMEFORMAT="[$(tput setaf 6)%F %T$(tput sgr0)]: " # colorful date

This worked perfectly up to and including bash version 5.1.0(16).

However, since bash version 5.2.0(2), I have observed the following problem:

When I connect via SSH to a system with bash >=5.2.0 and execute 
"shorewall compile" directly (which generates a shell script in 
/var/lib/shorewall/firewall), the script becomes corrupted:

> # /bin/sh /var/lib/shorewall/firewall help
> /var/lib/shorewall/firewall: line 2239: syntax error near unexpected token `('
> /var/lib/shorewall/firewall: line 2239: `    GCC_SPECS='export 
> HISTTIMEFORMAT=$'[\E[36m%F %T\E(B\E[m]: '''

This does not occur when I run "shorewall compile" via tty or, for 
example, in a tmux session. It only happens when I compile the firewall 
script directly via SSH.

When I compare the script generated with bash <5.2 to that generated 
with bash >=5.2, this is the difference:

> # diff -u /var/lib/shorewall/.safe /var/lib/shorewall/.restart
> --- /var/lib/shorewall/.safe    2024-06-03 02:53:02.687389414 +0200
> +++ /var/lib/shorewall/.restart 2024-06-03 02:53:02.564050925 +0200
> @@ -1,6 +1,6 @@
>  #!/bin/sh
>  #
> -# Compiled firewall script generated by Shorewall 5.2.8 - Mon Jun 3 02:50:56 
> AM CEST 2024
> +# Compiled firewall script generated by Shorewall 5.2.8 - Mon Jun 3 02:53:02 
> AM CEST 2024
>  #
>  #   (c) 1999-2019 - Tom Eastep (teastep@shorewall.net)
>  #
> @@ -2234,6 +2234,10 @@
>      IP=ip
>      TC=tc
>      IPSET=ipset
> +    #
> +    # From the params file
> +    #
> +    GCC_SPECS='export HISTTIMEFORMAT=$'[\E[36m%F %T\E(B\E[m]: ''
>
>      g_stopping=


The responsible function is a Perl function, which can be found here: 
https://gitlab.com/shorewall/code/-/blob/2673e6e60cce99b4d9723ecd13d5d6f17e27d390/Shorewall/Perl/Shorewall/Config.pm#L6051

I have stepped through the function with the Perl debugger but still do 
not understand what is going wrong (the below for full debug output):

%ENV contains GCC_SPECS -- but as an empty variable. When I create 
another empty environment variable like "GZZ_TEST", the error shifts, 
i.e., the error message then reads:

> /var/lib/shorewall/firewall: line 2239: `    GZZ_TEST='export 
> HISTTIMEFORMAT=$'[\E[36m%F %T\E(B\E[m]: '''

"export HISTTIMEFORMAT..." is not visible within %ENV.

The variable is only written into the script because no such variable 
with this value exists in the current environment (lines 6081-6085). Of 
course this seems to reveal a quoting issue in the Shorewall code. 
However, I'll address this later in the Shorewall project. The 
underlying problem is hopefully not related to Shorewall.

If I remove the mentioned line from my .bashrc, the problem does not 
occur anymore.

As mentioned, this problem does not occur with bash versions <5.2. In 
these versions, the last variable starting with G is empty, exactly as 
defined -- empty.

So why does "export HISTTIMEFORMAT..." appear within Perl at all? What 
has changed in bash >=5.2? Is this a bug? Am I doing something wrong and 
was just lucky that it worked that way for so long?

Thank you!



PS: Here's the hopefully related Perl debug output (I set a break point 
in export_params function after line 6062):

> # shorewall compile -d
> Compiling using Shorewall 5.2.8...
>
> Loading DB routines from perl5db.pl version 1.77
> Editor support available.
>
> Enter h or 'h h' for help, or 'man perldebug' for more help.
>
> main::(/usr/share/shorewall/compiler.pl:85):
> 85:     my $export        = 0;
>   DB<1> c
> Processing /etc/shorewall/params ...
> Processing /etc/shorewall/shorewall.conf...
> Loading Modules...
> Compiling /etc/shorewall/zones...
> Compiling /etc/shorewall/interfaces...
> Determining Hosts in Zones...
> Locating Action Files...
> Compiling /etc/shorewall/policy...
> Running /etc/shorewall/initdone...
> Adding Anti-smurf Rules
> Adding rules for DHCP
> Compiling TCP Flags filtering...
> Compiling Kernel Route Filtering...
> Compiling Martian Logging...
> Compiling MAC Filtration -- Phase 1...
> Compiling /etc/shorewall/rules...
> Compiling /etc/shorewall/conntrack...
> Compiling MAC Filtration -- Phase 2...
> Applying Policies...
> Generating Rule Matrix...
> Optimizing Ruleset...
> Shorewall::Config::export_params(/usr/share/shorewall/Shorewall/Config.pm:6065):
> 6065:           if ( $shell == BASH ) {
>   DB<1> p $param
> CCACHE_DIR
>   DB<2> c
> Shorewall::Config::export_params(/usr/share/shorewall/Shorewall/Config.pm:6065):
> 6065:           if ( $shell == BASH ) {
>   DB<2> p $param
> CONFIG_PROTECT
>   DB<3> p $value
> /usr/share/gnupg/qualified.txt
>   DB<4> c
> Shorewall::Config::export_params(/usr/share/shorewall/Shorewall/Config.pm:6065):
> 6065:           if ( $shell == BASH ) {
>   DB<4> p $param
> CONFIG_PROTECT_MASK
>   DB<5> c
> Shorewall::Config::export_params(/usr/share/shorewall/Shorewall/Config.pm:6065):
> 6065:           if ( $shell == BASH ) {
>   DB<5> p $param
> EDITOR
>   DB<6> p $value
> /bin/nano
>   DB<7> c
> Shorewall::Config::export_params(/usr/share/shorewall/Shorewall/Config.pm:6065):
> 6065:           if ( $shell == BASH ) {
>   DB<7> p $param
> GCC_SPECS
>   DB<8> p $value
> export HISTTIMEFORMAT=$'[\E[36m%F %T\E(B\E[m]: '
>   DB<9> s
> Shorewall::Config::export_params(/usr/share/shorewall/Shorewall/Config.pm:6068):
> 6068:               $value =~ s/\\'/'/g;
>   DB<9>
> Shorewall::Config::export_params(/usr/share/shorewall/Shorewall/Config.pm:6069):
> 6069:               $value =~ s/\\"/"/g;
>   DB<9>
> Shorewall::Config::export_params(/usr/share/shorewall/Shorewall/Config.pm:6070):
> 6070:               $value =~ s/\\\\/\\/g;
>   DB<9>
> Shorewall::Config::export_params(/usr/share/shorewall/Shorewall/Config.pm:6077):
> 6077:           if ( defined $ENV{$param} ) {
>   DB<9>
> Shorewall::Config::export_params(/usr/share/shorewall/Shorewall/Config.pm:6078):
> 6078:               next if $value eq $ENV{$param};
>   DB<9> V ENV[param]
> $| = 1
> $^A = ''
> $^D = 0
> $1 = 'ENV[param]'
> %ENV = (
>    'CCACHE_DIR' => '/var/cache/ccache'
>    'CONFIG_PROTECT' => '/usr/share/gnupg/qualified.txt'
>    'CONFIG_PROTECT_MASK' => '/etc/sandbox.d /etc/terminfo'
>    'EDITOR' => '/bin/nano'
>    'GCC_SPECS' => ''
>    'HISTFILESIZE' => 10000
>    'HISTSIZE' => 10000
>    'HISTTIMEFORMAT' => "[\e[36m%F %T\e(B\e[m]: "
>    'HOME' => '/root'
> [...]


--
Regards,
Thomas