Re: [Help-bash] bash suitable for parsing big files?

[adrelanos]

Greg Wooledge:
> On Fri, Sep 13, 2013 at 02:55:36AM +0000, adrelanos wrote:
>> The script I am writing looks what files are installed, downloads the
>> package from $linux-distribution's repositories and compares them with
>> the ones on the disk.
> 
> On Fri, Sep 13, 2013 at 12:05:25AM +0000, adrelanos wrote:
>> For those who don't know /var/lib/dpkg/status, it's size is roughly 2 MB
>> and contains roughly 50.000 lines.
> 
> Ugh.  Parsing /var/lib/dpkg/status is not the best approach at all.
> 
> while read -r pkg _; do
>   ... process "$pkg" somehow ...
> done < <(dpkg --get-selections)
> 
>>From the man page:
> 
>        --get-selections [package-name-pattern...]
>               Get  list of package selections, and write it to stdout. Without
>               a pattern, non-installed packages (i.e. those  which  have  been
>               previously purged) will not be shown.
> 
> This saves you from having to write a finite state machine sort of thing
> to parse the multiple-lines-per-package status file (which you would have
> to do in order to filter out the non-installed ones).
> 
> 

Thanks to all of the list's ideas, the script has been speed up factor
100 or so.

This was another fine idea. I am now using:

   while read -r package status arch version_with; do
...
   done < <(dpkg-query --admindir="$dpkg_status_folder"
--showformat="\${Package}\t\${db:Status-Abbrev}\t\${Architecture}\t\${Version}\n"
--show \*)

Is this safe from a security perspective (this is untrusted input) of
could a malformed dpkg status file containing things like "`$(echo a)`"
lead to code execution in bash?