Quotes being stripped by "--csv"

[Ed Morton]

Someone posted a question on stackoverflow about how to print just the 
first 2 fields from a CSV so given this input:

   "foo,""bar""",2,3
   1,"foo,bar",3
   1,"foo,
   bar",3

the expected output would be:

   "foo,""bar""",2
   1,"foo,bar"
   1,"foo,
   bar"

I thought I'd answer with "--csv" but when I tried it I got this output:

   $ awk --csv -v OFS=',' '{print $1, $2}' file.csv
   foo,"bar",2
   1,foo,bar
   1,foo,
   bar

The quotes around the fields that need to be quoted (and were quoted in 
the input) are missing and the escaped double quotes (`""`) around the 
first `bar` have become individual (`"`) so the output is no longer 
valid CSV.

I could get it back to valid CSV and produce the expected output by 
writing this or similar:

   $ awk --csv -v OFS=',' '{for (i=1; i<=NF; i++) {
   gsub(/"/,"\"\"",$i); if ($i ~ /[,\n"]/) { $i="\"" $i "\""} }; print
   $1, $2}' file.csv
   "foo,""bar""",2
   1,"foo,bar"
   1,"foo,
   bar"

but that's counter-intuitive and frustrating to have to write and I 
think many users wouldn't know how to, or understand why they need to, 
write that code to get valid CSV output.

I understand there is a benefit to stripping double quotes for working 
on field contents and I appreciate that you need to make this work with 
existing functionality (`OFS` values, etc.) so I understand why `--csv` 
can't simply always output valid CSV and I also understand the "don't 
provide constructs to do things that are easy to do with existing 
constructs" awk mantra to avoid code bloat, but there has to be a way to 
make it easier for people to just print a couple of fields from valid 
CSV input and have the output still be valid CSV.

If there was a way to have `--csv` optionally NOT strip double quotes 
when reading the fields then that'd solve the problem, e.g. `--csv=q` or 
`--csvq` or similar to indicate quotes in and around fields should be 
retained. If we had that then I could write something like:

    awk --csv=q -v OFS=',' '{print $1, $2}' file.csv

or, less desirably as it's longer and can't be set on the command line 
but would be better than nothing:

    awk --csv -v OFS=',' 'BEGIN{PROCINFO["CSV"]="q"} {print $1, $2}' 
file.csv

to get the desired output above and there are almost certainly other use 
cases for people wanting to retain the quotes and there is no simple 
alternative today (not using --csv but instead setting FPAT and counting 
double quotes to know if a newline is inside or outside of a field, and 
adding lines to $0 until you have a complete record).

I don't think that would be hard for users to understand or result in 
language bloat or introduce any additional complexity working with 
existing constructs - you simply wouldn't strip quotes when reading the 
input and so they'd still be there when producing output.

Regards,

    Ed.