Suggestions from a new GNATS/gnatsweb user

[William M. (Mike) Miller]

We've just switched to GNATS from a proprietary defect tracking
system, and I wanted to share some observations and suggested
changes that came out of that experience.  First, though, I'd
like to thank the GNATS team for the product; the transition
was as close to painless as I could have hoped for, and we're
very pleased with the outcome.

Let me begin by saying that we are a small (4-person),
geographically-distributed company.  Futhermore, we are the
only ones interacting directly with GNATS; our customers send
us free-form email about questions and problems, and we digest
that information before entering it into the defect tracking
system.  We use gnatsweb almost exclusively, the only exception
being a few canned queries that use query-pr directly; we don't
use the email interface to GNATS at all.  These factors
naturally affect what we need from GNATS and are different from
many other users, I'm sure.

All of the suggestions and changes below deal with gnatsweb.pl;
we found no issues at all with GNATS proper.  I'll describe them
in no particular order.  All diffs are against gnatsweb-4.00.
Some are included just to point out where the change would be
needed, because doing it "right" here was more work than I had
time to commit at the point I was making the change.

One caveat regarding these suggestions: although I've generally
included diffs for the changes I describe, my Perl skills are
pretty rudimentary, so take the code with a grain of salt.  (I
won't be offended at all if someone sends me a more Perl-esque
substitute for some bit or another. :-)

1. We use the Category field to separate PRs among our three
   products.  Nearly all of the support traffic is for one of
   those products, so it's very convenient to have the Category
   field default to that product.  However, gnatsweb goes out
   of its way to require the user to set the Category field
   explicitly for each new PR.  In the following patch I just
   commented out the offending code, but it would be nice if
   this were configurable in gnatsweb-site.pl.

*** 1081,1095 ****
          "</small></td><td>\n", $intro, "\n";

      if (fieldinfo($_, 'fieldtype') eq "enum")
      {
        # Force user to choose a category.
!       if ($_ eq $CATEGORY_FIELD)
!       {
!         push(@$values, "unknown") if (!grep /^unknown$/, @$values);
!         $default = "unknown";
!       }
        if ($_ eq $SUBMITTER_ID_FIELD)
        {
            $default = $global_prefs{$SUBMITTER_ID_FIELD} || '';
        }
        print popup_or_scrolling_menu($_, $values, $default),
--- 1081,1095 ----
          "</small></td><td>\n", $intro, "\n";

      if (fieldinfo($_, 'fieldtype') eq "enum")
      {
        # Force user to choose a category.
! #      if ($_ eq $CATEGORY_FIELD)
! #      {
! #        push(@$values, "unknown") if (!grep /^unknown$/, @$values);
! #        $default = "unknown";
! #      }
        if ($_ eq $SUBMITTER_ID_FIELD)
        {
            $default = $global_prefs{$SUBMITTER_ID_FIELD} || '';
        }
        print popup_or_scrolling_menu($_, $values, $default),
***************

2. We found the 4-line default height of all multitext fields
   (except Description in the Edit page) to be constraining.  It
   would be nice to be able to set the heights of different
   fields (in different pages) explicitly in gnatsweb-site.pl.
   In this patch, I've just made the Description and Notes
   fields ("Notes" is a locally-added multitext field) 8 lines
   high in both the Create and Edit pages.  (Note: the original
   code assumes that the Description field is named
   "Description", both for the height calculation and to
   determine where to put the list of attachments.  This should
   be done via a variable, as is the case with most field names,
   like $CATEGORY_FIELD in the preceding patch.  I just used the
   field names directly in this patch.)

*** 1102,1111 ****
--- 1102,1112 ----
              "</td>\n</tr>\n";
      }
      elsif (fieldinfo($_, 'fieldtype') eq "multitext")
      {
        my $rows = 4;
+       $rows = 8 if /Description|Notes/;
        print $q->textarea(-name=>$_,
                           -cols=>$textwidth,
                           -rows=>$rows,
                           -default=>$default),
              "</td>\n</tr>\n";
***************
*** 1604,1614 ****
        "</td>\n</tr>\n";
      }
      elsif (fieldinfo ($_, 'fieldtype') eq 'multitext')
      {
        my $rows = 4;
!       $rows = 8 if /Description/;
        $rows = 2 if /Environment/;
        print $q->textarea(-name=>$_,
                           -cols=>$textwidth,
                           -rows=>$rows,
                           -default=>$fields{$_}),
--- 1609,1619 ----
        "</td>\n</tr>\n";
      }
      elsif (fieldinfo ($_, 'fieldtype') eq 'multitext')
      {
        my $rows = 4;
!       $rows = 8 if /Description|Notes/;
        $rows = 2 if /Environment/;
        print $q->textarea(-name=>$_,
                           -cols=>$textwidth,
                           -rows=>$rows,
                           -default=>$fields{$_}),
***************

3. Although gnatsweb removes empty query fields in many of the
   links it displays, the ones that go through get_pr_url do
   not.  This affects, notably, the "view" and "edit" links in
   each row of query output, so the extra baggage can add up,
   especially for the result of an "advanced query", which will
   typically have lots of empty fields.

   (On a more global note, this URL is used in the return_url
   field, which allows an automatic return to the query results
   page after editing a PR on that page.  This mechanism is
   really wasteful of bandwidth, adding many, many KB to each
   query results page for all the return_url fields.  This
   really ought to be implemented by using a cookie, not by
   embedding the query in every link on the page!  If I get the
   time, I'm going to look into changing this into using
   cookies.)

*** 1311,1322 ****
  sub get_pr_url
  {
    my($cmd, $pr, $include_return_url) = @_;
    my $url = $q->url() . "?cmd=$cmd&database=$global_prefs{'database'}";
    $url .= "&pr=$pr" if $pr;
!   $url .= "&return_url=" . $q->escape($q->self_url())
!         if $include_return_url;
    return $url;
  }

  # Return a URL to edit the given pr.  See get_pr_url().
  #
--- 1312,1327 ----
  sub get_pr_url
  {
    my($cmd, $pr, $include_return_url) = @_;
    my $url = $q->url() . "?cmd=$cmd&database=$global_prefs{'database'}";
    $url .= "&pr=$pr" if $pr;
!   if ($include_return_url) {
!     my ($query_string) = $q->self_url();
!     # Remove empty fields
!     $query_string =~ s/(\w|-)+=;//g;
!     $url .= "&return_url=" . $q->escape($query_string);
!   }
    return $url;
  }

  # Return a URL to edit the given pr.  See get_pr_url().
  #
***************

4. There is an assumption in gnatsweb that the first field in
   the "responsible" configuration is a user id, so that a
   "mailto:"; can be constructed by appending $site_mail_domain
   to it.  That's not true in our configuration (for reasons
   that aren't worth getting into), and it's an unnecessary
   assumption in any case: the third "responsible" field is an
   email address, so if it's there, it ought to be used in the
   "mailto:";.  (Note that parse_responsible defaults
   $responsible_address to the person name, so a configuration
   in which the first field is, in fact, an email user id will
   still work the same after these changes.)

*** 1449,1459 ****

        # make links in various fields
        if ($_ =~ /responsible/i) {
          # values in the responsible field are likely to be bare usernames,
          # so mark_urls won't work on them.
!         $val = make_mailto($val);
        } elsif ($_ =~ /related-prs/i) {
           # make the Related-PRs field show links to the PRs
  # dtb - this is juniper specific, but i think it's a good field to have in
  # the dbconfig...
          $val =~ s{(\b|PR)(\d+)\b}{'<a
href="'.get_viewpr_url($2)."\">$1$2</a>"}egi;
--- 1454,1464 ----

        # make links in various fields
        if ($_ =~ /responsible/i) {
          # values in the responsible field are likely to be bare usernames,
          # so mark_urls won't work on them.
!         $val = make_mailto($val, $responsible_address{$val});
        } elsif ($_ =~ /related-prs/i) {
           # make the Related-PRs field show links to the PRs
  # dtb - this is juniper specific, but i think it's a good field to have in
  # the dbconfig...
          $val =~ s{(\b|PR)(\d+)\b}{'<a
href="'.get_viewpr_url($2)."\">$1$2</a>"}egi;
***************
*** 2141,2155 ****


  # takes a string, and turns it into a mailto: link
  # if it's not a full address, $site_mail_domain is appended first
  sub make_mailto {
!     my $string = shift;
!     if ($string !~ /@/) {
!       $string = qq*<a href="mailto:${string}${site_mail_domain}";>$string</a>*;
      } else {
!       $string = qq*<a href="mailto:$string";>$string</a>*;
      }
      return $string;
  }

  # takes a string, attempts to make urls, PR references and email
--- 2146,2162 ----


  # takes a string, and turns it into a mailto: link
  # if it's not a full address, $site_mail_domain is appended first
  sub make_mailto {
!     my $name = shift;
!     my $addr = shift || $name;
!     my $string;
!     if ($addr !~ /@/) {
!       $string = qq*<a href="mailto:${addr}${site_mail_domain}";>$name</a>*;
      } else {
!       $string = qq*<a href="mailto:$addr";>$name</a>*;
      }
      return $string;
  }

  # takes a string, attempts to make urls, PR references and email
***************
*** 2482,2492 ****
        }
        $fieldcontents = $q->escapeHTML($fieldcontents);
        $fieldcontents = nonempty($fieldcontents);

        if ($columns[$whichfield] =~ /responsible/i) {
!         $fieldcontents = make_mailto($fieldcontents);
        } else {
          # make urls and email addresses into live hrefs
          $fieldcontents = mark_urls($fieldcontents);
        }

--- 2489,2500 ----
        }
        $fieldcontents = $q->escapeHTML($fieldcontents);
        $fieldcontents = nonempty($fieldcontents);

        if ($columns[$whichfield] =~ /responsible/i) {
!         $fieldcontents = make_mailto($fieldcontents,
!                                      $responsible_address{$fieldcontents});
        } else {
          # make urls and email addresses into live hrefs
          $fieldcontents = mark_urls($fieldcontents);
        }

***************

5. Query output can get very wide if you include a number of
   fields.  Dates are a major contributor; gnatsweb displays
   them with times and UTC deltas.  We don't need that level of
   detail in query output, given the relatively low volume of
   traffic that we handle; the day a PR arrived or was closed is
   really sufficient.  The format used to display dates in query
   output should be configurable in gnatsweb-site.pl.  In the
   following patch, I've just changed the format directly to
   reduce the clutter.

*** 2345,2355 ****
    # fields in the query output. Note that the format strings are
    # interpolated (quoted with ""'s), so make sure to escape any $ or @
    # signs.
    foreach (@columns) {
        if (fieldinfo ($_, 'fieldtype') eq 'date') {
!       $format .= "\037%{%Y-%m-%d %H:%M:%S %Z}D";
        } elsif (fieldinfo ($_, 'fieldtype') eq 'enum') {
        $format .= "\037%d";
        } else {
        $format .= "\037%s";
      }
--- 2352,2362 ----
    # fields in the query output. Note that the format strings are
    # interpolated (quoted with ""'s), so make sure to escape any $ or @
    # signs.
    foreach (@columns) {
        if (fieldinfo ($_, 'fieldtype') eq 'date') {
!       $format .= "\037%{%Y-%m-%d}D";
        } elsif (fieldinfo ($_, 'fieldtype') eq 'enum') {
        $format .= "\037%d";
        } else {
        $format .= "\037%s";
      }
***************

6. There is a bug in gnatsweb-4.00 that causes some stored
   queries (notably "advanced queries") not to work.  The
   reason is that the regular expression used to eliminate
   empty query fields in the cookie is defective, not allowing
   for "-" in field names.  This affects advanced queries in
   particular because the date fields are at the beginning of
   the query URL and the field names for the dates typically
   include "-" characters.

*** 2584,2594 ****
    my $query_string = $q->query_string();

    # strip empty params out of $query_string.  in a gnats db with many
    # fields, the query-string will become very long, and may exceed the
    # 4K limit for cookies.
!   $query_string =~ s/\w+=;//g;

    if (length($query_string . $global_cookie_path .
"gnatsweb-query-$queryname") > 4050) {
      # this cookie is going to be longer than 4K, so we'll have to punt
      error_page('Cannot store query -- cookie too large',
                 "Gnatsweb cannot store this query as a cookie because"
--- 2592,2602 ----
    my $query_string = $q->query_string();

    # strip empty params out of $query_string.  in a gnats db with many
    # fields, the query-string will become very long, and may exceed the
    # 4K limit for cookies.
!   $query_string =~ s/(\w|-)+=;//g;

    if (length($query_string . $global_cookie_path .
"gnatsweb-query-$queryname") > 4050) {
      # this cookie is going to be longer than 4K, so we'll have to punt
      error_page('Cannot store query -- cookie too large',
                 "Gnatsweb cannot store this query as a cookie because"
***************

7. Because we're a small company and only the four of us are
   accessing GNATS, we don't use the "full name" fields in
   submitters and responsible (we know each other by first name
   :-).  However, the trailing "-" that results from this
   configuration is unesthetic.

*** 3271,3281 ****
    foreach $_ (sort @lines)
    {
      my($submitter, $fullname, $type, $response_time, $contact, $notify)
            = split(/:/);
      push(@submitter_id, $submitter);
!     $submitter_complete{$submitter} = $submitter .' - ' . $fullname;
      $submitter_contact{$submitter} = $contact;
      $submitter_notify{$submitter} = $notify;
    }
  }

--- 3279,3290 ----
    foreach $_ (sort @lines)
    {
      my($submitter, $fullname, $type, $response_time, $contact, $notify)
            = split(/:/);
      push(@submitter_id, $submitter);
!     $submitter_complete{$submitter} = $submitter;
!     $submitter_complete{$submitter} .= ' - ' . $fullname if $fullname;
      $submitter_contact{$submitter} = $contact;
      $submitter_notify{$submitter} = $notify;
    }
  }

***************
*** 3291,3302 ****

    foreach $_ (sort @lines)
    {
      my($person, $fullname, $address) = split(/:/);
      push(@responsible, $person);
!     $responsible_fullname{$person} = $fullname;
!     $responsible_complete{$person} = $person . ' - ' . $fullname;
      $responsible_address{$person} = $address || $person;
    }
  }

  # initialize -
--- 3300,3312 ----

    foreach $_ (sort @lines)
    {
      my($person, $fullname, $address) = split(/:/);
      push(@responsible, $person);
!     $responsible_fullname{$person} = $fullname || $person;
!     $responsible_complete{$person} = $person;
!     $responsible_complete{$person} .= ' - ' . $fullname if $fullname;
      $responsible_address{$person} = $address || $person;
    }
  }

  # initialize -
***************

8. I wasn't sufficiently motivated to change this, but there's
   an annoying assumption regarding regular expressions for
   enumeration values in the following code:

      # Most (?) people expect queries on enums to be of the
      # exact, not the substring type.
      # Hence, provide explicit anchoring for enums. This
      # still leaves the user the possibility of inserting
      # ".*" before and/or after regular expression searches
      # on the advanced query page.
      if (fieldinfo($field, 'fieldtype') =~ "enum|multienum")
      {
        $subexp = appendexpr ($subexp, '|', "$field~\"^$sval\$\"");
      }
      else
      {
        $subexp = appendexpr ($subexp, '|', "$field~\"$sval\"");
      }

   We aren't among the "most (?) people," apparently.  We have
   defined the enumeration for Priority to have values like
   "1 - showstopper", "2 - urgent", "3 - important", etc., so
   we'd like to use a regular expression like "[12]" to match
   high-priority PRs and not have to type "[12].*".  We've set
   "query-default inexact-regexp" in dbconfig for the Priority
   field, but gnatsweb ignores it.  Either gnatsweb should
   honor that setting or it ought to be possible to override
   this assumption about enumerated fields by a setting in
   gnatsweb-site.pl.

Well, that's it.  I hope someone finds this helpful.  If I get
around to coding up the change to use a cookie instead of
return_url, I'll post that diff, too.

--
William M. (Mike) Miller | Edison Design Group
address@hidden