Re: [Repo-criteria-discuss] What's needed to publish the evaluations (aka the longest email ever {aka two specific tasks})}

[Mike Gerwitz]

On Wed, Feb 17, 2016 at 21:43:27 -0500, Mike Gerwitz wrote:
> I did find it, so I'll just make sure everything looks accurate and
> send it over soon.

I think that this might be in the old format.  But it does not change
the grade vs. the refined criteria.

This is the same one I sent quite a while back, but I have updated it to
include the work done by GitLab to ensure that features work without
JavaScript, and the removal of GA for Piwik.

signature.asc
Description: PGP signature

# PAGE TITLE: Criteria for evaluating code hosting services for use with GNU 
packages

*A list of code currently active code hosting services evaluated with these 
criteria is coming soon. If you are interested in helping with the evaluation, 
please contact the Free Software Foundation at <address@hidden>.*

## Acceptable hosting for a GNU package

* All important site functionality that's enabled for use with that
package works correctly (though it need not look as nice) in free
browsers, including [IceCat](https://www.gnu.org/software/gnuzilla/), without 
running any nonfree software
sent by the site.

  All GitLab JS is free.

  [PASS]


* (Elaborating the first requirement for JavaScript.) Regarding
sending code that runs on the JavaScript platform, any such code
used by an important site function either (1) is free software, and
labeled properly for [LibreJS](https://www.gnu.org/software/librejs/) to 
recognize as free, or (2) isn't
necessary, so that the function works properly even if JavaScript is
disabled in the browser.

  Although all JS is free, it is not LibreJS-compatible.  If we could make
  note on the published webpage for these evaluations that I'm looking for
  people to help out with that (I've been tight on time), that'd be great.

  That said, I tested important functionality without JS:
    - Browsing projects and code [PASS]
    - New Project                [PASS]
    - Delete Project             [PASS]
    - Change project settings    [PASS]
    - Update profile             [PASS]
    - Add account e-mail         [PASS]
    - Add SSH key                [PASS]
    - Add issue                  [PASS]
    - Create merge request       [PASS]

  [PASS]

* (Elaborating the first requirement for other platforms.) Regarding
sending code that executes based on a platform other than
JavaScript, those conditions apply, mutatis mutandis. In addition,
a free implementation of that platform, and a free program to check
for free licensing (comparable to LibreJS for JavaScript), must be
available for the principal GNU browser IceCat, and the site must
work adequately with them.

  [PASS]


* No other nonfree software is required to use the site
(thus, no Flash).

  [PASS]

* Permits access from any country, by anyone.

  I use Tor exclusively and have never had a problem.  Tested numerous
  nodes.

  [PASS]

* Permits access via Tor (and such access is considered an important
site function).

  [PASS]

* Terms of service do not cover anything but the direct use of the
service and its non-software materials.

  [PASS]

* Recommends and encourages GPL 3-or-later licensing
at least as much as any other kind of licensing.

  Does not recommend _any_ licenses via it's "Add License" project link.

  [PASS]

## Good enough to recommend

The above criteria, plus:

* All code sent to the user's browser must be free software and
labeled for LibreJS or other suitable free automatic license
analyzer, regardless of whether the site functions when the user
disables this code.

  [FAIL]

* Does not report visitors to other organizations;
in particular, no tracking tags in the pages.

  GitLab recently replaced Google Analytics with Piwik.

  [PASS]

* Does not encourage bad licensing practices
(no license, unclear licensing, GPL N only).

  Does not encourage adding a license.

  Also recognizes only LICENSE as the license file, not COPYING, and so does
  not recognize all existing license files.

  [FAIL]

* Does not recommend nonfree licenses for works for practical use.

  Doesn't recommend anything.

  [PASS]


## Excellent

The above criteria, plus:

* All important site functions work correctly (though may not look
as nice) when the user disables execution of JavaScript and other
code sent by the site.

  See above.

  [FAIL]

* Server code released as free software.

  In part, but contains some proprietary modifications (GitLab EE).

  [FAIL]

* Encourages use of GPL 3-or-later as preferred option.

  Does not encourage any license.

  [FAIL]

* Offers use of AGPL 3-or-later as an option.

  Offers no options.  (Bruno clarified with Zak: this is okay, because
  Gitlab doesn't prohibit its use---it just doesn't offer license options.)

  [PASS]

* Does not permit nonfree licenses (or lack of license) for works for
practical use.

  No check on license.

  [FAIL]

* Does not recommend services that are
[SaaSS](https://gnu.org/philosophy/who-does-that-server-really-serve.html).

  Offers CI services.

  [FAIL]

* Says "free software", not "open source."

  Uses primarily "open source".

  [FAIL]

* Avoids saying "Linux" without "GNU" when referring to GNU/Linux.

  It seems to avoid the terminology by stating specific distributions
  without "Linux" or "GNU"...impressively consistent.  I ended up having to
  grep the GitLab CE repository to find instances, where there were also few
  results: the couple mentions in the actual documentation was
  "GNU/Linux".  All other references to "Linux" appear to be either in the
  name of the distribution "e.g. Arch Linux" (as stated by the distribution
  itself), or build flags for third-party systems (e.g. Docker).

  Does saying "Arch Linux" (for example) result in a fail here?  This is a
  difficult one, since the name of the distribution is indeed "Arch Linux".

  [PASS?]

## Extra credit

* Allows anonymous visitors to look and download, and does not log
anything about about those visitors. The site should follow the
criteria in The Electronic Frontier Foundation's [best practices for
online service providers](https://www.eff.org/wp/osp). (We cannot
verify this so we accept the site maintainer's word.

  See e-mail thread.
  Message-ID: <address@hidden>

  [FAIL]

* Follows accessibility standards: the [Web "Content" Accessibility
Guidelines 2.0](https://www.w3.org/WAI/WCAG20/quickref/) and
[WAI-ARIA](https://www.w3.org/TR/wai-aria-practices/).

  I found only 'aria-hidden' in the case of ARIA attributes.  Grepping the
  repository yielded only aria-{hidden,describedby}.  This is far from
  sufficient.

  [FAIL]

-- 
Mike Gerwitz
Free Software Hacker | GNU Maintainer
https://mikegerwitz.com
FSF Member #5804 | GPG Key ID: 0x8EE30EAB