[task #15497] Adding a verification step

[Mohammad Akhlaghi]

URL:
  <https://savannah.nongnu.org/task/?15497>

                 Summary: Adding a verification step
                 Project: Reproducible paper template
            Submitted by: makhlaghi
            Submitted on: Mon 23 Dec 2019 06:36:53 PM UTC
         Should Start On: Mon 23 Dec 2019 12:00:00 AM UTC
   Should be Finished on: Mon 23 Dec 2019 12:00:00 AM UTC
                Category: Analysis
                Priority: 5 - Normal
                  Status: Postponed
                 Privacy: Public
        Percent Complete: 0%
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
                  Effort: 0.00

    _______________________________________________________

Details:

Currently the only verification that the template provides is the published
PDF. Users will have to manually look into the published and generated PDFs
(numbers, plots, tables) and see if they obtained the same result.

Verification is an important step that we should also add after the analysis
is complete (possibly as a `verify.mk' file that is loaded before `paper.mk').
Users can use that Makefile to add rules for verifying their output datasets
(that go into the paper's plots and figures for example).

We'll just have to add some guiding comments on best practices in verifying,
for example if the file is timestamped (the date and time that it was created
is included in it, for example as commented lines in a table), its important
to remove the timestamp (maybe all commented lines) when verifying it with a
saved checksum.

This came up to me after reviewing a paper on The Popper Convention
<https://users.soe.ucsc.edu/~msevilla/papers/jimenez-ipdpsw17.pdf> for a paper
I am writing on this template. It introduces concepts very much like those
within this template, but I only recently learnt about it (early versions of
this template date back to the 2015 NoiseChisel paper
<https://arxiv.org/abs/1505.01664>). Its great to show that other people (and
they are computer scientists!) have independently come up with the same
principles/convention as ours (implemented in a very different way!).




    _______________________________________________________

Reply to this item at:

  <https://savannah.nongnu.org/task/?15497>

_______________________________________________
  Message sent via Savannah
  https://savannah.nongnu.org/