[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #58393] (buildbot) Tests are marked as "pass"
From: |
Markus Mützel |
Subject: |
[Octave-bug-tracker] [bug #58393] (buildbot) Tests are marked as "pass" if Octave crashes |
Date: |
Mon, 18 May 2020 11:02:19 -0400 (EDT) |
User-agent: |
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0 |
URL:
<https://savannah.gnu.org/bugs/?58393>
Summary: (buildbot) Tests are marked as "pass" if Octave
crashes
Project: GNU Octave
Submitted by: mmuetzel
Submitted on: Mon 18 May 2020 05:02:18 PM CEST
Category: Configuration and Build System
Severity: 3 - Normal
Priority: 5 - Normal
Item Group: Incorrect Result
Status: None
Assigned to: None
Originator Name:
Originator Email:
Open/Closed: Open
Release: other
Discussion Lock: Any
Operating System: GNU/Linux
_______________________________________________________
Details:
Like already reported on the mailing list a while ago:
https://octave.1599824.n4.nabble.com/buildbots-False-pass-results-for-segmentation-fault-in-test-td4695266.html
Quoting here:
> It looks like a segmentation fault in the last step of running the test
suite on the build bots is still marked as "success" if there were no failed
test up to that point. E.g.:
>
http://buildbot.octave.org:8010/#/builders/12/builds/1525/steps/6/logs/stdio
>
> Maybe any output to stderr in that step should result in it to be marked as
"failure"?
And a potential fix (but keep in mind that I don't know much about buildbot
configuration):
diff -r e02079ee69d1 master.cfg
--- a/master.cfg Fri Feb 08 15:58:06 2019 +0000
+++ b/master.cfg Mon Jan 13 19:44:13 2020 +0100
@@ -324,6 +324,7 @@
self.passed = 0
self.failed = 0
self.warnings = 0
+ self.errors = 0
self.summary_found = False
self.complete = False
@@ -352,6 +353,11 @@
self.rc = FAILURE
self.failed += num
+ def errLineRecived (self, line)
+ self.total += 1
+ self.errors += 1
+ self.rc = FAILURE
+
class octave_test (Test):
@@ -365,7 +371,7 @@
passed = self.observer.total - self.observer.failed
self.setTestResults (total = self.observer.total,
- failed = self.observer.failed,
+ failed = self.observer.failed +
self.observer.errors,
passed = self.observer.passed,
warnings = self.observer.warnings)
_______________________________________________________
Reply to this item at:
<https://savannah.gnu.org/bugs/?58393>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/
- [Octave-bug-tracker] [bug #58393] (buildbot) Tests are marked as "pass" if Octave crashes,
Markus Mützel <=