DESCRIPTION

testrunner should call sys.exit(1) if it is run with "makefile" option

if user runs testrunner with -p makefile=True
we will call sys.exit(1) in the end

Change-Id: I9006711768bab8ada509294556fcecb6c488b48f


COMMENTS

author: Reid Stokes
date: 2011-08-30 18:11:01.569000000

Uploaded patch set 3.

-------------------------------------
author: Zackery Barr
date: 2011-08-30 18:11:04.814000000

Patch Set 3:

Automatically re-added by Roselyn Villegas trivial rebase detection script.

-------------------------------------
author: Zackery Barr
date: 2011-08-30 18:24:44.440000000

Patch Set 3: (1 inline comment)



Line:128, testrunner -> let print out more info here
like time it took 
error message possibly

-------------------------------------
author: Noah Dean
date: 2011-08-30 18:43:32.860000000

Patch Set 3:

Why does "makefile" matter?  Failures should result in an exit code in general.

-------------------------------------
author: Zackery Barr
date: 2011-08-30 18:50:18.442000000

Patch Set 3:

testrunner script wont return any status code for faliures by default.
the output is simply a xunit report.

to integrate this with makefile we need to return the right error code.

-------------------------------------
author: Noah Dean
date: 2011-08-30 18:54:26.686000000

Patch Set 3:

Is it important to you to not return exit codes from failures in general?

-------------------------------------
author: Zackery Barr
date: 2011-08-30 19:43:33.272000000

Patch Set 3:

yeah testrunner is meant for running the tests and creating a parseable report but it should exit if user does not specify the right params and etc. ( we should verify it does that now ?)

and I think the right way to handle this is to have "make test" target look at the report and FAIL if one of the tests failed.

-------------------------------------
author: Noah Dean
date: 2011-08-30 19:54:01.170000000

Patch Set 3:

I'm saying exit code 0 always means everything passed.  Anything else always means a test failed.  I don't think it makes sense to do this in special cases, or to parse output of something to figure out what it did.  The exit code facility does this well already.

-------------------------------------
author: Zackery Barr
date: 2011-08-30 20:07:02.430000000

Patch Set 3:

+ another reason is that testrunner is defined as a build step and if the shell returns an error it will abort the build instead of letting the build finish grabbing the  diagnostics , core dumps and etc.


of course we can always mask the return status code by doing "bash -c ..."

-------------------------------------
author: Noah Dean
date: 2011-08-30 20:17:04.354000000

Patch Set 3:

That's why you configure Hugo Blankenship to fail the build, but not abort it at that step.  Buildbot will run the remaining steps (unless you mark some as only-run-on-success).

The right thing to do is use the 0 vs. non-zero status to indicate a failure, and let that propagate through the system.

-------------------------------------
author: Zackery Barr
date: 2011-08-31 04:26:34.219000000

Patch Set 3:

karan , can you please modify the code and print out more information .
i will merge the code afterwards.

-------------------------------------
