path: root/piglit-run.py
AgeCommit message (Collapse)Author
2013-03-10Move re.compile for regex into CoreDylan Baker
Move the re.compile into the core.Enivironment constructor, which reduces code duplication. It also allows us to pass environment data on initilization of the object, rather that having edit it's attributes individually. V2: - Does not remove deprecated options, only marks them as such V3: - Fixes deperecated warning for tests from V2 always being triggered Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2013-03-10Replaces getopt with argparseDylan Baker
This change gets us cleaner, more readable argument parsing code. Another advantage of this approach is that it automatically generates help menus, which means options will not be implemented without a help entry (like --resume) V2: - Does not remove deprecated options, only marks them as deprecated V3: - Fixes deprecated warning for tests from V2 always being triggered Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2013-03-10piglit-run.py: Add --resume to the help menuDylan Baker
This option was implemented, however, it was not added to the help menu, meaning that a user needed to know that it existed before they could even attempt to use it. Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2013-03-10piglit-run.py: Replace Piglit's --concurent flag with --no-concurrencyDylan Baker
Since piglit defaults to using concurrency, and since concurrency is generally the desired behavior, it seems more appropriate to have a flag that turns off concurrency than one that takes an argument to set it on or off. V2: - Leaves --concurrency option, but marks it as deprecated Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2013-03-10Change --tests to --include-testsDylan Baker
Excluding tests already uses a similar syntax (--exclude-tests), so this is more consistent. V2: - Leaves --tests option, but marks it as deprecated Signed-off-by: Dylan Baker <baker.dylan.c@gmail.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2013-02-08piglit-run: Fix resume path resolvingKenney Phillis
Fix resolution of relative paths to absolute paths. This will help keep piglit from crashing out when resume path is not within source path. This happens regularly when using distributed testing. Reviewed-and-tested-by: Chad Versace <chad.versace@linux.intel.com>
2013-01-03glean: Remove support for writing results databases.Kenneth Graunke
At this point, Glean wasn't writing anything interesting anyway; it was just clutter. Since there's no need to specify a results directory on the command line any longer, this patch also removes the -r option, making "run tests" the default action. This also allows us to simplify the Piglit runner framework a little: it no longer has to pass around the results directory just to pass to Glean.
2012-10-19piglit-run.py: add -p|--platform switch to select the piglit platformJordan Justen
This can be useful with gbm when running piglit-run.py under sudo. Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chad Versace <chad.versace@linux.intel.com>
2012-07-16Changed piglit-run.py to automatically correct previous test results on a ↵Kenney Phillis
platform where the os seperator is not '\'. Reviewed-by: Jose Fonseca <jfonseca@vmware.com>
2012-03-12Make valgrind testing a command line option rather than extra tests.Kenneth Graunke
Valgrind testing is useful, but really should be done as a separate exercise from the usual regression testing, as it takes way too long. Rather than including it by default in all.tests and making people exclude it with the -x valgrind option (or by using quick.tests), it makes sense to explicitly request valgrind testing with --valgrind. To perform valgrind testing: $ piglit-run.py --valgrind <options> tests/quick.tests results/vg-1 The ExecTest class now handles Valgrind wrapping natively, rather than relying on the tests/valgrind-test/valgrind-test shell script wrapper. This provides a huge benefit: we can leverage the interpretResult() function to make it work properly for any subclass of ExecTest. The old shell script only worked for PlainExecTest (piglit) and GleanTest. Another small benefit is that you can now use --valgrind with any test profile (such as quick.tests). Also, you can use all.tests without having to remember to specify "-x valgrind". Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2012-03-02Add the ability to resume an interrupted test run where it left off.Kenneth Graunke
GPUs like to hang, especially when barraged with lots of mean Piglit tests. Usually this results in the poor developer having to figure out what test hung, blacklist it via -x, and start the whole test run over. This can waste a huge amount of time, especially when many tests hang. This patch adds the ability to resume a Piglit run where you left off. The workflow is: $ piglit-run.py -t foo tests/quick.tests results/foobar-1 <interrupt the test run somehow> $ piglit-run.py -r -x bad-test results/foobar-1 To accomplish this, piglit-run.py now stores the test profile (quick.tests) and -t/-x options in the JSON results file so it can tell what you were originally running. When run with the --resume option, it re-reads the results file to obtain this information (repairing broken JSON if necessary), rewrites the existing results, and runs any remaining tests. WARNING: Results files produced after this commit are incompatible with older piglit-summary-html.py (due to the extra "option" section.) Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-03-02Move writing of JSON 'tests' dictionary section out of TestProfile.run.Kenneth Graunke
When resuming an interrupted piglit run, we'll want to output both existing and new results into the same 'tests' section. Since TestProfile.run only handles newly run tests, it can't open/close the JSON dictionary. So, move it to the caller in piglit-run.py. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-02-24piglit-run.py: Use list.append(item) instead of list[:0] = [item].Kenneth Graunke
list[:0] = [item] is a strange way to add an item to a list. 1. It forces the list to be copied, which can be inefficient. 2. It produces a strange ordering: >>> x = [1, 2, 3] >>> x[:0] = [4] >>> x [4, 1, 2, 3] ...whereas list.append(item) produces [1, 2, 3, 4]. 3. Most importantly, list.append is easier to understand at a glance. Reported-by: Paul Berry <stereotype441@gmail.com> Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2012-01-17piglit-run: support storing results outside of piglit_dirScott Tsai
piglit-run.py takes the directory to store the results in as a command line argument, yet always chdir's to piglit_dir first before accessing it. When running piglit from outside of piglit_dir, this is surprising to the user and will fail when piglit is packaged in a distribution and the user can't write to piglit_dir. This came up while packaging piglit for Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=773021 where all piglit code is placed under /usr/lib64/piglit/ and symbolic links to the piglit-* scripts are created in /usr/bin/. Reviewed-by: Chad Versace <chad.versace@linux.intel.com> Signed-off-by: Scott Tsai <scottt.tw@gmail.com>
2011-10-13Make Python scripts independent of the current working directoryMatěj Cepl
The main purpose of this patch is to make piglit independent of the current working directory, so it is possible to package piglit as a RPM package (with binaries symlinked to /usr/bin, most of the files in read-only /usr/lib*/piglit directory, and results somewhere else). So it is now possible to run $ piglit-run tests/quick-driver.tests /tmp/piglit and then with this command $ piglit-summary-html --overwrite /tmp/piglit/results /tmp/piglit/main generate a report in /tmp/piglit/results/index.html & al. Signed-off-by: Matěj Cepl <mcepl@redhat.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2011-08-05piglit-run.py: Remove SyncFileWriter as it's not necessary with JSON.Kenneth Graunke
Prior to JSON-ification, piglit wrote each test's results in a single call, and SyncFileWriter ensured that these were done sequentially. Now, the JSON writer internally handles locking and concurrency, so SyncFileWriter is unnecessary. Furthermore, outputting a single test's results now takes multiple write calls, so SyncFileWriter wouldn't actually guard against concurrency issues anyway. This also removes a fsync() call on each write, fixing a major performance regression on machines with non-SSDs. Prior to the JSON work, since each test mapped to a single write call, we were doing one fsync() per test case. With JSON, we started doing many more fsyncs. But none of them are actually necessary, so just scrap them all. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=39737 Cc: Chad Versace <chad@chad-versace.us> Cc: Ian Romanick <idr@freedesktop.org> Cc: Dave Airlie <airlied@gmail.com> Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chad Versace <chad@chad-versace.us> Tested-by: Ian Romanick <ian.d.romanick@intel.com>
2011-07-29piglit-run: Add option to enable/disable concurrent test runsChad Versace
-c bool, --concurrent=bool Enable/disable concurrent test runs. Valid option values are: 0, 1, on, off. (default: on) CC: Ben Widawsky <ben@bwidawsk.net> Signed-off-by: Chad Versace <chad@chad-versace.us> Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2011-07-29framework: Write each test result to the result file as the test completesChad Versace
When a test run is interrupted, perhaps by a system crash, we often want the test results. To accomplish this, Piglit must write each test result to the result file as the test completes. If the test run is interrupted, the result file will be corrupt. This is corrected in a subsequent commit. CC: Ian Romanick <ian.d.romanick@intel.com> Signed-off-by: Chad Versace <chad@chad-versace.us>
2011-07-14framework: Replace custom serialization format with jsonChad Versace
The results file produced by piglit-run.py contains a serialized TestrunResult, and the serialization format was horridly homebrew. This commit replaces that insanity with json. Benefits: - Net loss of 113 lines of code (ignoring comments and empty lines). - By using the json module in the Python standard library, serializing and unserializing is nearly as simple as `json.dump(object, file)` and `json.load(object, file)`. - By using a format that is easy to manipulate, it is now simple to extend Piglit to allow users to embed custom data into the results file. As a side effect, the summary file is no longer needed, so it is no longer produced. Reviewed-by: Paul Berry <stereotype441@gmail.com> Signed-off-by: Chad Versace <chad@chad-versace.us>
2011-02-10Add SyncFileWriter for write/close synchronization on results file.U. Artie Eoff
Add SyncFileWriter class to synchronize writes to the 'main' results file from multiple threads. This helps to ensure that writes to this file are not intermingled.
2010-07-29Add -x|--exclude-tests= option to specify tests not to run.Carl Worth
This is a blacklist to complement the existing -t|--tests whitelist. It works similarly, (accepts a regular expression and can be specified multiple times).
2009-08-23Python: Add spaces after commas.Vinson Lee
2009-08-21Python: Remove unused imports.Vinson Lee
2008-06-12Generate abbreviated resultsNicolai Haehnle
The results files can get rather huge when tests fail, because tests like glean/blendFunc output a lot of debugging data, which we all record. Now, we generate an additional .../summary file, in which the info string is simply truncated. Pretty stupid, but it should give enough info to get an idea of the rough kind of failure. Add a new option for piglit-summary-html.py, to choose between full or abbreviated info when both are present.
2008-06-06Collect output of glxinfo and lspciNicolai Haehnle
2008-02-21Use /usr/bin/env python instead of /usr/bin/python for portability.Eric Anholt
2007-04-03Nonsensical reference in licenseNicolai Haehnle
2007-03-25Store results in a directory instead of a single file.Nicolai Haehnle
The idea is to allow accompanying data (like screenshots) in the future.
2007-03-25RefactoringNicolai Haehnle
2007-03-25piglit: Fix calls to exit()Jeff Muizelaar
Fix calls to exit() to use sys.exit(). 'exit' is a string. We want to use sys.exit instead. Signed-off-by: Nicolai Haehnle <nhaehnle@gmail.com>
2007-03-24Initial commitNicolai Hähnle