summaryrefslogtreecommitdiff
path: root/run_project_tests.py
AgeCommit message (Collapse)Author
2021-06-09tests: Force colorize CI outputDaniel Mensinger
2021-06-09tests: Don't even start running skipped testsDaniel Mensinger
2021-06-09tests: Fix global state issueDaniel Mensinger
2021-06-09tests: Add option to limit the number of workersDaniel Mensinger
Also, increase the minimum ammount of workers from 1 to 2 when the core count can not be determined.
2021-06-09tests: Optional progress bar with tqdmDaniel Mensinger
2021-06-09tests: Refactored test runnerDaniel Mensinger
2021-06-09tests: Reduce complexity of the run_tests signatureDaniel Mensinger
2021-06-09tests: reduce global variablesDaniel Mensinger
2021-06-09typing: Fully annotate run_project_tests.pyDaniel Mensinger
2021-06-07cython: add 1 basic testDylan Baker
2021-05-13Set up VS environment automatically when run.Jussi Pakkanen
2021-05-02Add validation for --only in `run_project_tests.py`Ralf Gommers
This now gives a clear error rather than silently passes for unrecognized categories, like: python run_project_tests.py --only nonexisting or python run_project_tests.py --only objc # should be 'objective c'
2021-03-24Merge pull request #8568 from dcbaker/submit/qt-dependency-factoryJussi Pakkanen
QT: use a proper dependency factory
2021-03-23run_project_tests: add annotationsDylan Baker
2021-03-23run_project_tests.py: Allow qt tests to skip on !macOSDylan Baker
We don't always have qmake installed (and it's good to test failure paths too!) so we can't expect this to succeed in all cases. With the following commit we'll use a test.json to test both pkg-config and qmake, so we need to be able to skip.
2021-03-23test.json: Add support for not matching stdout linesXavier Claessens
By default expected line must be matched in order. When an expected line is matched it does not matter if it's matched again later or not. When defining "count", it means that line must be matched exactly that many times before matching the next expected line. Once all occurences have been matched for an expected line, it not must appear any more in all next lines.
2021-03-23run_project_tests.py: Allow "--only common/240" syntaxXavier Claessens
2021-03-16Update VS module version check.Jussi Pakkanen
2021-03-16Delete leftover files from some tests.Jussi Pakkanen
2021-03-04mass rewrite of string formatting to use f-strings everywhereEli Schwartz
performed by running "pyupgrade --py36-plus" and committing the results
2021-03-04various python neatness cleanupsEli Schwartz
All changes were created by running "pyupgrade --py3-only --keep-percent-format" and committing the results. I have not touched string formatting for now. - use set literals - simplify .format() parameter naming - remove __future__ - remove default "r" mode for open() - use OSError rather than compatibility aliases - remove stray parentheses in function(generator) scopes
2021-02-23run_*_tests: Add some type annotationsDylan Baker
2021-02-23run_project_tests: Delete a build directory if it already existsDylan Baker
This can happen if you abort a run before the cleanup finishes
2021-02-23run_project_tests: split json parsing code from test gathering codeDylan Baker
2021-02-06ci: Add bindgen to CI imagesDylan Baker
2021-01-21mlog: add __str__ method to AnsiDecoratorPaolo Bonzini
Automatically colorize the text when printing the AnsiDecorator, based on the result of mlog.colorize_console(). This is how AnsiDecorator is used most of the time anyway.
2021-01-10cmake: add PATH logic to preliminary dep check (fixes #8133)Daniel Mensinger
2020-11-23Remove AutoDeletedDirChristoph Reiter
Replace the only usage with a simple try/finally and remove the class.
2020-11-23Replace various calls to AutoDeletedDir with TemporaryDirectoryWinProofChristoph Reiter
AutoDeletedDir was a workaround to TemporaryDirectory not using windows_proof_rmtree(). TemporaryDirectoryWinProof does now, so replace it.
2020-11-22Merge pull request #8011 from dcbaker/submit/post-python36-cleanupsJussi Pakkanen
Python 3.6 cleanups
2020-11-21run_project_tests: Use the test environment for install and cleanJason Ekstrand
2020-11-20use real pathlib moduleDylan Baker
We added the _pathlib module to work around defeciencies in python 3.5's implementation, since we now rely on 3.6 lets drop this
2020-11-13gnome: Handle libraries that are not in the current build dirSam Thursfield
The generate_gir() function previously assumed all library inputs were in the current build dir. This would fail if they weren't.
2020-10-30Bump minimum supported Python version to 3.6. Closes #6297.Jussi Pakkanen
2020-10-22depenencies/llvm: Handle llvm-config --shared-mode failing (#7379)Dylan Baker
* depenencies/llvm: Handle llvm-config --shared-mode failing Fixes: #7371 Fixes: #7878 * test cases/llvm: Refactor to use test.json Instead of trying to cover everything internally
2020-10-13cmake: Add cross testsDaniel Mensinger
2020-10-04pathlib: Fix resolve() by overriding it in Python 3.5Daniel Mensinger
2020-09-17project tests: Also check whether introspect worksNirbheek Chauhan
2020-09-17project tests: Rename a variable for clarityNirbheek Chauhan
2020-09-11Add build machine compilers to project tests compiler reportJon Turney
When cross-compiling, also report build machine compilers in the compiler report produced by run_project_tests.py
2020-09-10Split tests out from 'common' which require a native compilerJon Turney
Split out tests (and parts of tests) which require a native compiler from the 'common' suite to a new suite called 'native', so we can selectively avoid running those tests when only a cross-compiler is available. Also move test '211 cmake module' to 'cmake' suite, since it appears that the way we use cmake requires a native compiler.
2020-09-08typing: fix code reviewDaniel Mensinger
2020-08-27test: Add 'dir' support for installed files in test.jsonDaniel Mensinger
This is useful for automatically generated docs (doxygen, hotdoc) with a lot of generated files that may differ with different versions of the generator.
2020-07-23Merge pull request #7460 from mensinda/fixDeepcpyJussi Pakkanen
deps: Do not deepcopy internal libraries (fixes #7457)
2020-07-21Add boost_root support to properties files (#7210)cmcneish
* Add boost_root support to properties files This commit implements `boost_root`, `boost_includedir`, and `boost_librarydir` variable support to native and cross properties files. The search order is currently environment variables, then these variables, and finally a platform-dependent search. * Add preliminary boost_root / boost_includedir tests Each test contains a fake "version.hpp", as that's how boost detection is currently being done. We look for this file relative to the root directory, which probably shouldn't be allowed (it previously was for BOOST_LIBRARYDIR but not for BOOST_ROOT). It also cannot help with breakage detection in libraries, however it looks like this wasn't getting tested beforehand. I've given the two unique version numbers that shouldn't be present in any stock version of boost (001 and 002). * Add return type to detect_split_root * Return empty list when nothing found in BOOST_ROOT, rather than None * Update boost_root tests * Create nativefile.ini based on location of run_project_tests.py * Add fake libraries to ensure boost_librarydir is being used * Require all search paths for boost to be absolute * Redo boost search ordering To better match things like pkg-config, we now look through native/cross files, then environment variables, then system locations for boost installations. Path detection does not fall back from one method to the next for properties or environment variables--if boost_root, boost_librarydir, or boost_includedir is specified, they must be sufficient to find boost. Likewise for BOOST_ROOT and friends. pkg-config detection is still optional falling back to system-wide detection, for Conan. (Also, fix a typo in test 33's nativefile) * Correct return type for detect_roots * Correct boost dependency search order in documentation * Print debug information for boost library finding, to resolve CI issues * Handle native/cross file templates in a more consistent way All tests can now create a `nativefile.ini.in` if they need to use some parameter that the testing framework knows about but they can't. * Pass str--rather than PosixPath--to os.path.exists, for Python35 * Look for boost minor versions, rather than boost patch versions in test cases * Drop fake dylib versions of boost_regex * Prefer get_env_var to use of os.environ * Correct error reporting for relative BOOST_ROOT paths * Bump version this appears in. Also, change "properties file" to "machine file" as that appears to be the more common language.
2020-07-16Make virtualenv inside the project workDaniel Mensinger
2020-07-12remove redundant syntaxMichael Hirsch
2020-07-12some python test cases don't care about backend, so run them in any caseMichael Hirsch
2020-06-15run_*tests*.py: print Meson version at start of test suiteMichael Hirsch
on some systems, tests may take over an hour to run--only to find you might have used an unintended Meson version (e.g. release instead of dev). This change prints the Meson version at the start of the run_*tests*.py scripts. Also, raise SystemExit(main()) is preferred in general over sys.exit(main())
2020-06-15ci: Use test.json to skip VS2017 azure testsDaniel Mensinger
See also #7307 #7314 #7316 cc @nirbheek