summaryrefslogtreecommitdiff
path: root/run_project_tests.py
AgeCommit message (Collapse)Author
2017-08-02Turned mesontest into on internal module.Jussi Pakkanen
2017-08-02Handle objc compiler missing more reliably.Jussi Pakkanen
2017-07-20Support implibs for executables on WindowsJon Turney
Add a boolean 'implib' kwarg to executable(). If true, it is permitted to use the returned build target object in link_with: On platforms where this makes sense (e.g. Windows), an implib is generated for the executable and used when linking. Otherwise, it has no effect. (Rather than checking if it is a StaticLibrary or SharedLibary, BuildTarget subclasses gain the is_linkable_target method to test if they can appear in link_with:) Also install any executable implib in a similar way to a shared library implib, i.e. placing the implib in the appropriate place Add tests of: - a shared_module containing a reference to a symbol which is known (at link time) to be provided by the executable - trying to link with non-implib executables (should fail) - installing the implib (This last one needs a little enhancement of the installed file checking as this is the first install test we have which needs to work with either MSVC-style or GCC-style implib filenames)
2017-07-18Close files reliably.Jussi Pakkanen
2017-06-23Split out languages from compilers.pyAlistair Thomas
2017-06-22Add some colour to test output.Elliott Sales de Andrade
Bold the section names and colourize errors&skips.
2017-06-21Merge pull request #1924 from mesonbuild/tingping/yelp-fixesJussi Pakkanen
Various yelp fixes
2017-06-17tests: Always consider symlink as valid installed filePatrick Griffis
An installed symlink might point to a non-existing file (because DESTDIR is set) so just always accept them.
2017-06-10tests: Add Boost unit tests and project tests on WindowsNirbheek Chauhan
Boost tests are disabled on Windows for now because the detection is actually completely broken. Once that's fixed (after the release) we can enable it again.
2017-06-10tests: Increase dependencies coverage a bit moreNirbheek Chauhan
2017-06-09unit tests: Add class to generate failing testsNirbheek Chauhan
It is not feasible to test all failure modes by creating projects in `test cases/failing` that would be an explosion of files, and that mechanism is too coarse anyway. We have no way to ensure that the expected error is being raised. See FailureTests.test_dependency for an example.
2017-05-21Create helper function for a rmtree that works reliably on Windows.Jussi Pakkanen
2017-05-17Remove unused variables.Elliott Sales de Andrade
2017-05-17Remove unnecessary __init__ methods.Elliott Sales de Andrade
2017-05-13Skip ObjectiveC/C++ tests if ObjC compiler is not availableTim-Philipp Müller
2017-05-09project tests: Also regen before buildingNirbheek Chauhan
This actually caught a cached-dependency related bug for me that the test-time regen did not. I also increased the ninja wait time to 1 second because that's actually how long you need to sleep to be guaranteed that a change will be detected. Must poke upstream about https://github.com/ninja-build/ninja/issues/371
2017-05-05Store and print test logs of executed projects.Jussi Pakkanen
2017-05-02Don't use len() to test emptiness vs not emptinessDylan Baker
Meson has a common pattern of using 'if len(foo) == 0:' or 'if len(foo) != 0:', however, this is a common anti-pattern in python. Instead tests for emptiness/non-emptiness should be done with a simple 'if foo:' or 'if not foo:' Consider the following: >>> import timeit >>> timeit.timeit('if len([]) == 0: pass') 0.10730923599840025 >>> timeit.timeit('if not []: pass') 0.030033907998586074 >>> timeit.timeit('if len(['a', 'b', 'c', 'd']) == 0: pass') 0.1154778649979562 >>> timeit.timeit("if not ['a', 'b', 'c', 'd']: pass") 0.08259823200205574 >>> timeit.timeit('if len("") == 0: pass') 0.089759664999292 >>> timeit.timeit('if not "": pass') 0.02340641999762738 >>> timeit.timeit('if len("foo") == 0: pass') 0.08848102600313723 >>> timeit.timeit('if not "foo": pass') 0.04032287199879647 And for the one additional case of 'if len(foo.strip()) == 0', which can be replaced with 'if not foo.isspace()' >>> timeit.timeit('if len(" ".strip()) == 0: pass') 0.15294511600222904 >>> timeit.timeit('if " ".isspace(): pass') 0.09413968399894657 >>> timeit.timeit('if len(" abc".strip()) == 0: pass') 0.2023209120015963 >>> timeit.timeit('if " abc".isspace(): pass') 0.09571301700270851 In other words, it's always a win to not use len(), when you don't actually want to check the length.
2017-04-29Warn when directory numbers are non-unique.Elliott Sales de Andrade
2017-04-29Shorten detect_tests_to_run.Elliott Sales de Andrade
Remove some duplication with a list comprehension, and invert conditions instead of long ternary statements.
2017-04-09tests: Move appveyor platform workaround to run_tests.pyNirbheek Chauhan
This is also needed for the unit tests
2017-04-09unit tests: Fix running specific targets with MSBuildNirbheek Chauhan
/t:targetname syntax doesn't work, but running the vcxproj does work Also use the Backend enum everywhere.
2017-04-09run_project_tests: Clean on backends that can't installNirbheek Chauhan
Also sets more groundwork for running unit tests with backends other that Ninja. Transferring global state to executors is totally broken in Python 3.4 so just serialize all the commands.
2017-04-09project tests: Add a new BuildStep called validateNirbheek Chauhan
2017-04-09run_project_tests: Clean-up backend detection a bitNirbheek Chauhan
Set a global variable that has it so other places can use it, and error out on unknown backends
2017-04-09tests: Factor out common code to run_testsNirbheek Chauhan
And use generic build/clean/test/install commands in the unit tests, just like project tests. This sets the groundwork for running the unit tests with all backends.
2017-04-06Ignore missing .pdb files when not using MSVSJon Turney
This fixes 'test cases/windows/1 basic' on Cygwin
2017-04-06Don't run framework tests on CygwinJon Turney
2017-04-06Run test cases/windows/ tests on CygwinJon Turney
Also, always run 'test cases/windows/5 resources' on Cygwin since it can handle args containing spaces correctly.
2017-04-06Use '.exe' extension for executables for CygwinJon Turney
Use '.exe' extension for executables for Cygwin when building and installing
2017-04-06Use correct shared library naming for CygwinJon Turney
Use correct shared library naming for Cygwin when building and installing
2017-03-28project tests: Don't look for PDB files on MinGW/GCCNirbheek Chauhan
2017-03-28project tests: DummyExecutor for MSYS2 and OpenBSDNirbheek Chauhan
Added and tested on MSYS2/MinGW which doesn't implement the required semaphore locks in the multiprocessing module: Traceback (most recent call last): File "C:/msys64/mingw64/lib/python3.5\multiprocessing\synchronize.py", line 29, in <module> from _multiprocessing import SemLock, sem_unlink ImportError: cannot import name 'sem_unlink' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "run_project_tests.py", line 560, in <module> (passing_tests, failing_tests, skipped_tests) = run_tests(all_tests, 'meson-test-run', options.extra_args) File "run_project_tests.py", line 406, in run_tests executor = conc.ProcessPoolExecutor(max_workers=num_workers) File "F:/msys64/mingw64/lib/python3.5\concurrent\futures\process.py", line 390, in __init__ EXTRA_QUEUED_CALLS) File "F:/msys64/mingw64/lib/python3.5\multiprocessing\context.py", line 101, in Queue return Queue(maxsize, ctx=self.get_context()) File "F:/msys64/mingw64/lib/python3.5\multiprocessing\queues.py", line 42, in __init__ self._rlock = ctx.Lock() File "F:/msys64/mingw64/lib/python3.5\multiprocessing\context.py", line 65, in Lock from .synchronize import Lock File "F:/msys64/mingw64/lib/python3.5\multiprocessing\synchronize.py", line 34, in <module> " function, see issue 3770.") ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770. See also: https://bugs.python.org/issue3770 https://github.com/mesonbuild/meson/issues/1323 According to 3770, the same problem also exists on OpenBSD, so this will potentially also be useful there.
2017-03-14project tests: Print meson log instead of stdout for configure failuresNirbheek Chauhan
2017-03-03tests/windows: Tests for #1444Nirbheek Chauhan
Check that pdb files are installed
2017-01-28vs: Fix running of tests to use mesontest.pyNirbheek Chauhan
Back in November when this broke, we didn't notice because our tests are run in-process, so we don't check that `msbuild RUN_TESTS.vcxproj` and `ninja test` actually work. Now we do.
2017-01-23No longer require a binary called python3 on Windows. The default name is ↵Jussi Pakkanen
"python".
2017-01-18cleanup: Remove redundant parenthesesMike Sinkovsky
2017-01-11style: [E251] unexpected spaces around keyword / parameter equalsMike Sinkovsky
2017-01-11style: [E301] expected 1 blank line, found 0Mike Sinkovsky
2017-01-09We only need Ninja 1.5.x for the core codeNirbheek Chauhan
This change helps us run on older distros such as Ubuntu LTS which is very lazy in updating even non-core and stable packages such as Ninja. Ninja 1.6.x is only needed for running the tests.
2017-01-04Project tests: Also catch ValueErrorNirbheek Chauhan
It seems on Windows, deleting in a loop can cause a race where the following error is raised: Traceback (most recent call last): File "run_project_tests.py", line 550, in <module> (passing_tests, failing_tests, skipped_tests) = run_tests(all_tests, 'meson-test-run', options.extra_args) File "run_project_tests.py", line 416, in run_tests result = result.result() File "C:\python34-x64\lib\concurrent\futures\_base.py", line 402, in result return self.__get_result() File "C:\python34-x64\lib\concurrent\futures\_base.py", line 354, in __get_result raise self._exception ValueError: I/O operation on closed file. https://ci.appveyor.com/project/jpakkane/meson/build/1.0.1559/job/vsek754eu000kg3e
2017-01-02project tests: Always print the logs of failing testsNirbheek Chauhan
There is never any reason to not do this since this script is supposed to be run by developers and testers who are concerned with the details of the problems. It also helps with intermittent or hard-to-reproduce errors.
2017-01-02Catch permissionerrors too, since Windows seems to produce them occasionally.Jussi Pakkanen
2017-01-01style: fix E128 violationsIgor Gnatenko
E128: continuation line under-indented for visual indent Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
2017-01-01style: fix E265 violationsIgor Gnatenko
E265: block comment should start with '# ' Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
2017-01-01style: fix E226 violationsIgor Gnatenko
E226: missing whitespace around arithmetic operator Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
2016-12-31Fix space before :.Jussi Pakkanen
2016-12-23Force a Ninja regeneration before each project testNirbheek Chauhan
We have no test coverage for regeneration at all, which is why issues like #1246 slide by without us noticing. With this, we will run a regen on every test during `ninja test` after it has been compiled. This will not affect test times too much since the regen will not rebuild anything at all since there have been no source changes.
2016-12-21Merge pull request #1233 from mesonbuild/wip/ignatenko/code-styleJussi Pakkanen
Trivial cleanups in code