summaryrefslogtreecommitdiff
path: root/mesonbuild/scripts
AgeCommit message (Collapse)Author
2024-09-24clang-tidy: Avoid spawning too many threadsJonathon Anderson
The clang-tidy-fix target uses run-clang-tidy to do the fixing, however this script itself spawns `os.cpu_count()` threads as part of its internal parallelism. When combined with Meson's parallelism this results in the creation of potentially thousands of unecessary threads. This commit rewrites the clang-tidy-fix to perform the same task run-clang-tidy does but exclusively on Meson's thread pool. "Fix-it" snippets are saved to `meson-private/clang-tidy-fix/` by a parallel clang-tidy phase, afterwards (to avoid races) all collected fixes are applied with a single call to clang-apply-replacements.
2024-08-25Better handle CTRL-C during clang-tidy/formatMark A. Tsuchida
It was possible (with some frequency) for the clang-tidy/format target to continue starting new subprocesses after a CTRL-C, because we were not canceling the already queued tasks and waiting for all of them. This makes a best-effort attempt to cancel all further subprocesses. It is not 100%, because there is a race that is hard to avoid (without major restructuring, at least): new subprocesses may be started after KeyboardInterrupt (or any other exception) is raised but before we get to the cancellation. When the race happens, the calling ninja may exit before Meson exits, causing some output (from clang-tidy/format and Meson's traceback) to be printed after returning to the shell prompt. But this is an improvement over potentially launching all the remaining tasks after having returned to the shell, which is what used to happen rather often. In practice, it appears that we cleanly exit with a pretty high probability unless CTRL-C is hit very early after starting (presumably before the thread pool has launched subprocesses on all its threads).
2024-07-23depfixer: Add missing annotation that breaks mypy checkDylan Baker
2024-07-23depfixer: deduplicate rpaths on darwinMonson Shao
Duplicated -delete_rpath arguments will cause macOS's install_name_tool failed.
2024-07-11Move OptionKey in the option source file.Jussi Pakkanen
2024-07-10Replace exe_exists function with shutil.which()Mads Andreasen
The documentation for subprocess.run at https://docs.python.org/3/library/subprocess.html#popen-constructor has a warning, pointing to using shutil.which() instead of subprocess.run for detecting if exe files exists on the path. shutil.which() is used in many places already.
2024-06-26depfixer: fix darwin regression when install rpaths are usedEli Schwartz
Fixes regression in commit 78e9009ff9d36925e04f329f9082841002ddd848. new_rpath is only set when install_rpath appears in meson.build. Before this commit, we treated new_rpath as a single string to pass to -add_rpath. This worked as long as new_rpath had a single rpath in it, though for ELF we explicitly supported multiple rpaths separated with ":" (as binutils ld is quite happy with that too). install_name_tool does not support paths with colons in it: ``` 21:12 <awilfox> Load command 19 cmd LC_RPATH cmdsize 40 path /bar:/baz:/eli:/ldap (offset 12) 21:12 <awilfox> Load command 20 cmd LC_RPATH cmdsize 24 path /foo (offset 12) 21:14 <awilfox> so the result is: do not use colons ``` After commit 78e9009ff9d36925e04f329f9082841002ddd848, we simply ended up with one load command for LC_RPATH per char in new_rpath, which was wrong in every possible case. What we actually need to do is pass every distinct rpath as a separate `-add_rpath` argument, so we split it on the colons instead. We do the same splitting to ensure proper diff'ability for ELF anyways... Fixes #13355
2024-06-23Add support for detecting free-threaded Python on WindowsRalf Gommers
This does a couple of things: 1. Scrape the `Py_GIL_DISABLED` sysconfig var, which is the best way as of today to determine whether the target interpreter was built with `--disable-gil` 2. link against the correct libpython 3. On Windows, work around a known issue in the python.org installer with a missing define in `pyconfig.h`, which the CPython devs have said is unlikely to be fixed since headers are shared between the regular and free-threaded builds in a single NSIS installer.
2024-06-23clang-tidy: use -quietMark A. Tsuchida
This adds the `-quiet` option when invoking clang-tidy for the generated `clang-tidy` target. (Note that the `clang-tidy-fix` target already does so.) This prevents messages like ``` Suppressed 1084 warnings (1084 in non-user code). Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well. ``` from being repeated for every file, which drowns out the actual warnings/errors from clang-tidy when more than a few files are processed. Also the tip about `-header-fileter` and `-system-headers` is not very useful here because Meson doesn't currently provide a way to supply these options to clang-tidy. Even with `-quiet`, clang-tidy still prints a line like `1084 warnings generated.` for each file.
2024-06-23coverage: Change --html-details to --html-nested (gcovr)Walkusz
--html-nested option is used to create a separate web page for each file and directory. Each of these web pages includes the contents of file with annotations that summarize code coverage. Signed-off-by: Ewelina Walkusz <ewelinax.walkusz@intel.com>
2024-06-18Improve `nm` usage in symbolextractor script on macOSRalf Gommers
This fixes the unit test `TestAllPlatformTests.test_noop_changes_cause_no_rebuilds`, when run with an `nm` binary from `cctools-port` (as shipped by conda-forge, see https://github.com/conda-forge/cctools-and-ld64-feedstock). It also addresses the issue discussed in https://github.com/mesonbuild/meson/discussions/11131, and this build warning: ``` [48/1383] Generating symbol file scipy/special/libsf_error_state.dylib.p/libsf_error_state.dylib.symbols WARNING: ['arm64-apple-darwin20.0.0-nm'] does not work. Relinking will always happen on source changes. error: arm64-apple-darwin20.0.0-nm: invalid argument -- ``` as reported in scipy#20740. The unit test traceback was: ``` > self.assertBuildRelinkedOnlyTarget('mylib') E AssertionError: Lists differ: ['mylib', 'prog'] != ['mylib'] E E First list contains 1 additional elements. E First extra element 1: E 'prog' E E - ['mylib', 'prog'] E + ['mylib'] unittests/allplatformstests.py:1292: AssertionError ``` The `nm` shipped by Apple yields the exact same results either way; the man page for `nm` only lists the single-character form so this seems preferred either way.
2024-06-04Extract native file parser to machinefile source file.Jussi Pakkanen
2024-04-04env2mfile: Only consume LDFLAGS for relevant languagesOle André Vadla Ravnås
It makes no sense to add them into vala_link_args, for example.
2024-04-01depfixer: change fix_darwin to act like fix_rpathtype_entryFabian Groffen
This somewhat aligns "darwin" (Mach-O) with how ELF RPATHs are treated. Instead of blindly removing all RPATHs, only remove the ones that are in the rpath_dirs_to_remove set. This way, RPATHs that were added by the toolchain or user are left untouched. It is important not to remove RPATHs as they may be vital for the executable at runtime. Issues #12045 and #12288 are examples of this. Issue: https://github.com/mesonbuild/meson/issues/12045 Signed-off-by: Fabian Groffen <grobian@gentoo.org> Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
2024-03-29scripts/depscan: remove unnecessary functionDylan Baker
This basically existed for an assert which we don't need, as mypy would catch that issue for us anyway. Removing the function entirely has some small performance advantages
2024-03-29scripts/depscan: pick language once, at configure timeDylan Baker
We already have to decide whether to scan a file at configure time, so we don't want to have to do it again at compile time, every time the depscan rule is run. We can do this by saving and passing the language to use in the pickle, so depscan doesn't have to re-calculate it. As an added bonus, this removes an import from depscan
2024-03-29scripts/depscan: combine pickle and JSON data into a single fileDylan Baker
We don't need to write and pass two separate files to the depscanner, I've used the pickle because the pickle serializer/deserializer should be faster than JSON, thought I haven't tested.
2024-03-28depscan: use a defaultdict to simplify append actionDylan Baker
2024-02-24scripts/coverage: Implemented handling for passing the gcovr and llvm-cov ↵dragonmux
binaries to use to the script
2024-02-23env2mfile: move language args to `built-in options`Ole André Vadla Ravnås
As putting them in the `properties` section is considered deprecated, and Meson would then complain when trying to consume the resulting machine file.
2024-02-16coverage: pass .lcovrc config file to genhtmlEsther Dalhuisen
The .lcovrc file can contain several directives for genhtml, mostly pertaining the visualisation of the generated HTML report. Passing the config file to genhtml allows the user to customize their report. Fixes #12863
2024-01-28python module: correct the embedded path for bytecompiled filesEli Schwartz
In recursive scanning, a script variable was overwritten that caused the logic *intended* to make the embedded path be trimmed based on $DESTDIR, to no longer apply. This resulted in embedding the staging path, but only when install_subdir() was used instead of specifying each file as the argument to install_sources.
2023-12-13Use SPDX-License-Identifier consistentlyDylan Baker
This replaces all of the Apache blurbs at the start of each file with an `# SPDX-License-Identifier: Apache-2.0` string. It also fixes existing uses to be consistent in capitalization, and to be placed above any copyright notices. This removes nearly 3000 lines of boilerplate from the project (only python files), which no developer cares to look at. SPDX is in common use, particularly in the Linux kernel, and is the recommended format for Meson's own `project(license: )` field
2023-12-06Fix coverage with lcov 2.0 and uncovered subprojectsDavid Robillard
Version 2.0 of lcov triggers an error when an exclude pattern is unused. This can happen when the project has subprojects, but no code in them ends up covered: lcov: ERROR: 'exclude' pattern '/[...]/subprojects/*' is unused. (use "lcov --ignore-errors unused ..." to bypass this error) So, simply do as it says. Unused patterns doesn't seem problematic (or even interesting) here, so I don't think there's any risk in simply turning this off, which matches what happened with earlier versions of lcov anyway.
2023-11-02scan-build: Exclude subprojects from scan-build reportAndrew McNulty
When a user invokes the scan-build target that Meson generates all subprojects are included in the resulting report. This commit modifies the invocation of scan-build to exclude all bugs that scan-build finds in the subprojects from the final report. A release note has also been added describing the changed behaviour.
2023-11-01Add comments suggesting to keep shell completion scripts up-to-date near cmd ↵Luke Elliott
line argument code
2023-10-26i18n module: Invoke itstool with the full command lineChun-wei Fan
Certain envs may not support invoking itstool by itself directly as a script as shebang lines are not supported, such as under cmd.exe shells on Windows, that are normally used for Visual Studio (and the like, such as clang-cl) builds. This will call the corresponding interpreter to invoke itstool when needed, so that itstool can be properly run, even if shebang lines are not supported by the env. This will fix building appstream on Windows using clang-cl, for instance.
2023-10-20add missing imports for future annotationsEli Schwartz
2023-10-08Add support for lcov 2.0Nigel Kukard
lcov 2.0 deprecates `--rc lcov_branch_coverage=1` for `--rc branch_coverage=1` and gives an error when an exclude is used on a non existing directory. I added a version check for lcov and removed the subprojects directory from the exclusion list if it does not exist. Fixes #11995
2023-10-02python module: stop using distutils "link to libpython" probe on recent pythonEli Schwartz
On python >=3.8, this information is expected to be encoded in the sysconfig vars. In distutils, it is always necessary to link to libpython on Windows; for posix platforms, it depends on the value of LIBPYTHON (which is the library to link to, possibly the empty string) as generated by configure.ac and embedded into python.pc and python-config.sh, and then coded a second time in the distutils python sources. There are a couple of caveats which have ramifications for Cygwin and Android: - python.pc and python-config.sh disagree with distutils when python is not built shared. In that case, the former act the same as a shared build, while the latter *never* links to libpython - python.pc disagrees with python-config.sh and distutils when python is built shared. The former never links to libpython, while the latter do The disagreement is resolved in favor of distutils' behavior in all cases, and python.pc is correct for our purposes on python 3.12; see: https://github.com/python/cpython/pull/100356 https://github.com/python/cpython/pull/100967 Although it was not backported to older releases, Cygwin at least has always patched in a fix for python.pc, which behavior is now declared canonical. We can reliably assume it is always correct. This is the other half of the fix for #7702
2023-10-02python module: refactor pypy detection into a consistent variableEli Schwartz
2023-10-02python module: stop using distutils schemes on sufficiently new DebianEli Schwartz
Since 3.10.3, Debian finally started patching sysconfig with custom paths, instead of just distutils. This means we can now go use that instead. It reduces our reliance on the deprecated distutils module. Partial fix for #7702
2023-09-26mypy: replace fchmod by chmodCharles Brunet
The former is unix-only, which complains when running on the Windows platform -- even though this code is in a unix-specific branch. But per the docs: > As of Python 3.3, this is equivalent to os.chmod(fd, mode). So we can just use the identical function that is available in more places.
2023-09-25Add clang-tidy-fix targetLei YU
Add the `clang-tidy-fix` target to apply clang-tidy fixes to the source code. This is done by calling `run-clang-tidy` with `-fix` argument. Add a test case to run `clang-tidy-fix` and verify the file is changed. Signed-off-by: Lei YU <yulei.sh@bytedance.com>
2023-09-15environment, env2mfile: Don't shell split paths if they point to a valid ↵L. E. Segovia
executable Fixes #11128
2023-08-14Python: Add 'limited_api' kwarg to extension_moduleAndrew McNulty
This commit adds a new keyword arg to extension_module() that enables a user to target the Python Limited API, declaring the version of the limited API that they wish to target. Two new unittests have been added to test this functionality.
2023-08-11treewide: automatic rewriting of all comment-style type annotationsEli Schwartz
Performed using https://github.com/ilevkivskyi/com2ann This has no actual effect on the codebase as type checkers (still) support both and negligible effect on runtime performance since __future__ annotations ameliorates that. Technically, the bytecode would be bigger for non function-local annotations, of which we have many either way. So if it doesn't really matter, why do a large-scale refactor? Simple: because people keep wanting to, but it's getting nickle-and-dimed. If we're going to do this we might as well do it consistently in one shot, using tooling that guarantees repeatability and correctness. Repeat with: ``` com2ann mesonbuild/ ```
2023-08-02treewide: internally avoid deprecated machine file uses of "pkgconfig"Eli Schwartz
We support this in a machine file: ``` [binaries] pkgconfig = 'pkg-config' pkg-config = 'pkg-config' ``` and you can use either one, because internally we look up both. If you only set *one* of them, this plays awkwardly with setting $PKG_CONFIG, since we don't know which one you set in the machine file and the *other* one will be initialized from the environment instead. In commit 22df45a31981874310a78dde0df59a6a7c5ebb29 we changed program lookup of config-tool style dependencies to use the regular tool names and only fall back on the strange internal names. This affected the pkg-config class too. The result is that instead of preferring `pkgconfig =` followed by $PKG_CONFIG followed by `pkg-config =`, we inverted the lookup order. This is a good idea anyway, because now it behaves consistently with `find_program('pkg-config')`. Unfortunately, we documented the wrong name in a bunch of places, and also used the wrong name in various testsuite bits, which meant that if you set $PKG_CONFIG and then ran the testsuite, it would fail. Correct these references, because they are buggy. One test case expected to find_program() a native copy for convenience of testing against natively installed glib. Force it to resolve a native copy.
2023-06-19Add kernel and subsystem properties to machine objects.Jussi Pakkanen
2023-06-12env2mfile: Take pkg-config properties from envXavier Claessens
2023-06-12env2mfile: Take binaries from env for cross file tooXavier Claessens
2023-06-12env2mfile: Not all compilers have env for flagsXavier Claessens
2023-05-16meson_exe: print suitable debug information for DLL not found errorsEli Schwartz
It's particularly inscrutable if you do not know where DLLs weren't found because you don't know what the PATH was in the child process.
2023-05-02bytecompile: switch to handling destdir in the script launcher envEli Schwartz
2023-05-02python module: add an automatic byte-compilation stepEli Schwartz
For all source `*.py` files installed via either py.install_sources() or an `install_dir: py.get_install_dir()`, produce `*.pyc` files at install time. Controllable via a module option.
2023-04-25Fix html coverage report generation when using clang on linuxJakob Widauer
2023-04-11fix various spelling issuesJosh Soref
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-03-28Import cpu detection fix from Debian.Jussi Pakkanen
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1033579
2023-03-21Fix run_tool() when git is not installedAlyssa Ross
Previously, it would raise an exception.
2023-03-16hotdoc: Install devhelp files at the right locationXavier Claessens
When devhelp is enabled, hotdoc generates a devhelp/ subdir that needs to be installed to /usr/share/devhelp/. Otherwise, the html/ subdir needs to be installed to /usr/share/doc/<project>/html/