h/fish-shell
1
Fork 0
mirror of https://github.com/fish-shell/fish-shell.git synced 2026-05-19 09:51:16 -03:00
Code Issues Packages Projects Releases Wiki Activity
Files
3478f78a0536414af623467c2addda4e9700a75d
fish-shell/cmake/Tests.cmake

71 lines
2.4 KiB
CMake
Raw Normal View History

Run each test fully independently in own environment
2021-03-15 12:42:53 -05:00
FILE(GLOB FISH_CHECKS CONFIGURE_DEPENDS ${CMAKE_SOURCE_DIR}/tests/checks/*.fish)
foreach(CHECK ${FISH_CHECKS})
get_filename_component(CHECK_NAME ${CHECK} NAME)
Simplify CMake Tests Remove dependency on CTest. Parallel execution is handled by `test_driver.py` internally now, so CTest is no longer relevant for performance. This also removes CMake targets for single tests. As a replacement, `test_driver.py` can be called directly with the path to the build directory as the first argument and the path to the desired test as the second argument. Ensuring that the executables in the build directory are up to date needs to be done separately. For a pure cargo build, an example of running a single test would be: `cargo b && tests/test_driver.py target/debug tests/checks/abbr.fish` The recommended way of running tests is `build_tools/check.sh`, which runs more extensive tests and does not depend on CMake. That script does not work in CI yet, so CMake testing is retained for now. Update CI config to use the new `FISH_TEST_MAX_CONCURRENCY`. Also update the FreeBSD version, since the previous one is outdated and does not support the semaphore logic in `test_driver.py`.
2025-06-19 22:55:58 +02:00
add_custom_target(
test_${CHECK_NAME}
Remove `--cachedir` for `test_driver.py` This was only used to cache `fish_test_driver`, which can be built in a few tens of milliseconds on most systems. This avoids using outdated binaries for testing and simplifies the code and its usage.
2025-06-10 16:43:01 +02:00
COMMAND ${CMAKE_SOURCE_DIR}/tests/test_driver.py ${CMAKE_CURRENT_BINARY_DIR}
Simplify CMake Tests Remove dependency on CTest. Parallel execution is handled by `test_driver.py` internally now, so CTest is no longer relevant for performance. This also removes CMake targets for single tests. As a replacement, `test_driver.py` can be called directly with the path to the build directory as the first argument and the path to the desired test as the second argument. Ensuring that the executables in the build directory are up to date needs to be done separately. For a pure cargo build, an example of running a single test would be: `cargo b && tests/test_driver.py target/debug tests/checks/abbr.fish` The recommended way of running tests is `build_tools/check.sh`, which runs more extensive tests and does not depend on CMake. That script does not work in CI yet, so CMake testing is retained for now. Update CI config to use the new `FISH_TEST_MAX_CONCURRENCY`. Also update the FreeBSD version, since the previous one is outdated and does not support the semaphore logic in `test_driver.py`.
2025-06-19 22:55:58 +02:00
checks/${CHECK_NAME}
cmake: Stop setting up temporary test directory We already set up a temporary directory for each test in the test_driver, so let's stop doing that here. It took a weirdly long time anyway.
2025-01-12 17:44:31 +01:00
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/tests
cmake: don't build fish_test_helper Our test driver (`tests/test_driver.py`) already builds this unconditionally on each run, so there is no point in having CMake build the binary as well. If we want to reduce the effort of rebuilding the test helper on each invocation of the test driver, we could consider some other caching approach, but it should work for non-CMake builds as well. I consider this a low priority, since building the executable only takes a few 10s of milliseconds on relatively modern hardware. Closes #12364
2026-01-21 16:29:22 +01:00
DEPENDS fish fish_indent fish_key_reader
Simplify CMake Tests Remove dependency on CTest. Parallel execution is handled by `test_driver.py` internally now, so CTest is no longer relevant for performance. This also removes CMake targets for single tests. As a replacement, `test_driver.py` can be called directly with the path to the build directory as the first argument and the path to the desired test as the second argument. Ensuring that the executables in the build directory are up to date needs to be done separately. For a pure cargo build, an example of running a single test would be: `cargo b && tests/test_driver.py target/debug tests/checks/abbr.fish` The recommended way of running tests is `build_tools/check.sh`, which runs more extensive tests and does not depend on CMake. That script does not work in CI yet, so CMake testing is retained for now. Update CI config to use the new `FISH_TEST_MAX_CONCURRENCY`. Also update the FreeBSD version, since the previous one is outdated and does not support the semaphore logic in `test_driver.py`.
2025-06-19 22:55:58 +02:00
USES_TERMINAL
Run each test fully independently in own environment
2021-03-15 12:42:53 -05:00
)
endforeach(CHECK)
Add pexpect tests to new test driver
2021-03-18 13:23:31 -05:00
FILE(GLOB PEXPECTS CONFIGURE_DEPENDS ${CMAKE_SOURCE_DIR}/tests/pexpects/*.py)
foreach(PEXPECT ${PEXPECTS})
get_filename_component(PEXPECT ${PEXPECT} NAME)
Simplify CMake Tests Remove dependency on CTest. Parallel execution is handled by `test_driver.py` internally now, so CTest is no longer relevant for performance. This also removes CMake targets for single tests. As a replacement, `test_driver.py` can be called directly with the path to the build directory as the first argument and the path to the desired test as the second argument. Ensuring that the executables in the build directory are up to date needs to be done separately. For a pure cargo build, an example of running a single test would be: `cargo b && tests/test_driver.py target/debug tests/checks/abbr.fish` The recommended way of running tests is `build_tools/check.sh`, which runs more extensive tests and does not depend on CMake. That script does not work in CI yet, so CMake testing is retained for now. Update CI config to use the new `FISH_TEST_MAX_CONCURRENCY`. Also update the FreeBSD version, since the previous one is outdated and does not support the semaphore logic in `test_driver.py`.
2025-06-19 22:55:58 +02:00
add_custom_target(
test_${PEXPECT}
Remove `--cachedir` for `test_driver.py` This was only used to cache `fish_test_driver`, which can be built in a few tens of milliseconds on most systems. This avoids using outdated binaries for testing and simplifies the code and its usage.
2025-06-10 16:43:01 +02:00
COMMAND ${CMAKE_SOURCE_DIR}/tests/test_driver.py ${CMAKE_CURRENT_BINARY_DIR}
Rewrite test driver in python (#11028) This replaces the test_driver.sh/test.fish/interactive.fish system with a test driver written in python that calls into littlecheck directly and runs pexpect in a subprocess. This means we reduce the reliance on the fish that we're testing, and we remove a posix sh script that is a weird stumbling block (see my recent quest to make it work on directories with spaces). To run specific tests, e.g. all the tmux tests and bind.py: tests/test_driver.py target/release/ tests/checks/tmux*.fish tests/pexpects/bind.py
2025-01-11 21:13:19 +01:00
pexpects/${PEXPECT}
cmake: Stop setting up temporary test directory We already set up a temporary directory for each test in the test_driver, so let's stop doing that here. It took a weirdly long time anyway.
2025-01-12 17:44:31 +01:00
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/tests
cmake: don't build fish_test_helper Our test driver (`tests/test_driver.py`) already builds this unconditionally on each run, so there is no point in having CMake build the binary as well. If we want to reduce the effort of rebuilding the test helper on each invocation of the test driver, we could consider some other caching approach, but it should work for non-CMake builds as well. I consider this a low priority, since building the executable only takes a few 10s of milliseconds on relatively modern hardware. Closes #12364
2026-01-21 16:29:22 +01:00
DEPENDS fish fish_indent fish_key_reader
Simplify CMake Tests Remove dependency on CTest. Parallel execution is handled by `test_driver.py` internally now, so CTest is no longer relevant for performance. This also removes CMake targets for single tests. As a replacement, `test_driver.py` can be called directly with the path to the build directory as the first argument and the path to the desired test as the second argument. Ensuring that the executables in the build directory are up to date needs to be done separately. For a pure cargo build, an example of running a single test would be: `cargo b && tests/test_driver.py target/debug tests/checks/abbr.fish` The recommended way of running tests is `build_tools/check.sh`, which runs more extensive tests and does not depend on CMake. That script does not work in CI yet, so CMake testing is retained for now. Update CI config to use the new `FISH_TEST_MAX_CONCURRENCY`. Also update the FreeBSD version, since the previous one is outdated and does not support the semaphore logic in `test_driver.py`.
2025-06-19 22:55:58 +02:00
USES_TERMINAL
Add pexpect tests to new test driver
2021-03-18 13:23:31 -05:00
)
endforeach(PEXPECT)
Initial Rust commit
2023-01-14 14:56:24 -08:00
# Rust stuff.
Simplify CMake Tests Remove dependency on CTest. Parallel execution is handled by `test_driver.py` internally now, so CTest is no longer relevant for performance. This also removes CMake targets for single tests. As a replacement, `test_driver.py` can be called directly with the path to the build directory as the first argument and the path to the desired test as the second argument. Ensuring that the executables in the build directory are up to date needs to be done separately. For a pure cargo build, an example of running a single test would be: `cargo b && tests/test_driver.py target/debug tests/checks/abbr.fish` The recommended way of running tests is `build_tools/check.sh`, which runs more extensive tests and does not depend on CMake. That script does not work in CI yet, so CMake testing is retained for now. Update CI config to use the new `FISH_TEST_MAX_CONCURRENCY`. Also update the FreeBSD version, since the previous one is outdated and does not support the semaphore logic in `test_driver.py`.
2025-06-19 22:55:58 +02:00
set(cargo_test_flags)
Enable rust address sanitizer for asan ci job (#9643) Rust has multiple sanitizers available (with llvm integration). -Zsanitizer=address catches the most likely culprits but we may want to set up a separate job w/ -Zsanitizer=memory to catch uninitialized reads. It might be necessary to execute `cargo build` as `cargo build -Zbuild-std` to get full coverage. When we're linking against the hybrid C++ codebase, the sanitizer library is injected into the binary by also include `-fsanitize=address` in CXXFLAGS - we do *not* want to manually opt-into `-lasan`. We also need to manually specify the desired target triple as a CMake variable and then explicitly pass it to all `cargo` invocations if building with ASAN. Corrosion has been patched to make sure it follows these rules. The `cargo-test` target is failing to link under ASAN. For some reason it has autocxx/ffi dependencies even though only rust-native, ffi-free code should be tested (and one would think the situation wouldn't change depending on the presence of the sanitizer flag). It's been disabled under ASAN for now.
2023-03-06 18:15:36 -06:00
if(DEFINED ASAN)
# Rust w/ -Zsanitizer=address requires explicitly specifying the --target triple or else linker
# errors pertaining to asan symbols will ensue.
if(NOT DEFINED Rust_CARGO_TARGET)
message(FATAL_ERROR "ASAN requires defining the CMake variable Rust_CARGO_TARGET to the
intended target triple")
endif()
Always pass `--target` to `cargo test` when CMake Rust_CARGO_TARGET is set The CMake `cargo test` integration was broken if Rust_CARGO_TARGET were used with `CARGO_FLAGS` set to `-Zbuild-std` (e.g. to target i586 under i686 without the i586 toolchain installed).
2024-06-21 11:56:50 -05:00
endif()
Update docker files and cirrus config - Ubuntu focal is the lowest LTS release that we can support with only distro packages (e.g. no rustup). - Remove tsan from Cirrus (it's not working currently, and also not really important). - Remove Centos (it passes tests but I'm not sure it's worth adding; there isn't even an official docker image for CentOS Stream).
2024-08-04 10:18:06 +02:00
if(DEFINED TSAN)
if(NOT DEFINED Rust_CARGO_TARGET)
message(FATAL_ERROR "TSAN requires defining the CMake variable Rust_CARGO_TARGET to the
intended target triple")
endif()
endif()
Always pass `--target` to `cargo test` when CMake Rust_CARGO_TARGET is set The CMake `cargo test` integration was broken if Rust_CARGO_TARGET were used with `CARGO_FLAGS` set to `-Zbuild-std` (e.g. to target i586 under i686 without the i586 toolchain installed).
2024-06-21 11:56:50 -05:00
if(DEFINED Rust_CARGO_TARGET)
Enable doctests They are probably not terribly useful for us but let's see what happens. Unfortunately cargo does not properly forward the combination of "RUSTFLAGS" and "--target" that is currently required to build with ASan [1]. Hence doctests will fail to link on ASan builds. Let's disable doctests when ASan is active. [1]: https://github.com/rust-lang/cargo/issues/10666 et al
2024-01-13 01:26:28 +01:00
list(APPEND cargo_test_flags "--target" ${Rust_CARGO_TARGET})
list(APPEND cargo_test_flags "--lib")
Enable rust address sanitizer for asan ci job (#9643) Rust has multiple sanitizers available (with llvm integration). -Zsanitizer=address catches the most likely culprits but we may want to set up a separate job w/ -Zsanitizer=memory to catch uninitialized reads. It might be necessary to execute `cargo build` as `cargo build -Zbuild-std` to get full coverage. When we're linking against the hybrid C++ codebase, the sanitizer library is injected into the binary by also include `-fsanitize=address` in CXXFLAGS - we do *not* want to manually opt-into `-lasan`. We also need to manually specify the desired target triple as a CMake variable and then explicitly pass it to all `cargo` invocations if building with ASAN. Corrosion has been patched to make sure it follows these rules. The `cargo-test` target is failing to link under ASAN. For some reason it has autocxx/ffi dependencies even though only rust-native, ffi-free code should be tested (and one would think the situation wouldn't change depending on the presence of the sanitizer flag). It's been disabled under ASAN for now.
2023-03-06 18:15:36 -06:00
endif()
Simplify CMake Tests Remove dependency on CTest. Parallel execution is handled by `test_driver.py` internally now, so CTest is no longer relevant for performance. This also removes CMake targets for single tests. As a replacement, `test_driver.py` can be called directly with the path to the build directory as the first argument and the path to the desired test as the second argument. Ensuring that the executables in the build directory are up to date needs to be done separately. For a pure cargo build, an example of running a single test would be: `cargo b && tests/test_driver.py target/debug tests/checks/abbr.fish` The recommended way of running tests is `build_tools/check.sh`, which runs more extensive tests and does not depend on CMake. That script does not work in CI yet, so CMake testing is retained for now. Update CI config to use the new `FISH_TEST_MAX_CONCURRENCY`. Also update the FreeBSD version, since the previous one is outdated and does not support the semaphore logic in `test_driver.py`.
2025-06-19 22:55:58 +02:00
set(max_concurrency_flag)
if(DEFINED ENV{FISH_TEST_MAX_CONCURRENCY})
list(APPEND max_concurrency_flag "--max-concurrency" $ENV{FISH_TEST_MAX_CONCURRENCY})
endif()
# The top-level test target is "fish_run_tests".
add_custom_target(fish_run_tests
# TODO: This should be replaced with a unified solution, possibly build_tools/check.sh.
COMMAND ${CMAKE_SOURCE_DIR}/tests/test_driver.py ${max_concurrency_flag} ${CMAKE_CURRENT_BINARY_DIR}
cmake/Tests: split long line
2025-12-09 19:29:23 +01:00
COMMAND env ${VARS_FOR_CARGO}
${Rust_CARGO}
test
--no-default-features
cmake/Tests: enable Cargo features For unknown reasons, we don't enable Cargo features on the test target. Let's try to, simplifying things overall. Ref: https://github.com/fish-shell/fish-shell/issues/12120#issuecomment-3627842316
2025-12-09 19:29:23 +01:00
--features=${FISH_CARGO_FEATURES}
cmake/Tests: split long line
2025-12-09 19:29:23 +01:00
${CARGO_FLAGS}
--workspace
--target-dir ${rust_target_dir}
${cargo_test_flags}
Simplify CMake Tests Remove dependency on CTest. Parallel execution is handled by `test_driver.py` internally now, so CTest is no longer relevant for performance. This also removes CMake targets for single tests. As a replacement, `test_driver.py` can be called directly with the path to the build directory as the first argument and the path to the desired test as the second argument. Ensuring that the executables in the build directory are up to date needs to be done separately. For a pure cargo build, an example of running a single test would be: `cargo b && tests/test_driver.py target/debug tests/checks/abbr.fish` The recommended way of running tests is `build_tools/check.sh`, which runs more extensive tests and does not depend on CMake. That script does not work in CI yet, so CMake testing is retained for now. Update CI config to use the new `FISH_TEST_MAX_CONCURRENCY`. Also update the FreeBSD version, since the previous one is outdated and does not support the semaphore logic in `test_driver.py`.
2025-06-19 22:55:58 +02:00
WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}"
cmake: don't build fish_test_helper Our test driver (`tests/test_driver.py`) already builds this unconditionally on each run, so there is no point in having CMake build the binary as well. If we want to reduce the effort of rebuilding the test helper on each invocation of the test driver, we could consider some other caching approach, but it should work for non-CMake builds as well. I consider this a low priority, since building the executable only takes a few 10s of milliseconds on relatively modern hardware. Closes #12364
2026-01-21 16:29:22 +01:00
DEPENDS fish fish_indent fish_key_reader
Simplify CMake Tests Remove dependency on CTest. Parallel execution is handled by `test_driver.py` internally now, so CTest is no longer relevant for performance. This also removes CMake targets for single tests. As a replacement, `test_driver.py` can be called directly with the path to the build directory as the first argument and the path to the desired test as the second argument. Ensuring that the executables in the build directory are up to date needs to be done separately. For a pure cargo build, an example of running a single test would be: `cargo b && tests/test_driver.py target/debug tests/checks/abbr.fish` The recommended way of running tests is `build_tools/check.sh`, which runs more extensive tests and does not depend on CMake. That script does not work in CI yet, so CMake testing is retained for now. Update CI config to use the new `FISH_TEST_MAX_CONCURRENCY`. Also update the FreeBSD version, since the previous one is outdated and does not support the semaphore logic in `test_driver.py`.
2025-06-19 22:55:58 +02:00
USES_TERMINAL
CMake: re-enable cargo-test under ASAN With autocxx/cxx gone, this seems to work fine now.
2024-01-07 21:38:49 +01:00
)
Reference in New Issue Copy Permalink