On Cygwin, check.sh was running all the tests twice, one with symlinks
enabled, and one without. But most tests do no use symlinks so
re-running those does not test anything new and it's just a major waste
of resources and time (and cygwin is quite slow already).
So only re-run the tests that use symlinks, i.e. that use `ln`.
Filter on mentions of `ln` and not `cygwin_nosymlink` because the latter
is only needed when a test would fail in one of the two scenarios, and
not all tests with symlinks do.
Rewrite the PO file handling logic in Rust and make it available via an
xtask. Replaces the
`build_tools/{update_translations,fish_xgettext}.fish` scripts.
Main benefits:
- Better ergonomics
- Better error handling
- Eliminates the need for a fish executable for updating PO files,
which is particularly useful in CI
- Improved performance, mainly due to concurrent threads working on the
PO files in parallel
The behavior is mostly unchanged, with the minor exception that section
headers for empty sections are now omitted in PO files.
The interface for invoking the tooling is quite different. Instead of
working with flags, `cargo xtask gettext` has 3 subcommands:
- `update` modifies the PO files to match the current sources
- `check` is like update, but instead of modifying the PO files, it
shows diffs between the current version of the PO files and what they
would look like after updating. When there is a difference, the xtask
exits non-zero, making it useful for checks to detect outdated PO
files.
- `new` creates a new PO file for the given language.
Both the `update` and `check` command take any number of file paths to
specify the PO files to consider. If none are specified, all files in
`localization/po/` are considered.
Extracting gettext messages from Rust still requires compiling with the
`gettext-extract` feature active. In situations where compilation is
needed for other purposes as well, it can make sense to only build once
and then tell the gettext xtask about the directory into which the
messages have been extracted. This can be done via the
`--rust-extraction-dir` flag. If we stop having gettext messages in
Rust, this logic can be removed.
Closes#12676
As discussed in #12649, we should check builds with all Cargo features
enabled. Previously, this did cause issues with the `benchmark` feature,
since that only works with nightly Rust. #12653 resolves that by only
enabling the `benchmark` feature with the nightly toolchain, so now we
can use `--all-features` with stable Rust.
Closes#12657
This is not a functional change, since the variable names don't have
spaces, but it is more robust to changes and removes ShellCheck warnings
Part of #12636
When `system_tests` is called without arguments, `[ -n "$@" ]` becomes
`[ -n ]`, which is true, resulting in running `export`, which lists all
exported variables, unnecessarily cluttering the output.
If `system_tests` is called with more than one argument, the check would
fail because having more than one argument after `-n` is invalid syntax.
Fix this by using `$*`, which concatenates all positional arguments to
`system_tests` into a single value.
Part of #12636
The main changes are:
- disabling some checks related to POSIX file permissions when a filesystem is
mounted with "noacl" (default on MSYS2)
- disabling some checks related to symlinks when using fake ones (file copy)
Windows with acl hasn't been tested because 1) Cygwin itself does not have any
Rust package yet to compile fish, and 2) MSYS2 defaults to `noacl`
Part of #12171
Replace the `build_tools/style.fish` script by an xtask. This eliminates
the need for a fish binary for performing the formatting/checking. The
`fish_indent` binary is still needed. Eventually, this should be made
available as a library function, so the xtask can use that instead of
requiring a `fish_indent` binary in the `$PATH`.
The new xtask is called `format` rather than `style`, because that's a
more fitting description of what it does (and what the script it
replaces did).
The old script's behavior is not replicated exactly:
- Specifying `--all` and explicit paths is supported within a single
invocation.
- Explicit arguments no longer have to be files. If a directory is
specified, all files within it will be considered.
- The git check for un-staged changes is no longer filtered by file
names, mainly to simplify the implementation.
- A warning is now printed if neither the `--all` flag nor a path are
provided as arguments. The reason for this is that one might assume
that omitting these arguments would default to formatting everything
in the current directory, but instead no formatting will happen in
this case.
- The wording of some messages is different.
The design of the new code tries to make it easy to add formatters for
additional languages, or change the ones we already have. This is
achieved by separating the code into one function per language, which
can be modified without touching the code for the other languages.
Adding support for a new formatter/language only requires adding a
function which builds the formatter command line based on the arguments
to the xtask, and calling that function from the main `format` function.
Closes#12467
This is useful for running the checks with a toolchain which is
different from the default toolchain, for example to check if everything
works with our MSRV, or on beta/nightly toolchains. Additionally,
providing a way to run using the nightly toolchain allows writing
wrappers around `check.sh` which make use of nightly-only features.
The toolchain could be changed using `rustup toolchain default`, but if
the toolchain should only be used for a specific run, this is
inconvenient, and it does not allow for concurrent builds using
different toolchains.
Closes#12281
Multiple gettext-extraction proc macro instances can run at the same
time due to Rust's compilation model. In the previous implementation,
where every instance appended to the same file, this has resulted in
corruption of the file. This was reported and discussed in
https://github.com/fish-shell/fish-shell/pull/11928#discussion_r2488047964
for the equivalent macro for Fluent message ID extraction. The
underlying problem is the same.
The best way we have found to avoid such race condition is to write each
entry to a new file, and concatenate them together before using them.
It's not a beautiful approach, but it should be fairly robust and
portable.
Closes#12125
In future, we should ask "renovatebot" to update these version. I
don't have an opinion on whether to use "uv" or something else, but
I think we do want lockfiles, and I don't know of a natural way to
install Sphinx via Cargo.
No particular reason for this Python version.
Part of #11996
- Convert update checks in check.sh to mechanical updates.
- Use https://www.updatecli.io/ for now, which is not as
full-featured as renovatebot or dependabot, but I found it easier
to plug arbitrary shell scripts into.
- Add updaters for
- ubuntu-latest-lts (which is similar to GitHub Action's "ubuntu-latest").
- FreeBSD image used in Cirrus (requires "gcloud auth login" for now,
see https://github.com/cirruslabs/cirrus-ci-docs/issues/1315)
- littlecheck and widecharwidth
- Update all dependencies except Cargo ones.
- As a reminder, our version policies are arbitrary and can be changed
as needed.
- To-do:
- Add updaters for GitHub Actions (such as "actions/checkout").
Renovatebot could do that.
Opt-in because we don't want failures not related to local changes,
see d93fc5eded (Revert "build_tools/check.sh: check that stable rust
is up-to-date", 2025-08-10).
For the same reason, it's not currently checked by CI, though we should
definitely have "renovatebot" do this in future.
Also, document the minimum supported Rust version (MSRV) policy. There's no
particular reason for this specific MSRV policy, but it's better than nothing,
since this will allow us to always update without thinking about it.
Closes#11960
Some versions of `/bin/sh`, e.g. the one on FreeBSD, do not propagate
variables set on a command through shell functions. This results in
`FISH_GETTEXT_EXTRACTION_FILE` being set in the `cargo` function, but
not for the actual `cargo` process spawned from the function, which
breaks our localization scripts and tests. Exporting the variable
prevents that.
Fixes#11896Closes#11899
That configuration is already tested, but not clippy-checked yet.
This sometimes causes things like unused imports linger on master.
Let's at least enable clippy for stable Rust.
Also do the same build_tools/check.sh; since that script already runs
"cargo test --no-default-features", this shouldn't add much work,
though I didn't check that.
This allows having the proc macro crate as an optional dependency and speeds up
compilation in situations where `FISH_GETTEXT_EXTRACTION_FILE` changes, such as
the `build_tools/check.sh` script. Because we don't need to recompile on changes
to the environment variable when the feature is disabled, cargo can reuse
earlier compilation results instead of recompiling everything.
This speeds up the compilation work in `build_tools/check.sh` when no changes
were made which necessitate recompilation.
For such runs of `build_tools/check.sh`, these changes reduce the runtime on my
system by about 10 seconds, from 70 to 60, approximately.
The difference comes from the following two commands recompiling code without
the changes in this commit, but not with them:
- `cargo test --doc --workspace`
- `cargo doc --workspace`
As reported in #11711 and #11712, the update-checks make check.sh automatically
fail every 6 weeks, so it pressures people into updating Rust, and (what's
worse), updating fish's pinned Rust version, even when that's not relevant
to their intent (which is to run `clippy -Dwarnings` and all other checks).
The update-checks were added as a "temporary" solution to make sure that
our pinned version doesn't lag too far behind stable, which gives us an
opportunity to fix new warnings before most contributors see them.
As suggested in #11584, reasonable solutions might be either of:
1. stop pinning stable Rust and rely on beta-nightlies to fix CI failures early
2. use renovatebot or similar to automate Rust updates
Until then, remove the update check to reduce friction.
I'll still run it on my machine.
This reverts commit 6d061daa91.
I sometimes want to run this script in multiple docker containers concurrently,
and possibly modify it while another instance is already running. The behavior
after modification is unpredictable; let's change it to read the whole script
up-front (like Python/fish do).
Use test_driver directly instead of CMake in the docker tests.
Deal with the read-only "/fish-source" by exporting
"CARGO_TARGET_DIR=$HOME/fish-build". It seems correct to also inject this
environment variable into the interactive debugging shells. Add some logging
to make this override more obvious to the user.
Adopt "build_tools/check.sh", because that defines the full set of checks
that we (eventually) want to run in CI.
In particular, this will also run "tests/checks/po-files-up-to-date.fish"
which "cargo b && cargo t && tests/test_driver.py" does not, due to the
REQUIRES clause.
Since most docker images have some lints/warnings today, disable those for
now. Use "docker_run_tests.sh --lint" to override. The default may be changed
in future.
We have a mixture of 2 and 4 space indent.
4 benchmarks/driver.sh
2 build_tools/check.sh
4 build_tools/git_version_gen.sh
4 build_tools/mac_notarize.sh
2 build_tools/make_pkg.sh
2 build_tools/make_tarball.sh
2 build_tools/make_vendor_tarball.sh
4 docker/docker_run_tests.sh
4 osx/install.sh
2 tests/test_functions/sphinx-shared.sh
Our editorconfig file specifies 2, with no explicit reason.
Our fish and Python scripts use 4, so let's use that.
The PO file updates can now run in a normal test, eliminating the need for
special handling.
Rename the `check-translations.fish` script, to clarify which part of the checks
happens in it.
This uses Python's `asyncio` to run tests in parallel, which speeds up test
execution significantly.
The timeout is removed. It would be possible to add a timeout to
`asyncio.as_completed()` if we want that.
This is intended as a way to run all available checks with a single command.
The script can be used locally and in CI. It is intended to replace
`cmake/Tests.cmake` (but this script also runs checks not present there).
At the moment, `ctest` is not used, which could be added to speed up tests.
Address and thread sanitizers are not run by this script.
