The new names are consistently formulated as commands.
`main` is not very descriptive, so change it to `test`, which is more
informative and accurate.
`rust_checks` are replaced be the more general `lint` in preparation for
non-Rust-related checks.
These changes were suggested in
https://github.com/fish-shell/fish-shell/pull/11918#discussion_r2415957733Closes#11922
sphinx-build fails with
sphinx.errors.SphinxError: Builder name markdown not registered or available through entry point
Apparently this issue was hidden locally by caching, and not checked
in CI because of this error causing
tests/checks/sphinx-markdown-changelog.fish to be skipped.
sphinx-build 7.2.6
runner@runnervm3ublj:~/work/fish-shell/fish-shell$ python -c 'import sphinx_markdown_builder'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/runner/.local/lib/python3.12/site-packages/sphinx_markdown_builder/__init__.py", line 6, in <module>
from sphinx.util.typing import ExtensionMetadata
ImportError: cannot import name 'ExtensionMetadata' from 'sphinx.util.typing' (/usr/lib/python3/dist-packages/sphinx/util/typing.py)
This reverts commit 7b495497d7.
While at it, fail the test earlier if something went wrong, because the
remaining check will likely also fail and confuse.
GitHub-flavored Markdown translates line breaks to <br/>, which does
not match our intent. Work around that by joining lines when producing
Markdown output.
System tests typically run outside the workspace directory, but they
still have read-only access to the workspace; fix it accordingly.
This test only works on git checkouts, not in tarballs, so skip it
if .git doesn't exist.
The previous version results in an immediate workflow failure due to a
syntax error in the YAML. `workflow_dispatch` should be a dictionary
key, with its value being another dictionary (whose only key is `inputs`
at the moment).
Release automation can be tested on any GitHub fork, using
build_tools/release.sh $version $repository_owner $git_remote
which should work perfectly except for macOS packages (which fail
unless provided GitHub secrets).
People might push tags to their forks, both non-release tags (which
would trigger an early failure in "is-release-tag") or replicas of
our actual release tags (which would create a draft release etc. and
only fail when building macOS packages).
Run on explicit workflow dispatch to make sure it's not triggered by
accident like that.
This means that we'll use the .github/workflows/release.yml from
the default branch (i.e. master), so try to make sure it matches the
version in the release, to prevent accidents.
Closes#11816
These tests require building with the `localize-messages` feature.
If certain translations are updated, this test might fail, either
because a message was changed in the source, or because a translation of
a message was changed, or because a translation was added in a language
which previously did not have a translation for a particular message,
and we rely on that in the test. If any of these happen, the tests need
to be updated accordingly.
Closes#11726
This completely removes our runtime dependency on gettext. As a
replacement, we have our own code for runtime localization in
`src/wutil/gettext.rs`. It considers the relevant locale variables to
decide which message catalogs to take localizations from. The use of
locale variables is mostly the same as in gettext, with the notable
exception that we do not support "default dialects". If `LANGUAGE=ll` is
set and we don't have a `ll` catalog but a `ll_CC` catalog, we will use
the catalog with the country code suffix. If multiple such catalogs
exist, we use an arbitrary one. (At the moment we have at most one
catalog per language, so this is not particularly relevant.)
By using an `EnvStack` to pass variables to gettext at runtime, we now
respect locale variables which are not exported.
For early output, we don't have an `EnvStack` to pass, so we add an
initialization function which constructs an `EnvStack` containing the
relevant locale variables from the corresponding Environment variables.
Treat `LANGUAGE` as path variable. This add automatic colon-splitting.
The sourcing of catalogs is completely reworked. Instead of looking for
MO files at runtime, we create catalogs as Rust maps at build time, by
converting PO files into MO data, which is not stored, but immediately
parsed to extract the mappings. From the mappings, we create Rust source
code as a build artifact, which is then macro-included in the crate's
library, i.e. `crates/gettext-maps/src/lib.rs`. The code in
`src/wutil/gettext.rs` includes the message catalogs from this library,
resulting in the message catalogs being built into the executable.
The `localize-messages` feature can now be used to control whether to
build with gettext support. By default, it is enabled. If `msgfmt` is
not available at build time, and `gettext` is enabled, a warning will be
emitted and fish is built with gettext support, but without any message
catalogs, so localization will not work then.
As a performance optimization, for each language we cache a separate
Rust source file containing its catalog as a map. This allows us to
reuse parsing results if the corresponding PO files have not changed
since we cached the parsing result.
Note that this approach does not eliminate our build-time dependency on
gettext. The process for generating PO files (which uses `msguniq` and
`msgmerge`) is unchanged, and we still need `msgfmt` to translate from
PO to MO. We could parse PO files directly, but these are significantly
more complex to parse, so we use `msgfmt` to do it for us and parse the
resulting MO data.
Advantages of the new approach:
- We have no runtime dependency on gettext anymore.
- The implementation has the same behavior everywhere.
- Our implementation is significantly simpler than GNU gettext.
- We can have localization in cargo-only builds by embedding
localizations into the code.
Previously, localization in such builds could only work reliably as
long as the binary was not moved from the build directory.
- We no longer have to take care of building and installing MO files in
build systems; everything we need for localization to work happens
automatically when building fish.
- Reduced overhead when disabling localization, both in compilation time
and binary size.
Disadvantages of this approach:
- Our own runtime implementation of gettext needs to be maintained.
- The implementation has a more limited feature set (but I don't think
it lacks any features which have been in use by fish).
Part of #11726Closes#11583Closes#11725Closes#11683
The extracted release notes trigger a sphinx warning
/tmp/tmp.V6RGP92nc2/src/index.rst:6:Document may not end with a transition.
which we don't seem to get on the full CHANGELOG.rst.
Let's work around that for now.
That configuration is already tested, but not clippy-checked yet.
This sometimes causes things like unused imports linger on master.
Let's at least enable clippy for stable Rust.
Also do the same build_tools/check.sh; since that script already runs
"cargo test --no-default-features", this shouldn't add much work,
though I didn't check that.
Things that are not currently happening in this workflow:
- No GPG-signature on the Git tag
- No *.asc signature file for the tarball (or for any other release assets)
- No GPG-signed Debian and other OBS packages
To-do:
- remove the corresponding entries from
https://github.com/fish-shell/fish-shell/wiki/Release-checklist
and link to this workflow.
- Maybe add some testing (for the Linux packages)?.
- Let's hope that this doesn't cause security issues.
Usage:
1. run "build_tools/release.sh $version"; this will create and push
a tag, which kicks off .github/workflows/release.yml
2. wait for the draft release to be created at
https://github.com/fish-shell/fish-shell/releases/tags/$version
3. publish the draft (manually, for now). This should unblock the
last part of the workflow (website updates).
Closes#10449
Incremental usage example:
version=4.0.3
repository_owner=fish-shell
remote=origin
cd ../fish-shell-secondary-worktree
git tag -d $version ||:
git push $remote :$version ||:
git reset --hard origin/Integration_$version
for d in .github build_tools; do {
rm -rf $d
cp -r ../fish-shell/$d .
git add $d
} done
git commit -m 'Backport CI/CD'
echo "See https://github.com/$repository_owner/fish-shell/actions"
echo "See the draft release at https://github.com/$repository_owner/fish-shell/releases/$version"
../fish-shell/build_tools/release.sh $version $repository_owner $remote
We use the MSRV for CI checks, and for deploying to old macOS.
But for static Linux builds , there should be no reason to use an
old Rust version. Let's track stable.
By default, we make every rustup user use our pinned version. This might
not be ideal at this point, for a few reasons:
1. we don't have automatic Rust updates yet (see the parent commit),
so this might unnecessarily install an old version. As a contributor,
this feels irritating (newer versions are usually strictly better).
2. it will use more bandwidth and perhaps other resources during "git-bisect"
scenarios
3. somehow rustup will download things redundantly; it will download "1.89.0"
and "stable" even if they are identical. The user will need to clean
those up at some point, even if they didn't add them explicitly.
See also
https://github.com/fish-shell/fish-shell/pull/11712#issuecomment-3165388330
Part of the motivation for rust-toolchain.toml is probably the regular
(every 6 weeks) failures due to the update check, but that failure has been
removed in the parent commit.
The other motivation ("fix the issue of local compiles running into lint
warnings from newer compilers") is a fair point but I think we should rather
fix warnings quickly.
Let's remove rust-toolchain.toml again until we have more agreement on what
we should do.
This reverts commits
* f806d35af8 (Ignore rust-toolchain.toml in CI, 2025-08-07)
* 9714b98262 (Explicitly use fully qualified rust version numbers, 2025-08-07)
* 921aaa0786 (Add rust-toolchain.toml, 2025-08-07)
Closes#11718
This ensures that, by default, developers use the toolchain that is also tested
in CI, avoiding spurious warnings from lints added in new compiler versions.
Today, when a change introduces warnings, the change author might not see
them. Fix that by making clippy fail on warnings.
AFAICT, "clippy --deny-warnings" will also fail on rustc warnings.
I'd imagine this is what most respectable Rust projects do.
Pin stable rust so we won't get unrelated failures. Alternatively, we could
keep using "dtolnay/rust-toolchain@stable", that should be fine too (as long
as we have the capacity to quickly silence/iron out clippy failures).
While at it, remove some unneeded dependencies. Keep gettext because that
one might enable some cfg-directives (?).
Other cfgs like feature="benchmark" and target_os != "linux" are not yet checked in CI.
See #11584
Extract a github action to reduce the number of references to our MSRV and
stable (to be pinned in the next commit).
While at it, use the MSRV for macOS builds; this means that we'll be less
like accidentally to break the macOS build when bumping the MSRV. I don't
think there is a reason for using 1.73 specifically, other than "it's the
highest we can use on old macOS", so using an even older one should be fine.
Remove dependency on CTest. Parallel execution is handled by `test_driver.py`
internally now, so CTest is no longer relevant for performance.
This also removes CMake targets for single tests. As a replacement,
`test_driver.py` can be called directly with the path to the build directory as
the first argument and the path to the desired test as the second argument.
Ensuring that the executables in the build directory are up to date needs to be
done separately.
For a pure cargo build, an example of running a single test would be:
`cargo b && tests/test_driver.py target/debug tests/checks/abbr.fish`
The recommended way of running tests is `build_tools/check.sh`, which runs more
extensive tests and does not depend on CMake. That script does not work in CI
yet, so CMake testing is retained for now.
Update CI config to use the new `FISH_TEST_MAX_CONCURRENCY`.
Also update the FreeBSD version, since the previous one is outdated and does not
support the semaphore logic in `test_driver.py`.
This new wrapper type can be constructed via macros which invoke the
`gettext_extract` proc macro to extract the string literals for PO file
generation.
The type checking enabled by this wrapper should prevent trying to obtain
translations for a string for which none exist.
Because some strings (e.g. for completions) are not defined in Rust, but rather
in fish scripts, the `LocalizableString` type can also be constructed from
non-literals, in which case no extraction happens.
In such cases, it is the programmer's responsibility to only construct the type
for strings which are available for localization.
This approach is a replacement for the `cargo-expand`-based extraction.
When building with the `FISH_GETTEXT_EXTRACTION_FILE` environment variable set,
the `gettext_extract` proc macro will write the messages marked for extraction
to a file in the directory specified by the variable.
Updates to the po files:
- This is the result of running the `update_translations.fish` script using the
new proc_macro extraction. It finds additional messages compared to the
`cargo-expand` based approach.
- Messages IDs corresponding to paths are removed. The do not have localizations
in any language and localizing paths would not make sense. I have not
investigated how they made it into the po files in the first place.
- Some messages are reordered due to `msguniq` sorting differing from `sort`.
Remove docs about installing `cargo-expand`
These are no longer needed due to the switch to our extraction macro.
