get_version() in build.rs duplicates some logic in
build_tools/git_version_gen.sh. There are some differences
1. When computing the Git hash, get_version() falls back to reading
.git/HEAD directly if "git describe" fails
1.1. Git is not installed
Not sure if this is a good strong reason. If you don't have
Git installed, how would you have created ".git"? If the exact
Git SHA is important, maybe we should use something like gitoxide
for this rather than implementing our own.
1.2. there is a permission problem
The case mentiond in 0083192fcb (Read git SHA ourselves if it
is unavailable, 2024-12-09) doesn't seem to happen with current
versions of Git: "sudo git describe" works fine. Something like
"sudo -u postgres git describe" doesn't. We could support that
but let's wait until we know of a use case.
1.3 there are no tags
(when doing "cargo install --git", as mentioned in 0dfc490721
(build.rs: Use Cargo_PKG_VERSION if no version could be found,
2024-06-10)).
Missing tags are no longer a problem because we read the version
from Cargo.toml now. Tweak the script to make sure that the
version is 4.3.3-g${git_sha} instead of just ${git_sha}.
2. get_version() falls back to jj too.
That was added for jj workspaces that don't have a Git worktree;
but those should be one their way out; when using jj's Git backend,
all workspaces will get an associated worktre.
Use the version in Cargo.toml instead.
Print a warning if the Cargo.toml version is not a prefix of the Git
version. This can happen legit scenarios, see 0dfc490721 (build.rs:
Use Cargo_PKG_VERSION if no version could be found, 2024-06-10)
but the next commit will fix that.
Also remove stale comments in git_version_gen.sh.
Man pages used to be built by "build.rs" but now are built by a
dependent "crates/build-man-pages/build.rs". This means that changing
the environment of build.rs is ineffective.
In future, "fn get_version" should probably be a part of
"crates/build-helper/", so Cargo builds only need to compute the
version once.
Lack of this dependency means that "build-man-pages" does not
pass FISH_BUILD_VERSION, which means that Sphinx will fall back to
build_tools/git_version_gen.sh. This acceptable for now given that
"build-man-pages" is not used in CMake builds.
A subsequent commit will need to test for cygwin in a new crate. On
current stable Rust (1.92) this works via `#[cfg(target_os = "cygwin)]`,
but our MSRV (1.85) does not support this. To avoid code duplication,
the OS detection logic is extracted into the build helper crate. For
now, only `detect_cygwin` is needed, but it would be inconsistent to
extract that but not the same functions for other operating systems.
Part of #12183
Advantages of prebuilt docs:
- convenient for users who compile the tarball
- convenient for packagers who don't have sphinx-build packaged
(but packaging/installing that should be easy nowadays?
see https://github.com/fish-shell/fish-shell/issues/12052#issuecomment-3520336984)
Disadvantages:
- Extra build stage / code path
- Users who switch from building from tarball to building from source
might be surprised to lose docs.
- People put the [tarballs into Git repositories](https://salsa.debian.org/debian/fish), which seems weird.
Remove the tarball.
Let's hope this is not too annoying to users who build on outdated
distros that don't have sphinx -- but those users can probably use
our static builds, skipping all compilation.
To avoid breaking packagers who use `-DBUILD_DOCS=OFF` (which still
installs prebuilt docs), rename the option.
Closes#12088
While it's not necessary to rebuild the proc macro for extraction when
the env var controlling the output location changes, this way everything
using the macro will automatically be rebuilt when this env var changes,
making it impossible to forget adding this to other `build.rs` files.
For now, keeping the rebuild instructions in fish's main crate's
`build.rs` would be fine as well, but if we start breaking this crate
into smaller parts, it would become annoying to add the rebuild command
in every crate depending on gettext extraction.
Closes#12107
For paths embedded via `rust-embed`, we only need to rebuild on path
changes if the files are actually embedded.
To avoid having to remember and duplicate this logic for all embedded
paths, extract it into the build helper.
Closes#12083
This mode still has some problems (see the next commit) but having
it behind a feature flag doesn't serve us. Let's commit to the parts
we want to keep and drop anything that we don't want.
To reduce diff noise, use "if false" in the code paths used by man
pages; a following commit will use them.
This might help cross-compilation; even if it doesn't matter in
practice, this is more correct. While at it, get rid of $TARGET in
favor of the more accurate $CARGO_CFG_TARGET_OS.
Not yet sure how to do it for "has_small_stack".
Historically, Sphinx was required when building "standalone" builds,
else they would have no man pages.
This means that commit 0709e4be8b (Use standalone code paths by
default, 2025-10-26) broke man pages for users who build from a
tarball where non-standalone builds would use prebuilt docs.
Add a hack to use prebuilt docs again.
In future, we'll remove prebuilt docs, see #12052.
As mentioned in a previous commit ("Don't special-case __fish_data_dir
in standalone builds"), we want to enable embed-data by default.
To reduce the amount of breakage, we'll keep installing files,
and keep defining those variables at least for some time.
We have no use for $__fish_data_dir apart from helping plugins and
improving upgrade experience, but we still use $__fish_help_dir;
so this will allow installed "standalone" builds to use local docs
via our "help" function.
(This doesn't matter in practice because we always override SYSCONFDIR
in CMake builds.)
According to https://cmake.org/cmake/help/latest/command/install.html
default paths look like
Type variable default
INCLUDE ${CMAKE_INSTALL_INCLUDEDIR} include
SYSCONF ${CMAKE_INSTALL_SYSCONFDIR} etc
DATA ${CMAKE_INSTALL_DATADIR} <DATAROOT dir>
MAN ${CMAKE_INSTALL_MANDIR} <DATAROOT dir>/man
so "etc" is supposed to be a sibling of "include", not a child of DATA
(aka "share/").
This allows us to remove a hack where we needed to know the data dir
in non-standalone builds.
It's always the CMake output directory, so call it
FISH_CMAKE_BINARY_DIR. It's possible to set it via some other build
system but if such builds exist, they are likely subtly broken, or
at least with the following commit which adds the assumption that
"share/__fish_build_paths.fish.in" exists in this directory.
We could even call it CMAKE_BINARY_DIR but let's namespace it to make
our use more obvious. Also, stop using the $CMAKE environment variable,
it's not in our namespace.
As mentioned in 6896898769 (Add [lints] table to suppress lints
across all our crates, 2024-01-12), we can use workspace lints in
Cargo.toml now that we have MSRV >=1.74 and since we probably don't
support building without cargo.
This implies moving some lints from src/lib.rs to "workspace.lints".
While at it, address some of them insrtead.
This commit adds `style_edition = "2024"` as a rustfmt config setting.
All other changes are automatically generated by `cargo fmt`.
The 2024 style edition fixes several bugs and changes some defaults.
https://doc.rust-lang.org/edition-guide/rust-2024/rustfmt-style-edition.html
Most of the changes made to our code result from a different sorting
method for `use` statements, improved ability to split long lines, and
contraction of short trailing expressions into single-line expressions.
While our MSRV is still 1.70, we use more recent toolchains for
development, so we can already benefit from the improvements of the new
style edition. Formatting is not require for building fish, so builds
with Rust 1.70 are not affected by this change.
More context can be found at
https://github.com/fish-shell/fish-shell/issues/11630#issuecomment-3406937077Closes#11959
When running a debug build, rust-embed always sources files from disk.
This is currently broken with on Cygwin.
As a temporary workaround, use the "debug-embed" feature to actually
embed the files into the binary, like we do for release builds.
We can probably fix the rust-embed issue fairly easily.
I haven't checked. For now, I think this hack is preferrable to
not having an easy way to make debug builds on Cygwin. (CMake
files would need some changes, and I also hit some problems with
installation). At least this would have helped with investigating
https://github.com/msys2/msys2-runtime/issues/308
This completely removes our runtime dependency on gettext. As a
replacement, we have our own code for runtime localization in
`src/wutil/gettext.rs`. It considers the relevant locale variables to
decide which message catalogs to take localizations from. The use of
locale variables is mostly the same as in gettext, with the notable
exception that we do not support "default dialects". If `LANGUAGE=ll` is
set and we don't have a `ll` catalog but a `ll_CC` catalog, we will use
the catalog with the country code suffix. If multiple such catalogs
exist, we use an arbitrary one. (At the moment we have at most one
catalog per language, so this is not particularly relevant.)
By using an `EnvStack` to pass variables to gettext at runtime, we now
respect locale variables which are not exported.
For early output, we don't have an `EnvStack` to pass, so we add an
initialization function which constructs an `EnvStack` containing the
relevant locale variables from the corresponding Environment variables.
Treat `LANGUAGE` as path variable. This add automatic colon-splitting.
The sourcing of catalogs is completely reworked. Instead of looking for
MO files at runtime, we create catalogs as Rust maps at build time, by
converting PO files into MO data, which is not stored, but immediately
parsed to extract the mappings. From the mappings, we create Rust source
code as a build artifact, which is then macro-included in the crate's
library, i.e. `crates/gettext-maps/src/lib.rs`. The code in
`src/wutil/gettext.rs` includes the message catalogs from this library,
resulting in the message catalogs being built into the executable.
The `localize-messages` feature can now be used to control whether to
build with gettext support. By default, it is enabled. If `msgfmt` is
not available at build time, and `gettext` is enabled, a warning will be
emitted and fish is built with gettext support, but without any message
catalogs, so localization will not work then.
As a performance optimization, for each language we cache a separate
Rust source file containing its catalog as a map. This allows us to
reuse parsing results if the corresponding PO files have not changed
since we cached the parsing result.
Note that this approach does not eliminate our build-time dependency on
gettext. The process for generating PO files (which uses `msguniq` and
`msgmerge`) is unchanged, and we still need `msgfmt` to translate from
PO to MO. We could parse PO files directly, but these are significantly
more complex to parse, so we use `msgfmt` to do it for us and parse the
resulting MO data.
Advantages of the new approach:
- We have no runtime dependency on gettext anymore.
- The implementation has the same behavior everywhere.
- Our implementation is significantly simpler than GNU gettext.
- We can have localization in cargo-only builds by embedding
localizations into the code.
Previously, localization in such builds could only work reliably as
long as the binary was not moved from the build directory.
- We no longer have to take care of building and installing MO files in
build systems; everything we need for localization to work happens
automatically when building fish.
- Reduced overhead when disabling localization, both in compilation time
and binary size.
Disadvantages of this approach:
- Our own runtime implementation of gettext needs to be maintained.
- The implementation has a more limited feature set (but I don't think
it lacks any features which have been in use by fish).
Part of #11726Closes#11583Closes#11725Closes#11683
On a system where _CS_PATH is not defined (tested by removing that
code path), we get:
$ cargo b && env -u PATH HOME=$PWD target/debug/fish -c 'set -S PATH'
$PATH[1]: |.local//bin|
$PATH[2]: |/usr/bin|
$PATH[3]: |/bin|
The relative $PATH[1] makes no sense; probably it's an
accident. Restore the traditional $PATH[1]=/usr/local/bin.
Commit bf65b9e3a7 (Change `gettext` paths to be relocatable (#11195),
2025-03-30) made the locale directory (/usr/local/share/locale)
relocatable as well, so that "mv /usr/local /usr/local2" would not
break translations.
But this introduces a weird circular dependency, which was probably
the reason why the locale directory hadn't been relocatable:
1. option parsing might fail and print error messages, which should
be localized, hence require detection of config paths
2. detection of config paths calls `FLOG`, which depends on options
parsing (e.g. "-d config -o /tmp/log")
Since commit bf65b9e3a7, fish initializes the config paths
lazily, as soon as needed by translations.
When initializing config paths, we produce logs. The logs are off by
default so its' fine in practical cases, but technically we should only
log things after we have handled things like FISH_DEBUG_OUTPUT.
Here's an example where the config directory initialization sneakily
injected by an error message's "wgettext_fmt!" causes logs to be
printed to the wrong file spuriously:
$ FISH_DEBUG='*' FISH_DEBUG_OUTPUT=/tmp/log build/fish --unknown-arg
config: exec_path: "./build/fish", argv[0]: "./build/fish"
config: paths.sysconf: ./etc
config: paths.bin: ./build
config: paths.data: ./share
config: paths.doc: ./user_doc/html
config: paths.locale: ./share/locale
fish: --unknown-arg: unknown option
Now we could handle "-d config", "-o", and FISH_DEBUG later, but
that would mean that in this example we don't get any logs at all,
which doesn't seem correct either.
Break the circular dependency by determining config paths earlier,
while making sure to log the config path locations only after parsing
options, because options might affect whether we want to log the
"config" category.
The global variable is only needed for locale, so use explicit argument
passing for everything else, as before.
While at it, make other binaries (fish_key_reader, fish_indent) use
the same localization logic as fish. This means that we need to tweak
the «ends_with("bin/fish")» check.
Closes#11785
Commit 8b102f2571 (Stop using Cargo's OUT_DIR,
2025-06-22) accidentally removed canonicalization of
FISH_BUILD_DIR=${CMAKE_BINARY_DIR}. This means that if the path to
${CMAKE_BINARY_DIR} includes a symlink, ${CMAKE_BINARY_DIR}/fish will
wrongly use /usr/share/fish instead of ${CARGO_MANIFEST_DIR}/share.
Fix this and reintroduce the comment.
Building man pages takes significant time due to Sphinx running for several
seconds, even when no updates are required. Previously, we added custom logic to
avoid calling `sphinx-build` if the inputs to `sphinx-build` had not changed
since a cached timestamp.
By moving this into its own crate, we can tell cargo to rebuild when the input
files changed and unrelated changes will have no effect on this crate. This
allows us to get rid of the custom code for tracking whether to recompile, while
keeping the effect of only calling `sphinx-build` when appropriate.
In order to avoid code duplication, a new `build-helper` crate is added,
which contains some functionality for use in `build.rs`.
Closes#11737
Cargo tracks normal Rust dependencies and automatically figures out if changes
to Rust source files, `Cargo.{toml,lock}`, and `build.rs` necessitate a rebuild,
and if so of what. In some cases these checks are smarter than just comparing
file modification times, so not specifying such paths explicitly can reduce the
amount of rebuilding which happens, without skipping necessary rebuilding.
ja: this reverts b2aaf1db52 (Rebuild if src changed, 2025-03-28) and
460b93a (Rebuild on changes relevant to build artifacts, 2025-03-30)
which tried to fix#11332 ("moving Git HEAD does not invalidate cargo
build results"). But that expectation is overbearing. It's better
to only rebuild if something else of relevance to the build output
has changed.
Closes#11736
With an increasing number of local dependencies, the repo root is getting
somewhat bloated. This commit moves the two current local dependencies into the
newly created `crates` directory, with the intention of using it for all future
local dependencies as well.
Some dependencies which are introduced by currently in-progress pull requests
will need modifications in order for relative paths to work correctly.
This allows having the proc macro crate as an optional dependency and speeds up
compilation in situations where `FISH_GETTEXT_EXTRACTION_FILE` changes, such as
the `build_tools/check.sh` script. Because we don't need to recompile on changes
to the environment variable when the feature is disabled, cargo can reuse
earlier compilation results instead of recompiling everything.
This speeds up the compilation work in `build_tools/check.sh` when no changes
were made which necessitate recompilation.
For such runs of `build_tools/check.sh`, these changes reduce the runtime on my
system by about 10 seconds, from 70 to 60, approximately.
The difference comes from the following two commands recompiling code without
the changes in this commit, but not with them:
- `cargo test --doc --workspace`
- `cargo doc --workspace`
Despite the caching in `sphinx-build`, it takes several seconds to run
`sphinx-build` when no rebuilding is necessary, which slows down build times
significantly.
Add custom logic to `build.rs` to avoid calling `sphinx-build` if deemed
unnecessary based on the mtime of the source files.
This is done by writing the most recent timestamp of the source files into a
dedicated file, and only calling `sphinx-build` (and updating the timestamp)
when a cached timestamp is older than the most recent source file mtime.
The sources for Sphinx documentation builds include `CHANGELOG.rst` and
`CONTRIBUTING.rst`. Use `SPHINX_DOC_SOURCES` to clarify this and avoid
repetition.
Rebuilds should happen for debug builds as well, since rebuilding is required
for updating the man files.
If cargo ever wants to write to "target/man", it would collide with our
use of this path. Let's make this less likely by prefixing the name with
"fish-". This also makes it more obvious that this is fish's invention.
