Make use of the new `Error::from_fluent_message` function.
This message in particular was chosen to demonstrate Fluent's ability to
use variables in arbitrary order. The suggestion in
https://github.com/fish-shell/fish-shell/pull/11833#issuecomment-3343835252
to allow reordering variables was one of the reasons for switching to a
more flexible localization system. `zh_CN.ftl` has been modified
accordingly. The effect can be seen with
```fish
for LC_MESSAGES in fr zh_CN zh_TW
argparse h-
end
```
Part of #11928
This migrates the fish version info message from gettext to Fluent. It
can be used to see Fluent-based localization in action.
Because this commit adds new FTL files, these languages show up in the
Fluent language precedence, requiring an update to the corresponding
tests.
Part of #11928
Add an implementation allowing to use Fluent for localization in Rust.
Fluent is significantly more expressive than gettext. It uses message
IDs which, unlike in gettext, are not necessarily the default message
string. This allows for proper support of messages which happen to be
identical in English, but not in other languages. In gettext, this could
be solved to some extent with contexts, but our gettext implementation
does not support that. In Fluent, arguments to the message are specified
as key-value pairs, which gives translators more semantic information
and allows reordering the arguments in the translation, which is
impossible with gettext. Fluent also allows for more complex grammatical
features, such as different plural forms, grammatical cases, and
adapting phrases to the correct gender.
This commit only introduces the infrastructure for using Fluent instead
of gettext, with the goal of eventually replacing gettext for
localization in Rust. Making use of the new infrastructure is left to
follow-up commits.
To localize a message with Fluent, the new `localize!` macro should be
used. Its arguments are key-value pairs. The first pair must consist of
two string literals. The key is a Fluent message ID and the value the
corresponding definition of the message in English. It must be valid
Fluent syntax for a message definition. The remaining key-value pairs
are used for Fluent variables which appear in the message definition.
Each key must match the name of a variable present in the message
definition, and the value is some Rust value which can be displayed by
Fluent. The variable in the Fluent message definition will be replaced
by that value. Example syntax:
```rust
localize!(
"fish-version" = "{ $package_name }, version { $version }",
package_name = "fish",
version = crate::BUILD_VERSION,
)
````
If a message should be available in more than one place, define a
function containing the `localize!` macro and call that function from
the different locations needing access to the message, instead of
putting multiple `localize!` macros with the same message ID into the
code. The `localize_fn!` macro can help with that.
By having the message definition in the Rust sources, we have all the
relevant information for showing English messages in Rust, without
needing to rely on external sources. However, the Fluent library expects
data for all languages in the FTL format, so we have tooling to
automatically generate `en.ftl` from the Rust sources. Having this file
is also useful for other Fluent tooling, as well as being able to track
modifications to messages, which we also have tooling for.
To add translations for a new language, translators can create a new FTL
file in the `localization/fluent` directory with the appropriate file
name. No additional setup is needed. Translations for an individual
message can be added by copying the message from `en.ftl` and adjusting
the definition for the respective language.
These FTL files are included via `rust-embed` and parsed on demand at
runtime, ensuring that only languages specified in the user's language
settings are considered, saving the effort or parsing unneeded files.
`en.ftl` is always parsed, as it implicitly is the last fallback
option, when no other language in the user's precedence list has a
translation for a message ID. We know that `en.ftl` contains a
translation for all message IDs we use because message IDs can only be
used by putting them into the `localize!` macro with a corresponding
English message definition, and `en.ftl` is auto-generated from these
definitions. This generation happens as a 2-step process: First, we
compile with the `fluent-extract` feature, with the
`FISH_FLUENT_EXTRACTION_DIR` environment variable set to point to an
empty directory. This will result in the `localize!` macro passing the
message ID, definition, and variable names specified in subsequent
key-value pairs, to a proc macro which performs some checks on the
provided data and writes the message definition in the FTL format to a
fresh file in `FISH_FLUENT_EXTRACTION_DIR`. This results in one file per
`localize!` macro invocation. In the second step, the data from all
these files is combined into a single file, `en.ftl`, which is
automatically formatted. Before overwriting the old `en.ftl`, checks are
performed to look for changes to message definitions. If any change is
detected, translations of the message might have to be updated.
Some changes, like removal of a message, can be handled automatically,
by deleting all translations. For changes which cannot be handled fully
automatically, the affected translations are marked with an annotation,
which causes our checks to fail while it is present. It is the
responsibility of the developer who changed the message definition to
handle updates to its translations. This can be done manually, by
editing the affected translations and removing the annotation, which
should be the preferred option when translations need to be modified and
the developer knows the language. Otherwise, there is automation in the
form of `cargo xtask fluent resolve-outdated`, which allows specifying
what should happen to some or all translations of a message, with the
available option being:
- delete the translation, used when the meaning of the English message
changed so much that keeping the old translations would be misleading
- delete the annotation, used when changes to the English message have
no impact on translations, e.g. for typo fixes
- change the annotation to indicate that translator review is desired,
used when the meaning or wording of the English message changed
slightly, but not enough to invalidate the translations
There are several tools available as subcommands of `cargo xtask
fluent`:
- `check`: Checks the FTL files, ensuring that they can be parsed
without errors, that no duplicate IDs are specified, that they are
formatted correctly, and that there are no extra IDs, i.e. IDs not
present in `en.ftl`, which is expected to be complete. More rigorous
checks could be added, such as checking whether the same set of
variables are used for a certain ID in all languages. The complexity
of Fluent's syntax makes this non-trivial, which is the reason it's
not already implemented.
- `format`: Formats the specified FTL files (or all by default). Also
has a mode suitable for editor integration to format files from the
editor. Examples for setting that up in Vim are provided in the
`CONTRIBUTING.rst` docs.
- `rename`: Renames IDs or associated variables across all FTL files.
- `resolve-outdated`: Described above.
- `show-missing`: Shows which IDs don't have a translation yet.
- `update`: Runs the generation pipeline for `en.ftl` and potential
translation updates/annotations.
There is one additional tool, which is designed to help with porting
existing messages localized via gettext to the new Fluent
implementation. This is intentionally not added to fish, because it
is only useful for the transition. Once we have ported all messages to
Fluent we won't have a use for it anymore. If you are interested in
using it to port messages, it's the `po-convert` binary in the
`fluent-ftl-tools` package. The CLI is somewhat convoluted, but can be
simplified by wrapping it with a script which hard-codes the path to the
relevant PO and FTL file directories. Then, the remaining information
which needs to be specified is:
- a line number in a PO file to identify the message to be ported
- the new message ID
- the name of each variable, in the order the formatting specifiers
appear in the gettext msgid.
Specifying the line number and invoking the wrapper script can be
partially automated by using a custom editor shortcut.
The tool will port the msgstr for each language which has one defined,
and always generate and entry for `en.po` based on the msgid.
The tool does not edit Rust code, but suggests a Rust code snippet on
stdout based on the specified message ID and variable names.
Some of our tooling relies on features of the `fluent` package which
are not exported by default, so we use a fork which changes that until
our PR for adding it upstream is accepted.
Part of #11928
Commit 2f6ed61833 (parse_util_cmdsubst_extent to return an exclusive
range, 2024-04-27) used the "Parentheses" naming in preparation for
raw quotes. We still don't have them, so let's clean that up for now.
Also, reuse the Result and Option types.
Add missing options, descriptions, y/n completions, and correct short
flags throughout. Drop --changelog (removed from portage). Fix several
wrong option names in stubs (--ignore-built-slot-operator-deps,
--rebuilt-binaries, --sync-submodule, etc.). Use -xa for mandatory
boolean args and -a for optional ones.
Closes#12810
Add a test for the following behavior: if we try to `cd` to a directory,
and it fails, and so we call realpath and discover that our $PWD is stale,
and correct it and then try to `cd` again, and it STILL fails, then:
1. We report the error for the second (better) failure.
2. $PWD is not modified.
It does not make sense to change these.
If anything, we should make them computed electric variable, so they
change whenever one of $HOME, $XDG_CONFIG_HOME and friends change.
Make them electric variables, purely because that's our only
way of marking variables as read-only. In future we can add a
EnvVarFlags::READ_ONLY instead. We'll want this anyway for "set
--read-only".
As of 7640e95bd7 (Create user config file/directories only on
first startup again, 2025-12-29) we interpret absence of the
__fish_initialized variable as "this is the first run, so create
~/.config/fish/conf.d etc.".
As reported in #11226, the presence of this universal variable causes
friction to users who track ~/.config/fish/fish_variables in version
control.
Also, __fish_initialized is the only universal variable we create
by default.
Use another way to detect the first run: since we already create
~/.config/fish on every run, assume that a successful mkdir() means
we should also create the subdirectories.
This has false negatives (if the user already created the directory)
and false positives (if the user doesn't want ~/.config/fish to exist)
but at least the latter should not really matter because historically
we always created it, at least for ~/.config/fish/fish_variables.
Alternatively, we could create a file at ~/.cache/fish/first-run-done.
But let's not add more state unless there's a good reason.
This was added in d1fd3d5825 (Detect at startup whether config and
data paths are remote, 2021-05-08); I don't understand why
this would be necessary. On my system, statfs("/tmp") and
statfs("/tmp/") both yields f_type=TMPFS_MAGIC.
In addition to the entry in ELECTRIC_VARIABLES, each computed electric
variable has a piece of code in "try_get_computed()". Makes more
sense to store this directly in the electric variable itself, no?
use consts for the getter functions, so we can reuse the type.
This is a bit weird..
try_get_computed() returns None if the variable is not computed.
That's the case for "$history" on (future) background threads.
This is a weird edge case, because we don't ever return None for any
other electric variable. This will become especially apparent when
an upcoming commit adds the computing-function to ElectricVar.
Given that the main thread already can't use "$history" from being
used as general purpose variable, there seems to be little harm in
banning it for all of them.
parse_escape_sequence() can self-recurse once, to parse sequences of
the form "\e\e...", where we attempt to interpret the first byte as
either legacy alt (if the "\e..." suffix decodes to a key), or legacy
escape (if it doesn't).
On "\e\e]11;rgb:ffff/ffff/ffff\e\\", i.e. with the inner sequence
being an OSC which doesn't produce a key event, we (tragically)
stop parsing the sequence and assume legacy escape.
Rewrite the logic to keep parsing whole escape sequences whenever
possible. Also, don't drop any leading \e even if we later error.
Also, make sure each key event gets the correct "originating sequence".
This makes
tmux new-session fish -C '
bind alt-b "echo alt-b"
bind escape "echo escape"
bind alt-escape "echo alt-escape"
sleep 2
'
output the following after quickly pressing "escape" five times
followed by "b" one time:
escape
alt-escape
escape
alt-b
which is weird but fine because "alt-escape" is rarely used.
If this proves to be a problem, we could disable translation of "\e\e"
to "alt-escape"?
Closes#12379
Electric variables are largely an implementation detail, and a following
commit wants to add a circular dependency between the electric var
impl and environment_impl. Specifically, "ElectricVar" will know
about "EnvScopedImpl". Move both impls into a common impl module,
to make this dependency less surprising.
A following commit wants to make "on_byte_read()" potentially return
multiple CharEvent values, in case there is a legacy escape.
Today "parse_csi()" and friends simply insert into the input queue.
This makes it hard to get the ordering right when we return multiple
events.
Change the parsing functions to return their result rather than
queuing directly.
This directory is no longer since 0fafff2c89 (CMake: stop installing
embedded files, 2026-05-08). Instead of telling users to see "status
list-files", refer them to the source directly.
Gets rid of another use of XDG_CACHE_HOME; this gets us closer to
a single source of truth. Not super important right now, but will
be helpful if we allow users to override XDG_CACHE_HOME in future.
I guess we could also remove it from create_manpage_completions.py,
that's unnecessary breakage but probably fine since this is not
public-facing.
A following commit wants to inject mock input bytes in unit tests.
Allow this in a quick-and-dirty way by making the input-byte-yielding an
overridable trait method again. Unfortunately this needs to be in the
shared trait; we can't specialize a private trait since specialization
is not supported in stable Rust.
