This makes them more general than the previous versions which expected
`&wstr`. It comes at the cost of additional eager calls to `chars()`.
To implement `appendln` without having to call `append` twice, implement
`IntoCharIter` for chained iterators whose elements are both the kind of
iterator specified by `IntoCharIter`.
Because `IntoCharIter` is not implemented for owned types to avoid
allocations, some call sites which used to pass `WString` need to be
updated to pass references instead, which constitutes the bulk of the
changes in this commit.
Part of #12396
- Don't use `WString::from_str` for `str`s which are available at
compile-time. Use `L!(input).to_owned()` instead. The reason for this
is that `WString::from_str` can cause problems if the input contains
PUA bytes which we use for our custom encoding scheme. In such cases,
`bytes2wcstring` should be used, to avoid problems when decoding the
`WString`. Removing harmless usages of `WString::from_str` allows us
to focus on the potentially dangerous ones which don't convert
`str`'s that are compiled into the binary.
- Make `cstr2wcstring` actually take `CStr` as its input. The former
version was only used in one place, and the conversion to `CStr`
should happen there, where it can be checked that the conversion makes
sense and is safe. The new version is used in
`src/env/environmant.rs`, to avoid `to_bytes()` calls cluttering the
code there.
- Add `osstr2wcstring` function. This function also works for `Path`.
Now, these types can be converted to widestrings with much less
syntactic clutter.
There was only a single usage of `EMPTY_STRING`, and there it was
immediately dereferenced, so use an empty static `&wstr` instead and
remove the `WString` constant.
Closes#12309
Rust 1.80 stabilized `std::sync::LazyLock`, which replaces
`once_cell::sync::Lazy`. There is one exception in
`src/env_dispatch.rs`, which still uses the `once_cell` variant, since
the code there relies on `Lazy::get`, which also exists for `LazyLock`,
but will only be stabilized in Rust 1.94, so we can't use it yet.
Part of #12289
Rust has this annoying design where all of the syscall conveniences on
File assume that it owns its fd; in particular this means that we can't
easily construct File from stdin, a raw file descriptor, etc.
The usual workarounds are to construct a File and then mem::forget it
(this is apparently idiomatic Rust!). But this has problems of its own:
for example it can't easily be used in Drop.
Introduce BorrowedFdFile which wraps File with ManuallyDrop and then
never drops the file (i.e. it's always forgotten). Replace some raw FDs
with BorrowedFdFile.
Change the input of some functions to take `impl IntoCharIter`, allowing
them to accept more input. Implementing this efficiently means that no
owned types should be passed into these functions, because their
`IntoCharIter` implementation would require unnecessary allocations.
Instead, convert the uses which previously passed `WString` by prefixing
an `&`, so the borrowed `&WString` is passed instead.
To allow for wider use of the modified functions, `IntoCharIter`
implementations are added for `&String`, `&Cow<str>`, and `&Cow<wstr>`.
Closes#12207
This command
echo $(/bin/echo -n 1; echo -n 2)
sometimes outputs "21" because we implement this as
let bufferfill = IoBufferfill::create_opts(...);
...
let eval_res = parser.eval_with(...);
let buffer = IoBufferfill::finish(bufferfill);
i.e. /bin/echo and builtin echo both output to the same buffer; the
builtin does inside parser.eval_with(), and the external process may
or may not output before that, depending on when the FD monitor thread
gets scheduled (to run item_callback).
(Unrelated to that we make sure to consume all available input in
"IoBufferfill::finish(bufferfill)" but that doesn't help with
ordering.)
Fix this by reading all available data from stdout after the child
process has exited.
This means we need to pass the BufferFill down to
process_mark_finished_children().
We don't need to do this for builtins like "fg" or "wait",
because commands that buffer output do not get job control, see
2ca66cff53 (Disable job control inside command substitutions,
2021-07-26).
We also don't need to do it when reaping from reader because there
should be no buffering(?).
fish still deviates from other shells in that it doesn't wait for
it's child's stdout to be closed, meaning that this will behave
non-deterministically.
fish -c '
echo -n $(
sh -c " ( for i in \$(seq 10000); do printf .; done ) & "
)
' | wc -c
We should fix that later.
Closes#12018
The existing functionality of converting a `&wstr` to bytes (unescaping
PUA codepoints) and writing these to a file descriptor can be reused for
Rust's built-in strings by making the input type generic. This is
simple, because the only functionality we need is converting the input
into a `char` iterator, which is available for both types.
While at it, rename the functions to have more accurate and informative
names.
Closes#12206
Having the prelude in wchar is not great. The wchar module was empty
except for the prelude, and its prelude included things from wutil.
Having a top-level prelude module in the main crate resolves this. It
allows us to completely remove the wchar module, and a top-level prelude
module makes more sense conceptually. Putting non-wchar things into the
prelude also becomes more sensible, if we ever want to do that.
Closes#12182
functions/help and completions/help duplicate a lot of information
from doc_src. Get this information from Sphinx.
Drop short section titles such as "help globbing" in favor of the
full HTML anchor:
help language#wildcards-globbing
I think the verbosity is no big deal because we have tab completion,
we're trading in conciseness for consistency and better searchability.
In future, we can add back shorter invocations like "help globbing"
(especially given that completion descriptions often already repeated
the anchor path), but it should be checked by CI.
Also
- Remove some unused Sphinx anchors
- Remove an obsoleted script.
- Test that completions are in sync with Sphinx sources.
(note that an alternative would be to check
in the generated help_sections.rs file, see
https://internals.rust-lang.org/t/how-fail-on-cargo-warning-warnings-from-build-rs/23590/5)
Here's a list of deleted msgids. Some of them were unused, for others
there was a better message (+ translation).
$variable $variable 变量
(command) command substitution (命令) 命令替换
< and > redirections < 和 > 重定向
Autoloading functions 自动加载函数
Background jobs 后台作业
Builtin commands 内建命令
Combining different expansions 合并不同的展开
Command substitution (SUBCOMMAND) 命令替换 (子命令)
Defining aliases 定义别名
Escaping characters 转义字符
Help on how to reuse previously entered commands 关于如何重复使用先前输入的命令的帮助
How lists combine 列表如何组合
Job control 作业控制
Local, global and universal scope 局域、全局和通用作用域
Other features 其他功能
Programmable prompt 可编程提示符
Shell variable and function names Shell 变量和函数名
Some common words 一些常用词
The status variable 状况变量
Variable scope for functions 函数的变量作用域
Vi mode commands Vi 模式命令
What set -x does `set -x` 做什么
Writing your own completions 自己写补全
ifs and elses if 和 else
var[x..y] slices var[x..y] 切片
{a,b} brace expansion {a,b} 大括号展开
~ expansion ~ 展开
Closes#11796
As mentioned in 6896898769 (Add [lints] table to suppress lints
across all our crates, 2024-01-12), we can use workspace lints in
Cargo.toml now that we have MSRV >=1.74 and since we probably don't
support building without cargo.
This implies moving some lints from src/lib.rs to "workspace.lints".
While at it, address some of them insrtead.
Some string handling functions deal with `Vec<u8>` or `&[u8]`, which
have been referred to as `string` or `str` in the function names. This
is confusing, since they don't deal with Rust's `String` type. Use
`bytes` in the function names instead to reduce confusion.
Closes#11969
This commit adds `style_edition = "2024"` as a rustfmt config setting.
All other changes are automatically generated by `cargo fmt`.
The 2024 style edition fixes several bugs and changes some defaults.
https://doc.rust-lang.org/edition-guide/rust-2024/rustfmt-style-edition.html
Most of the changes made to our code result from a different sorting
method for `use` statements, improved ability to split long lines, and
contraction of short trailing expressions into single-line expressions.
While our MSRV is still 1.70, we use more recent toolchains for
development, so we can already benefit from the improvements of the new
style edition. Formatting is not require for building fish, so builds
with Rust 1.70 are not affected by this change.
More context can be found at
https://github.com/fish-shell/fish-shell/issues/11630#issuecomment-3406937077Closes#11959
Length modifiers are useless. This simplifies the code a bit, results in
more consistency, and allows removing a few PO messages which only
differed in the use of length modifiers.
Closes#11878
Our use of the terminfo database in /usr/share/terminfo/$TERM is both
1. a way for users to configure app behavior in their terminal (by
setting TERM, copying around and modifying terminfo files)
2. a way for terminal emulator developers to advertise support for
backwards-incompatible features that are not otherwise easily observable.
To 1: this is not ideal (it's very easy to break things). There's not many
things that realistically need configuration; let's use shell variables
instead.
To 2: in practice, feature-probing via terminfo is often wrong. There's not
many backwards-incompatible features that need this; for the ones that do
we can still use terminfo capabilities but query the terminal via XTGETTCAP
directly, skipping the file (which may not exist on the same system as
the terminal).
---
Get rid of terminfo. If anyone finds a $TERM where we need different behavior,
we can hardcode that into fish.
* Allow to override this with `fish_features=no-ignore-terminfo fish`
Not sure if we should document this, since it's supposed to be removed soon,
and if someone needs this (which we don't expect), we'd like to know.
* This is supported on a best-effort basis; it doesn't match the previous
behavior exactly. For simplicity of implementation, it will not change
the fact that we now:
* use parm_left_cursor (CSI Ps D) instead of cursor_left (CSI D) if
terminfo claims the former is supported
* no longer support eat_newline_glitch, which seems no longer present
on today's ConEmu and ConHost
* Tested as described in https://github.com/fish-shell/fish-shell/pull/11345#discussion_r2030121580
* add `man fish-terminal-compatibility` to state our assumptions.
This could help terminal emulator developers.
* assume `parm_up_cursor` is supported if the terminal supports XTGETTCAP
* Extract all control sequences to src/terminal_command.rs.
* Remove the "\x1b(B" prefix from EXIT_ATTRIBUTE_MODE. I doubt it's really
needed.
* assume it's generally okay to output 256 colors
Things have improved since commit 3669805627 (Improve compatibility with
0-16 color terminals., 2016-07-21).
Apparently almost every actively developed terminal supports it, including
Terminal.app and GNU screen.
* That is, we default `fish_term256` to true and keep it only as a way to
opt out of the the full 256 palette (e.g. switching to the 16-color
palette).
* `TERM=xterm-16color` has the same opt-out effect.
* `TERM` is generally ignored but add back basic compatiblity by turning
off color for "ansi-m", "linux-m" and "xterm-mono"; these are probably
not set accidentally.
* Since `TERM` is (mostly) ignored, we don't need the magic "xterm" in
tests. Unset it instead.
* Note that our pexpect tests used a dumb terminal because:
1. it makes fish do a full redraw of the commandline everytime, making it
easier to write assertions.
2. it disables all control sequences for colors, etc, which we usually
don't want to test explicitly.
I don't think TERM=dumb has any other use, so it would be better
to print escape sequences unconditionally, and strip them in
the test driver (leaving this for later, since it's a bit more involved).
Closes#11344Closes#11345
[Do NOT cherry-pick to 4.0 - this needs more time to be tested]
fish sometimes needs to capture the output of a command or block of
commands. Examples include fish_prompt or any command substitution
("cmdsubs"). It does this the obvious way: by creating a pipe, using dup2
to replace stdout of the command with the write end of the pipe, and then
reading from the read end into a buffer, until EOF or the command
substitution completes. Importantly, this task also overlaps with waiting
for the process to exit; that is when executing:
set var (some_cmd)
fish needs to both wait on `some_cmd` and ALSO read its output into memory.
This is awkward to do in a portable way in a single thread (though maybe
doable on Linux with pidfd). So we wait and read on different threads.
To make things worse, command substitutions may themselves create
additional command substitutions (recursion, etc). Creating a read thread
for every command substitution would result in excessive threads. So rather
than a thread per cmdsub, we have a single dedicated thread that handles
ALL command substitutions, by multiplexing multiple file descriptors via
select/poll. This is the "fd monitor." You hand it a file descriptor and it
lets you know when it's readable, and then you can read from it (via a
callback). Also, it has a "wakeup" fd: if you write to that then the fd
monitor wakes up, figures out what it has to do, and resumes.
When the command substitution ends, we need to remove the fd from the fd
monitor, because we intend to close it. You might object "the commands in
the cmdsub have all completed so the write end of the pipe has been closed
so the fd monitor can just notice that the pipe is closed" but it's not so:
consider the horrible case of `set var (yes &)` and abandon all hope.
The current mechanism for removing the fd from the monitor is called a
"poke." We tell the fd monitor (through a "control" self-pipe) to
explicitly wake up the item. It then invokes the callback ("pokes") the
item on the dedicated fd monitor thread. The item notices that the command
substitution is complete, and it returns a value meaning "remove me" and
the fd monitor does so. The client thread is stuck waiting for this process
to complete.
So basically removing a fd from the monitor requires a round trip to its
dedicated thread. This is slow and also complicated (Rust doesn't have
futures)!
So let's not do that.
The big idea is to remove this round-trip synchronization. That is, when we
intend to remove the fd from the fd monitor, we _just do it_ and then close
the fd. Use a lock rather than a round-trip to the thread. Crucially that
lock is unlocked while the monitor thread waits in select/poll.
This invites all sorts of races:
1. fish might remove and close the fd right before the monitor polls it. It
will thus attempt to poll a closed fd.
2. fish might remove and close the fd, and then something else opens a file
and receives the same fd. Now the fd monitor will poll an fd that was
never added.
3. fish might remove and close the fd _while the fd monitor is polling it_.
What happens then? (Turns out on macOS we get EBADF, and on Linux the fd is
marked readable).
The Big Idea is that *all of these races are benign*. As long as
poll/select doesn't crash or hang, we don't care *what* it returns, because
the source of truth are the set of items stored in the fd monitor and these
item IDs are never recycled. (This also assumes that it's OK to select/poll
on random file descriptors; there ought to be no side effects).
Not only is this a large simplification since we no longer need that round
trip, it's a substantial performance improvement as well. The
"aliases.fish" benchmark goes from 164 to 154 msec on my Mac, and from 124
to 112 msec on my Linux machine - nearly 10%.
Add some tests to verify our assumptions about the behavior of closing or
replacing a file descriptor during poll. But even if these fail, all we
care about is that poll/select doesn't crash or hang.
FdMonitor is used to monitor a set of file descriptors and invoke a callback
when one becomes readable. Prior to this commit, they coudl also have the
callback invoked on timeout. fish used to use this feature but no longer does;
remove it.
This tries to open the given file to use as stdin, and if it fails,
for any reason, it uses /dev/null instead.
This is useful in cases where we would otherwise do either of these:
```fish
test -r /path/to/file
and string match foo < /path/to/file
cat /path/to/file 2>/dev/null | string match foo
```
This both makes it nicer and shorter, *and* helps with TOCTTOU - what if the file is removed/changed after the check?
The reason for reading /dev/null instead of a closed fd is that a closed fd will often cause an error.
In case opening /dev/null fails, it still skips the command.
That's really a last resort for when the operating system
has turned out to be a platypus and not a unix.
Fixes#4865
(cherry picked from commit df8b9b7095)
`intersects()` is "any of" while `contains()` is "all of" and while it makes no
difference when testing a single bit, I believe `contains()` is less brittle
for future maintenance and updates as its meaning is clearer.
</pedantic>
The lines of code I commented on in #10254 were meant to serve only as examples
of the changes I was requesting, not the only instances.
Also just use `Mode::from_bits_truncate()` instead of unsafe or unwrapping since
we know the modes are correct.
