Release 3.1.2

Commit 5fccfd83ec, with the fix for #6806,
switched eval to buffer its output (like other builtins do). But this
prevents using eval with commands that wants to see the tty, especially
fzf. So only buffer the output if the output is piped to the next process.

CHANGELOG.md

@@ -1,3 +1,69 @@
+# fish 3.1.2 (released April 29, 2020)
+
+This release of fish fixes a major issue discovered in fish 3.1.1:
+
+- Commands such as `fzf` and `enhancd`, when used with `eval`, would hang. `eval` buffered output too aggressively, which has been fixed (#6955).
+
+If you are upgrading from version 3.0.0 or before, please also review the release notes for 3.1.1, 3.1.0 and 3.1b1 (included below).
+
+---
+
+# fish 3.1.1 (released April 27, 2020)
+
+This release of fish fixes a number of major issues discovered in fish 3.1.0.
+
+- Commands which involve `. ( ... | psub)` now work correctly, as a bug in the `function --on-job-exit` option has been fixed (#6613).
+- Conflicts between upstream packages for ripgrep and bat, and the fish packages, have been resolved (#5822).
+- Starting fish in a directory without read access, such as via `su`, no longer crashes (#6597).
+- Glob ordering changes which were introduced in 3.1.0 have been reverted, returning the order of globs to the previous state (#6593).
+- Redirections using the deprecated caret syntax to a file descriptor (eg `^&2`) work correctly (#6591).
+- Redirections that append to a file descriptor (eg `2>>&1`) work correctly (#6614).
+- Building fish on macOS (#6602) or with new versions of GCC (#6604, #6609) is now successful.
+- `time` is now correctly listed in the output of `builtin -n`, and `time --help` works correctly (#6598).
+- Exported universal variables now update properly (#6612).
+- `status current-command` gives the expected output when used with an environment override - that is, `F=B status current-command` returns `status` instead of `F=B` (#6635).
+- `test` no longer crashes when used with "`nan`" or "`inf`" arguments, erroring out instead (#6655).
+- Copying from the end of the command line no longer crashes fish (#6680).
+- `read` no longer removes multiple separators when splitting a variable into a list, restoring the previous behaviour from fish 3.0 and before (#6650).
+- Functions using `--on-job-exit` and `--on-process-exit` work reliably again (#6679).
+- Functions using `--on-signal INT` work reliably in interactive sessions, as they did in fish 2.7 and before (#6649). These handlers have never worked in non-interactive sessions, and making them work is an ongoing process.
+- Functions using `--on-variable` work reliably with variables which are set implicitly (rather than with `set`), such as "`fish_bind_mode`" and "`PWD`" (#6653).
+- 256 colors are properly enabled under certain conditions that were incorrectly detected in fish 3.1.0 (`$TERM` begins with xterm, does not include "`256color`", and `$TERM_PROGRAM` is not set) (#6701).
+- The Mercurial (`hg`) prompt no longer produces an error when the current working directory is removed (#6699). Also, for performance reasons it shows only basic information by default; to restore the detailed status, set `$fish_prompt_hg_show_informative_status`.
+- The VCS prompt, `fish_vcs_prompt`, no longer displays Subversion (`svn`) status by default, due to the potential slowness of this operation (#6681).
+- Pasting of commands has been sped up (#6713).
+- Using extended Unicode characters, such as emoji, in a non-Unicode capable locale (such as the `C` or `POSIX` locale) no longer renders all output blank (#6736).
+- `help` prefers to use `xdg-open`, avoiding the use of `open` on Debian systems where this command is actually `openvt` (#6739).
+- Command lines starting with a space, which are not saved in history, now do not get autosuggestions. This fixes an issue with Midnight Commander integration (#6763), but may be changed in a future version.
+- Copying to the clipboard no longer inserts a newline at the end of the content, matching fish 2.7 and earlier (#6927).
+- `fzf` in complex pipes no longer hangs. More generally, code run as part of command substitutions or `eval` will no longer have separate process groups. (#6624, #6806).
+
+This release also includes:
+- a number of changes to improve macOS compatibility with code signing and notarization;
+- a number of improvements to completions; and
+- a number of content and formatting improvements to the documentation.
+
+If you are upgrading from version 3.0.0 or before, please also review the release notes for 3.1.0 and 3.1b1 (included below).
+
+## Errata for fish 3.1
+
+A new builtin, `time`, was introduced in the fish 3.1 releases. This builtin is a reserved word (like `test`, `function`, and others) because of the way it is implemented, and functions can no longer be named `time`. This was not clear in the fish 3.1b1 changelog.
+
+---
+
+# fish 3.1.0 (released February 12, 2020)
+
+Compared to the beta release of fish 3.1b1, fish version 3.1.0:
+- fixes a regression where spaces after a brace were removed despite brace expansion not occurring (#6564)
+- fixes a number of problems in compiling and testing on Cygwin (#6549) and Solaris-derived systems such as Illumos (#6553, #6554, #6555, #6556, and #6558); 
+- fixes the process for building macOS packages;
+- fixes a regression where excessive error messages are printed if Unicode characters are emitted in non-Unicode-capable locales (#6584); and
+- contains some improvements to the documentation and a small number of completions.
+
+If you are upgrading from version 3.0.0 or before, please also review the release notes for 3.1b1 (included below).
+
+---
+
 # fish 3.1b1 (released January 26, 2020)
 
 ## Notable improvements and fixes

CMakeLists.txt

@@ -7,6 +7,8 @@ IF(POLICY CMP0067)
   CMAKE_POLICY(SET CMP0067 NEW)
 ENDIF()
 
+set(CMAKE_OSX_DEPLOYMENT_TARGET "10.9" CACHE STRING "Minimum OS X deployment version")
+
 PROJECT(fish)
 
 # We are C++11.
@@ -163,10 +165,26 @@ ADD_DEFINITIONS(-D_REENTRANT)
 # Set up PCRE2
 INCLUDE(cmake/PCRE2.cmake)
 
+# Code signing ID on Mac. A default '-' is ad-hoc codesign.
+SET(MAC_CODESIGN_ID "-" CACHE STRING "Mac code-signing identity")
+
+FUNCTION(CODESIGN_ON_MAC target)
+  IF(APPLE)
+    ADD_CUSTOM_COMMAND(
+      TARGET ${target}
+      POST_BUILD
+      COMMAND codesign --force --deep --options runtime --sign "${MAC_CODESIGN_ID}" $<TARGET_FILE:${target}>
+      VERBATIM
+    )
+  ENDIF()
+ENDFUNCTION(CODESIGN_ON_MAC target)
+
+
 # Define a function to link dependencies.
-FUNCTION(FISH_LINK_DEPS target)
+FUNCTION(FISH_LINK_DEPS_AND_SIGN target)
   TARGET_LINK_LIBRARIES(${target} fishlib)
-ENDFUNCTION(FISH_LINK_DEPS)
+  CODESIGN_ON_MAC(${target})
+ENDFUNCTION(FISH_LINK_DEPS_AND_SIGN)
 
 # Define libfish.a.
 ADD_LIBRARY(fishlib STATIC ${FISH_SRCS})
@@ -177,17 +195,17 @@ TARGET_LINK_LIBRARIES(fishlib
 
 # Define fish.
 ADD_EXECUTABLE(fish src/fish.cpp)
-FISH_LINK_DEPS(fish)
+FISH_LINK_DEPS_AND_SIGN(fish)
 
 # Define fish_indent.
 ADD_EXECUTABLE(fish_indent
                src/fish_indent.cpp src/print_help.cpp)
-FISH_LINK_DEPS(fish_indent)
+FISH_LINK_DEPS_AND_SIGN(fish_indent)
 
 # Define fish_key_reader.
 ADD_EXECUTABLE(fish_key_reader
                src/fish_key_reader.cpp src/print_help.cpp)
-FISH_LINK_DEPS(fish_key_reader)
+FISH_LINK_DEPS_AND_SIGN(fish_key_reader)
 
 # Set up the docs.
 INCLUDE(cmake/Docs.cmake)

build_tools/mac_notarize.sh

@@ -0,0 +1,99 @@
+#!/usr/bin/env bash
+
+# Helper to notarize an .app.zip or .pkg file.
+# Based on https://www.logcg.com/en/archives/3222.html
+
+set -e
+
+die() { echo "$*" 1>&2 ; exit 1; }
+
+check_status() {
+    echo "STATUS" $1
+}
+
+get_req_uuid() {
+    RESPONSE=$(</dev/stdin)
+    if echo "$RESPONSE" | egrep -q "RequestUUID"; then
+        echo "$RESPONSE" | egrep RequestUUID | awk '{print $3'}
+    elif echo "$RESPONSE" | egrep -q "The upload ID is "; then
+        echo "$RESPONSE" | egrep -p "The upload ID is [-a-z0-9]+" | awk '{print $5}'
+    else
+        die "Could not get Request UUID"
+    fi
+}
+
+INPUT=$1
+AC_USER=$2
+
+test -z "$AC_USER" && die "AC_USER not specified as second param"
+test -z "$INPUT" && die "No path specified"
+test -f "$INPUT" || die "Not a file: $INPUT"
+
+ext="${INPUT##*.}"
+(test "$ext" = "zip" || test "$ext" = "pkg") || die "Unrecognized extension: $ext"
+
+LOGFILE=$(mktemp -t mac_notarize_log)
+AC_PASS="@keychain:AC_PASSWORD"
+echo "Logs at $LOGFILE"
+
+NOTARIZE_UUID=$(xcrun altool --notarize-app \
+                --primary-bundle-id "com.ridiculousfish.fish-shell" \
+                --username "$AC_USER" \
+                --password "$AC_PASS" \
+                --file "$INPUT" 2>&1 |
+                    tee -a "$LOGFILE" |
+                    get_req_uuid)
+
+test -z "$NOTARIZE_UUID" && cat "$LOGFILE" && die "Could not get RequestUUID"
+echo "RequestUUID: $NOTARIZE_UUID"
+
+success=0
+for i in $(seq 20); do
+    echo "Checking progress..."
+    PROGRESS=$(xcrun altool --notarization-info "${NOTARIZE_UUID}" \
+                -u "$AC_USER" \
+                -p "$AC_PASS" 2>&1 |
+                    tee -a "$LOGFILE")
+    echo "${PROGRESS}" | tail -n 1
+
+    if [ $? -ne 0 ] || [[ "${PROGRESS}" =~ "Invalid" ]] ; then
+        echo "Error with notarization. Exiting"
+        break
+    fi
+
+    if ! [[ "${PROGRESS}" =~ "in progress" ]]; then
+        success=1
+        break
+    else
+        echo "Not completed yet. Sleeping for 30 seconds."
+    fi
+    sleep 30
+done
+
+if [ $success -eq 1 ] ; then
+    if test "$ext" = "zip"; then
+        TMPDIR=$(mktemp -d)
+        echo "Extracting to $TMPDIR"
+        unzip -q "$INPUT" -d "$TMPDIR"
+        # Force glob expansion.
+        STAPLE_TARGET="$TMPDIR"/*
+        STAPLE_TARGET=$(echo $STAPLE_TARGET)
+    else
+        STAPLE_TARGET="$INPUT"
+    fi
+    echo "Stapling $STAPLE_TARGET"
+    xcrun stapler staple "$STAPLE_TARGET"
+
+    if test "$ext" = "zip"; then
+        # Zip it back up.
+        INPUT_FULL=$(realpath "$INPUT")
+        rm -f "$INPUT"
+        cd "$(dirname "$STAPLE_TARGET")"
+        zip -r -q "$INPUT_FULL" $(basename "$STAPLE_TARGET")
+    fi
+fi
+echo "Processed $INPUT"
+
+if test "$ext" = "zip"; then
+    spctl -a -v "$STAPLE_TARGET"
+fi

build_tools/make_pkg.sh

@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/usr/bin/env bash
 
 # Script to produce an OS X installer .pkg and .app(.zip)
 
@@ -17,19 +17,23 @@ set -x
 #Exit on error
 set -e
 
+# Respect MAC_CODESIGN_ID and MAC_PRODUCTSIGN_ID, or default for ad-hoc.
+# Note the :- means "or default" and the following - is the value.
+MAC_CODESIGN_ID=${MAC_CODESIGN_ID:--}
+MAC_PRODUCTSIGN_ID=${MAC_PRODUCTSIGN_ID:--}
+
 PKGDIR=$(mktemp -d)
 
 SRC_DIR=$PWD
 OUTPUT_PATH=${FISH_ARTEFACT_PATH:-~/fish_built}
 
 mkdir -p "$PKGDIR/build" "$PKGDIR/root" "$PKGDIR/intermediates" "$PKGDIR/dst"
-{ cd "$PKGDIR/build" && cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo "$SRC_DIR" && make -j 4 && env DESTDIR="$PKGDIR/root/" make install; }
-pkgbuild --scripts build_tools/osx_package_scripts --root "$PKGDIR/root/" --identifier 'com.ridiculousfish.fish-shell-pkg' --version "$VERSION" "$PKGDIR/intermediates/fish.pkg"
-
-productbuild  --package-path "$PKGDIR/intermediates" --distribution build_tools/osx_distribution.xml --resources build_tools/osx_package_resources/ "$OUTPUT_PATH/fish-$VERSION.pkg"
-
+{ cd "$PKGDIR/build" && cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DMAC_CODESIGN_ID="${MAC_CODESIGN_ID}" "$SRC_DIR" && make -j 12 && env DESTDIR="$PKGDIR/root/" make install; }
+pkgbuild --scripts "$SRC_DIR/build_tools/osx_package_scripts" --root "$PKGDIR/root/" --identifier 'com.ridiculousfish.fish-shell-pkg' --version "$VERSION" "$PKGDIR/intermediates/fish.pkg"
+productbuild  --package-path "$PKGDIR/intermediates" --distribution "$SRC_DIR/build_tools/osx_distribution.xml" --resources "$SRC_DIR/build_tools/osx_package_resources/" "$OUTPUT_PATH/fish-$VERSION.pkg"
+productsign --sign "${MAC_PRODUCTSIGN_ID}" "$OUTPUT_PATH/fish-$VERSION.pkg" "$OUTPUT_PATH/fish-$VERSION-signed.pkg" && mv "$OUTPUT_PATH/fish-$VERSION-signed.pkg" "$OUTPUT_PATH/fish-$VERSION.pkg"
 
 # Make the app
-{ cd "$PKGDIR/build" && make fish_macapp && zip -r "$OUTPUT_PATH/fish-$VERSION.app.zip" fish.app; }
+{ cd "$PKGDIR/build" && make signed_fish_macapp && zip -r "$OUTPUT_PATH/fish-$VERSION.app.zip" fish.app; }
 
 rm -r "$PKGDIR"

cmake/MacApp.cmake

@@ -55,4 +55,21 @@ ADD_CUSTOM_COMMAND(TARGET fish_macapp POST_BUILD
             --build ${CMAKE_CURRENT_BINARY_DIR} --target install
     COMMAND ${CMAKE_COMMAND} -E copy_directory ${MACAPP_FISH_BUILDROOT}/..
             $<TARGET_BUNDLE_CONTENT_DIR:fish_macapp>/Resources/
+    VERBATIM
+)
+
+# The entitlements file.
+SET(MACAPP_ENTITLEMENTS "${CMAKE_SOURCE_DIR}/osx/MacApp.entitlements")
+
+# Target to sign the macapp.
+# Note that a POST_BUILD step happens before resources are copied,
+# and therefore would be too early.
+ADD_CUSTOM_TARGET(signed_fish_macapp
+    DEPENDS fish_macapp "${MACAPP_ENTITLEMENTS}"
+    COMMAND codesign --force --deep
+            --options runtime
+            --entitlements "${MACAPP_ENTITLEMENTS}"
+            --sign "${MAC_CODESIGN_ID}"
+            $<TARGET_BUNDLE_DIR:fish_macapp>
+    VERBATIM
 )

cmake/PCRE2.cmake

@@ -7,8 +7,10 @@ SET(PCRE2_BUILD_PCRE2GREP OFF CACHE BOOL "Build pcre2grep")
 
 
 SET(PCRE2_MIN_VERSION 10.21)
-FIND_LIBRARY(PCRE2_LIB pcre2-${PCRE2_WIDTH})
-FIND_PATH(PCRE2_INCLUDE_DIR pcre2.h)
+IF (NOT APPLE)
+  FIND_LIBRARY(PCRE2_LIB pcre2-${PCRE2_WIDTH})
+  FIND_PATH(PCRE2_INCLUDE_DIR pcre2.h)
+ENDIF()
 IF (PCRE2_LIB AND PCRE2_INCLUDE_DIR)
   MESSAGE(STATUS "Found system PCRE2 library ${PCRE2_INCLUDE_DIR}")
 ELSE()
@@ -16,5 +18,5 @@ ELSE()
   ADD_SUBDIRECTORY(pcre2-10.32 EXCLUDE_FROM_ALL)
   SET(PCRE2_INCLUDE_DIR ${CMAKE_BINARY_DIR}/pcre2-10.32/)
   SET(PCRE2_LIB pcre2-${PCRE2_WIDTH})
-endif(PCRE2_LIB AND PCRE2_INCLUDE_DIR)
+ENDIF(PCRE2_LIB AND PCRE2_INCLUDE_DIR)
 INCLUDE_DIRECTORIES(${PCRE2_INCLUDE_DIR})

cmake/Tests.cmake

@@ -1,7 +1,7 @@
 # Define fish_tests.
 ADD_EXECUTABLE(fish_tests EXCLUDE_FROM_ALL
                src/fish_tests.cpp)
-FISH_LINK_DEPS(fish_tests)
+FISH_LINK_DEPS_AND_SIGN(fish_tests)
 
 # The "test" directory.
 SET(TEST_DIR ${CMAKE_CURRENT_BINARY_DIR}/test)

osx/MacApp.entitlements

@@ -0,0 +1,8 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+	<key>com.apple.security.automation.apple-events</key>
+	<true/>
+</dict>
+</plist>

share/completions/bat.fish

@@ -1,88 +0,0 @@
-# Generated by hand against bat 0.12.0.
-# bat is at <https://github.com/sharkdp/bat>.
-
-# TODO: Have --map-syntax properly suggest from- and to-languages.
-# NOTE: The completion for --style won’t help the user write anything in the comma-separated-list form.
-
-function __bat_complete_languages_and_extensions -d 'All languages and their filenames/extensions, one per line'
-    command bat --list-languages | string split : | string split ,
-end
-
-function __bat_complete_language_extensions -d 'All language extensions/names, one per line'
-    command bat --list-languages | cut -d : -f 2 | string split ,
-end
-
-function __bat_complete_themes -d 'All themes, one per line'
-    command bat --list-themes
-end
-
-set -l style_opts '
-    auto\tdefault
-    full\t
-    plain\t
-    changes\t
-    header\t
-    grid\t
-    numbers\t
-'
-
-set -l color_opts '
-    auto\tdefault
-    never\t
-    always\t
-'
-
-set -l italic_text_opts '
-    always\t
-    never\tdefault
-'
-
-set -l decorations_opts "$color_opts"
-set -l paging_opts "$color_opts"
-
-set -l wrap_opts '
-    auto\tdefault
-    never\t
-    character\t
-'
-
-# While --tabs theoretically takes any number, most people should be OK with these. Specifying a list lets me explain what 0 does.
-set -l tabs_opts '
-    0\tpasses tabs through directly
-    1\t
-    2\t
-    4\t
-    8\t
-'
-
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -s l -l language -a '(__bat_complete_languages_and_extensions)' -d 'Set language for syntax highlighting'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -f -s L -l list-language -d 'List supported languages for syntax highlighting'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -s m -l map-syntax -d 'Map file name/extension to existing syntax'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -l theme -a '(__bat_complete_themes)' -d 'Set theme for syntax highlighting'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -f -l list-themes -d 'List syntax-highlighting themes'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -l style -a "$style_opts" -d 'Choose what to add to the file contents'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -s p -l plain -d 'Show no decorations; -pp disables paging too'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -s n -l number -d 'Only show line numbers and no other decorations'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -s A -l show-all -d 'Show non-printable characters'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -s r -l line-range -d 'Only print lines from [M]:[N] (either optional)'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -s H -l highlight-line -d 'Highlight the given line'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -l color -ka "$color_opts" -d 'Specify when to use colors'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -l italic-text -ka "$italic_text_opts" -d 'Specify when to use ANSI italic-text sequences'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -l decorations -ka "$decorations_opts" -d 'Specify when to use --style decorations'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -l paging -ka "$paging_opts" -d 'Specify when to use a pager'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -l pager -d 'Specify what pager to use'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -l wrap -ka "$wrap_opts" -d 'Specify text-wrapping mode'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -l tabs -a "$tabs_opts" -d 'Sets tab width to N spaces'
-# Don’t suggest the always-ignored added-only-for-POSIX-compliance -u/--unbuffered. Output is always unbuffered.
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -x -l terminal-width -d 'Explicitly set terminal width, optionally prefixed with +/-'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -s h -l help -d 'Print help'
-complete -c bat -n 'not __fish_seen_subcommand_from cache' -s V -l version -d 'Show version information'
-
-# cache things
-complete -c bat -n '__fish_use_subcommand' -a cache -d 'Manage syntax-definition and theme cache'
-complete -c bat -n '__fish_seen_subcommand_from cache' -f -s b -l build -d 'Initialize/update cache from the source directory'
-complete -c bat -n '__fish_seen_subcommand_from cache' -f -s c -l clear -d 'Remove cached definitions and themes'
-complete -c bat -n '__fish_seen_subcommand_from cache' -r -l source -d 'Specify directory to load syntaxes/themes from'
-complete -c bat -n '__fish_seen_subcommand_from cache' -r -l target -d 'Specify directory to store cached syntaxes/themes'
-complete -c bat -n '__fish_seen_subcommand_from cache' -f -l blank -d 'Create completely new syntax/theme sets instead of appending to the defaults'
-complete -c bat -n '__fish_seen_subcommand_from cache' -f -s h -l help -d 'Print help about cache management'

share/completions/conda.fish

@@ -31,7 +31,7 @@ function __fish_conda_config_keys
 end
 
 function __fish_conda_environments
-    conda env list | string match -rv '^#' | string match -r '^\w+'
+    conda env list | string match -rv '^#' | string match -r '^\S+'
 end
 
 # common options

share/completions/flatpak.fish

@@ -1,26 +1,50 @@
 # Completions for flatpak, an "Application deployment framework for desktop apps"
 # (http://flatpak.org)
-set -l commands install update uninstall list info run override make-current enter document-{export,unexport,info,list} \
-    remote-{add,modify,delete,list,ls} build build-{init,finish,export,bundle,import-bundle,sign,update-repo}
+set -l flatversion (flatpak --version | string match -r '[\d.]+' | cut -f -2 -d .)
+set -l commands install update uninstall list info config repair create-usb search run override make-current enter ps \
+    documents document-{export,unexport,info} permissions permission-{show,reset} remotes remote-{add,modify,delete,ls,info} \
+    build build-{init,finish,export,bundle,import-bundle,sign,update-repo,commit-from} repo
+
+if test $flatversion -ge 1.1 2>/dev/null
+    set commands $commands history kill
+    complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a history -d 'Show history'
+    complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a kill -d 'Stop a running application'
+end
+
+if test $flatversion -ge 1.5 2>/dev/null
+    set commands $commands mask permission-{remove,set}
+    complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a mask -d 'Mask out updates and automatic installation'
+    complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a permission-remove -d 'Remove item from permission store'
+    complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a permission-set -d 'Set permissions'
+end
 
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a install -d 'Install an application or runtime from a remote'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a update -d 'Update an installed application or runtime'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a uninstall -d 'Uninstall an installed application or runtime'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a list -d 'List installed apps and/or runtimes'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a info -d 'Show info for installed app or runtime'
+complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a config -d 'Configure flatpak'
+complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a repair -d 'Repair flatpak installation'
+complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a create-usb -d 'Put applications or runtimes onto removable media'
+complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a search -d 'Search for remote apps/runtimes'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a run -d 'Run an application'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a override -d 'Override permissions for an application'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a make-current -d 'Specify default version to run'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a enter -d 'Enter the namespace of a running application'
+complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a ps -d 'Enumerate running applications'
+complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a documents -d 'List exported files'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a document-export -d 'Grant an application access to a specific file'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a document-unexport -d 'Revoke access to a specific file'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a document-info -d 'Show information about a specific file'
-complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a document-list -d 'List exported files'
+complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a permissions -d 'List permissions'
+complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a permission-show -d 'Show app permissions'
+complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a permission-reset -d 'Reset app permissions'
+complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a remotes -d 'List all configured remotes'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a remote-add -d 'Add a new remote repository (by URL)'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a remote-modify -d 'Modify properties of a configured remote'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a remote-delete -d 'Delete a configured remote'
-complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a remote-list -d 'List all configured remotes'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a remote-ls -d 'List contents of a configured remote'
+complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a remote-info -d 'Show information about a remote app or runtime'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a build-init -d 'Initialize a directory for building'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a build -d 'Run a build command inside the build dir'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a build-finish -d 'Finish a build dir for export'
@@ -29,10 +53,19 @@ complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a build-b
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a build-import-bundle -d 'Import a bundle file'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a build-sign -d 'Sign an application or runtime'
 complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a build-update-repo -d 'Update the summary file in a repository'
+complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a build-commit-from -d 'Create new commit based on existing ref'
+complete -f -c flatpak -n "not __fish_seen_subcommand_from $commands" -a repo -d 'Show information about a repo'
 
-complete -f -c flatpak -n "__fish_seen_subcommand_from run" -a "(flatpak list --app | string match -r '\S+')"
-complete -f -c flatpak -n "__fish_seen_subcommand_from info uninstall" -a "(flatpak list | string match -r '\S+')"
-complete -f -c flatpak -n "__fish_seen_subcommand_from remote-ls remote-modify remote-delete" -a "(flatpak remote-list | string match -r '\S+')"
+if test $flatversion -ge 1.2 2>/dev/null
+    complete -f -c flatpak -n "__fish_seen_subcommand_from run" -a "(flatpak list --app --columns=application,name)"
+    complete -f -c flatpak -n "__fish_seen_subcommand_from info uninstall" -a "(flatpak list --columns=application,name)"
+    complete -f -c flatpak -n "__fish_seen_subcommand_from enter kill" -a "(flatpak ps --columns=instance,application)"
+    complete -f -c flatpak -n "__fish_seen_subcommand_from remote-info remote-ls remote-modify remote-delete" -a "(flatpak remotes --columns=name,title)"
+else
+    complete -f -c flatpak -n "__fish_seen_subcommand_from run" -a "(flatpak list --app | string match -r '\S+')"
+    complete -f -c flatpak -n "__fish_seen_subcommand_from info uninstall" -a "(flatpak list | string match -r '\S+')"
+    complete -f -c flatpak -n "__fish_seen_subcommand_from remote-info remote-ls remote-modify remote-delete" -a "(flatpak remotes | string match -r '\S+')"
+end
 
 # Note that "remote-ls" opens an internet connection, so we don't want to complete "install"
 # Plenty of the other stuff is too free-form to complete (e.g. remote-add).

share/completions/git.fish

@@ -339,7 +339,7 @@ function __fish_git_files
         # Note that we can't use space as a delimiter between status and filename, because
         # the status can contain spaces - " M" is different from "M ".
         __fish_git $git_opt status --porcelain -z $status_opt \
-            | while read -lz line
+            | while read -lz -d' ' line
             set -l desc
             # The entire line is the "from" from a rename.
             if set -q use_next[1]
@@ -621,7 +621,9 @@ function __fish_git_aliases
     __fish_git config -z --get-regexp '^alias\.' 2>/dev/null | while read -lz key value
         begin
             set -l name (string replace -r '^.*\.' '' -- $key)
-            printf "%s\t%s\n" $name "Alias for $value"
+            # Only use the first line of the value as the description.
+            set -l val (printf '%s\n' $value)[1]
+            printf "%s\t%s\n" $name "Alias for $val"
         end
     end
 end

share/completions/kill.fish

@@ -1,8 +1,8 @@
 __fish_make_completion_signals
 
 for sig in $__kill_signals[-1..1]
-    set number (string split ' ' $sig)[1]
-    set name (string split ' ' $sig)[2]
+    set -l number (string split ' ' $sig)[1]
+    set -l name (string split ' ' $sig)[2]
     complete -c kill -o $number -d $name
     complete -c kill -o $name -d $number
     complete -c kill -k -s s -x -a "$name\t$number"

share/completions/npm.fish

@@ -128,10 +128,10 @@ complete -f -c npm -n "__fish_npm_needs_command" -a 'set' -d 'Sets the config ke
 
 # install
 for c in 'install' 'isntall' 'i'
-    complete -f -c npm -n '__fish_npm_needs_command' -a "$c" -d 'install a package'
-    complete -f -c npm -n "__fish_npm_using_command $c" -l save-dev -d 'Save to devDependencies in package.json'
-    complete -f -c npm -n "__fish_npm_using_command $c" -l save -d 'Save to dependencies in package.json'
-    complete -f -c npm -n "__fish_npm_using_command $c" -s g -l global -d 'Install package globally'
+    complete -c npm -n '__fish_npm_needs_command' -a "$c" -d 'install a package'
+    complete -c npm -n "__fish_npm_using_command $c" -l save-dev -d 'Save to devDependencies in package.json'
+    complete -c npm -n "__fish_npm_using_command $c" -l save -d 'Save to dependencies in package.json'
+    complete -c npm -n "__fish_npm_using_command $c" -s g -l global -d 'Install package globally'
 end
 
 # list

share/completions/optipng.fish

@@ -37,6 +37,6 @@ complete -x -c optipng -n '__fish_should_complete_switches' -a '-dir\t"write out
 complete -x -c optipng -n '__fish_should_complete_switches' -a '-log\t"log messages to <file>"'
 
 complete -x -c optipng -n 'not __fish_prev_arg_in -out -dir -log' \
-    -a '(__fish_complete_suffix .png;  __fish_complete_suffix .pnm;  __fish_complete_suffix .tiff;  __fish_complete_suffix .bmp)
+    -a '(__fish_complete_suffix .png;  __fish_complete_suffix .pnm;  __fish_complete_suffix .tiff;  __fish_complete_suffix .bmp)'
 
 complete -x -c optipng -n '__fish_prev_arg_in -dir' -a '(__fish_complete_directories)'

share/completions/rg.fish

@@ -1,117 +0,0 @@
-# The following ripgrep (rg) completions were generated from the rg sources via
-# clap's completion file generator, from the MIT-licensed ripgrep 0.10.0 release.
-# To avoid licensing confusion, this file is released under the MIT license.
-
-complete -c rg -n "__fish_use_subcommand" -s A -l after-context -d 'Show NUM lines after each match.'
-complete -c rg -n "__fish_use_subcommand" -s B -l before-context -d 'Show NUM lines before each match.'
-complete -c rg -n "__fish_use_subcommand" -l color -d 'Controls when to use color.' -r -f -a "never auto always ansi"
-complete -c rg -n "__fish_use_subcommand" -l colors -d 'Configure color settings and styles.'
-complete -c rg -n "__fish_use_subcommand" -s C -l context -d 'Show NUM lines before and after each match.'
-complete -c rg -n "__fish_use_subcommand" -l context-separator -d 'Set the context separator string.'
-complete -c rg -n "__fish_use_subcommand" -l dfa-size-limit -d 'The upper size limit of the regex DFA.'
-complete -c rg -n "__fish_use_subcommand" -s E -l encoding -d 'Specify the text encoding of files to search.'
-complete -c rg -n "__fish_use_subcommand" -s f -l file -d 'Search for patterns from the given file.'
-complete -c rg -n "__fish_use_subcommand" -s g -l glob -d 'Include or exclude files.'
-complete -c rg -n "__fish_use_subcommand" -l iglob -d 'Include or exclude files case insensitively.'
-complete -c rg -n "__fish_use_subcommand" -l ignore-file -d 'Specify additional ignore files.'
-complete -c rg -n "__fish_use_subcommand" -s M -l max-columns -d 'Don\'t print lines longer than this limit.'
-complete -c rg -n "__fish_use_subcommand" -s m -l max-count -d 'Limit the number of matches.'
-complete -c rg -n "__fish_use_subcommand" -l max-depth -d 'Descend at most NUM directories.'
-complete -c rg -n "__fish_use_subcommand" -l max-filesize -d 'Ignore files larger than NUM in size.'
-complete -c rg -n "__fish_use_subcommand" -l path-separator -d 'Set the path separator.'
-complete -c rg -n "__fish_use_subcommand" -l pre -d 'search outputs of COMMAND FILE for each FILE'
-complete -c rg -n "__fish_use_subcommand" -l pre-glob -d 'Include or exclude files from a preprocessing command.'
-complete -c rg -n "__fish_use_subcommand" -l regex-size-limit -d 'The upper size limit of the compiled regex.'
-complete -c rg -n "__fish_use_subcommand" -s e -l regexp -d 'A pattern to search for.'
-complete -c rg -n "__fish_use_subcommand" -s r -l replace -d 'Replace matches with the given text.'
-complete -c rg -n "__fish_use_subcommand" -l sort -d 'Sort results in ascending order. Implies --threads=1.' -r -f -a "path modified accessed created none"
-complete -c rg -n "__fish_use_subcommand" -l sortr -d 'Sort results in descending order. Implies --threads=1.' -r -f -a "path modified accessed created none"
-complete -c rg -n "__fish_use_subcommand" -s j -l threads -d 'The approximate number of threads to use.'
-complete -c rg -n "__fish_use_subcommand" -s t -l type -d 'Only search files matching TYPE.'
-complete -c rg -n "__fish_use_subcommand" -l type-add -d 'Add a new glob for a file type.'
-complete -c rg -n "__fish_use_subcommand" -l type-clear -d 'Clear globs for a file type.'
-complete -c rg -n "__fish_use_subcommand" -s T -l type-not -d 'Do not search files matching TYPE.'
-complete -c rg -n "__fish_use_subcommand" -l block-buffered -d 'Force block buffering.'
-complete -c rg -n "__fish_use_subcommand" -l no-block-buffered
-complete -c rg -n "__fish_use_subcommand" -s b -l byte-offset -d 'Print the 0-based byte offset for each matching line.'
-complete -c rg -n "__fish_use_subcommand" -s s -l case-sensitive -d 'Search case sensitively (default).'
-complete -c rg -n "__fish_use_subcommand" -l column -d 'Show column numbers.'
-complete -c rg -n "__fish_use_subcommand" -l no-column
-complete -c rg -n "__fish_use_subcommand" -s c -l count -d 'Only show the count of matching lines for each file.'
-complete -c rg -n "__fish_use_subcommand" -l count-matches -d 'Only show the count of individual matches for each file.'
-complete -c rg -n "__fish_use_subcommand" -l crlf -d 'Support CRLF line terminators (useful on Windows).'
-complete -c rg -n "__fish_use_subcommand" -l no-crlf
-complete -c rg -n "__fish_use_subcommand" -l debug -d 'Show debug messages.'
-complete -c rg -n "__fish_use_subcommand" -l trace
-complete -c rg -n "__fish_use_subcommand" -l no-encoding
-complete -c rg -n "__fish_use_subcommand" -l files -d 'Print each file that would be searched.'
-complete -c rg -n "__fish_use_subcommand" -s l -l files-with-matches -d 'Only print the paths with at least one match.'
-complete -c rg -n "__fish_use_subcommand" -l files-without-match -d 'Only print the paths that contain zero matches.'
-complete -c rg -n "__fish_use_subcommand" -s F -l fixed-strings -d 'Treat the pattern as a literal string.'
-complete -c rg -n "__fish_use_subcommand" -l no-fixed-strings
-complete -c rg -n "__fish_use_subcommand" -s L -l follow -d 'Follow symbolic links.'
-complete -c rg -n "__fish_use_subcommand" -l no-follow
-complete -c rg -n "__fish_use_subcommand" -l heading -d 'Print matches grouped by each file.'
-complete -c rg -n "__fish_use_subcommand" -l no-heading -d 'Don\'t group matches by each file.'
-complete -c rg -n "__fish_use_subcommand" -l hidden -d 'Search hidden files and directories.'
-complete -c rg -n "__fish_use_subcommand" -l no-hidden
-complete -c rg -n "__fish_use_subcommand" -s i -l ignore-case -d 'Case insensitive search.'
-complete -c rg -n "__fish_use_subcommand" -s v -l invert-match -d 'Invert matching.'
-complete -c rg -n "__fish_use_subcommand" -l json -d 'Show search results in a JSON Lines format.'
-complete -c rg -n "__fish_use_subcommand" -l no-json
-complete -c rg -n "__fish_use_subcommand" -l line-buffered -d 'Force line buffering.'
-complete -c rg -n "__fish_use_subcommand" -l no-line-buffered
-complete -c rg -n "__fish_use_subcommand" -s n -l line-number -d 'Show line numbers.'
-complete -c rg -n "__fish_use_subcommand" -s N -l no-line-number -d 'Suppress line numbers.'
-complete -c rg -n "__fish_use_subcommand" -s x -l line-regexp -d 'Only show matches surrounded by line boundaries.'
-complete -c rg -n "__fish_use_subcommand" -l mmap -d 'Search using memory maps when possible.'
-complete -c rg -n "__fish_use_subcommand" -l no-mmap -d 'Never use memory maps.'
-complete -c rg -n "__fish_use_subcommand" -s U -l multiline -d 'Enable matching across multiple lines.'
-complete -c rg -n "__fish_use_subcommand" -l no-multiline
-complete -c rg -n "__fish_use_subcommand" -l multiline-dotall -d 'Make \'.\' match new lines when multiline is enabled.'
-complete -c rg -n "__fish_use_subcommand" -l no-multiline-dotall
-complete -c rg -n "__fish_use_subcommand" -l no-config -d 'Never read configuration files.'
-complete -c rg -n "__fish_use_subcommand" -l no-ignore -d 'Don\'t respect ignore files.'
-complete -c rg -n "__fish_use_subcommand" -l ignore
-complete -c rg -n "__fish_use_subcommand" -l no-ignore-global -d 'Don\'t respect global ignore files.'
-complete -c rg -n "__fish_use_subcommand" -l ignore-global
-complete -c rg -n "__fish_use_subcommand" -l no-ignore-messages -d 'Suppress gitignore parse error messages.'
-complete -c rg -n "__fish_use_subcommand" -l ignore-messages
-complete -c rg -n "__fish_use_subcommand" -l no-ignore-parent -d 'Don\'t respect ignore files in parent directories.'
-complete -c rg -n "__fish_use_subcommand" -l ignore-parent
-complete -c rg -n "__fish_use_subcommand" -l no-ignore-vcs -d 'Don\'t respect VCS ignore files.'
-complete -c rg -n "__fish_use_subcommand" -l ignore-vcs
-complete -c rg -n "__fish_use_subcommand" -l no-messages -d 'Suppress some error messages.'
-complete -c rg -n "__fish_use_subcommand" -l messages
-complete -c rg -n "__fish_use_subcommand" -l no-pcre2-unicode -d 'Disable Unicode mode for PCRE2 matching.'
-complete -c rg -n "__fish_use_subcommand" -l pcre2-unicode
-complete -c rg -n "__fish_use_subcommand" -s 0 -l null -d 'Print a NUL byte after file paths.'
-complete -c rg -n "__fish_use_subcommand" -l null-data -d 'Use NUL as a line terminator instead of \\n.'
-complete -c rg -n "__fish_use_subcommand" -l one-file-system -d 'Do not descend into directories on other file systems.'
-complete -c rg -n "__fish_use_subcommand" -l no-one-file-system
-complete -c rg -n "__fish_use_subcommand" -s o -l only-matching -d 'Print only matches parts of a line.'
-complete -c rg -n "__fish_use_subcommand" -l passthru -d 'Print both matching and non-matching lines.'
-complete -c rg -n "__fish_use_subcommand" -s P -l pcre2 -d 'Enable PCRE2 matching.'
-complete -c rg -n "__fish_use_subcommand" -l no-pcre2
-complete -c rg -n "__fish_use_subcommand" -l no-pre
-complete -c rg -n "__fish_use_subcommand" -s p -l pretty -d 'Alias for --color always --heading --line-number.'
-complete -c rg -n "__fish_use_subcommand" -s q -l quiet -d 'Do not print anything to stdout.'
-complete -c rg -n "__fish_use_subcommand" -s z -l search-zip -d 'Search in compressed files.'
-complete -c rg -n "__fish_use_subcommand" -l no-search-zip
-complete -c rg -n "__fish_use_subcommand" -s S -l smart-case -d 'Smart case search.'
-complete -c rg -n "__fish_use_subcommand" -l sort-files -d 'DEPRECATED'
-complete -c rg -n "__fish_use_subcommand" -l no-sort-files
-complete -c rg -n "__fish_use_subcommand" -l stats -d 'Print statistics about this ripgrep search.'
-complete -c rg -n "__fish_use_subcommand" -l no-stats
-complete -c rg -n "__fish_use_subcommand" -s a -l text -d 'Search binary files as if they were text.'
-complete -c rg -n "__fish_use_subcommand" -l no-text
-complete -c rg -n "__fish_use_subcommand" -l trim -d 'Trim prefixed whitespace from matches.'
-complete -c rg -n "__fish_use_subcommand" -l no-trim
-complete -c rg -n "__fish_use_subcommand" -l type-list -d 'Show all supported file types.'
-complete -c rg -n "__fish_use_subcommand" -s u -l unrestricted -d 'Reduce the level of "smart" searching.'
-complete -c rg -n "__fish_use_subcommand" -l vimgrep -d 'Show results in vim compatible format.'
-complete -c rg -n "__fish_use_subcommand" -s H -l with-filename -d 'Print the file path with the matched lines.'
-complete -c rg -n "__fish_use_subcommand" -l no-filename -d 'Never print the file path with the matched lines.'
-complete -c rg -n "__fish_use_subcommand" -s w -l word-regexp -d 'Only show matches surrounded by word boundaries.'
-complete -c rg -n "__fish_use_subcommand" -s h -l help -d 'Prints help information. Use --help for more details.'
-complete -c rg -n "__fish_use_subcommand" -s V -l version -d 'Prints version information'

share/completions/ssh.fish

@@ -17,10 +17,8 @@ complete -c ssh -n 'test (__fish_number_of_cmd_args_wo_opts) -ge 2' -d "Command
 
 complete -c ssh -s a -d "Disables forwarding of the authentication agent"
 complete -c ssh -s A -d "Enables forwarding of the authentication agent"
-# TODO: Improve this since /proc/net/arp is not POSIX compliant.
-complete -x -c ssh -s b -d "Interface to transmit from" -a "
-(cut -d ' ' -f 1 /proc/net/arp 2>/dev/null | string match -r -v '^IP')
-"
+
+complete -x -c ssh -s b -d "Local address to bind to" -a "(__fish_print_addresses)"
 
 complete -x -c ssh -s e -d "Escape character" -a "\^ none"
 complete -c ssh -s f -d "Go to background"
@@ -43,13 +41,13 @@ complete -c ssh -s R -d "Remotely forwarded ports"
 complete -c ssh -s D -d "Dynamic port forwarding"
 complete -c ssh -s c -d "Encryption cipher" -xa "(ssh -Q cipher)"
 
-# Also look up hosts from the history
+# Also retrieve `user@host` entries from history
 function __ssh_history_completions --argument limit
     if string match -q ""
         set limit 100
     end
 
-    history --prefix ssh | sed -n "s/.* \([A-Za-z0-9._:-]\+@[A-Za-z0-9._:-]\+\).*/\1/p" | head -n $limit
+    history --prefix ssh | string replace -rf '.* ([A-Za-z0-9._:-]+@[A-Za-z0-9._:-]+).*' '$1' | head -n $limit
 end
 
 complete -k -c ssh -a '(__ssh_history_completions 100)' -f -d "Remote"

share/completions/zfs.fish

@@ -438,17 +438,17 @@ if contains -- $OS SunOS FreeBSD
     complete -c zfs -f -n '__fish_zfs_using_command allow; and __fish_not_contain_opt -s u -s g -s e' -a 'everyone' -d 'Delegate permission to everyone'
 end
 complete -c zfs -x -n '__fish_zfs_using_command allow; and __fish_not_contain_opt -s u -s g everyone' -s e -d 'Delegate permission to everyone'
-complete -c zfs -x -n '__fish_zfs_using_command allow; and __fish_not_contain_opt -s l -s d -s e -s g -s u -s s' -s c -d 'Delegate permissions only to the creator of later descendent datasets' -a '(__fish_zfs_append , (__fish_zfs_list_permissions))'
+complete -c zfs -x -n '__fish_zfs_using_command allow; and __fish_not_contain_opt -s l -s d -s e -s g -s u -s s' -s c -d 'Delegate permissions only to the creator of later descendent datasets' -a '(__fish_append , (__fish_zfs_list_permissions))'
 complete -c zfs -x -n '__fish_zfs_using_command allow; and __fish_not_contain_opt -s l -s d -s e -s g -s u -s c' -s s -d 'Create a permission set or add permissions to an existing one'
 complete -c zfs -x -n '__fish_zfs_using_command allow' -d 'Dataset on which delegation is to be applied' -a '(__fish_print_zfs_filesystems; __fish_print_zfs_volumes)'
 
 # unallow completions
 complete -c zfs -f -n '__fish_zfs_using_command unallow; and __fish_not_contain_opt -s d' -s l -d 'Remove permissions only on the specified dataset'
 complete -c zfs -f -n '__fish_zfs_using_command unallow; and __fish_not_contain_opt -s l' -s d -d 'Remove permissions only on the descendents dataset'
-complete -c zfs -x -n '__fish_zfs_using_command unallow; and __fish_not_contain_opt -s e' -s u -d 'User to remove permissions from' -a '(__fish_zfs_append , (__fish_complete_users))'
-complete -c zfs -x -n '__fish_zfs_using_command unallow; and __fish_not_contain_opt -s e' -s g -d 'Group to remove permissions from' -a '(__fish_zfs_append , (__fish_complete_groups))'
+complete -c zfs -x -n '__fish_zfs_using_command unallow; and __fish_not_contain_opt -s e' -s u -d 'User to remove permissions from' -a '(__fish_append , (__fish_complete_users))'
+complete -c zfs -x -n '__fish_zfs_using_command unallow; and __fish_not_contain_opt -s e' -s g -d 'Group to remove permissions from' -a '(__fish_append , (__fish_complete_groups))'
 complete -c zfs -x -n '__fish_zfs_using_command unallow; and __fish_not_contain_opt -s u -s g everyone' -s e -d 'Remove permission from everyone'
-complete -c zfs -x -n '__fish_zfs_using_command unallow; and __fish_not_contain_opt -s l -s d -s e -s g -s u -s s' -s c -d 'Remove permissions only on later created descendent datasets' -a '(__fish_zfs_append , (__fish_zfs_list_permissions))'
+complete -c zfs -x -n '__fish_zfs_using_command unallow; and __fish_not_contain_opt -s l -s d -s e -s g -s u -s s' -s c -d 'Remove permissions only on later created descendent datasets' -a '(__fish_append , (__fish_zfs_list_permissions))'
 complete -c zfs -x -n '__fish_zfs_using_command unallow; and __fish_not_contain_opt -s l -s d -s e -s g -s u -s c' -s s -d 'Remove a permission set or remove permissions from an existing one'
 if test $OS = 'SunOS'
     complete -c zfs -f -n '__fish_zfs_using_command unallow' -s r -d 'Remove permissions recursively'

share/config.fish

@@ -123,7 +123,7 @@ end
 # system utilities.
 #
 
-if test -d /usr/xpg4/bin
+if begin; not set -q FISH_UNIT_TESTS_RUNNING; and test -d /usr/xpg4/bin; end
     not contains -- /usr/xpg4/bin $PATH
     and set PATH /usr/xpg4/bin $PATH
 end

share/functions/__fish_complete_external_command.fish

@@ -1,3 +1,3 @@
 function __fish_complete_external_command
-    command find $PATH/ -maxdepth 1 -perm -u+x 2>&- | string match -r '[^/]*$'
+    complete -C "$argv[1]"
 end

share/functions/__fish_complete_man.fish

@@ -1,3 +1,20 @@
+# macOS 10.15 "Catalina" has some major issues.
+# The whatis database is non-existent, so apropos tries (and fails) to create it every time,
+# which takes about half a second.
+#
+# So we disable this entirely in that case.
+if test (uname) = Darwin
+    set -l darwin_version (uname -r | string split .)
+    # macOS 15 is Darwin 19, this is an issue at least up to 10.15.3.
+    # If this is fixed in later versions uncomment the second check.
+    if test "$darwin_version[1]" = 19 # -a "$darwin_version[2]" -le 3
+        function __fish_complete_man
+        end
+        # (remember: exit when `source`ing only exits the file, not the shell)
+        exit
+    end
+end
+
 function __fish_complete_man
     # Try to guess what section to search in. If we don't know, we
     # use [^)]*, which should match any section.

share/functions/__fish_complete_subcommand.fish

@@ -49,7 +49,7 @@ function __fish_complete_subcommand -d "Complete subcommand" --no-scope-shadowin
     end
 
     if test $allow_functions_and_builtins = false && test (count $subcommand) -eq 1
-        __fish_complete_external_command
+        __fish_complete_external_command "$subcommand"
     else
         printf "%s\n" (complete -C "$subcommand")
     end

share/functions/__fish_print_hostnames.fish

@@ -108,7 +108,8 @@ function __fish_print_hostnames -d "Print a list of known hostnames"
             read -alz -d \n contents <$file
 
             # Print hosts from system wide ssh configuration file
-            string replace -rfi '^\s*Host\s+(\S.*?)\s*$' '$1' -- $contents | string match -v '*\**'
+            # Multiple names for a single host can be given separated by spaces, so just split it explicitly (#6698).
+            string replace -rfi '^\s*Host\s+(\S.*?)\s*$' '$1' -- $contents | string split " " | string match -v '*\**'
             # Also extract known_host paths.
             set known_hosts $known_hosts (string replace -rfi '.*KnownHostsFile\s*' '' -- $contents)
         end

share/functions/__fish_shared_key_bindings.fish

@@ -161,7 +161,7 @@ function __fish_shared_key_bindings -d "Bindings shared between emacs and vi mod
     bind --preset -M paste \\ "__fish_commandline_insert_escaped \\\ \$__fish_paste_quoted"
     # Only insert spaces if we're either quoted or not at the beginning of the commandline
     # - this strips leading spaces if they would trigger histignore.
-    bind --preset -M paste \  'if set -q __fish_paste_quoted[1]; or string length -q -- (commandline -c); commandline -i " "; end'
+    bind --preset -M paste " " self-insert-notfirst
 end
 
 function __fish_commandline_insert_escaped --description 'Insert the first arg escaped if a second arg is given'

share/functions/__fish_systemctl_services.fish

@@ -1,13 +1,13 @@
 function __fish_systemctl_services
     if type -q systemctl
         if __fish_contains_opt user
-            systemctl --user list-unit-files --no-legend --type=service 2>/dev/null $argv | cut -f 1 -d ' '
-            systemctl --user list-units --state=loaded --no-legend --type=service 2>/dev/null | cut -f 1 -d ' '
+            systemctl --user list-unit-files --full --no-legend --no-pager --plain --type=service 2>/dev/null $argv | cut -f 1 -d ' '
+            systemctl --user list-units --state=loaded --full --no-legend --no-pager --plain --type=service 2>/dev/null | cut -f 1 -d ' '
         else
             # list-unit-files will also show disabled units
-            systemctl list-unit-files --no-legend --type=service 2>/dev/null $argv | cut -f 1 -d ' '
+            systemctl list-unit-files --full --no-legend --no-pager --plain --type=service 2>/dev/null $argv | cut -f 1 -d ' '
             # list-units will not show disabled units but will show instances (like wpa_supplicant@wlan0.service)
-            systemctl list-units --state=loaded --no-legend --type=service 2>/dev/null | cut -f 1 -d ' '
+            systemctl list-units --state=loaded --full --no-legend --no-pager --plain --type=service 2>/dev/null | cut -f 1 -d ' '
         end
     end
 end

share/functions/_fish_systemctl.fish

@@ -74,11 +74,11 @@ function _fish_systemctl --description 'Call systemctl with some options from th
     # Output looks like
     # systemd-tmpfiles-clean.timer      [more whitespace] loaded    active     waiting   Daily Cleanup[...]
     # Use the last part as the description.
-    systemctl --no-legend --no-pager --all list-units $passflags | string replace -r "(?: +(\S+)){4}" \t'$1'
+    systemctl --full --no-legend --no-pager --plain --all list-units $passflags | string replace -r "(?: +(\S+)){4}" \t'$1'
     # We need this for disabled/static units. Also instance units without an active instance.
     # Output looks like
     # systemd-tmpfiles-clean.timer               static
     # Just use the state as the description, since we won't get it here.
     # This is an issue for units that appear in both.
-    systemctl --no-legend --no-pager --all list-unit-files $passflags | string replace -r "(?: +(\S+)){1}" \t'$1'
+    systemctl --full --no-legend --no-pager --plain --all list-unit-files $passflags | string replace -r "(?: +(\S+)){1}" \t'$1'
 end

share/functions/fish_clipboard_copy.fish

@@ -3,14 +3,14 @@ function fish_clipboard_copy
     set -l cmdline (commandline --current-selection)
     test -n "$cmdline"; or set cmdline (commandline)
     if type -q pbcopy
-        printf '%s\n' $cmdline | pbcopy
+        printf '%s' $cmdline | pbcopy
     else if set -q WAYLAND_DISPLAY; and type -q wl-copy
-        printf '%s\n' $cmdline | wl-copy
+        printf '%s' $cmdline | wl-copy
     else if type -q xsel
         # Silence error so no error message shows up
         # if e.g. X isn't running.
-        printf '%s\n' $cmdline | xsel --clipboard 2>/dev/null
+        printf '%s' $cmdline | xsel --clipboard 2>/dev/null
     else if type -q xclip
-        printf '%s\n' $cmdline | xclip -selection clipboard 2>/dev/null
+        printf '%s' $cmdline | xclip -selection clipboard 2>/dev/null
     end
 end

share/functions/fish_hg_prompt.fish

@@ -36,6 +36,12 @@ function fish_hg_prompt --description 'Write out the hg prompt'
         set branch "$branch|$bookmark"
     end
 
+    if not set -q fish_prompt_hg_show_informative_status
+        set_color normal
+        echo -n " ($branch)"
+        return
+    end
+
     echo -n '|'
 
     # Disabling color and pager is always a good idea.

share/functions/fish_print_hg_root.fish

@@ -7,7 +7,9 @@ function fish_print_hg_root
     # Find an hg directory above $PWD
     # without calling `hg root` because that's too slow
     set -l root
-    set -l dir (pwd -P)
+    set -l dir (pwd -P 2>/dev/null)
+    or return 1
+
     while test $dir != "/"
         if test -f $dir'/.hg/dirstate'
             echo $dir/.hg

share/functions/fish_vcs_prompt.fish

@@ -3,5 +3,8 @@ function fish_vcs_prompt --description "Print the prompts for all available vcse
     # This is so we don't try svn if git already worked.
     fish_git_prompt
     or fish_hg_prompt
-    or fish_svn_prompt
+    # The svn prompt is disabled by default because it's quite slow on common svn repositories.
+    # To enable it uncomment it.
+    # You can also only use it in specific directories by checking $PWD.
+    # or fish_svn_prompt
 end

share/functions/help.fish

@@ -68,8 +68,7 @@ function help --description 'Show help for the fish shell'
             if type -q cygstart
                 set fish_browser cygstart
                 # If xdg-open is available, just use that
-                # but only if an X session is running
-            else if type -q xdg-open; and set -q -x DISPLAY
+            else if type -q xdg-open
                 set fish_browser xdg-open
             end
 

sphinx_doc_src/_static/pygments.css

@@ -79,3 +79,6 @@
 .highlight .il { color: #0000cf; font-weight: bold } /* Literal.Number.Integer.Long */
 .purple { color: #551a8b }
 .yellow { color: #FFFF00 }
+.red { color: #FF0000 }
+.gray { color: #555555 }
+.underline { text-decoration: underline }

sphinx_doc_src/cmds/abbr.rst

@@ -9,7 +9,7 @@ Synopsis
 ::
 
     abbr --add [SCOPE] WORD EXPANSION
-    abbr --erase word
+    abbr --erase WORD
     abbr --rename [SCOPE] OLD_WORD NEW_WORD
     abbr --show
     abbr --list
@@ -27,7 +27,7 @@ Options
 
 The following options are available:
 
-- ``-a WORD EXPANSION`` or ``--add WORD EXPANSION`` Adds a new abbreviation, causing WORD to be expanded to PHRASE.
+- ``-a WORD EXPANSION`` or ``--add WORD EXPANSION`` Adds a new abbreviation, causing WORD to be expanded to EXPANSION.
 
 - ``-r OLD_WORD NEW_WORD`` or ``--rename OLD_WORD NEW_WORD`` Renames an abbreviation, from OLD_WORD to NEW_WORD.
 
@@ -39,7 +39,7 @@ The following options are available:
 
 - ``-q`` or ``--query`` Return 0 (true) if one of the WORDs is an abbreviation.
 
-In addition, when adding abbreviations:
+In addition, when adding or renaming abbreviations:
 
 - ``-g`` or ``--global`` to use a global variable.
 - ``-U`` or ``--universal`` to use a universal variable (default).

sphinx_doc_src/cmds/and.rst

@@ -10,7 +10,6 @@ Synopsis
 
     COMMAND1; and COMMAND2
 
-
 Description
 -----------
 
@@ -18,17 +17,18 @@ Description
 
 ``and`` statements may be used as part of the condition in an :ref:`while <cmd-while>` or :ref:`if <cmd-if>` block.
 
-``and`` does not change the current exit status itself, but the command it runs most likely will. The exit status of the last foreground command to exit can always be accessed using the `$status <index.html#variables-status>`__ variable.
+``and`` does not change the current exit status itself, but the command it runs most likely will. The exit status of the last foreground command to exit can always be accessed using the :ref:`$status <variables-status>` variable.
 
 Example
 -------
 
 The following code runs the ``make`` command to build a program. If the build succeeds, ``make``'s exit status is 0, and the program is installed. If either step fails, the exit status is 1, and ``make clean`` is run, which removes the files created by the build process.
 
-
-
 ::
 
     make; and make install; or make clean
 
+See Also
+--------
 
+- :ref:`or <cmd-or>` command

sphinx_doc_src/cmds/argparse.rst

@@ -135,7 +135,7 @@ Some OPTION_SPEC examples:
 
 - ``n-name=+`` means that only ``--name`` is valid. It requires a value and can be used more than once. If the flag is seen then ``_flag_n`` and ``_flag_name`` will be set with the values associated with each occurrence of the flag.
 
-- ``x`` means that only ``-x`` is valid. It is a boolean can can be used more than once. If it is seen then ``_flag_x`` will be set to the count of how many times the flag was seen.
+- ``x`` means that only ``-x`` is valid. It is a boolean that can be used more than once. If it is seen then ``_flag_x`` will be set to the count of how many times the flag was seen.
 
 - ``x=``, ``x=?``, and ``x=+`` are similar to the n/name examples above but there is no long flag alternative to the short flag ``-x``.
 

sphinx_doc_src/cmds/bg.rst

@@ -13,7 +13,7 @@ Synopsis
 Description
 -----------
 
-``bg`` sends `jobs <index.html#syntax-job-control>`__ to the background, resuming them if they are stopped.
+``bg`` sends :ref:`jobs <syntax-job-control>` to the background, resuming them if they are stopped.
 
 A background job is executed simultaneously with fish, and does not have access to the keyboard. If no job is specified, the last job to be used is put in the background. If PID is specified, the jobs with the specified process group IDs are put in the background.
 

sphinx_doc_src/cmds/bind.rst

@@ -20,7 +20,7 @@ Description
 
 ``bind`` adds a binding for the specified key sequence to the specified command.
 
-SEQUENCE is the character sequence to bind to. These should be written as `fish escape sequences <index.html#escapes>`__. For example, because pressing the Alt key and another character sends that character prefixed with an escape character, Alt-based key bindings can be written using the ``\e`` escape. For example, :kbd:`Alt+w` can be written as ``\ew``. The control character can be written in much the same way using the ``\c`` escape, for example :kbd:`Control+X` (^X) can be written as ``\cx``. Note that Alt-based key bindings are case sensitive and Control-based key bindings are not. This is a constraint of text-based terminals, not ``fish``.
+SEQUENCE is the character sequence to bind to. These should be written as :ref:`fish escape sequences <escapes>`. For example, because pressing the Alt key and another character sends that character prefixed with an escape character, Alt-based key bindings can be written using the ``\e`` escape. For example, :kbd:`Alt+w` can be written as ``\ew``. The control character can be written in much the same way using the ``\c`` escape, for example :kbd:`Control+X` (^X) can be written as ``\cx``. Note that Alt-based key bindings are case sensitive and Control-based key bindings are not. This is a constraint of text-based terminals, not ``fish``.
 
 The default key binding can be set by specifying a ``SEQUENCE`` of the empty string (that is, ``''`` ). It will be used whenever no other binding matches. For most key bindings, it makes sense to use the ``self-insert`` function (i.e. ``bind '' self-insert``) as the default keybinding. This will insert any keystrokes not specifically bound to into the editor. Non- printable characters are ignored by the editor, so this will not result in control sequences being printable.
 
@@ -36,7 +36,7 @@ When multiple ``COMMAND``\s are provided, they are all run in the specified orde
 
 If no ``SEQUENCE`` is provided, all bindings (or just the bindings in the specified ``MODE``) are printed. If ``SEQUENCE`` is provided without ``COMMAND``, just the binding matching that sequence is printed.
 
-To save custom keybindings, put the ``bind`` statements into `config.fish <index.html#initialization>`__. Alternatively, fish also automatically executes a function called ``fish_user_key_bindings`` if it exists.
+To save custom keybindings, put the ``bind`` statements into :ref:`config.fish <initialization>`. Alternatively, fish also automatically executes a function called ``fish_user_key_bindings`` if it exists.
 
 Key bindings may use "modes", which mimics Vi's modal input behavior. The default mode is "default", and every bind applies to a single mode. The mode can be viewed/changed with the ``$fish_bind_mode`` variable.
 
@@ -120,6 +120,8 @@ The following special input functions are available:
 
 - ``execute`` run the current commandline
 
+- ``force-repaint`` reexecute the prompt functions without coalescing
+
 - ``forward-bigword``, move one whitespace-delimited word to the right
 
 - ``forward-char``, move one character to the right
@@ -160,7 +162,9 @@ The following special input functions are available:
 
 - ``repaint-mode`` reexecutes the fish_mode_prompt function and redraws the prompt. This is useful for vi-mode. If no fish_mode_prompt exists, it acts like a normal repaint.
 
-- ``force-repaint`` reexecute the prompt functions without coalescing.
+- ``self-insert``, inserts the matching sequence into the command line
+
+- ``self-insert-notfirst``, inserts the matching sequence into the command line, unless the cursor is at the beginning
 
 - ``suppress-autosuggestion``, remove the current autosuggestion
 
@@ -211,8 +215,8 @@ Turns on Vi key bindings and rebinds :kbd:`Control+C` to clear the input line.
 Special Case: The escape Character
 ----------------------------------
 
-The escape key can be used standalone, for example, to switch from insertion mode to normal mode when using Vi keybindings. Escape may also be used as a "meta" key, to indicate the start of an escape sequence, such as function or arrow keys. Custom bindings can also be defined that begin with an escape character.
+The escape key can be used standalone, for example, to switch from insertion mode to normal mode when using Vi keybindings. Escape can also be used as a "meta" key, to indicate the start of an escape sequence, like for function or arrow keys. Custom bindings can also be defined that begin with an escape character.
 
-fish waits for a period after receiving the escape character, to determine whether it is standalone or part of an escape sequence. While waiting, additional key presses make the escape key behave as a meta key. If no other key presses come in, it is handled as a standalone escape. The waiting period is set to 300 milliseconds (0.3 seconds) in the default key bindings and 10 milliseconds in the vi key bindings. It can be configured by setting the ``fish_escape_delay_ms`` variable to a value between 10 and 5000 ms. It is recommended that this be a universal variable that you set once from an interactive session.
+Holding alt and something else also typically sends escape, for example holding alt+a will send an escape character and then an "a".
 
-Note: fish 2.2.0 and earlier used a default of 10 milliseconds, and provided no way to configure it. That effectively made it impossible to use escape as a meta key.
+fish waits for a period after receiving the escape character, to determine whether it is standalone or part of an escape sequence. While waiting, additional key presses make the escape key behave as a meta key. If no other key presses come in, it is handled as a standalone escape. The waiting period is set to 30 milliseconds (0.03 seconds). It can be configured by setting the ``fish_escape_delay_ms`` variable to a value between 10 and 5000 ms. This can be a universal variable that you set once from an interactive session.

sphinx_doc_src/cmds/breakpoint.rst

@@ -16,6 +16,6 @@ Description
 
 ``breakpoint`` is used to halt a running script and launch an interactive debugging prompt.
 
-For more details, see `Debugging fish scripts <index.html#debugging>`__ in the ``fish`` manual.
+For more details, see :ref:`Debugging fish scripts <debugging>` in the ``fish`` manual.
 
 There are no parameters for ``breakpoint``.

sphinx_doc_src/cmds/disown.rst

@@ -13,7 +13,7 @@ Synopsis
 Description
 -----------
 
-``disown`` removes the specified `job <index.html#syntax-job-control>`__ from the list of jobs. The job itself continues to exist, but fish does not keep track of it any longer.
+``disown`` removes the specified :ref:`job <syntax-job-control>` from the list of jobs. The job itself continues to exist, but fish does not keep track of it any longer.
 
 Jobs in the list of jobs are sent a hang-up signal when fish terminates, which usually causes the job to terminate; ``disown`` allows these processes to continue regardless.
 

sphinx_doc_src/cmds/false.rst

@@ -14,3 +14,9 @@ Description
 -----------
 
 ``false`` sets the exit status to 1.
+
+See Also
+--------
+
+- :ref:`true <cmd-true>` command
+- :ref:`$status <variables-status>` variable

sphinx_doc_src/cmds/fg.rst

@@ -13,7 +13,7 @@ Synopsis
 Description
 -----------
 
-``fg`` brings the specified `job <index.html#syntax-job-control>`__ to the foreground, resuming it if it is stopped. While a foreground job is executed, fish is suspended. If no job is specified, the last job to be used is put in the foreground. If PID is specified, the job with the specified group ID is put in the foreground.
+``fg`` brings the specified :ref:`job <syntax-job-control>` to the foreground, resuming it if it is stopped. While a foreground job is executed, fish is suspended. If no job is specified, the last job to be used is put in the foreground. If PID is specified, the job with the specified group ID is put in the foreground.
 
 
 Example

sphinx_doc_src/cmds/fish_breakpoint_prompt.rst

@@ -18,7 +18,7 @@ Description
 
 By defining the ``fish_breakpoint_prompt`` function, the user can choose a custom prompt when asking for input in response to a :ref:`breakpoint <cmd-breakpoint>` command. The ``fish_breakpoint_prompt`` function is executed when the prompt is to be shown, and the output is used as a prompt.
 
-The exit status of commands within ``fish_breakpoint_prompt`` will not modify the value of `$status <index.html#variables-status>`__ outside of the ``fish_breakpoint_prompt`` function.
+The exit status of commands within ``fish_breakpoint_prompt`` will not modify the value of :ref:`$status <variables-status>` outside of the ``fish_breakpoint_prompt`` function.
 
 ``fish`` ships with a default version of this function that displays the function name and line number of the current execution context.
 

sphinx_doc_src/cmds/fish_hg_prompt.rst

@@ -19,7 +19,11 @@ The fish_hg_prompt function displays information about the current Mercurial rep
 
 `Mercurial <https://www.mercurial-scm.org/>`_ (``hg``) must be installed.
 
-There are numerous customization options, which can be controlled with fish variables.
+By default, only the current branch is shown because ``hg status`` can take be slow on large repository. You can enable a more informative prompt by setting the variable ``$fish_prompt_hg_show_informative_status``, for example::
+
+	set --universal fish_prompt_hg_show_informative_status
+
+If you enabled the informative status, there are numerous customization options, which can be controlled with fish variables.
 
 - ``$fish_color_hg_clean``, ``$fish_color_hg_modified`` and ``$fish_color_hg_dirty`` are colors used when the repository has the respective status.
 
@@ -52,6 +56,7 @@ A simple prompt that displays hg info::
 
     function fish_prompt
         ...
+        set -g fish_prompt_hg_show_informative_status
         printf '%s %s$' $PWD (fish_hg_prompt)
     end
 

sphinx_doc_src/cmds/fish_prompt.rst

@@ -18,7 +18,7 @@ Description
 
 By defining the ``fish_prompt`` function, the user can choose a custom prompt. The ``fish_prompt`` function is executed when the prompt is to be shown, and the output is used as a prompt.
 
-The exit status of commands within ``fish_prompt`` will not modify the value of `$status <index.html#variables-status>`__ outside of the ``fish_prompt`` function.
+The exit status of commands within ``fish_prompt`` will not modify the value of :ref:`$status <variables-status>` outside of the ``fish_prompt`` function.
 
 ``fish`` ships with a number of example prompts that can be chosen with the ``fish_config`` command.
 

sphinx_doc_src/cmds/fish_vcs_prompt.rst

@@ -25,6 +25,8 @@ It calls out to VCS-specific functions. The currently supported systems are:
 
 If a VCS isn't installed, the respective function does nothing.
 
+The svn prompt is disabled by default because it's slow on large svn repositories. To enable it, modify fish_vcs_prompt to uncomment it. See :ref:`funced <cmd-funced>`.
+
 For more information, see the documentation for each of the functions above.
 
 Example

sphinx_doc_src/cmds/function.rst

@@ -44,7 +44,7 @@ The following options are available:
 
 - ``-V`` or ``--inherit-variable NAME`` snapshots the value of the variable ``NAME`` and defines a local variable with that same name and value when the function is defined. This is similar to a closure in other languages like Python but a bit different. Note the word "snapshot" in the first sentence. If you change the value of the variable after defining the function, even if you do so in the same scope (typically another function) the new value will not be used by the function you just created using this option. See the ``function notify`` example below for how this might be used.
 
-If the user enters any additional arguments after the function, they are inserted into the environment `variable list <index.html#variables-lists>`__ ``$argv``. If the ``--argument-names`` option is provided, the arguments are also assigned to names specified in that option.
+If the user enters any additional arguments after the function, they are inserted into the environment :ref:`variable list <variables-lists>` ``$argv``. If the ``--argument-names`` option is provided, the arguments are also assigned to names specified in that option.
 
 By using one of the event handler switches, a function can be made to run automatically at specific events. The user may generate new events using the :ref:`emit <cmd-emit>` builtin. Fish generates the following named events:
 

sphinx_doc_src/cmds/jobs.rst

@@ -14,7 +14,7 @@ Synopsis
 Description
 -----------
 
-``jobs`` prints a list of the currently running `jobs <index.html#syntax-job-control>`__ and their status.
+``jobs`` prints a list of the currently running :ref:`jobs <syntax-job-control>` and their status.
 
 jobs accepts the following switches:
 

sphinx_doc_src/cmds/or.rst

@@ -15,8 +15,7 @@ Description
 
 ``or`` is used to execute a command if the previous command was not successful (returned a status of something other than 0).
 
-``or`` statements may be used as part of the condition in an :ref:`and <cmd-if>` or :ref:`while <cmd-while>` block. See the documentation
-for :ref:`if <cmd-if>` and :ref:`while <cmd-while>` for examples.
+``or`` statements may be used as part of the condition in an :ref:`and <cmd-if>` or :ref:`while <cmd-while>` block.
 
 ``or`` does not change the current exit status itself, but the command it runs most likely will. The exit status of the last foreground command to exit can always be accessed using the :ref:`$status <variables-status>` variable.
 
@@ -25,9 +24,11 @@ Example
 
 The following code runs the ``make`` command to build a program. If the build succeeds, the program is installed. If either step fails, ``make clean`` is run, which removes the files created by the build process.
 
-
-
 ::
 
     make; and make install; or make clean
 
+See Also
+--------
+
+- :ref:`and <cmd-and>` command

sphinx_doc_src/cmds/test.rst

@@ -191,7 +191,7 @@ which is logically equivalent to the following:
 Standards
 ---------
 
-``test`` implements a subset of the `IEEE Std 1003.1-2008 (POSIX.1) standard <http://www.unix.com/man-page/POSIX/1/test/>`__. The following exceptions apply:
+``test`` implements a subset of the `IEEE Std 1003.1-2008 (POSIX.1) standard <https://www.unix.com/man-page/posix/1p/test/>`__. The following exceptions apply:
 
 - The ``<`` and ``>`` operators for comparing strings are not implemented.
 

sphinx_doc_src/cmds/true.rst

@@ -10,8 +10,13 @@ Synopsis
 
     true
 
-
 Description
 -----------
 
 ``true`` sets the exit status to 0.
+
+See Also
+--------
+
+- :ref:`false <cmd-false>` command
+- :ref:`$status <variables-status>` variable

sphinx_doc_src/cmds/ulimit.rst

@@ -50,7 +50,7 @@ If limit is given, it is the new value of the specified resource. If no option i
 
 - ``-S`` or ``--soft`` sets soft resource limit
 
-A hard limit can only be decreased. Once it is set it cannot be increased; a soft limit may be increased up to the value of the hard limit. If neither -H nor -S is specified, both the soft and hard limits are updated when assigning a new limit value, and the soft limit is used when reporting the current value.
+A hard limit can only be decreased. Once it is set it cannot be increased; a soft limit may be increased up to the value of the hard limit. If neither ``-H`` nor ``-S`` is specified, both the soft and hard limits are updated when assigning a new limit value, and the soft limit is used when reporting the current value.
 
 The following additional options are also understood by ``ulimit``:
 
@@ -62,7 +62,7 @@ The ``fish`` implementation of ``ulimit`` should behave identically to the imple
 
 - Fish ``ulimit`` does not support the ``-p`` option for getting the pipe size. The bash implementation consists of a compile-time check that empirically guesses this number by writing to a pipe and waiting for SIGPIPE. Fish does not do this because it this method of determining pipe size is unreliable. Depending on bash version, there may also be further additional limits to set in bash that do not exist in fish.
 
-- Fish ``ulimit`` does not support getting or setting multiple limits in one command, except reporting all values using the -a switch
+- Fish ``ulimit`` does not support getting or setting multiple limits in one command, except reporting all values using the ``-a`` switch
 
 
 Example

sphinx_doc_src/faq.rst

@@ -233,7 +233,7 @@ Fish history recall is very simple yet effective:
 
   - If you want to reuse several arguments from the same line ("!!:3*" and the like), consider recalling the whole line and removing what you don't need (:kbd:`Alt+D` and :kbd:`Alt+Backspace` are your friends).
 
-See `documentation <index.html#editor>`__ for more details about line editing in fish.
+See :ref:`documentation <editor>` for more details about line editing in fish.
 
 
 How can I use ``-`` as a shortcut for ``cd -``?

sphinx_doc_src/index.rst

@@ -77,7 +77,7 @@ You can make fish your default shell by adding fish's  executable in two places:
 - add ``/usr/local/bin/fish``  to  ``/etc/shells``
 - change your default shell with ``chsh -s`` to ``/usr/local/bin/fish``
 
-For for detailed instructions see `Switching to fish <tutorial.html#tut_switching_to_fish>`_.
+For for detailed instructions see :ref:`Switching to fish <switching-to-fish>`.
 
 Uninstalling
 ------------
@@ -1080,7 +1080,7 @@ If a process exits through a signal, the exit status will be 128 plus the number
 Variables for changing highlighting colors
 ------------------------------------------
 
-The colors used by fish for syntax highlighting can be configured by changing the values of a various variables. The value of these variables can be one of the colors accepted by the `set_color <cmds/set.html_color>`_ command. The ``--bold`` or ``-b`` switches accepted by ``set_color`` are also accepted.
+The colors used by fish for syntax highlighting can be configured by changing the values of a various variables. The value of these variables can be one of the colors accepted by the :ref:`set_color <cmd-set_color>` command. The ``--bold`` or ``-b`` switches accepted by ``set_color`` are also accepted.
 
 The following variables are available to change the highlighting colors in fish:
 
@@ -1379,7 +1379,7 @@ Shared bindings
 
 Some bindings are shared between emacs- and vi-mode because they aren't text editing bindings or because what Vi/Vim does for a particular key doesn't make sense for a shell.
 
-- :kbd:`Tab` `completes <#completion>`_ the current token. :kbd:`Shift, Tab` completes the current token and starts the pager's search mode.
+- :kbd:`Tab` `completes <#tab-completion>`_ the current token. :kbd:`Shift, Tab` completes the current token and starts the pager's search mode.
 
 - :kbd:`Alt+←,Left` and :kbd:`Alt+→,Right` move the cursor one word left or right (to the next space or punctuation mark), or moves forward/backward in the directory history if the command line is empty. If the cursor is already at the end of the line, and an autosuggestion is available, :kbd:`Alt+→,Right` (or :kbd:`Alt+F`) accepts the first word in the suggestion.
 
@@ -1628,7 +1628,7 @@ Configuration files are evaluated in the following order:
   If there are multiple files with the same name in these directories, only the first will be executed.
   They are executed in order of their filename, sorted (like globs) in a natural order (i.e. "01" sorts before "2").
 
-- System-wide configuration files, where administrators can include initialization that should be run for all users on the system - similar to ``/etc/profile`` for POSIX-style shells - in ``$__fish_sysconf_dir` (usually /etc/fish/config.fish``);
+- System-wide configuration files, where administrators can include initialization that should be run for all users on the system - similar to ``/etc/profile`` for POSIX-style shells - in ``$__fish_sysconf_dir`` (usually ``/etc/fish/config.fish``).
 - User initialization, usually in `~/.config/fish/config.fish` (controlled by the ``XDG_CONFIG_HOME`` environment variable, and accessible as ``$__fish_config_dir``).
 
 These paths are controlled by parameters set at build, install, or run time, and may vary from the defaults listed above.

sphinx_doc_src/tutorial.rst

@@ -28,7 +28,7 @@ which means you are all set up and can start using fish::
 
 
 This prompt that you see above is the ``fish`` default prompt: it shows your username, hostname, and working directory.
-- to change this prompt see `how to change your prompt <prompt>`_
+- to change this prompt see `how to change your prompt <#prompt>`_
 - to switch to fish permanently see `switch your default shell to fish <#switching-to-fish>`_.
 
 From now on, we'll pretend your prompt is just a '``>``' to save space.
@@ -82,19 +82,23 @@ Getting Help
 Syntax Highlighting
 -------------------
 
-You'll quickly notice that ``fish`` performs syntax highlighting as you type. Invalid commands are colored red by default::
+.. role:: red
+.. role:: gray
+.. role:: underline
 
-    > <red>/bin/mkd</red>
+You'll quickly notice that ``fish`` performs syntax highlighting as you type. Invalid commands are colored red by default
 
 
+> :red:`/bin/mkd`
+
 A command may be invalid because it does not exist, or refers to a file that you cannot execute. When the command becomes valid, it is shown in a different color::
 
     > /bin/mkdir
 
 
-``fish`` will underline valid file paths as you type them::
+``fish`` will underline valid file paths as you type them
 
-    > cat <underline>~/somefi</underline>
+> cat :underline:`~/somefi`
 
 
 This tells you that there exists a file that starts with '``somefi``', which is useful feedback as you type.
@@ -139,7 +143,7 @@ You can pipe between commands with the usual vertical bar::
           1       2      12
 
 
-stdin and stdout can be redirected via the familiar `<` and `<`. stderr is redirected with a `2>`.
+stdin and stdout can be redirected via the familiar `<` and `>`. stderr is redirected with a `2>`.
 
 
 
@@ -156,19 +160,19 @@ To redirect stdout and stderr into one file, you need to first redirect stdout,
 Autosuggestions
 ---------------
 
-``fish`` suggests commands as you type, and shows the suggestion to the right of the cursor, in gray. For example::
+``fish`` suggests commands as you type, and shows the suggestion to the right of the cursor, in gray. For example
 
-    > <red>/bin/h</red><gray>ostname</gray>
+> :red:`/bin/h`:gray:`ostname`
 
 
-It knows about paths and options::
+It knows about paths and options
 
-    > grep --i<gray>gnore-case</gray>
+`> grep --i`:gray:`gnore-case`
 
 
-And history too. Type a command once, and you can re-summon it by just typing a few letters::
+And history too. Type a command once, and you can re-summon it by just typing a few letters
 
-    > <red>r</red><gray>sync -avze ssh . myname@somelonghost.com:/some/long/path/doo/dee/doo/dee/doo</gray>
+> :red:`r`:gray:`sync -avze ssh . myname@somelonghost.com:/some/long/path/doo/dee/doo/dee/doo`
 
 
 To accept the autosuggestion, hit :kbd:`→` (right arrow) or :kbd:`Control+F`. To accept a single word of the autosuggestion, :kbd:`Alt+→` (right arrow). If the autosuggestion is not what you want, just ignore it.
@@ -178,15 +182,15 @@ Tab Completions
 
 ``fish`` comes with a rich set of tab completions, that work "out of the box."
 
-Press :kbd:`Tab`, and ``fish`` will attempt to complete the command, argument, or path::
+Press :kbd:`Tab`, and ``fish`` will attempt to complete the command, argument, or path
 
-    > <red>/pri</red> :kbd:`Tab` => /private/
+> :red:`/pri` :kbd:`Tab` => /private/
 
 
-If there's more than one possibility, it will list them::
+If there's more than one possibility, it will list them
 
-    > <red>~/stuff/s</red> :kbd:`Tab`
-    ~/stuff/script.sh  (Executable, 4.8kB)  ~/stuff/sources/  (Directory)
+> :red:`~/stuff/s` :kbd:`Tab`
+~/stuff/script.sh  (Executable, 4.8kB)  ~/stuff/sources/  (Directory)
 
 
 Hit tab again to cycle through the possibilities.

src/builtin.cpp

@@ -394,6 +394,7 @@ static const builtin_data_t builtin_datas[] = {
     {L"string", &builtin_string, N_(L"Manipulate strings")},
     {L"switch", &builtin_generic, N_(L"Conditionally execute a block of commands")},
     {L"test", &builtin_test, N_(L"Test a condition")},
+    {L"time", &builtin_generic, N_(L"Measure how long a command or block takes")},
     {L"true", &builtin_true, N_(L"Return a successful result")},
     {L"ulimit", &builtin_ulimit, N_(L"Set or get the shells resource usage limits")},
     {L"wait", &builtin_wait, N_(L"Wait for background processes completed")},

src/builtin_eval.cpp

@@ -17,6 +17,9 @@
 /// Implementation of eval builtin.
 int builtin_eval(parser_t &parser, io_streams_t &streams, wchar_t **argv) {
     int argc = builtin_count_args(argv);
+    if (argc <= 1) {
+        return STATUS_CMD_OK;
+    }
 
     wcstring new_cmd;
     for (int i = 1; i < argc; ++i) {
@@ -24,19 +27,61 @@ int builtin_eval(parser_t &parser, io_streams_t &streams, wchar_t **argv) {
         new_cmd += argv[i];
     }
 
-    const auto cached_exec_count = parser.libdata().exec_count;
-    int status = STATUS_CMD_OK;
-    if (argc > 1) {
-        if (parser.eval(new_cmd, *streams.io_chain) != eval_result_t::ok) {
-            status = STATUS_CMD_ERROR;
-        } else if (cached_exec_count == parser.libdata().exec_count) {
-            // Issue #5692, in particular, to catch `eval ""`, `eval "begin; end;"`, etc.
-            // where we have an argument but nothing is executed.
-            status = STATUS_CMD_OK;
-        } else {
-            status = parser.get_last_status();
+    // If stdout is piped, then its output must go to the streams, not to the io_chain in our
+    // streams, because the pipe may be intended to be consumed by a process which
+    // is not yet launched (#6806). If stdout is NOT redirected, it must see the tty (#6955). So
+    // create a bufferfill for stdout if and only if stdout is piped.
+    // Note do not do this if stdout is merely redirected (say, to a file); we don't want to
+    // buffer in that case.
+    shared_ptr<io_bufferfill_t> stdout_fill{};
+    if (streams.out_is_piped) {
+        stdout_fill =
+            io_bufferfill_t::create(fd_set_t{}, parser.libdata().read_limit, STDOUT_FILENO);
+        if (!stdout_fill) {
+            // We were unable to create a pipe, probably fd exhaustion.
+            return STATUS_CMD_ERROR;
         }
     }
 
+    // Of course the same applies to stderr.
+    shared_ptr<io_bufferfill_t> stderr_fill{};
+    if (streams.err_is_piped) {
+        stderr_fill =
+            io_bufferfill_t::create(fd_set_t{}, parser.libdata().read_limit, STDERR_FILENO);
+        if (!stderr_fill) {
+            return STATUS_CMD_ERROR;
+        }
+    }
+
+    // Construct the full io chain, perhaps with our bufferfills appended.
+    io_chain_t ios = *streams.io_chain;
+    if (stdout_fill) ios.push_back(stdout_fill);
+    if (stderr_fill) ios.push_back(stderr_fill);
+
+    const auto cached_exec_count = parser.libdata().exec_count;
+    int status = STATUS_CMD_OK;
+    if (parser.eval(new_cmd, ios, block_type_t::top, streams.parent_pgid) != eval_result_t::ok) {
+        status = STATUS_CMD_ERROR;
+    } else if (cached_exec_count == parser.libdata().exec_count) {
+        // Issue #5692, in particular, to catch `eval ""`, `eval "begin; end;"`, etc.
+        // where we have an argument but nothing is executed.
+        status = STATUS_CMD_OK;
+    } else {
+        status = parser.get_last_status();
+    }
+
+    // Finish the bufferfills - exhaust and close our pipes.
+    // Copy the output from the bufferfill back to the streams.
+    // Note it is important that we hold no other references to the bufferfills here - they need to
+    // deallocate to close.
+    ios.clear();
+    if (stdout_fill) {
+        std::shared_ptr<io_buffer_t> output = io_bufferfill_t::finish(std::move(stdout_fill));
+        streams.out.append_narrow_buffer(output->buffer());
+    }
+    if (stderr_fill) {
+        std::shared_ptr<io_buffer_t> errput = io_bufferfill_t::finish(std::move(stderr_fill));
+        streams.err.append_narrow_buffer(errput->buffer());
+    }
     return status;
 }

src/builtin_fg.cpp

@@ -98,9 +98,9 @@ int builtin_fg(parser_t &parser, io_streams_t &streams, wchar_t **argv) {
         std::fwprintf(stderr, FG_MSG, job->job_id(), job->command_wcstr());
     }
 
-    const wcstring ft = tok_first(job->command());
+    wcstring ft = tok_command(job->command());
     // For compatibility with fish 2.0's $_, now replaced with `status current-command`
-    if (!ft.empty()) parser.vars().set_one(L"_", ENV_EXPORT, ft);
+    if (!ft.empty()) parser.set_var_and_fire(L"_", ENV_EXPORT, std::move(ft));
     reader_write_title(job->command(), parser);
 
     parser.job_promote(job);

src/builtin_function.cpp

@@ -104,19 +104,15 @@ static int parse_cmd_opts(function_cmd_opts_t &opts, int *optind,  //!OCLINT(hig
                 event_description_t e(event_type_t::any);
 
                 if ((opt == 'j') && (wcscasecmp(w.woptarg, L"caller") == 0)) {
-                    job_id_t job_id = -1;
-
-                    if (parser.libdata().is_subshell) {
-                        job_id = parser.libdata().caller_job_id;
-                    }
-
-                    if (job_id == -1) {
+                    internal_job_id_t caller_id =
+                        parser.libdata().is_subshell ? parser.libdata().caller_id : 0;
+                    if (caller_id == 0) {
                         streams.err.append_format(
                             _(L"%ls: Cannot find calling job for event handler"), cmd);
                         return STATUS_INVALID_ARGS;
                     }
-                    e.type = event_type_t::job_exit;
-                    e.param1.job_id = job_id;
+                    e.type = event_type_t::caller_exit;
+                    e.param1.caller_id = caller_id;
                 } else if ((opt == 'p') && (wcscasecmp(w.woptarg, L"%self") == 0)) {
                     pid = getpid();
                     e.type = event_type_t::exit;

src/builtin_functions.cpp

@@ -183,8 +183,8 @@ static wcstring functions_def(const wcstring &name) {
                     append_format(out, L" --on-job-exit %d", -d.param1.pid);
                 break;
             }
-            case event_type_t::job_exit: {
-                const job_t *j = job_t::from_job_id(d.param1.job_id);
+            case event_type_t::caller_exit: {
+                const job_t *j = job_t::from_job_id(d.param1.caller_id);
                 if (j) append_format(out, L" --on-job-exit %d", j->pgid);
                 break;
             }

src/builtin_read.cpp

@@ -429,7 +429,6 @@ static int validate_read_args(const wchar_t *cmd, read_cmd_opts_t &opts, int arg
 
 /// The read builtin. Reads from stdin and stores the values in environment variables.
 int builtin_read(parser_t &parser, io_streams_t &streams, wchar_t **argv) {
-    auto &vars = parser.vars();
     wchar_t *cmd = argv[0];
     int argc = builtin_count_args(argv);
     wcstring buff;
@@ -519,22 +518,22 @@ int builtin_read(parser_t &parser, io_streams_t &streams, wchar_t **argv) {
                     }
                 }
 
-                vars.set(*var_ptr++, opts.place, tokens);
+                parser.set_var_and_fire(*var_ptr++, opts.place, std::move(tokens));
             } else {
                 maybe_t<tok_t> t;
                 while ((vars_left() - 1 > 0) && (t = tok.next())) {
                     auto text = tok.text_of(*t);
                     if (unescape_string(text, &out, UNESCAPE_DEFAULT)) {
-                        vars.set_one(*var_ptr++, opts.place, out);
+                        parser.set_var_and_fire(*var_ptr++, opts.place, out);
                     } else {
-                        vars.set_one(*var_ptr++, opts.place, text);
+                        parser.set_var_and_fire(*var_ptr++, opts.place, text);
                     }
                 }
 
                 // If we still have tokens, set the last variable to them.
                 if ((t = tok.next())) {
                     wcstring rest = wcstring(buff, t->offset);
-                    vars.set_one(*var_ptr++, opts.place, rest);
+                    parser.set_var_and_fire(*var_ptr++, opts.place, std::move(rest));
                 }
             }
             // The rest of the loop is other split-modes, we don't care about those.
@@ -567,12 +566,12 @@ int builtin_read(parser_t &parser, io_streams_t &streams, wchar_t **argv) {
 
             if (opts.array) {
                 // Array mode: assign each char as a separate element of the sole var.
-                vars.set(*var_ptr++, opts.place, chars);
+                parser.set_var_and_fire(*var_ptr++, opts.place, chars);
             } else {
                 // Not array mode: assign each char to a separate var with the remainder being
                 // assigned to the last var.
                 for (const auto &c : chars) {
-                    vars.set_one(*var_ptr++, opts.place, c);
+                    parser.set_var_and_fire(*var_ptr++, opts.place, c);
                 }
             }
         } else if (opts.array) {
@@ -588,14 +587,14 @@ int builtin_read(parser_t &parser, io_streams_t &streams, wchar_t **argv) {
                      loc.first != wcstring::npos; loc = wcstring_tok(buff, opts.delimiter, loc)) {
                     tokens.emplace_back(wcstring(buff, loc.first, loc.second));
                 }
-                vars.set(*var_ptr++, opts.place, tokens);
+                parser.set_var_and_fire(*var_ptr++, opts.place, tokens);
             } else {
                 // We're using a delimiter provided by the user so use the `string split` behavior.
                 wcstring_list_t splits;
                 split_about(buff.begin(), buff.end(), opts.delimiter.begin(), opts.delimiter.end(),
                             &splits);
 
-                vars.set(*var_ptr++, opts.place, splits);
+                parser.set_var_and_fire(*var_ptr++, opts.place, splits);
             }
         } else {
             // Not array mode. Split the input into tokens and assign each to the vars in sequence.
@@ -607,17 +606,9 @@ int builtin_read(parser_t &parser, io_streams_t &streams, wchar_t **argv) {
                     wcstring substr;
                     loc = wcstring_tok(buff, (vars_left() > 1) ? opts.delimiter : wcstring(), loc);
                     if (loc.first != wcstring::npos) {
-                        if (vars_left() == 1) {  // Discard trailing delimiters, see #6406
-                            loc.first =
-                                std::find_if(buff.begin() + loc.first, buff.end(),
-                                             [&opts](wchar_t c) {
-                                                 return opts.delimiter.find(c) == wcstring::npos;
-                                             }) -
-                                buff.begin();
-                        }
                         substr = wcstring(buff, loc.first, loc.second);
                     }
-                    vars.set_one(*var_ptr++, opts.place, substr);
+                    parser.set_var_and_fire(*var_ptr++, opts.place, substr);
                 }
             } else {
                 // We're using a delimiter provided by the user so use the `string split` behavior.
@@ -628,7 +619,7 @@ int builtin_read(parser_t &parser, io_streams_t &streams, wchar_t **argv) {
                             &splits, argc - 1);
                 assert(splits.size() <= (size_t)vars_left());
                 for (const auto &split : splits) {
-                    vars.set_one(*var_ptr++, opts.place, split);
+                    parser.set_var_and_fire(*var_ptr++, opts.place, split);
                 }
             }
         }

src/builtin_test.cpp

@@ -665,10 +665,12 @@ static bool parse_number(const wcstring &arg, number_t *number, wcstring_list_t
         *number = number_t{integral, 0.0};
 
         return true;
-    } else if (got_float && errno != ERANGE) {
+    } else if (got_float && errno != ERANGE && std::isfinite(floating)) {
         // Here we parsed an (in range) floating point value that could not be parsed as an integer.
         // Break the floating point value into base and delta. Ensure that base is <= the floating
         // point value.
+        //
+        // Note that a non-finite number like infinity or NaN doesn't work for us, so we checked above.
         double intpart = std::floor(floating);
         double delta = floating - intpart;
         *number = number_t{static_cast<long long>(intpart), delta};
@@ -680,6 +682,11 @@ static bool parse_number(const wcstring &arg, number_t *number, wcstring_list_t
         if (errno == -1) {
             errors.push_back(
                 format_string(_(L"Integer %lld in '%ls' followed by non-digit"), integral, argcs));
+        } else if (std::isnan(floating)) {
+            // NaN is an error as far as we're concerned.
+            errors.push_back(_(L"Not a number"));
+        } else if (std::isinf(floating)) {
+            errors.push_back(_(L"Number is infinite"));
         } else {
             errors.push_back(format_string(L"%s: '%ls'", std::strerror(errno), argcs));
         }

src/common.cpp

@@ -1435,7 +1435,6 @@ static bool unescape_string_internal(const wchar_t *const input, const size_t in
     // The positions of variable expansions or brace ","s.
     // We only read braces as expanders if there's a variable expansion or "," in them.
     std::vector<size_t> vars_or_seps;
-    bool brace_text_start = false;
     int brace_count = 0;
 
     bool errored = false;
@@ -1531,7 +1530,6 @@ static bool unescape_string_internal(const wchar_t *const input, const size_t in
                         // assert(brace_count > 0 && "imbalanced brackets are a tokenizer error, we
                         // shouldn't be able to get here");
                         brace_count--;
-                        brace_text_start = brace_text_start && brace_count > 0;
                         to_append_or_none = BRACE_END;
                         if (!braces.empty()) {
                             // If we didn't have a var or separator since the last '{',
@@ -1559,17 +1557,13 @@ static bool unescape_string_internal(const wchar_t *const input, const size_t in
                 case L',': {
                     if (unescape_special && brace_count > 0) {
                         to_append_or_none = BRACE_SEP;
-                        brace_text_start = false;
                         vars_or_seps.push_back(input_position);
                     }
                     break;
                 }
-                case L'\n':
-                case L'\t':
                 case L' ': {
                     if (unescape_special && brace_count > 0) {
-                        to_append_or_none =
-                            brace_text_start ? maybe_t<wchar_t>(BRACE_SPACE) : none();
+                        to_append_or_none = BRACE_SPACE;
                     }
                     break;
                 }
@@ -1586,9 +1580,6 @@ static bool unescape_string_internal(const wchar_t *const input, const size_t in
                     break;
                 }
                 default: {
-                    if (unescape_special && brace_count > 0) {
-                        brace_text_start = true;
-                    }
                     break;
                 }
             }

src/env_dispatch.cpp

@@ -347,16 +347,16 @@ static void update_fish_color_support(const environment_t &vars) {
         // Assume that all 'xterm's can handle 256, except for Terminal.app from Snow Leopard
         wcstring term_program;
         if (auto tp = vars.get(L"TERM_PROGRAM")) term_program = tp->as_string();
-        if (auto tpv = vars.get(L"TERM_PROGRAM_VERSION")) {
-            if (term_program == L"Apple_Terminal" &&
-                fish_wcstod(tpv->as_string().c_str(), nullptr) > 299) {
+        if (term_program == L"Apple_Terminal") {
+            auto tpv = vars.get(L"TERM_PROGRAM_VERSION");
+            if (tpv && fish_wcstod(tpv->as_string().c_str(), nullptr) > 299) {
                 // OS X Lion is version 299+, it has 256 color support (see github Wiki)
                 support_term256 = true;
                 debug(2, L"256 color support enabled for TERM=%ls on Terminal.app", term.c_str());
-            } else {
-                support_term256 = true;
-                debug(2, L"256 color support enabled for TERM=%ls", term.c_str());
             }
+        } else {
+            support_term256 = true;
+            debug(2, L"256 color support enabled for TERM=%ls", term.c_str());
         }
     } else if (cur_term != nullptr) {
         // See if terminfo happens to identify 256 colors

src/env_universal_common.cpp

@@ -346,11 +346,12 @@ void env_universal_t::generate_callbacks_and_update_exports(const var_table_t &n
 
         bool old_exports = (existing != this->vars.end() && existing->second.exports());
         bool export_changed = (old_exports != new_entry.exports());
-        if (export_changed) {
+        bool value_changed = existing != this->vars.end() && existing->second != new_entry;
+        if (export_changed || value_changed) {
             export_generation += 1;
         }
-        if (existing == this->vars.end() || export_changed || existing->second != new_entry) {
-            // Value has changed.
+        if (existing == this->vars.end() || export_changed || value_changed) {
+            // Value is set for the first time, or has changed.
             callbacks.push_back(callback_data_t(key, new_entry.as_string()));
         }
     }

src/event.cpp

@@ -113,8 +113,8 @@ static bool handler_matches(const event_handler_t &classv, const event_t &instan
             if (classv.desc.param1.pid == EVENT_ANY_PID) return true;
             return classv.desc.param1.pid == instance.desc.param1.pid;
         }
-        case event_type_t::job_exit: {
-            return classv.desc.param1.job_id == instance.desc.param1.job_id;
+        case event_type_t::caller_exit: {
+            return classv.desc.param1.caller_id == instance.desc.param1.caller_id;
         }
         case event_type_t::generic: {
             return classv.desc.str_param1 == instance.desc.str_param1;
@@ -167,13 +167,13 @@ wcstring event_get_desc(const event_t &evt) {
             DIE("Unreachable");
         }
 
-        case event_type_t::job_exit: {
-            job_t *j = job_t::from_job_id(ed.param1.job_id);
+        case event_type_t::caller_exit: {
+            job_t *j = job_t::from_job_id(ed.param1.caller_id);
             if (j) {
                 return format_string(_(L"exit handler for job %d, '%ls'"), j->job_id(),
                                      j->command_wcstr());
             } else {
-                return format_string(_(L"exit handler for job with job id %d"), ed.param1.job_id);
+                return format_string(_(L"exit handler for job with job id %d"), ed.param1.caller_id);
             }
             break;
         }
@@ -297,6 +297,8 @@ void event_fire_delayed(parser_t &parser) {
     auto &ld = parser.libdata();
     // Do not invoke new event handlers from within event handlers.
     if (ld.is_event) return;
+    // Do not invoke new event handlers if we are unwinding (#6649).
+    if (parser.get_cancel_signal()) return;
 
     std::vector<shared_ptr<event_t>> to_send;
     to_send.swap(ld.blocked_events);
@@ -351,7 +353,7 @@ struct event_type_name_t {
 static const event_type_name_t events_mapping[] = {{event_type_t::signal, L"signal"},
                                                    {event_type_t::variable, L"variable"},
                                                    {event_type_t::exit, L"exit"},
-                                                   {event_type_t::job_exit, L"job-id"},
+                                                   {event_type_t::caller_exit, L"job-id"},
                                                    {event_type_t::generic, L"generic"}};
 
 maybe_t<event_type_t> event_type_for_name(const wcstring &name) {
@@ -386,8 +388,8 @@ void event_print(io_streams_t &streams, maybe_t<event_type_t> type_filter) {
                           return d1.signal < d2.signal;
                       case event_type_t::exit:
                           return d1.param1.pid < d2.param1.pid;
-                      case event_type_t::job_exit:
-                          return d1.param1.job_id < d2.param1.job_id;
+                      case event_type_t::caller_exit:
+                          return d1.param1.caller_id < d2.param1.caller_id;
                       case event_type_t::variable:
                       case event_type_t::any:
                       case event_type_t::generic:
@@ -414,7 +416,7 @@ void event_print(io_streams_t &streams, maybe_t<event_type_t> type_filter) {
                                           evt->function_name.c_str());
                 break;
             case event_type_t::exit:
-            case event_type_t::job_exit:
+            case event_type_t::caller_exit:
                 streams.out.append_format(L"%d %ls\n", evt->desc.param1,
                                           evt->function_name.c_str());
                 break;

src/event.h

@@ -29,8 +29,8 @@ enum class event_type_t {
     variable,
     /// An event triggered by a job or process exit.
     exit,
-    /// An event triggered by a job exit.
-    job_exit,
+    /// An event triggered by a caller exit.
+    caller_exit,
     /// A generic event.
     generic,
 };
@@ -44,10 +44,11 @@ struct event_description_t {
     ///
     /// signal: Signal number for signal-type events.Use EVENT_ANY_SIGNAL to match any signal
     /// pid: Process id for process-type events. Use EVENT_ANY_PID to match any pid. (Negative
-    /// values are used for PGIDs). job_id: Job id for EVENT_JOB_ID type events
+    /// values are used for PGIDs).
+    /// caller_id: Internal job id for caller_exit type events
     union {
         int signal;
-        int job_id;
+        uint64_t caller_id;
         pid_t pid;
     } param1{};
 

src/exec.cpp

@@ -428,9 +428,16 @@ static bool exec_internal_builtin_proc(parser_t &parser, const std::shared_ptr<j
         }
     }
 
+    // Pull out the IOs for stdout and stderr.
+    auto out_io = proc_io_chain.io_for_fd(STDOUT_FILENO);
+    auto err_io = proc_io_chain.io_for_fd(STDERR_FILENO);
+
+    // Set up our streams.
     streams.stdin_fd = local_builtin_stdin;
-    streams.out_is_redirected = proc_io_chain.io_for_fd(STDOUT_FILENO) != nullptr;
-    streams.err_is_redirected = proc_io_chain.io_for_fd(STDERR_FILENO) != nullptr;
+    streams.out_is_redirected = out_io != nullptr;
+    streams.err_is_redirected = err_io != nullptr;
+    streams.out_is_piped = (out_io != nullptr && out_io->io_mode == io_mode_t::pipe);
+    streams.err_is_piped = (err_io != nullptr && err_io->io_mode == io_mode_t::pipe);
     streams.stdin_is_directly_redirected = stdin_is_directly_redirected;
     streams.io_chain = &proc_io_chain;
 
@@ -911,6 +918,7 @@ static bool exec_process_in_job(parser_t &parser, process_t *p, const std::share
 
         case process_type_t::builtin: {
             io_streams_t builtin_io_streams{stdout_read_limit};
+            if (j->pgid != INVALID_PID) builtin_io_streams.parent_pgid = j->pgid;
             if (!exec_internal_builtin_proc(parser, j, p, pipe_read.get(), process_net_io_chain,
                                             builtin_io_streams)) {
                 return false;
@@ -1121,8 +1129,8 @@ bool exec_job(parser_t &parser, const shared_ptr<job_t> &j, const job_lineage_t
     return true;
 }
 
-static int exec_subshell_internal(const wcstring &cmd, parser_t &parser, wcstring_list_t *lst,
-                                  bool apply_exit_status, bool is_subcmd) {
+static int exec_subshell_internal(const wcstring &cmd, parser_t &parser, maybe_t<pid_t> parent_pgid,
+                                  wcstring_list_t *lst, bool apply_exit_status, bool is_subcmd) {
     ASSERT_IS_MAIN_THREAD();
     auto &ld = parser.libdata();
     bool prev_subshell = ld.is_subshell;
@@ -1144,7 +1152,8 @@ static int exec_subshell_internal(const wcstring &cmd, parser_t &parser, wcstrin
     // be null.
     std::shared_ptr<io_buffer_t> buffer;
     if (auto bufferfill = io_bufferfill_t::create(fd_set_t{}, ld.read_limit)) {
-        if (parser.eval(cmd, io_chain_t{bufferfill}, block_type_t::subst) == eval_result_t::ok) {
+        if (parser.eval(cmd, io_chain_t{bufferfill}, block_type_t::subst, parent_pgid) ==
+            eval_result_t::ok) {
             subcommand_statuses = parser.get_last_statuses();
         }
         buffer = io_bufferfill_t::finish(std::move(bufferfill));
@@ -1215,12 +1224,12 @@ static int exec_subshell_internal(const wcstring &cmd, parser_t &parser, wcstrin
 }
 
 int exec_subshell(const wcstring &cmd, parser_t &parser, wcstring_list_t &outputs,
-                  bool apply_exit_status, bool is_subcmd) {
+                  bool apply_exit_status, bool is_subcmd, maybe_t<pid_t> parent_pgid) {
     ASSERT_IS_MAIN_THREAD();
-    return exec_subshell_internal(cmd, parser, &outputs, apply_exit_status, is_subcmd);
+    return exec_subshell_internal(cmd, parser, parent_pgid, &outputs, apply_exit_status, is_subcmd);
 }
 
 int exec_subshell(const wcstring &cmd, parser_t &parser, bool apply_exit_status, bool is_subcmd) {
     ASSERT_IS_MAIN_THREAD();
-    return exec_subshell_internal(cmd, parser, nullptr, apply_exit_status, is_subcmd);
+    return exec_subshell_internal(cmd, parser, none(), nullptr, apply_exit_status, is_subcmd);
 }

src/exec.h

@@ -22,10 +22,12 @@ bool exec_job(parser_t &parser, const std::shared_ptr<job_t> &j, const job_linea
 ///
 /// \param cmd the command to execute
 /// \param outputs The list to insert output into.
+/// \param parent_pgid if set, the pgid for any spawned jobs
 ///
 /// \return the status of the last job to exit, or -1 if en error was encountered.
 int exec_subshell(const wcstring &cmd, parser_t &parser, wcstring_list_t &outputs,
-                  bool apply_exit_status, bool is_subcmd = false);
+                  bool apply_exit_status, bool is_subcmd = false,
+                  maybe_t<pid_t> parent_pgid = none());
 int exec_subshell(const wcstring &cmd, parser_t &parser, bool apply_exit_status,
                   bool is_subcmd = false);
 

src/expand.cpp

@@ -17,6 +17,7 @@
 #include <procfs.h>
 #endif
 #if __APPLE__
+#include <sys/time.h> // Required to build with old SDK versions
 #include <sys/proc.h>
 #else
 #include <dirent.h>
@@ -45,6 +46,7 @@
 #include "path.h"
 #include "proc.h"
 #include "reader.h"
+#include "util.h"
 #include "wcstringutil.h"
 #include "wildcard.h"
 #include "wutil.h"  // IWYU pragma: keep
@@ -575,8 +577,9 @@ static expand_result_t expand_braces(const wcstring &instr, expand_flags_t flags
 }
 
 /// Perform cmdsubst expansion.
-static bool expand_cmdsubst(wcstring input, parser_t &parser, completion_list_t *out_list,
-                            parse_error_list_t *errors) {
+static bool expand_cmdsubst(wcstring input, const operation_context_t &ctx,
+                            completion_list_t *out_list, parse_error_list_t *errors) {
+    assert(ctx.parser && "Cannot expand without a parser");
     wchar_t *paren_begin = nullptr, *paren_end = nullptr;
     wchar_t *tail_begin = nullptr;
     size_t i, j;
@@ -603,14 +606,14 @@ static bool expand_cmdsubst(wcstring input, parser_t &parser, completion_list_t
 
     wcstring_list_t sub_res;
     const wcstring subcmd(paren_begin + 1, paren_end - paren_begin - 1);
-    if (exec_subshell(subcmd, parser, sub_res, true /* apply_exit_status */,
-                      true /* is_subcmd */) == -1) {
+    if (exec_subshell(subcmd, *ctx.parser, sub_res, true /* apply_exit_status */,
+                      true /* is_subcmd */, ctx.parent_pgid) == -1) {
         append_cmdsub_error(errors, SOURCE_LOCATION_UNKNOWN,
                             L"Unknown error while evaluating command substitution");
         return false;
     }
 
-    if (parser.get_last_status() == STATUS_READ_TOO_MUCH) {
+    if (ctx.parser->get_last_status() == STATUS_READ_TOO_MUCH) {
         append_cmdsub_error(
             errors, in - paren_begin,
             _(L"Too much data emitted by command substitution so it was discarded\n"));
@@ -650,7 +653,7 @@ static bool expand_cmdsubst(wcstring input, parser_t &parser, completion_list_t
     // Recursively call ourselves to expand any remaining command substitutions. The result of this
     // recursive call using the tail of the string is inserted into the tail_expand array list
     completion_list_t tail_expand;
-    expand_cmdsubst(tail_begin, parser, &tail_expand, errors);  // TODO: offset error locations
+    expand_cmdsubst(tail_begin, ctx, &tail_expand, errors);  // TODO: offset error locations
 
     // Combine the result of the current command substitution with the result of the recursive tail
     // expansion.
@@ -684,7 +687,7 @@ static bool expand_cmdsubst(wcstring input, parser_t &parser, completion_list_t
         }
     }
 
-    return parser.get_last_status() != STATUS_READ_TOO_MUCH;
+    return ctx.parser->get_last_status() != STATUS_READ_TOO_MUCH;
 }
 
 // Given that input[0] is HOME_DIRECTORY or tilde (ugh), return the user's name. Return the empty
@@ -895,7 +898,7 @@ expand_result_t expander_t::stage_cmdsubst(wcstring input, completion_list_t *ou
         }
     } else {
         assert(ctx.parser && "Must have a parser to expand command substitutions");
-        bool cmdsubst_ok = expand_cmdsubst(std::move(input), *ctx.parser, out, errors);
+        bool cmdsubst_ok = expand_cmdsubst(std::move(input), ctx, out, errors);
         if (!cmdsubst_ok) return expand_result_t::error;
     }
 
@@ -1017,7 +1020,11 @@ expand_result_t expander_t::stage_wildcards(wcstring path_to_expand, completion_
             }
         }
 
-        std::sort(expanded.begin(), expanded.end(), completion_t::is_naturally_less_than);
+        std::sort(expanded.begin(), expanded.end(),
+                  [&](const completion_t &a, const completion_t &b) {
+                      return wcsfilecmp_glob(a.completion.c_str(), b.completion.c_str()) < 0;
+                  });
+
         std::move(expanded.begin(), expanded.end(), std::back_inserter(*out));
     } else {
         // Can't fully justify this check. I think it's that SKIP_WILDCARDS is used when completing

src/fish.cpp

@@ -505,7 +505,7 @@ int main(int argc, char **argv) {
                 for (char **ptr = argv + my_optind; *ptr; ptr++) {
                     list.push_back(str2wcstring(*ptr));
                 }
-                parser.vars().set(L"argv", ENV_DEFAULT, list);
+                parser.vars().set(L"argv", ENV_DEFAULT, std::move(list));
 
                 auto &ld = parser.libdata();
                 wcstring rel_filename = str2wcstring(file);

src/input.cpp

@@ -117,6 +117,7 @@ static const input_function_metadata_t input_function_metadata[] = {
     {readline_cmd_t::history_token_search_backward, L"history-token-search-backward"},
     {readline_cmd_t::history_token_search_forward, L"history-token-search-forward"},
     {readline_cmd_t::self_insert, L"self-insert"},
+    {readline_cmd_t::self_insert_notfirst, L"self-insert-notfirst"},
     {readline_cmd_t::transpose_chars, L"transpose-chars"},
     {readline_cmd_t::transpose_words, L"transpose-words"},
     {readline_cmd_t::upcase_word, L"upcase-word"},
@@ -181,12 +182,13 @@ static wcstring input_get_bind_mode(const environment_t &vars) {
 }
 
 /// Set the current bind mode.
-static void input_set_bind_mode(env_stack_t &vars, const wcstring &bm) {
+static void input_set_bind_mode(parser_t &parser, const wcstring &bm) {
     // Only set this if it differs to not execute variable handlers all the time.
     // modes may not be empty - empty is a sentinel value meaning to not change the mode
     assert(!bm.empty());
-    if (input_get_bind_mode(vars) != bm) {
-        vars.set_one(FISH_BIND_MODE_VAR, ENV_GLOBAL, bm);
+    if (input_get_bind_mode(parser.vars()) != bm) {
+        // Must send events here - see #6653.
+        parser.set_var_and_fire(FISH_BIND_MODE_VAR, ENV_GLOBAL, bm);
     }
 }
 
@@ -360,7 +362,7 @@ void inputter_t::mapping_execute(const input_mapping_t &m, bool allow_commands)
 
     // !has_functions && !has_commands: only set bind mode
     if (!has_commands && !has_functions) {
-        if (!m.sets_mode.empty()) input_set_bind_mode(parser_->vars(), m.sets_mode);
+        if (!m.sets_mode.empty()) input_set_bind_mode(*parser_, m.sets_mode);
         return;
     }
 
@@ -399,7 +401,7 @@ void inputter_t::mapping_execute(const input_mapping_t &m, bool allow_commands)
     }
 
     // Empty bind mode indicates to not reset the mode (#2871)
-    if (!m.sets_mode.empty()) input_set_bind_mode(parser_->vars(), m.sets_mode);
+    if (!m.sets_mode.empty()) input_set_bind_mode(*parser_, m.sets_mode);
 }
 
 /// Try reading the specified function mapping.
@@ -496,7 +498,8 @@ char_event_t inputter_t::readch(bool allow_commands) {
 
         if (evt.is_readline()) {
             switch (evt.get_readline()) {
-                case readline_cmd_t::self_insert: {
+                case readline_cmd_t::self_insert:
+                case readline_cmd_t::self_insert_notfirst: {
                     // Typically self-insert is generated by the generic (empty) binding.
                     // However if it is generated by a real sequence, then insert that sequence.
                     for (auto iter = evt.seq.crbegin(); iter != evt.seq.crend(); ++iter) {
@@ -504,7 +507,13 @@ char_event_t inputter_t::readch(bool allow_commands) {
                     }
                     // Issue #1595: ensure we only insert characters, not readline functions. The
                     // common case is that this will be empty.
-                    return read_characters_no_readline();
+                    char_event_t res = read_characters_no_readline();
+
+                    // Hackish: mark the input style.
+                    res.input_style = evt.get_readline() == readline_cmd_t::self_insert_notfirst
+                                          ? char_input_style_t::notfirst
+                                          : char_input_style_t::normal;
+                    return res;
                 }
                 case readline_cmd_t::func_and: {
                     if (function_status_) {

src/input_common.h

@@ -42,6 +42,7 @@ enum class readline_cmd_t {
     history_token_search_backward,
     history_token_search_forward,
     self_insert,
+    self_insert_notfirst,
     transpose_chars,
     transpose_words,
     upcase_word,
@@ -96,6 +97,16 @@ enum class char_event_type_t : uint8_t {
     check_exit,
 };
 
+/// Hackish: the input style, which describes how char events (only) are applied to the command
+/// line. Note this is set only after applying bindings; it is not set from readb().
+enum class char_input_style_t : uint8_t {
+    // Insert characters normally.
+    normal,
+
+    // Insert characters only if the cursor is not at the beginning. Otherwise, discard them.
+    notfirst,
+};
+
 class char_event_t {
     union {
         /// Set if the type is charc.
@@ -109,6 +120,9 @@ class char_event_t {
     /// The type of event.
     char_event_type_t type;
 
+    /// The style to use when inserting characters into the command line.
+    char_input_style_t input_style{char_input_style_t::normal};
+
     /// The sequence of characters in the input mapping which generated this event.
     /// Note that the generic self-insert case does not have any characters, so this would be empty.
     wcstring seq{};

src/io.cpp

@@ -174,8 +174,10 @@ void io_buffer_t::complete_background_fillthread() {
     fillthread_waiter_ = {};
 }
 
-shared_ptr<io_bufferfill_t> io_bufferfill_t::create(const fd_set_t &conflicts,
-                                                    size_t buffer_limit) {
+shared_ptr<io_bufferfill_t> io_bufferfill_t::create(const fd_set_t &conflicts, size_t buffer_limit,
+                                                    int target) {
+    assert(target >= 0 && "Invalid target fd");
+
     // Construct our pipes.
     auto pipes = make_autoclose_pipes(conflicts);
     if (!pipes) {
@@ -192,7 +194,7 @@ shared_ptr<io_bufferfill_t> io_bufferfill_t::create(const fd_set_t &conflicts,
     // Our fillthread gets the read end of the pipe; out_pipe gets the write end.
     auto buffer = std::make_shared<io_buffer_t>(buffer_limit);
     buffer->begin_background_fillthread(std::move(pipes->read));
-    return std::make_shared<io_bufferfill_t>(std::move(pipes->write), buffer);
+    return std::make_shared<io_bufferfill_t>(target, std::move(pipes->write), buffer);
 }
 
 std::shared_ptr<io_buffer_t> io_bufferfill_t::finish(std::shared_ptr<io_bufferfill_t> &&filler) {
@@ -346,3 +348,9 @@ shared_ptr<const io_data_t> io_chain_t::io_for_fd(int fd) const {
     }
     return nullptr;
 }
+
+void output_stream_t::append_narrow_buffer(const separated_buffer_t<std::string> &buffer) {
+    for (const auto &rhs_elem : buffer.elements()) {
+        buffer_.append(str2wcstring(rhs_elem.contents), rhs_elem.separation);
+    }
+}

src/io.h

@@ -252,7 +252,6 @@ class io_buffer_t;
 class io_chain_t;
 
 /// Represents filling an io_buffer_t. Very similar to io_pipe_t.
-/// Bufferfills always target stdout.
 class io_bufferfill_t : public io_data_t {
     /// Write end. The other end is connected to an io_buffer_t.
     const autoclose_fd_t write_fd_;
@@ -265,8 +264,8 @@ class io_bufferfill_t : public io_data_t {
 
     // The ctor is public to support make_shared() in the static create function below.
     // Do not invoke this directly.
-    io_bufferfill_t(autoclose_fd_t write_fd, std::shared_ptr<io_buffer_t> buffer)
-        : io_data_t(io_mode_t::bufferfill, STDOUT_FILENO, write_fd.fd()),
+    io_bufferfill_t(int target, autoclose_fd_t write_fd, std::shared_ptr<io_buffer_t> buffer)
+        : io_data_t(io_mode_t::bufferfill, target, write_fd.fd()),
           write_fd_(std::move(write_fd)),
           buffer_(std::move(buffer)) {
         assert(write_fd_.valid() && "fd is not valid");
@@ -279,9 +278,11 @@ class io_bufferfill_t : public io_data_t {
     /// Create an io_bufferfill_t which, when written from, fills a buffer with the contents.
     /// \returns nullptr on failure, e.g. too many open fds.
     ///
+    /// \param target the fd which this will be dup2'd to - typically stdout.
     /// \param conflicts A set of fds. The function ensures that any pipe it makes does
     /// not conflict with an fd redirection in this list.
-    static shared_ptr<io_bufferfill_t> create(const fd_set_t &conflicts, size_t buffer_limit = 0);
+    static shared_ptr<io_bufferfill_t> create(const fd_set_t &conflicts, size_t buffer_limit = 0,
+                                              int target = STDOUT_FILENO);
 
     /// Reset the receiver (possibly closing the write end of the pipe), and complete the fillthread
     /// of the buffer. \return the buffer.
@@ -421,6 +422,9 @@ class output_stream_t {
 
     void append(const wchar_t *s, size_t amt) { buffer_.append(s, s + amt); }
 
+    // Append data from a narrow buffer, widening it.
+    void append_narrow_buffer(const separated_buffer_t<std::string> &buffer);
+
     void push_back(wchar_t c) { append(c); }
 
     void append_format(const wchar_t *format, ...) {
@@ -447,13 +451,23 @@ struct io_streams_t {
     // < foo.txt
     bool stdin_is_directly_redirected{false};
 
-    // Indicates whether stdout and stderr are redirected (e.g. to a file or piped).
+    // Indicates whether stdout and stderr are specifically piped.
+    // If this is set, then the is_redirected flags must also be set.
+    bool out_is_piped{false};
+    bool err_is_piped{false};
+
+    // Indicates whether stdout and stderr are at all redirected (e.g. to a file or piped).
     bool out_is_redirected{false};
     bool err_is_redirected{false};
 
     // Actual IO redirections. This is only used by the source builtin. Unowned.
     const io_chain_t *io_chain{nullptr};
 
+    // The pgid of the job, if any. This enables builtins which run more code like eval() to share
+    // pgid.
+    // TODO: this is awkwardly placed, consider just embedding a lineage here.
+    maybe_t<pid_t> parent_pgid{};
+
     // io_streams_t cannot be copied.
     io_streams_t(const io_streams_t &) = delete;
     void operator=(const io_streams_t &) = delete;

src/iothread.cpp

@@ -6,6 +6,7 @@
 #include <pthread.h>
 #include <signal.h>
 #include <stdio.h>
+#include <string.h>
 #include <sys/select.h>
 #include <sys/time.h>
 #include <sys/types.h>

src/iothread.h

@@ -4,6 +4,7 @@
 
 #include <pthread.h>
 
+#include <cstdint> // for uint64_t
 #include <functional>
 #include <type_traits>
 

src/operation_context.h

@@ -29,6 +29,11 @@ class operation_context_t {
     // context itself.
     const environment_t &vars;
 
+    /// The pgid of the parental job.
+    /// This is used only when expanding command substitutions. If this is set, any jobs created by
+    /// the command substitions should use this pgid.
+    maybe_t<pid_t> parent_pgid{};
+
     // A function which may be used to poll for cancellation.
     cancel_checker_t cancel_checker;
 

src/output.cpp

@@ -402,9 +402,14 @@ int outputter_t::writech(wint_t ch) {
 void outputter_t::writestr(const wchar_t *str) {
     assert(str && "Empty input string");
 
+    if (MB_CUR_MAX == 1) {
+        // Single-byte locale (C/POSIX/ISO-8859).
+        while (*str) writech(*str++);
+        return;
+    }
     size_t len = wcstombs(nullptr, str, 0);  // figure amount of space needed
     if (len == static_cast<size_t>(-1)) {
-        debug(1, L"Tried to print invalid wide character string");
+        debug(3, L"Tried to print invalid wide character string");
         return;
     }
 

src/parse_execution.cpp

@@ -403,9 +403,9 @@ eval_result_t parse_execution_context_t::run_for_statement(
     }
     int retval;
     if (var) {
-        retval = parser->vars().set(for_var_name, ENV_LOCAL | ENV_USER, var->as_list());
+        retval = parser->set_var_and_fire(for_var_name, ENV_LOCAL | ENV_USER, var->as_list());
     } else {
-        retval = parser->vars().set_empty(for_var_name, ENV_LOCAL | ENV_USER);
+        retval = parser->set_empty_var_and_fire(for_var_name, ENV_LOCAL | ENV_USER);
     }
     assert(retval == ENV_OK);
 
@@ -424,7 +424,7 @@ eval_result_t parse_execution_context_t::run_for_statement(
             break;
         }
 
-        int retval = parser->vars().set_one(for_var_name, ENV_DEFAULT | ENV_USER, val);
+        int retval = parser->set_var_and_fire(for_var_name, ENV_DEFAULT | ENV_USER, val);
         assert(retval == ENV_OK && "for loop variable should have been successfully set");
         (void)retval;
 
@@ -1037,7 +1037,7 @@ eval_result_t parse_execution_context_t::apply_variable_assignments(
             vals.emplace_back(std::move(completion.completion));
         }
         if (proc) proc->variable_assignments.push_back({variable_name, vals});
-        parser->vars().set(variable_name, ENV_LOCAL | ENV_EXPORT, std::move(vals));
+        parser->set_var_and_fire(variable_name, ENV_LOCAL | ENV_EXPORT, std::move(vals));
     }
     return eval_result_t::ok;
 }
@@ -1283,17 +1283,13 @@ eval_result_t parse_execution_context_t::run_1_job(tnode_t<g::job> job_node,
     // We are about to populate a job. One possible argument to the job is a command substitution
     // which may be interested in the job that's populating it, via '--on-job-exit caller'. Record
     // the job ID here.
-    auto &libdata = parser->libdata();
-    const auto saved_caller_jid = libdata.caller_job_id;
-    libdata.caller_job_id = job->job_id();
+    scoped_push<internal_job_id_t> caller_id(&parser->libdata().caller_id, job->internal_job_id);
 
     // Populate the job. This may fail for reasons like command_not_found. If this fails, an error
     // will have been printed.
     eval_result_t pop_result =
         this->populate_job_from_job_node(job.get(), job_node, associated_block);
-
-    assert(libdata.caller_job_id == job->job_id() && "Caller job ID unexpectedly changed");
-    parser->libdata().caller_job_id = saved_caller_jid;
+    caller_id.restore();
 
     // Store time it took to 'parse' the command.
     if (profile_item != nullptr) {

src/parse_productions.cpp

@@ -397,16 +397,12 @@ RESOLVE(optional_background) {
 }
 
 RESOLVE(optional_time) {
-    UNUSED(token2);
-
-    switch (token1.keyword) {
-        case parse_keyword_time:
-            *out_tag = parse_optional_time_time;
-            return production_for<time>();
-        default:
-            *out_tag = parse_optional_time_no_time;
-            return production_for<empty>();
+    if (token1.keyword == parse_keyword_time && !token2.is_help_argument) {
+        *out_tag = parse_optional_time_time;
+        return production_for<time>();
     }
+    *out_tag = parse_optional_time_no_time;
+    return production_for<empty>();
 }
 
 const production_element_t *parse_productions::production_for_token(parse_token_type_t node_type,

src/parse_tree.cpp

@@ -1073,26 +1073,6 @@ static inline bool is_help_argument(const wcstring &txt) {
     return txt == L"-h" || txt == L"--help";
 }
 
-// Return the location of the equals sign, or npos if the string does
-// not look like a variable assignment like FOO=bar.  The detection
-// works similar as in some POSIX shells: only letters and numbers qre
-// allowed on the left hand side, no quotes or escaping.
-maybe_t<size_t> variable_assignment_equals_pos(const wcstring &txt) {
-    enum { init, has_some_variable_identifier } state = init;
-    // TODO bracket indexing
-    for (size_t i = 0; i < txt.size(); i++) {
-        wchar_t c = txt[i];
-        if (state == init) {
-            if (!valid_var_name_char(c)) return {};
-            state = has_some_variable_identifier;
-        } else {
-            if (c == '=') return {i};
-            if (!valid_var_name_char(c)) return {};
-        }
-    }
-    return {};
-}
-
 /// Return a new parse token, advancing the tokenizer.
 static inline parse_token_t next_parse_token(tokenizer_t *tok, maybe_t<tok_t> *out_token,
                                              wcstring *storage) {

src/parse_tree.h

@@ -235,9 +235,6 @@ using parsed_source_ref_t = std::shared_ptr<const parsed_source_t>;
 parsed_source_ref_t parse_source(wcstring src, parse_tree_flags_t flags, parse_error_list_t *errors,
                                  parse_token_type_t goal = symbol_job_list);
 
-/// The position of the equal sign in a variable assignment like foo=bar.
-maybe_t<size_t> variable_assignment_equals_pos(const wcstring &txt);
-
 /// Error message for improper use of the exec builtin.
 #define EXEC_ERR_MSG _(L"The '%ls' command can not be used in a pipeline")
 

src/parser.cpp

@@ -85,7 +85,7 @@ parser_t::parser_t(std::shared_ptr<env_stack_t> vars) : variables(std::move(vars
     int cwd = open_cloexec(".", O_RDONLY);
     if (cwd < 0) {
         perror("Unable to open the current working directory");
-        abort();
+        return;
     }
     libdata().cwd_fd = std::make_shared<const autoclose_fd_t>(cwd);
 }
@@ -107,6 +107,25 @@ void parser_t::cancel_requested(int sig) {
     principal->cancellation_signal = sig;
 }
 
+int parser_t::set_var_and_fire(const wcstring &key, env_mode_flags_t mode, wcstring_list_t vals) {
+    std::vector<event_t> events;
+    int res = vars().set(key, mode, std::move(vals), &events);
+    for (const auto &evt : events) {
+        event_fire(*this, evt);
+    }
+    return res;
+}
+
+int parser_t::set_var_and_fire(const wcstring &key, env_mode_flags_t mode, wcstring val) {
+    wcstring_list_t vals;
+    vals.push_back(std::move(val));
+    return set_var_and_fire(key, mode, std::move(vals));
+}
+
+int parser_t::set_empty_var_and_fire(const wcstring &key, env_mode_flags_t mode) {
+    return set_var_and_fire(key, mode, wcstring_list_t{});
+}
+
 // Given a new-allocated block, push it onto our block list, acquiring ownership.
 block_t *parser_t::push_block(block_t &&block) {
     block_t new_current{block};
@@ -615,11 +634,11 @@ profile_item_t *parser_t::create_profile_item() {
 }
 
 eval_result_t parser_t::eval(const wcstring &cmd, const io_chain_t &io,
-                             enum block_type_t block_type) {
+                             enum block_type_t block_type, maybe_t<pid_t> parent_pgid) {
     // Parse the source into a tree, if we can.
     parse_error_list_t error_list;
     if (parsed_source_ref_t ps = parse_source(cmd, parse_flag_none, &error_list)) {
-        return this->eval(ps, io, block_type);
+        return this->eval(ps, io, block_type, parent_pgid);
     } else {
         // Get a backtrace. This includes the message.
         wcstring backtrace_and_desc;
@@ -632,11 +651,12 @@ eval_result_t parser_t::eval(const wcstring &cmd, const io_chain_t &io,
 }
 
 eval_result_t parser_t::eval(const parsed_source_ref_t &ps, const io_chain_t &io,
-                             enum block_type_t block_type) {
+                             enum block_type_t block_type, maybe_t<pid_t> parent_pgid) {
     assert(block_type == block_type_t::top || block_type == block_type_t::subst);
     if (!ps->tree.empty()) {
         job_lineage_t lineage;
         lineage.block_io = io;
+        lineage.parent_pgid = parent_pgid;
         // Execute the first node.
         tnode_t<grammar::job_list> start{&ps->tree, &ps->tree.front()};
         return this->eval_node(ps, start, std::move(lineage), block_type);
@@ -670,6 +690,9 @@ eval_result_t parser_t::eval_node(const parsed_source_ref_t &ps, tnode_t<T> node
     operation_context_t op_ctx = this->context();
     block_t *scope_block = this->push_block(block_t::scope_block(block_type));
 
+    // Propogate any parent pgid.
+    op_ctx.parent_pgid = lineage.parent_pgid;
+
     // Create and set a new execution context.
     using exc_ctx_ref_t = std::unique_ptr<parse_execution_context_t>;
     scoped_push<exc_ctx_ref_t> exc(&execution_context, make_unique<parse_execution_context_t>(

src/parser.h

@@ -148,9 +148,9 @@ struct library_data_t {
     /// Whether we are currently cleaning processes.
     bool is_cleaning_procs{false};
 
-    /// The job id of the job being populated.
+    /// The internal job id of the job being populated, or 0 if none.
     /// This supports the '--on-job-exit caller' feature.
-    job_id_t caller_job_id{-1};
+    internal_job_id_t caller_id{0};
 
     /// Whether we are running a subshell command.
     bool is_subshell{false};
@@ -266,15 +266,18 @@ class parser_t : public std::enable_shared_from_this<parser_t> {
     /// \param io io redirections to perform on all started jobs
     /// \param block_type The type of block to push on the block stack, which must be either 'top'
     /// or 'subst'.
+    /// \param parent_pgid if set, the pgid to give to spawned jobs
     ///
     /// \return the eval result,
     eval_result_t eval(const wcstring &cmd, const io_chain_t &io,
-                       block_type_t block_type = block_type_t::top);
+                       block_type_t block_type = block_type_t::top,
+                       maybe_t<pid_t> parent_pgid = {});
 
     /// Evaluate the parsed source ps.
     /// Because the source has been parsed, a syntax error is impossible.
     eval_result_t eval(const parsed_source_ref_t &ps, const io_chain_t &io,
-                       block_type_t block_type = block_type_t::top);
+                       block_type_t block_type = block_type_t::top,
+                       maybe_t<pid_t> parent_pgid = {});
 
     /// Evaluates a node.
     /// The node type must be grammar::statement or grammar::job_list.
@@ -326,6 +329,12 @@ class parser_t : public std::enable_shared_from_this<parser_t> {
     statuses_t get_last_statuses() const { return vars().get_last_statuses(); }
     void set_last_statuses(statuses_t s) { vars().set_last_statuses(std::move(s)); }
 
+    /// Cover of vars().set(), which also fires any returned event handlers.
+    /// \return a value like ENV_OK.
+    int set_var_and_fire(const wcstring &key, env_mode_flags_t mode, wcstring val);
+    int set_var_and_fire(const wcstring &key, env_mode_flags_t mode, wcstring_list_t vals);
+    int set_empty_var_and_fire(const wcstring &key, env_mode_flags_t mode);
+
     /// Pushes a new block. Returns a pointer to the block, stored in the parser. The pointer is
     /// valid until the call to pop_block()
     block_t *push_block(block_t &&b);
@@ -355,6 +364,9 @@ class parser_t : public std::enable_shared_from_this<parser_t> {
     void get_backtrace(const wcstring &src, const parse_error_list_t &errors,
                        wcstring &output) const;
 
+    /// \return the signal triggering cancellation, or 0 if none.
+    int get_cancel_signal() const { return cancellation_signal; }
+
     /// Output profiling data to the given filename.
     void emit_profiling(const char *path) const;
 

src/proc.cpp

@@ -254,9 +254,15 @@ void process_t::check_generations_before_launch() {
     gens_ = topic_monitor_t::principal().current_generations();
 }
 
+static uint64_t next_internal_job_id() {
+    static std::atomic<uint64_t> s_next{};
+    return ++s_next;
+}
+
 job_t::job_t(job_id_t job_id, const properties_t &props, const job_lineage_t &lineage)
     : properties(props),
       job_id_(job_id),
+      internal_job_id(next_internal_job_id()),
       root_constructed(lineage.root_constructed ? lineage.root_constructed : this->constructed) {}
 
 job_t::~job_t() {
@@ -412,6 +418,7 @@ event_t proc_create_event(const wchar_t *msg, event_type_t type, pid_t pid, int
     event_t event{type};
     event.desc.param1.pid = pid;
 
+    event.arguments.reserve(3);
     event.arguments.push_back(msg);
     event.arguments.push_back(to_string(pid));
     event.arguments.push_back(to_string(status));
@@ -544,18 +551,18 @@ static bool process_clean_after_marking(parser_t &parser, bool allow_interactive
     // complete.
     std::vector<event_t> exit_events;
 
+    // A helper to indicate if we should process a job.
+    auto should_process_job = [=](const shared_ptr<job_t> &j) {
+        // Do not attempt to process jobs which are not yet constructed.
+        // Do not attempt to process jobs that need to print a status message,
+        // unless we are interactive, in which case printing is OK.
+        return j->is_constructed() && (interactive || !job_wants_message(j));
+    };
+
     // Print status messages for completed or stopped jobs.
     const bool only_one_job = parser.jobs().size() == 1;
     for (const auto &j : parser.jobs()) {
-        // Skip unconstructed jobs.
-        if (!j->is_constructed()) {
-            continue;
-        }
-
-        // If we are not interactive, skip cleaning jobs that want to print an interactive message.
-        if (!interactive && job_wants_message(j)) {
-            continue;
-        }
+        if (!should_process_job(j)) continue;
 
         // Clean processes within the job.
         // Note this may print the message on behalf of the job, affecting the result of
@@ -581,16 +588,19 @@ static bool process_clean_after_marking(parser_t &parser, bool allow_interactive
                     proc_create_event(L"JOB_EXIT", event_type_t::exit, -j->pgid, 0));
             }
             exit_events.push_back(
-                proc_create_event(L"JOB_EXIT", event_type_t::job_exit, j->job_id(), 0));
+                proc_create_event(L"JOB_EXIT", event_type_t::caller_exit, j->job_id(), 0));
+            exit_events.back().desc.param1.caller_id = j->internal_job_id;
         }
     }
 
     // Remove completed jobs.
     // Do this before calling out to user code in the event handler below, to ensure an event
     // handler doesn't remove jobs on our behalf.
-    auto is_complete = [](const shared_ptr<job_t> &j) { return j->is_completed(); };
+    auto should_remove = [&](const shared_ptr<job_t> &j) {
+        return should_process_job(j) && j->is_completed();
+    };
     auto &jobs = parser.jobs();
-    jobs.erase(std::remove_if(jobs.begin(), jobs.end(), is_complete), jobs.end());
+    jobs.erase(std::remove_if(jobs.begin(), jobs.end(), should_remove), jobs.end());
 
     // Post pending exit events.
     for (const auto &evt : exit_events) {

src/proc.h

@@ -266,7 +266,12 @@ class process_t {
 typedef std::unique_ptr<process_t> process_ptr_t;
 typedef std::vector<process_ptr_t> process_list_t;
 
-typedef int job_id_t;
+/// The non user-visible, never-recycled job ID.
+/// Every job has a unique positive value for this.
+using internal_job_id_t = uint64_t;
+
+/// The user-visible, optional, recycled job ID.
+using job_id_t = int;
 job_id_t acquire_job_id(void);
 void release_job_id(job_id_t jid);
 
@@ -381,8 +386,12 @@ class job_t {
     pid_t pgid{INVALID_PID};
 
     /// The id of this job.
+    /// This is user-visible, is recycled, and may be -1.
     job_id_t job_id() const { return job_id_; }
 
+    /// A non-user-visible, never-recycled job ID.
+    const internal_job_id_t internal_job_id;
+
     /// Mark this job as internal. Internal jobs' job_ids are removed from the
     /// list of jobs so that, among other things, they don't take a job_id
     /// entry.

src/reader.cpp

@@ -1060,6 +1060,7 @@ static bool command_ends_paging(readline_cmd_t c, bool focused_on_search_field)
         case rl::backward_kill_path_component:
         case rl::backward_kill_bigword:
         case rl::self_insert:
+        case rl::self_insert_notfirst:
         case rl::transpose_chars:
         case rl::transpose_words:
         case rl::upcase_word:
@@ -1271,6 +1272,10 @@ void reader_data_t::completion_insert(const wchar_t *val, size_t token_end,
     set_buffer_maintaining_pager(new_command_line, cursor);
 }
 
+static bool may_add_to_history(const wcstring &commandline_prefix) {
+    return !commandline_prefix.empty() && commandline_prefix.at(0) != L' ';
+}
+
 // Returns a function that can be invoked (potentially
 // on a background thread) to determine the autosuggestion
 static std::function<autosuggestion_result_t(void)> get_autosuggestion_performer(
@@ -1294,17 +1299,19 @@ static std::function<autosuggestion_result_t(void)> get_autosuggestion_performer
             return nothing;
         }
 
-        history_search_t searcher(*history, search_string, history_search_type_t::prefix,
-                                  history_search_flags_t{});
-        while (!ctx.check_cancel() && searcher.go_backwards()) {
-            const history_item_t &item = searcher.current_item();
+        if (may_add_to_history(search_string)) {
+            history_search_t searcher(*history, search_string, history_search_type_t::prefix,
+                                      history_search_flags_t{});
+            while (!ctx.check_cancel() && searcher.go_backwards()) {
+                const history_item_t &item = searcher.current_item();
 
-            // Skip items with newlines because they make terrible autosuggestions.
-            if (item.str().find(L'\n') != wcstring::npos) continue;
+                // Skip items with newlines because they make terrible autosuggestions.
+                if (item.str().find(L'\n') != wcstring::npos) continue;
 
-            if (autosuggest_validate_from_history(item, working_directory, ctx)) {
-                // The command autosuggestion was handled specially, so we're done.
-                return {searcher.current_string(), search_string};
+                if (autosuggest_validate_from_history(item, working_directory, ctx)) {
+                    // The command autosuggestion was handled specially, so we're done.
+                    return {searcher.current_string(), search_string};
+                }
             }
         }
 
@@ -1943,7 +1950,7 @@ void set_env_cmd_duration(struct timeval *after, struct timeval *before, env_sta
 void reader_run_command(parser_t &parser, const wcstring &cmd) {
     struct timeval time_before, time_after;
 
-    wcstring ft = tok_first(cmd);
+    wcstring ft = tok_command(cmd);
 
     // For compatibility with fish 2.0's $_, now replaced with `status current-command`
     if (!ft.empty()) parser.vars().set_one(L"_", ENV_GLOBAL, ft);
@@ -2390,47 +2397,36 @@ struct readline_loop_state_t {
 /// Read normal characters, inserting them into the command line.
 /// \return the next unhandled event.
 maybe_t<char_event_t> reader_data_t::read_normal_chars(readline_loop_state_t &rls) {
-    maybe_t<char_event_t> event_needing_handling = inputter.readch();
-
-    if (!event_is_normal_char(*event_needing_handling) || !can_read(STDIN_FILENO))
-        return event_needing_handling;
-
-    // This is a normal character input.
-    // We are going to handle it directly, accumulating more.
-    char_event_t evt = event_needing_handling.acquire();
+    maybe_t<char_event_t> event_needing_handling{};
+    wcstring accumulated_chars;
     size_t limit = std::min(rls.nchars - command_line.size(), READAHEAD_MAX);
-
-    wchar_t arr[READAHEAD_MAX + 1] = {};
-    arr[0] = evt.get_char();
-
-    for (size_t i = 1; i < limit; ++i) {
-        if (!can_read(0)) {
+    while (accumulated_chars.size() < limit) {
+        bool allow_commands = (accumulated_chars.empty());
+        auto evt = inputter.readch(allow_commands);
+        if (!event_is_normal_char(evt) || !can_read(STDIN_FILENO)) {
+            event_needing_handling = std::move(evt);
             break;
-        }
-        // Only allow commands on the first key; otherwise, we might have data we
-        // need to insert on the commandline that the command might need to be able
-        // to see.
-        auto next_event = inputter.readch(false);
-        if (event_is_normal_char(next_event)) {
-            arr[i] = next_event.get_char();
+        } else if (evt.input_style == char_input_style_t::notfirst && accumulated_chars.empty() &&
+                   active_edit_line()->position == 0) {
+            // The cursor is at the beginning and nothing is accumulated, so skip this character.
+            continue;
         } else {
-            // We need to process this in the outer loop.
-            assert(!event_needing_handling && "Should not have an unhandled event");
-            event_needing_handling = next_event;
-            break;
+            accumulated_chars.push_back(evt.get_char());
         }
     }
 
-    editable_line_t *el = active_edit_line();
-    insert_string(el, arr);
+    if (!accumulated_chars.empty()) {
+        editable_line_t *el = active_edit_line();
+        insert_string(el, accumulated_chars);
 
-    // End paging upon inserting into the normal command line.
-    if (el == &command_line) {
-        clear_pager();
+        // End paging upon inserting into the normal command line.
+        if (el == &command_line) {
+            clear_pager();
+        }
+
+        // Since we handled a normal character, we don't have a last command.
+        rls.last_cmd.reset();
     }
-
-    // Since we handled a normal character, we don't have a last command.
-    rls.last_cmd.reset();
     return event_needing_handling;
 }
 
@@ -2764,7 +2760,7 @@ void reader_data_t::handle_readline_command(readline_cmd_t c, readline_loop_stat
                 // Finished command, execute it. Don't add items that start with a leading
                 // space.
                 const editable_line_t *el = &command_line;
-                if (history != nullptr && !el->empty() && el->text.at(0) != L' ') {
+                if (history != nullptr && may_add_to_history(el->text)) {
                     history->add_pending_with_file_detection(el->text, vars.get_pwd_slash());
                 }
                 rls.finished = true;
@@ -3179,12 +3175,11 @@ void reader_data_t::handle_readline_command(readline_cmd_t c, readline_loop_stat
             }
             break;
         }
-            // Some commands should have been handled internally by input_readch().
-        case rl::self_insert: {
-            DIE("self-insert should have been handled by inputter_t::readch");
-        }
+            // Some commands should have been handled internally by inputter_t::readch().
+        case rl::self_insert:
+        case rl::self_insert_notfirst:
         case rl::func_and: {
-            DIE("self-insert should have been handled by inputter_t::readch");
+            DIE("should have been handled by inputter_t::readch");
         }
     }
 }
@@ -3298,8 +3293,11 @@ maybe_t<wcstring> reader_data_t::readline(int nchars_or_0) {
         } else {
             // Ordinary char.
             wchar_t c = event_needing_handling->get_char();
-            if (!fish_reserved_codepoint(c) && (c >= L' ' || c == L'\n' || c == L'\r') &&
-                c != 0x7F) {
+            if (event_needing_handling->input_style == char_input_style_t::notfirst &&
+                active_edit_line()->position == 0) {
+                // This character is skipped.
+            } else if (!fish_reserved_codepoint(c) && (c >= L' ' || c == L'\n' || c == L'\r') &&
+                       c != 0x7F) {
                 // Regular character.
                 editable_line_t *el = active_edit_line();
                 insert_char(active_edit_line(), c);
@@ -3477,7 +3475,7 @@ bool reader_get_selection(size_t *start, size_t *len) {
     reader_data_t *data = current_data_or_null();
     if (data != nullptr && data->sel_active) {
         *start = data->sel_start_pos;
-        *len = std::min(data->sel_stop_pos - data->sel_start_pos, data->command_line.size());
+        *len = std::min(data->sel_stop_pos, data->command_line.size()) - data->sel_start_pos;
         result = true;
     }
     return result;

src/tokenizer.cpp

@@ -362,6 +362,7 @@ maybe_t<pipe_or_redir_t> pipe_or_redir_t::from_string(const wchar_t *buff) {
         }
         case L'>': {
             consume(L'>');
+            if (try_consume(L'>')) result.mode = redirection_mode_t::append;
             if (try_consume(L'|')) {
                 // Note we differ from bash here.
                 // Consider `echo foo 2>| bar`
@@ -374,6 +375,7 @@ maybe_t<pipe_or_redir_t> pipe_or_redir_t::from_string(const wchar_t *buff) {
                                    : STDOUT_FILENO;              // like >|
             } else if (try_consume(L'&')) {
                 // This is a redirection to an fd.
+                // Note that we allow ">>&", but it's still just writing to the fd - "appending" to it doesn't make sense.
                 result.mode = redirection_mode_t::fd;
                 result.fd = has_fd ? parse_fd(fd_start, fd_end)  // like 1>&2
                                    : STDOUT_FILENO;              // like >&2
@@ -381,11 +383,10 @@ maybe_t<pipe_or_redir_t> pipe_or_redir_t::from_string(const wchar_t *buff) {
                 // This is a redirection to a file.
                 result.fd = has_fd ? parse_fd(fd_start, fd_end)  // like 1> file.txt
                                    : STDOUT_FILENO;              // like > file.txt
+                if (result.mode != redirection_mode_t::append) result.mode = redirection_mode_t::overwrite;
                 // Note 'echo abc >>? file' is valid: it means append and noclobber.
                 // But here "noclobber" means the file must not exist, so appending
                 // can be ignored.
-                result.mode = redirection_mode_t::overwrite;
-                if (try_consume(L'>')) result.mode = redirection_mode_t::append;
                 if (try_consume(L'?')) result.mode = redirection_mode_t::noclob;
             }
             break;
@@ -412,7 +413,12 @@ maybe_t<pipe_or_redir_t> pipe_or_redir_t::from_string(const wchar_t *buff) {
                 consume(L'^');
                 result.fd = STDERR_FILENO;
                 result.mode = redirection_mode_t::overwrite;
-                if (try_consume(L'^')) result.mode = redirection_mode_t::append;
+                if (try_consume(L'^')) {
+                    result.mode = redirection_mode_t::append;
+                } else if (try_consume(L'&')) {
+                    // This is a redirection to an fd.
+                    result.mode = redirection_mode_t::fd;
+                }
                 if (try_consume(L'?')) result.mode = redirection_mode_t::noclob;
                 break;
             }
@@ -657,6 +663,21 @@ wcstring tok_first(const wcstring &str) {
     return {};
 }
 
+wcstring tok_command(const wcstring &str) {
+    tokenizer_t t(str.c_str(), 0);
+    while (auto token = t.next()) {
+        if (token->type != token_type_t::string) {
+            return {};
+        }
+        wcstring text = t.text_of(*token);
+        if (variable_assignment_equals_pos(text)) {
+            continue;
+        }
+        return text;
+    }
+    return {};
+}
+
 bool move_word_state_machine_t::consume_char_punctuation(wchar_t c) {
     enum { s_always_one = 0, s_rest, s_whitespace_rest, s_whitespace, s_alphanumeric, s_end };
 
@@ -843,3 +864,23 @@ move_word_state_machine_t::move_word_state_machine_t(move_word_style_t syl)
     : state(0), style(syl) {}
 
 void move_word_state_machine_t::reset() { state = 0; }
+
+// Return the location of the equals sign, or npos if the string does
+// not look like a variable assignment like FOO=bar.  The detection
+// works similar as in some POSIX shells: only letters and numbers qre
+// allowed on the left hand side, no quotes or escaping.
+maybe_t<size_t> variable_assignment_equals_pos(const wcstring &txt) {
+    enum { init, has_some_variable_identifier } state = init;
+    // TODO bracket indexing
+    for (size_t i = 0; i < txt.size(); i++) {
+        wchar_t c = txt[i];
+        if (state == init) {
+            if (!valid_var_name_char(c)) return {};
+            state = has_some_variable_identifier;
+        } else {
+            if (c == '=') return {i};
+            if (!valid_var_name_char(c)) return {};
+        }
+    }
+    return {};
+}

src/tokenizer.h

@@ -144,6 +144,9 @@ class tokenizer_t {
 /// returns the empty string.
 wcstring tok_first(const wcstring &str);
 
+/// Like to tok_first, but skip variable assignments like A=B.
+wcstring tok_command(const wcstring &str);
+
 /// Struct wrapping up a parsed pipe or redirection.
 struct pipe_or_redir_t {
     // The redirected fd, or -1 on overflow.
@@ -210,4 +213,7 @@ class move_word_state_machine_t {
     void reset();
 };
 
+/// The position of the equal sign in a variable assignment like foo=bar.
+maybe_t<size_t> variable_assignment_equals_pos(const wcstring &txt);
+
 #endif

src/util.cpp

@@ -100,6 +100,55 @@ int wcsfilecmp(const wchar_t *a, const wchar_t *b) {
     return 1;  // string b is a prefix of a and a is longer
 }
 
+/// wcsfilecmp, but frozen in time for glob usage.
+int wcsfilecmp_glob(const wchar_t *a, const wchar_t *b) {
+    assert(a && b && "Null parameter");
+    const wchar_t *orig_a = a;
+    const wchar_t *orig_b = b;
+    int retval = 0;  // assume the strings will be equal
+
+    while (*a && *b) {
+        if (iswdigit(*a) && iswdigit(*b)) {
+            retval = wcsfilecmp_leading_digits(&a, &b);
+            // If we know the strings aren't logically equal or we've reached the end of one or both
+            // strings we can stop iterating over the chars in each string.
+            if (retval || *a == 0 || *b == 0) break;
+        }
+
+        wint_t al = towlower(*a);
+        wint_t bl = towlower(*b);
+        if (al < bl) {
+            retval = -1;
+            break;
+        } else if (al > bl) {
+            retval = 1;
+            break;
+        } else {
+            a++;
+            b++;
+        }
+    }
+
+    if (retval != 0) return retval;  // we already know the strings aren't logically equal
+
+    if (*a == 0) {
+        if (*b == 0) {
+            // The strings are logically equal. They may or may not be the same length depending on
+            // whether numbers were present but that doesn't matter. Disambiguate strings that
+            // differ by letter case or length. We don't bother optimizing the case where the file
+            // names are literally identical because that won't occur given how this function is
+            // used. And even if it were to occur (due to being reused in some other context) it
+            // would be so rare that it isn't worth optimizing for.
+            retval = wcscmp(orig_a, orig_b);
+            return retval < 0 ? -1 : retval == 0 ? 0 : 1;
+        }
+        return -1;  // string a is a prefix of b and b is longer
+    }
+
+    assert(*b == 0);
+    return 1;  // string b is a prefix of a and a is longer
+}
+
 /// Return microseconds since the epoch.
 long long get_time() {
     struct timeval time_struct;

src/util.h

@@ -31,6 +31,9 @@
 /// given above.
 int wcsfilecmp(const wchar_t *a, const wchar_t *b);
 
+/// wcsfilecmp, but frozen in time for glob usage.
+int wcsfilecmp_glob(const wchar_t *a, const wchar_t *b);
+
 /// Get the current time in microseconds since Jan 1, 1970.
 long long get_time();
 

src/widecharwidth/widechar_width.h

@@ -1,5 +1,5 @@
 /**
- * widechar_width.h, generated on 2019-05-14.
+ * widechar_width.h, generated on 2020-01-30.
  * See https://github.com/ridiculousfish/widecharwidth/
  *
  * SHA1 file hashes:
@@ -30,8 +30,8 @@ enum {
 
 /* An inclusive range of characters. */
 struct widechar_range {
-  wchar_t lo;
-  wchar_t hi;
+  uint32_t lo;
+  uint32_t hi;
 };
 
 /* Simple ASCII characters - used a lot, so we check them first. */
@@ -506,14 +506,14 @@ static const struct widechar_range widechar_widened_table[] = {
 };
 
 template<typename Collection>
-bool widechar_in_table(const Collection &arr, wchar_t c) {
+bool widechar_in_table(const Collection &arr, uint32_t c) {
     auto where = std::lower_bound(std::begin(arr), std::end(arr), c,
-        [](widechar_range p, wchar_t c) { return p.hi < c; });
+        [](widechar_range p, uint32_t c) { return p.hi < c; });
     return where != std::end(arr) && where->lo <= c;
 }
 
 /* Return the width of character c, or a special negative value. */
-int widechar_wcwidth(wchar_t c) {
+int widechar_wcwidth(uint32_t c) {
     if (widechar_in_table(widechar_ascii_table, c))
         return 1;
     if (widechar_in_table(widechar_private_table, c))

src/wutil.cpp

@@ -305,10 +305,11 @@ int fd_check_is_remote(int fd) {
     }
     // Linux has constants for these like NFS_SUPER_MAGIC, SMB_SUPER_MAGIC, CIFS_MAGIC_NUMBER but
     // these are in varying headers. Simply hard code them.
-    switch (buf.f_type) {
+    // NOTE: The cast is necessary for 32-bit systems because of the 4-byte CIFS_MAGIC_NUMBER
+    switch ((unsigned int)buf.f_type) {
         case 0x6969:      // NFS_SUPER_MAGIC
         case 0x517B:      // SMB_SUPER_MAGIC
-        case 0xFF534D42:  // CIFS_MAGIC_NUMBER
+        case 0xFF534D42u:  // CIFS_MAGIC_NUMBER
             return 1;
         default:
             // Other FSes are assumed local.

tests/bind.expect

@@ -299,3 +299,15 @@ expect_prompt -re {nul seen\r\nnul seen\r\nnul seen} {
 } unmatched {
     puts stderr "nul not seen"
 }
+
+# Test self-insert-notfirst. (#6603)
+# Here the leading 'q's should be stripped, but the trailing ones not.
+send "bind q self-insert-notfirst\r"
+expect_prompt
+send "qqqecho qqq"
+send "\r"
+expect_prompt -re {qqq} {
+    puts "Leading q properly stripped"
+} unmatched {
+    puts stderr "Leading qs not stripped"
+}

tests/bind.expect.out

@@ -23,3 +23,4 @@ ctrl-o seen
 ctrl-w stops at :
 ctrl-w stops at @
 nul seen
+Leading q properly stripped

tests/bind_mode_events.expect

@@ -0,0 +1,36 @@
+# vim: set filetype=expect:
+spawn $fish
+expect_prompt
+
+send "set -g fish_key_bindings fish_vi_key_bindings\r"
+expect_prompt
+
+send "echo ready to go\r"
+expect_prompt -re {\r\nready to go\r\n} {
+    puts "ready to go"
+}
+send "function add_change --on-variable fish_bind_mode ; set -g MODE_CHANGES \$MODE_CHANGES \$fish_bind_mode ; end\r"
+expect_prompt
+
+# normal mode
+send "\033"
+sleep 0.050
+
+# insert mode
+send "i"
+sleep 0.050
+
+# back to normal mode
+send "\033"
+sleep 0.050
+
+# insert mode again
+send "i"
+sleep 0.050
+
+send "echo mode changes: \$MODE_CHANGES\r"
+expect_prompt -re {\r\nmode changes: default insert default insert\r\n} {
+        puts "Correct mode changes"
+} unmatched {
+    puts "Incorrect mode changes"
+}

tests/bind_mode_events.expect.out

@@ -0,0 +1,2 @@
+ready to go
+Correct mode changes

tests/checks/bad-option.fish

@@ -1,2 +1,2 @@
 #RUN: %fish -Z
-#CHECKERR: {{.*fish}}: {{unrecognized option: Z|invalid option -- '?Z'?|unknown option -- Z}}
+#CHECKERR: {{.*fish}}: {{unrecognized option: Z|invalid option -- '?Z'?|unknown option -- Z}|illegal option -- Z}}

tests/checks/braces.fish

@@ -0,0 +1,50 @@
+#RUN: %fish %s
+
+echo x-{1}
+#CHECK: x-{1}
+
+echo x-{1,2}
+#CHECK: x-1 x-2
+
+echo foo-{1,2{3,4}}
+#CHECK: foo-1 foo-23 foo-24
+
+echo foo-{} # literal "{}" expands to itself
+#CHECK: foo-{}
+
+echo foo-{{},{}} # the inner "{}" expand to themselves, the outer pair expands normally.
+#CHECK: foo-{} foo-{}
+
+echo foo-{{a},{}} # also works with something in the braces.
+#CHECK: foo-{a} foo-{}
+
+echo foo-{""} # still expands to foo-{}
+#CHECK: foo-{}
+
+echo foo-{$undefinedvar} # still expands to nothing
+#CHECK: 
+
+echo foo-{,,,} # four empty items in the braces.
+#CHECK: foo- foo- foo- foo-
+
+echo foo-{,\,,} # an empty item, a "," and an empty item.
+#CHECK: foo- foo-, foo-
+
+echo .{  foo  bar  }. # see 6564
+#CHECK: .{  foo  bar  }.
+
+# whitespace within entries is retained
+for foo in {a, hello
+wo  rld }
+    echo \'$foo\'
+end
+# CHECK: 'a'
+# CHECK: 'hello
+# CHECK: wo  rld'
+
+for foo in {hello
+world}
+   echo \'$foo\'
+end
+#CHECK: '{hello
+#CHECK: world}'