Support FOO=bar syntax for passing variables to individual commands

This adds initial support for statements with prefixed variable assignments.
Statments like this are supported:

a=1 b=$a echo $b        # outputs 1

Just like in other shells, the left-hand side of each assignment must
be a valid variable identifier (no quoting/escaping).  Array indexing
(PATH[1]=/bin ls $PATH) is *not* yet supported, but can be added fairly
easily.

The right hand side may be any valid string token, like a command
substitution, or a brace expansion.

Since `a=* foo` is equivalent to `begin set -lx a *; foo; end`,
the assignment, like `set`, uses nullglob behavior, e.g. below command
can safely be used to check if a directory is empty.

x=/nothing/{,.}* test (count $x) -eq 0

Generic file completion is done after the equal sign, so for example
pressing tab after something like `HOME=/` completes files in the
root directory
Subcommand completion works, so something like
`GIT_DIR=repo.git and command git ` correctly calls git completions
(but the git completion does not use the variable as of now).

The variable assignment is highlighted like an argument.

Closes #6048

src/parse_tree.cpp

@@ -1007,6 +1007,26 @@ static inline bool is_help_argument(const wcstring &txt) {
     return txt == L"-h" || txt == L"--help";
 }
 
+// Return the location of the equals sign, or npos if the string does
+// not look like a variable assignment like FOO=bar.  The detection
+// works similar as in some POSIX shells: only letters and numbers qre
+// allowed on the left hand side, no quotes or escaping.
+maybe_t<size_t> variable_assignment_equals_pos(const wcstring &txt) {
+    enum { init, has_some_variable_identifier } state = init;
+    // TODO bracket indexing
+    for (size_t i = 0; i < txt.size(); i++) {
+        wchar_t c = txt[i];
+        if (state == init) {
+            if (!valid_var_name_char(c)) return {};
+            state = has_some_variable_identifier;
+        } else {
+            if (c == '=') return {i};
+            if (!valid_var_name_char(c)) return {};
+        }
+    }
+    return {};
+}
+
 /// Return a new parse token, advancing the tokenizer.
 static inline parse_token_t next_parse_token(tokenizer_t *tok, maybe_t<tok_t> *out_token,
                                              wcstring *storage) {
@@ -1028,6 +1048,7 @@ static inline parse_token_t next_parse_token(tokenizer_t *tok, maybe_t<tok_t> *o
     result.is_help_argument = result.has_dash_prefix && is_help_argument(text);
     result.is_newline = (result.type == parse_token_type_end && text == L"\n");
     result.preceding_escaped_nl = token.preceding_escaped_nl;
+    result.may_be_variable_assignment = bool(variable_assignment_equals_pos(text));
 
     // These assertions are totally bogus. Basically our tokenizer works in size_t but we work in
     // uint32_t to save some space. If we have a source file larger than 4 GB, we'll probably just