fast-syntax-highlighting highlighting of multi-word abbreviations

[olets]

fast-syntax-highlighting highlighting of single-word abbreviations was done in #24.

Useful links:

• fsh chroma docs: https://github.com/zdharma-continuum/fast-syntax-highlighting/blob/master/CHROMA_GUIDE.adoc
• Initial discussion of how to expand single-word support to multiple words fast-syntax-highlighting highlighting of single-word abbreviations #24 (comment)

[5A6F65]

Here to report.
I'm very sorry that it took so long to complete the work.
Some trivial matters in life have taken up all my energy.

---

How hard do you think it would be to support multi-word abbreviations? Is that possible with F-Sy-H?

In my opinion, it would be slightly difficult because F-Sy-H chroma function seems to be triggered by a single word. To support multi-word abbreviations, we need to create a chroma function for the first word only, and then the second and subsequent words should be treated like arguments of the first word.

I think therefore that it can be implemented roughly in the following flow.

1. list all abbreviations
2. group by the first word
3. generate chroma function for each first word dynamically using eval or something
4. register chroma function for each first word

At first, I followed Teppei Shimaji's approach, attempting to bind the first word of all abbreviations to the →chroma/-abbr.ch function for highlighting analysis. But this led to two issues:

1. Unable to handle multiple spaces
   For example:

   abbr 'git po'='git push -u origin'

   git po will expand to git push -u origin;
   git po will not expand.
   Unfortunately, for the chroma function, we cannot distinguish between the two.

2. Unable to handle glob prefixes
   Obviously, we cannot bind the chroma function to all globs.

---

A more elegant approach is to parse the buffer ourselves before f-sy-h intervenes. If cannot be expanded, pass it on to f-sy-h for further processing.
The advantage of this implementation is that it is not constrained by any framework, so any abbreviation supported by zsh-abbr can be highlighted correctly.

Based on this idea, I first attempted to repackage all the widgets bound by f-sy-h.
Although this achieved the desired effect, it turned out to be rather complex.

After reading the f-sy-h source code, I discovered that all widgets parse the buffer through the _zsh_highlight function.
Therefore, a simpler idea was to repackage the _zsh_highlight function. Thus, the following implementation was devised:

function _abbr_fsh_wrap(){
    (( ${+ABBR_SOURCE_PATH} )) && (( ${+FAST_HIGHLIGHT} )) || return 0

    typeset -g ABBR_FSH_SUBCMD_MAX_LENGTH=${ABBR_FSH_SUBCMD_MAX_LENGTH:-7}
    typeset -g ABBR_FSH_ARGUMENT_MAX_LENGTH=${ABBR_FSH_ARGUMENT_MAX_LENGTH:-7}

    typeset -g _ABBR_FSH_PRIOR_LBUFFER

    function _abbr_fsh_parse_regular_abbreviation() { ... }
    function _abbr_fsh_parse_global_abbreviation() { ... }

    (( ${+functions[_orig_zsh_highlight]} )) && return 0
    functions -c -- _zsh_highlight _orig_zsh_highlight
    function _zsh_highlight() {
        local ret=$?
        if [[ $WIDGET == zle-isearch-update ]] && ! (( $+ISEARCHMATCH_ACTIVE )) {
            region_highlight=()
            return $ret
        }
        [[ -n ${ZSH_HIGHLIGHT_MAXLENGTH:-} ]] && (( ${#BUFFER} > $ZSH_HIGHLIGHT_MAXLENGTH )) && return $ret
        (( $PENDING > 0 )) && return $ret

        local -a reply
        if [[ $LBUFFER != $_ABBR_FSH_PRIOR_LBUFFER ]] {
            _abbr_fsh_parse_regular_abbreviation $LBUFFER
            if (( $? == 0 )) {
                _ABBR_FSH_PRIOR_LBUFFER=$LBUFFER
                region_highlight=($reply)
                return 0
            }
            _ZSH_HIGHLIGHT_PRIOR_BUFFER=
        }
        _orig_zsh_highlight "$@"
        if [[ $LBUFFER != $_ABBR_FSH_PRIOR_LBUFFER ]] {
            _ABBR_FSH_PRIOR_LBUFFER=$LBUFFER
            _abbr_fsh_parse_global_abbreviation $LBUFFER && region_highlight+=($reply)
        }
        return 0
    }
    return 0
}

The _abbr_fsh_wrap should be executed after both zsh‑abbr and fast‑syntax‑highlighting have been loaded.

Here, _abbr_fsh_parse_regular_abbreviation will attempt to parse its first argument ($LBUFFER) and pass the highlighting rules via reply. Its implementation is as follows:

function _abbr_fsh_parse_regular_abbreviation() {
    emulate -LR zsh
    set -o localoptions -o localtraps

    local buffer=$1
    (( ${#${(z)buffer}} )) || return

    function _abbr_fsh_parse_regular_abbreviation:check() {
        local snippet=$1

        (( ! ${+ABBR_REGULAR_SESSION_ABBREVIATIONS[${(qqq)snippet}]} )) \
            && (( ! ${+ABBR_REGULAR_USER_ABBREVIATIONS[${(qqq)snippet}]} )) \
            && return 1
        return 0
    }

    function _abbr_fsh_parse_regular_abbreviation:transform() {
        local snippet=$1
        local type=$2
        local -i offset=$3

        result=(
            rule "$offset $((offset += ${#snippet})) ${FAST_HIGHLIGHT_STYLES[${FAST_THEME_NAME}$type]}"
            offset "$offset"
        )
        return 0
    }

    function _abbr_fsh_parse_regular_abbreviation:trim() {
        local buffer=$1
        local -i offset=$2

        local prefix
        local snippet
        local matches
        local -a prefixes
        local -i length
        local -i threshold
        local -i index

        prefixes=(
            $ABBR_REGULAR_ABBREVIATION_SCALAR_PREFIXES
            $ABBR_REGULAR_ABBREVIATION_GLOB_PREFIXES
        )
        length=${#prefixes}
        threshold=${#ABBR_REGULAR_ABBREVIATION_SCALAR_PREFIXES}
        index=1

        until {
            _abbr_fsh_parse_regular_abbreviation:check $buffer
        } {
            prefix=${prefixes[$index]}
            if (( index++ <= threshold )) {
                snippet=${(M)buffer#$prefix}
            } else {
                snippet=${(M)buffer#$~prefix}
            }
            if (( ! ${#snippet} )) {
                (( $index > $length )) && return 1
                continue
            }
            matches+=$snippet
            buffer=${buffer:${#snippet}}
            index=1
        }

        (( ${#matches} )) && _abbr_fsh_parse_regular_abbreviation:transform $matches precommand $offset
        return 0
    }

    function _abbr_fsh_parse_regular_abbreviation:highlight() {
        local buffer=$1
        local -i offset=$2

        local type
        local -a words
        local -a types
        local -i index

        _abbr_fsh_parse_regular_abbreviation:transform ${buffer:$offset} alias $offset
        reply=($result[rule])

        words=(${(z)${buffer:$offset}})
        index=1

        if [[ ${words[$index]} == "sudo" ]] {
            types+=(precommand)
            let index++
        }

        if (( ! ${#words:$index} )) {
            types+=(alias)
            let index++
        } elif (( ${+functions[${words[$index]}]} )) {
            types+=(function)
            let index++
        } elif (( ${+commands[${words[$index]}]} )) {
            types+=(command)
            let index++
        } else {
            return 1
        }

        if [[ ${words[$index]} =~ '^[A-Za-z]{1,'$ABBR_FSH_SUBCMD_MAX_LENGTH'}$' ]] {
            types+=(subcommand)
            let index++
            if [[ ${words[$index]} =~ '^[A-Za-z]{1,'$ABBR_FSH_ARGUMENT_MAX_LENGTH'}$' ]] {
                types+=(default)
                let index++
            }
        }

        if [[ ${words[$index]} =~ '^\-[A-Za-z]+$' ]] {
            types+=(single-hyphen-option)
            let index++
        } elif [[ ${words[$index]} =~ '^\-\-[A-Za-z]+$' ]] {
            types+=(double-hyphen-option)
            let index++
        }

        (( ${#words[$index]} )) && return 1

        reply=()
        index=1
        for type ($types) {
            offset+=${${buffer:$offset}[(i)${(q)words[$index]}]}-1
            _abbr_fsh_parse_regular_abbreviation:transform ${words[$((index++))]} $type $offset
            reply+=(${result[rule]})
            offset=${result[offset]}
        }
        return 0
    }

    function _abbr_fsh_parse_regular_abbreviation:cleanup() {
        unfunction -m _abbr_fsh_parse_regular_abbreviation:check
        unfunction -m _abbr_fsh_parse_regular_abbreviation:transform
        unfunction -m _abbr_fsh_parse_regular_abbreviation:trim
        unfunction -m _abbr_fsh_parse_regular_abbreviation:highlight
        unfunction -m _abbr_fsh_parse_regular_abbreviation:cleanup
        return 0
    }

    trap _abbr_fsh_parse_regular_abbreviation:cleanup EXIT

    local -A result

    local -a rules
    local -i offset

    rules=($reply)
    offset=0

    _abbr_fsh_parse_regular_abbreviation:trim $buffer $offset || return
    if (( ${#result[rule]} )) {
        rules+=(${result[rule]})
        offset=${result[offset]}
    }

    _abbr_fsh_parse_regular_abbreviation:highlight $buffer $offset
    rules+=($reply)
    reply=($rules)
    return 0
}

The behavior of _abbr_fsh_parse_global_abbreviation is similar, except that it will only pass highlighting rules for the global abbreviation region. Its reply should be appended to the end of region_highlight. Its implementation is as follows:

function _abbr_fsh_parse_global_abbreviation() {
    emulate -LR zsh

    local buffer=$1
    (( ${#${(z)buffer}} )) || return

    local snippet
    local -i offset
    local -i length
    local -i index
    local -a words

    offset=-1
    words=("${(@s: :)buffer}")
    length=${#words}
    for (( index = 1; index <= $length; index++ )) {
        snippet=${(j: :)words[$index,$length]}
        (( ${+ABBR_GLOBAL_SESSION_ABBREVIATIONS[${(qqq)snippet}]} )) \
            || (( ${+ABBR_GLOBAL_USER_ABBREVIATIONS[${(qqq)snippet}]} )) \
            || continue
        offset=$((${buffer[(I)${(q)snippet}]} - 1))
        break
    }
    (( $offset == -1 )) && return 1

    reply=("$offset $(($offset + ${#snippet})) ${FAST_HIGHLIGHT_STYLES[${FAST_THEME_NAME}global-alias]}")
    return 0
}

---

Now, abbreviations follow the following highlighting rules:

• Regular Prefixes:
  Prefixes will be highlighted as precommand, and both strings and globs can be processed normally.

• Regular Abbreviations:
  If it matches the format: [precommand] command/function [subcommand [argument]] [single/double-hyphen-option] (precommand only matches sudo), it will be highlighted according to the corresponding rules (the rule for argument is default); otherwise, it will be entirely highlighted as alias.
  Note: The subcommand or argument should match the regular expression [A-Za-z]+ and have a length less than $ABBR_REGULAR_ABBREVIATION_SUBCMD/ARGUMENT_MAX_LENGTH (both are 7 by default).
  Special case: if there is nothing after command/function, it will be highlighted as alias.

• Global Abbreviations:
  The global abbreviation located at the end of the LBUFFER would be highlighted as global-alias.

Of course, this is only a preliminary solution based on my personal usage. I believe there is plenty of room for further expansion, and I welcome anyone to continue improving my work.
For example, support highlighting brackets in abbreviation. The current implementation definitely supports parsing brackets, but I'm not sure what the related highlighting behavior should be because I don't use brackets in abbreviation. If anyone has good ideas, welcome to improve.

---

Known limitation:
None.

---

Additionally, while reading multiple-prefixes, I discovered an issue in it:

abbr a=b
e.xyz sudo a[SPACE] # expands to `e.xyz sudo bcd`
xyz e.sudo a[SPACE] # expands to `xyz e.sudo bcd`
xyz sudo e.a[SPACE] # expands to `xyz sudo e.bcd`

Here, the regular abbreviation should be abbr a=bcd.

[olets]

No need to apologize about timeline for a volunteer project you gave yourself!

Very cool stuff. I don't know when I'll be able to do serious testing given my abbreviations. If you feel like testing some additional abbreviations, I've written up some of my go-tos at https://olets.dev/posts/the-command-line-mostly-git-abbreviations-i-rely-on/.

The sample zshrc has enough going on that you might want to try making this into a stand alone plugin, so that users could do something like (pseudocode)

#zshrc

load olets/zsh-abbr
load zdharma-continuum/fast-syntax-highlighting
load 5A6F65/fast-syntax-highlighting-zsh-abbr-chroma # or whatever name

If you haven't done that before, zsh-window-title might be a useful reference — it has what is consider a solid minimal zsh plugin file setup.

Again I have no expectations and absolutely do not mean to pressure you to spend more time on this than you want to!

[5A6F65]

The sample zshrc has enough going on that you might want to try making this into a stand alone plugin, so that users could do something like (pseudocode)

As you suggested, I’ve created a simple plugin fast-abbr-highlighting.
This is my first time developing a zsh plugin, is there anything I need to improve? Any suggestions are very welcome!

[olets]

Cool!

I read through your code. It reads like something that grew out of Sebastian Gniazdowski code: I'd have to spend a lot of time with it before I understood it, but it has strong "brilliant" vibes. 😅 RIP Sebastian. Congratulations on figuring out some complicated stuff.

Won't be able to try it for a while. I hope someone else does! Interested to hear other people's experience.

[olets]

fast-abbr-highlighting is linked in the docs https://zsh-abbr.olets.dev/integrations.html#fast-syntax-highlighting