Post-release version number bump.

* upstream/master:
  Follow-up to 28d7056a7a: Fix test expectations.

.gitignore

@@ -1 +1,3 @@
 *.zwc*
+.pc/
+docs/all.md

.version

@@ -1 +1 @@
-0.3.0
+0.5.1-dev

COPYING.md

@@ -1,4 +1,4 @@
-Copyright (c) 2010-2015 zsh-syntax-highlighting contributors
+Copyright (c) 2010-2016 zsh-syntax-highlighting contributors
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without modification, are permitted

HACKING.md

@@ -0,0 +1,75 @@
+Hacking on zsh-syntax-highlighting itself
+=========================================
+
+This document includes information for people working on z-sy-h itself: on the
+core driver (`zsh-syntax-highlighting.zsh`), on the highlighters in the
+distribution, and on the test suite.  It does not target third-party
+highlighter authors (although they may find it an interesting read).
+
+The `main` highlighter
+----------------------
+
+The following function `pz` is useful when working on the `main` highlighting:
+
+    pq() {
+      (( $#argv )) || return 0
+      print -r -l -- ${(qqqq)argv}
+    }
+    pz() {
+      local arg
+      for arg; do
+        pq ${(z)arg}
+      done
+    }
+
+It prints, for each argument, its token breakdown, similar to how the main
+loop of the `main` highlighter sees it.
+
+Testing the `brackets` highlighter
+----------------------------------
+
+Since the test harness empties `ZSH_HIGHLIGHT_STYLES` and the `brackets`
+highlighter interrogates `ZSH_HIGHLIGHT_STYLES` to determine how to highlight,
+tests must set the `bracket-level-#` keys themselves.  For example:
+
+    ZSH_HIGHLIGHT_STYLES[bracket-level-1]=
+    ZSH_HIGHLIGHT_STYLES[bracket-level-2]=
+
+    BUFFER='echo ({x})'
+
+    expected_region_highlight=(
+      "6  6  bracket-level-1" # (
+      "7  7  bracket-level-2" # {
+      "9  9  bracket-level-2" # }
+      "10 10 bracket-level-1" # )
+    )
+
+Testing the `pattern` highlighter
+---------------------------------
+
+Because the `pattern` highlighter modifies `region_highlight` directly instead
+of using `_zsh_highlight_add_highlight`, the test harness cannot get the
+`ZSH_HIGHLIGHT_STYLES` keys.  Therefore, when writing tests, use the style
+itself as third word (cf. the [documentation for `expected_region_highlight`]
+(docs/highlighters.md)).  For example:
+
+    ZSH_HIGHLIGHT_PATTERNS+=('rm -rf *' 'fg=white,bold,bg=red')
+
+    BUFFER='rm -rf /'
+
+    expected_region_highlight=(
+      "1 8 fg=white,bold,bg=red" # rm -rf /
+    )
+
+Miscellany
+----------
+
+If you work on the driver (`zsh-syntax-highlighting.zsh`), you may find the following zstyle useful:
+
+    zstyle ':completion:*:*:*:*:globbed-files' ignored-patterns {'*/',}zsh-syntax-highlighting.plugin.zsh
+
+IRC channel
+-----------
+
+We're on #zsh-syntax-highlighting on freenode.
+

INSTALL.md

@@ -0,0 +1,101 @@
+How to install
+--------------
+
+### Using packages
+
+* Arch Linux: [community/zsh-syntax-highlighting][arch-package] / [AUR/zsh-syntax-highlighting-git][AUR-package]
+* Debian: `zsh-syntax-highlighting` package [in `stretch`][debian-package]
+* Gentoo: [mv overlay][gentoo-overlay]
+* Mac OS X / Homebrew: [brew install zsh-syntax-highlighting][brew-package]
+* Ubuntu: `zsh-syntax-highlighting` package [in Xenial][ubuntu-package]
+
+[arch-package]: https://www.archlinux.org/packages/zsh-syntax-highlighting
+[AUR-package]: https://aur.archlinux.org/packages/zsh-syntax-highlighting-git
+[debian-package]: https://packages.debian.org/zsh-syntax-highlighting
+[gentoo-overlay]: http://gpo.zugaina.org/app-shells/zsh-syntax-highlighting
+[brew-package]: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/zsh-syntax-highlighting.rb
+[ubuntu-package]: https://launchpad.net/ubuntu/+source/zsh-syntax-highlighting
+
+
+### In your ~/.zshrc
+
+Simply clone this repository and source the script:
+
+        git clone https://github.com/zsh-users/zsh-syntax-highlighting.git
+        echo "source ${(q-)PWD}/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh" >> ${ZDOTDIR:-$HOME}/.zshrc
+
+  Then, enable syntax highlighting in the current interactive shell:
+
+        source ./zsh-syntax-highlighting/zsh-syntax-highlighting.zsh
+
+  If `git` is not installed, download and extract a snapshot of the latest
+  development tree from:
+
+        https://github.com/zsh-users/zsh-syntax-highlighting/archive/master.tar.gz
+
+  Note the `source` command must be **at the end** of `~/.zshrc`.
+
+
+### With a plugin manager
+
+Note that `zsh-syntax-highlighting` must be the last plugin sourced.
+
+The zsh-syntax-highlighting authors recommend manual installation over the use
+of a framework or plugin manager.
+
+This list is incomplete as there are too many [frameworks / plugin managers]
+(https://github.com/unixorn/awesome-zsh-plugins#frameworks) to list them all
+here.
+
+#### [Antigen](https://github.com/zsh-users/antigen)
+
+Add `antigen bundle zsh-users/zsh-syntax-highlighting` as the last bundle in
+your `.zshrc`.
+
+#### [Oh-my-zsh](https://github.com/robbyrussell/oh-my-zsh)
+
+1. Clone this repository in oh-my-zsh's plugins directory:
+
+        git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting
+
+2. Activate the plugin in `~/.zshrc`:
+
+        plugins=( [plugins...] zsh-syntax-highlighting)
+
+3. Source `~/.zshrc`  to take changes into account:
+
+        source ~/.zshrc
+
+#### [Prezto](https://github.com/sorin-ionescu/prezto)
+
+Zsh-syntax-highlighting is included with Prezto. See the [Prezto documentation]
+(https://github.com/sorin-ionescu/prezto/tree/master/modules/syntax-highlighting)
+to enable and configure highlighters.
+
+#### [zgen](https://github.com/tarjoilija/zgen)
+
+Add `zgen load zsh-users/zsh-syntax-highlighting` to the end of your `.zshrc`.
+
+#### [zplug](https://github.com/zplug/zplug)
+
+Add `zplug "zsh-users/zsh-syntax-highlighting", nice:10` to your `.zshrc`.
+
+#### [zplugin](https://github.com/psprint/zplugin)
+
+Add `zplugin load zsh-users/zsh-syntax-highlighting` to the end of your
+`.zshrc`.
+
+
+### System-wide installation
+
+Any of the above methods is suitable for a single-user installation,
+which requires no special privileges.  If, however, you desire to install
+zsh-syntax-highlighting system-wide, you may do so by running
+
+    make install
+
+and directing your users to add
+
+    source /usr/local/share/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh
+
+to their `.zshrc`s.

Makefile

@@ -2,30 +2,61 @@ NAME=zsh-syntax-highlighting
 
 INSTALL?=install -c
 PREFIX?=/usr/local
-SHARE_DIR=$(DESTDIR)$(PREFIX)/share/$(NAME)
+SHARE_DIR?=$(DESTDIR)$(PREFIX)/share/$(NAME)
+DOC_DIR?=$(DESTDIR)$(PREFIX)/share/doc/$(NAME)
+ZSH?=zsh # zsh binary to run tests with
 
-# Have the default target do nothing.
 all:
-	@ :
+	cd docs && \
+	cp highlighters.md all.md && \
+	printf '\n\nIndividual highlighters documentation\n=====================================' >> all.md && \
+	for doc in highlighters/*.md; do printf '\n\n'; cat "$$doc"; done >> all.md
 
-install:
+install: all
 	$(INSTALL) -d $(SHARE_DIR)
-	cp -r .version zsh-syntax-highlighting.zsh highlighters $(SHARE_DIR)
+	$(INSTALL) -d $(DOC_DIR)
+	cp .version zsh-syntax-highlighting.zsh $(SHARE_DIR)
+	cp COPYING.md README.md changelog.md $(DOC_DIR)
 	if [ x"true" = x"`git rev-parse --is-inside-work-tree 2>/dev/null`" ]; then \
 		git rev-parse HEAD; \
 	else \
 		cat .revision-hash; \
 	fi > $(SHARE_DIR)/.revision-hash
+	:
+# The [ -e ] check below is to because sh evaluates this with (the moral
+# equivalent of) NONOMATCH in effect, and highlighters/*.zsh has no matches.
+	for dirname in highlighters highlighters/*/ ; do \
+		$(INSTALL) -d $(SHARE_DIR)/"$$dirname"; \
+		for fname in "$$dirname"/*.zsh ; do [ -e "$$fname" ] && cp "$$fname" $(SHARE_DIR)"/$$dirname"; done; \
+	done
+	cp -R docs/* $(DOC_DIR)
+
+clean:
+	rm -f docs/all.md
 
 test:
 	@result=0; \
 	for test in highlighters/*; do \
 		if [ -d $$test/test-data ]; then \
 			echo "Running test $${test##*/}"; \
-			zsh -f tests/test-highlighting.zsh "$${test##*/}"; \
+			$(ZSH) -f tests/test-highlighting.zsh "$${test##*/}"; \
 			: $$(( result |= $$? )); \
 		fi \
 	done; \
 	exit $$result
 
-.PHONY: all install test
+quiet-test:
+	$(MAKE) test QUIET=y
+
+perf:
+	@result=0; \
+	for test in highlighters/*; do \
+		if [ -d $$test/test-data ]; then \
+			echo "Running test $${test##*/}"; \
+			$(ZSH) -f tests/test-perfs.zsh "$${test##*/}"; \
+			: $$(( result |= $$? )); \
+		fi \
+	done; \
+	exit $$result
+
+.PHONY: all install clean test perf

README.md

@@ -1,53 +1,38 @@
 zsh-syntax-highlighting
 =======================
 
-**[Fish shell](http://www.fishshell.com) like syntax highlighting for [Zsh](http://www.zsh.org).**
+**[Fish shell][fish]-like like syntax highlighting for [Zsh][zsh].**
 
 *Requirements: zsh 4.3.17+.*
 
+[fish]: http://www.fishshell.com/
+[zsh]: http://www.zsh.org/
+
+This package provides syntax highlighing for the shell zsh.  It enables
+highlighing of commands whilst they are typed at a zsh prompt into an
+interactive terminal.  This helps in reviewing commands before running
+them, particularly in catching syntax errors.
+
+Some examples:
+
+Before: [![Screenshot #1.1](images/before1-smaller.png)](images/before1.png)
+<br/>
+After:&nbsp; [![Screenshot #1.2](images/after1-smaller.png)](images/after1.png)
+
+Before: [![Screenshot #2.1](images/before2-smaller.png)](images/before2.png)
+<br/>
+After:&nbsp; [![Screenshot #2.2](images/after2-smaller.png)](images/after2.png)
+
+Before: [![Screenshot #3.1](images/before3-smaller.png)](images/before3.png)
+<br/>
+After:&nbsp; [![Screenshot #3.2](images/after3-smaller.png)](images/after3.png)
+
 
 How to install
 --------------
 
-### Using packages
+See [INSTALL.md](INSTALL.md).
 
-* Arch Linux: [community/zsh-syntax-highlighting](https://www.archlinux.org/packages/zsh-syntax-highlighting) / [AUR/zsh-syntax-highlighting-git](https://aur.archlinux.org/packages/zsh-syntax-highlighting-git)
-* Gentoo: [mv overlay](http://gpo.zugaina.org/app-shells/zsh-syntax-highlighting)
-* Mac OS X / Homebrew: [brew install zsh-syntax-highlighting](https://github.com/Homebrew/homebrew/blob/master/Library/Formula/zsh-syntax-highlighting.rb)
-
-### In your ~/.zshrc
-
-* Clone this repository:
-
-        git clone git://github.com/zsh-users/zsh-syntax-highlighting.git
-
-  (or [download a snapshot](https://github.com/zsh-users/zsh-syntax-highlighting/archive/master.tar.gz))
-
-* Source the script **at the end** of `~/.zshrc`:
-
-        source /path/to/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh
-
-* Source `~/.zshrc`  to take changes into account:
-
-        source ~/.zshrc
-
-
-### With oh-my-zsh
-
-* Download the script or clone this repository in [oh-my-zsh](http://github.com/robbyrussell/oh-my-zsh) plugins directory:
-
-        git clone git://github.com/zsh-users/zsh-syntax-highlighting.git ~/.oh-my-zsh/custom/plugins/zsh-syntax-highlighting
-
-* Activate the plugin in `~/.zshrc`:
-
-        plugins=( [plugins...] zsh-syntax-highlighting)
-
-* Source `~/.zshrc`  to take changes into account:
-
-        source ~/.zshrc
-
-Note that `zsh-syntax-highlighting` must be the last plugin sourced,
-so make it the last element of the `$plugins` array.
 
 FAQ
 ---
@@ -59,15 +44,35 @@ custom widgets have been created (i.e., after all `zle -N` calls and after
 running `compinit`).  Widgets created later will work, but will not update the
 syntax highlighting.
 
+### Does syntax highlighting work during incremental history search?
+
+Highlighting the command line during an incremental history search
+(with the `history-incremental-search-backward` widget, which is
+bound by default to <kbd>Ctrl+R</kbd> in zsh's emacs keymap) requires zsh 5.3
+or newer.
+
+Under zsh 5.2 and older, the zsh-default [underlining][zshzle-Character-Highlighting]
+of the matched portion of the buffer remains available, but zsh-syntax-highlighting's
+additional highlighting is unavailable.  (Those versions of zsh do not provide
+enough information to allow computing the highlighting correctly.)
+
+See [issue #288][i288] for details.
+
+[zshzle-Character-Highlighting]: http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#Character-Highlighting
+[i288]: https://github.com/zsh-users/zsh-syntax-highlighting/pull/288
+
 ### How are new releases announced?
 
-There is currently no "push" announcements channel.  However, the following alternatives exist:
+There is currently no "push" announcements channel.  However, the following
+alternatives exist:
 
 - GitHub's RSS feed of releases: https://github.com/zsh-users/zsh-syntax-highlighting/releases.atom
 - An anitya entry: https://release-monitoring.org/project/7552/
 
+
 How to tweak
 ------------
 
-Syntax highlighting is done by pluggable highlighter scripts, see the [highlighters directory](highlighters)
+Syntax highlighting is done by pluggable highlighter scripts.  See the
-for documentation and configuration settings.
+[documentation on highlighters](docs/highlighters.md) for details and
+configuration settings.

changelog.md

@@ -1,3 +1,393 @@
+up to 0.5.0
+
+# Changes in version 0.5.0
+
+
+## Performance improvements:
+
+We thank Sebastian Gniazdowski and "m0viefreak" for significant contributions
+in this area.
+
+- Optimize string operations in the `main` (default) highlighter.
+  (#372/3cb58fd7d7b9, 02229ebd6328, ef4bfe5bcc14, #372/c6b6513ac0d6, #374/15461e7d21c3)
+
+- Command word highlighting:  Use the `zsh/parameter` module to avoid forks.
+  Memoize (cache) the results.
+  (#298, 3ce01076b521, 2f18ba64e397, 12b879caf7a6; #320, 3b67e656bff5)
+
+- Avoid forks in the driver and in the `root` highlighter.
+  (b9112aec798a, 38c8fbea2dd2)
+
+
+## Added highlighting of:
+
+- `pkexec` (a precommand).
+  (#248, 4f3910cbbaa5)
+
+- Aliases that cannot be defined normally nor invoked normally (highlighted as an error).
+  (#263 (in part), 28932316cca6)
+
+- Path separators (`/`) — the default behaviour remains to highlight path separators
+  and path components the same way.
+  (#136, #260, 6cd39e7c70d3, 9a934d291e7c, f3d3aaa00cc4)
+
+- Assignments to individual positional arguments (`42=foo` to assign to `$42`).
+  (f4036a09cee3)
+
+- Linewise region (the `visual-line-mode` widget, bound to `V` in zsh's `vi` keymap).
+  (#267, a7a7f8b42280, ee07588cfd9b)
+
+- Command-lines recalled by `isearch` mode; requires zsh≥5.3.
+  (#261 (in part); #257; 4ad311ec0a68)
+
+- Command-lines whilst the `IGNORE_BRACES` or `IGNORE_CLOSE_BRACES` option is in effect.
+  (a8a6384356af, 02807f1826a5)
+
+- Mismatched parentheses and braces (in the `main` highlighter).
+  (51b9d79c3bb6, 2fabf7ca64b7, a4196eda5e6f, and others)
+
+- Mismatched `do`/`done` keywords.
+  (b2733a64da93)
+
+- Mismatched `foreach`/`end` keywords.
+  (#96, 2bb8f0703d8f)
+
+- In Bourne-style function definitions, when the `MULTI_FUNC_DEF` option is set
+  (which is the default), highlight the first word in the function body as
+  a command word: `f() { g "$@" }`.
+  (6f91850a01e1)
+
+- `always` blocks.
+  (#335, e5782e4ddfb6)
+
+- Command substitutions inside double quotes, `"$(echo foo)"`.
+  (#139 (in part), c3913e0d8ead)
+
+- Non-alphabetic parameters inside double quotes (`"$$"`, `"$#"`, `"$*"`, `"$@"`, `"$?"`, `"$-"`).
+  (4afe670f7a1b, 44ef6e38e5a7)
+
+- Command words from future versions of zsh (forward compatibly).
+  This also adds an `arg0` style that all other command word styles fall back to.
+  (b4537a972eed, bccc3dc26943)
+
+- Escaped history expansions inside double quotes: `: "\!"`
+  (28d7056a7a06, et seq)
+
+
+## Fixed highlighting of:
+
+- Command separator tokens in syntactically-invalid positions.
+  (09c4114eb980)
+
+- Redirections with a file descriptor number at command word.
+  (#238 (in part), 73ee7c1f6c4a)
+
+- The `select` prompt, `$PS3`.
+  (#268, 451665cb2a8b)
+
+- Values of variables in `vared`.
+  (e500ca246286)
+
+- `!` as an argument (neither a history expansion nor a reserved word).
+  (4c23a2fd1b90)
+
+- "division by zero" error under the `brackets` highlighter when `$ZSH_HIGHLIGHT_STYLES` is empty.
+  (f73f3d53d3a6)
+
+- Process substitutions, `<(pwd)` and `>(wc -l)`.
+  (#302, 6889ff6bd2ad, bfabffbf975c, fc9c892a3f15)
+
+- The non-`SHORT_LOOPS` form of `repeat` loops: `repeat 42; do true; done`.
+  (#290, 4832f18c50a5, ef68f50c048f, 6362c757b6f7)
+
+- Broken symlinks (are now highlighted as files).
+  (#342, 95f7206a9373, 53083da8215e)
+
+- Lines accepted from `isearch` mode.
+  (#284; #257, #259, #288; 5bae6219008b, a8fe22d42251)
+
+- Work around upstream bug that triggered when the command word was a relative
+  path, that when interpreted relative to a $PATH directory denoted a command;
+  the effect of that upstream bug was that the relative path was cached as
+  a "valid external command name".
+  (#354, #355, 51614ca2c994, fdaeec45146b, 7d38d07255e4;
+  upstream fix slated to be released in 5.3 (workers/39104))
+
+- After accepting a line with the cursor on a bracket, the matching bracket
+  of the bracket under the cursor no longer remains highlighted (with the
+  `brackets` highlighter).
+  (4c4baede519a)
+
+- The first word on a new line within an array assignment or initialization is no
+  longer considered a command position.
+  (8bf423d16d46)
+
+- Subshells that end at command position, `(A=42)`, `(true;)`.
+  (#231, 7fb6f9979121; #344, 4fc35362ee5a)
+
+- Command word after array assignment, `a=(lorem ipsum) pwd`.
+  (#330, 7fb6f9979121)
+
+
+## API changes (for highlighter authors):
+
+- New interface `_zsh_highlight_add_highlight`.
+  (341a3ae1f015, c346f6eb6fb6)
+
+- tests: Specify the style key, not its value, in test expectations.
+  (a830613467af, fd061b5730bf, eaa4335c3441, among others)
+
+- Module author documentation improvements.
+  (#306 (in part), 217669270418, 0ff354b44b6e, 80148f6c8402, 364f206a547f, and others)
+
+- The driver no longer defines a `_zsh_highlight_${highlighter}_highlighter_cache`
+  variable, which is in the highlighters' namespace.
+  (3e59ab41b6b8, 80148f6c8402, f91a7b885e7d)
+
+- Rename highlighter entry points.  The old names remain supported for
+  backwards compatibility.
+  (a3d5dfcbdae9, c793e0dceab1)
+
+- tests: Add the "NONE" expectation.
+  (4da9889d1545, 13018f3dd735, d37c55c788cd)
+
+- tests: consider a test that writes to stderr to have failed.
+  (#291, 1082067f9315)
+
+
+## Developer-visible changes:
+
+- Add `make quiet-test`.
+  (9b64ad750f35)
+
+- test harness: Better quote replaceables in error messages.
+  (30d8f92df225)
+
+- test harness: Fix exit code for XPASS.
+  (bb8d325c0cbd)
+
+- Create [HACKING.md](HACKING.md).
+  (cef49752fd0e)
+
+- tests: Emit a description for PASS test points.
+  (6aa57d60aa64, f0bae44b76dd)
+
+- tests: Create a script that generates a test file.
+  (8013dc3b8db6, et seq; `tests/generate.zsh`)
+
+
+## Other changes:
+
+- Under zsh≤5.2, widgets whose names start with a `_` are no longer excluded
+  from highlighting.
+  (ed33d2cb1388; reverts part of 186d80054a40 which was for #65)
+
+- Under zsh≤5.2, widgets implemented by a function named after the widget are
+  no longer excluded from highlighting.
+  (487b122c480d; reverts part of 776453cb5b69)
+
+- Under zsh≤5.2, shell-unsafe widget names can now be wrapped.
+  (#278, 6a634fac9fb9, et seq)
+
+- Correct some test expectations.
+  (78290e043bc5)
+
+- `zsh-syntax-highlighting.plugin.zsh`: Convert from symlink to plain file
+  for msys2 compatibility.
+  (#292, d4f8edc9f3ad)
+
+- Document installation under some plugin managers.
+  (e635f766bef9, 9cab566f539b)
+
+- Don't leak the `PATH_DIRS` option.
+  (7b82b88a7166)
+
+- Don't require the `FUNCTION_ARGZERO` option to be set.
+  (#338, 750aebc553f2)
+
+- Under zsh≤5.2, support binding incomplete/nonexistent widgets.
+  (9e569bb0fe04, part of #288)
+
+- Make the driver reentrant, fixing possibility of infinite recursion
+  under zsh≤5.2 under interaction with theoretical third-party code.
+  (#305, d711563fe1bf, 295d62ec888d, f3242cbd6aba)
+
+- Fix warnings when `WARN_CREATE_GLOBAL` is set prior to sourcing zsh-syntax-highlighting.
+  (z-sy-h already sets `WARN_CREATE_GLOBAL` internally.)
+  (da60234fb236)
+
+- Warn only once, rather than once per keypress, when a highlighter is unavailable.
+  (0a9b347483ae)
+
+
+# Changes in version 0.4.1
+
+## Fixes:
+
+- Arguments to widgets were not properly dash-escaped.  Only matters for widgets
+  that take arguments (i.e., that are invoked as `zle ${widget} -- ${args}`).
+  (282c7134e8ac, reverts c808d2187a73)
+
+
+# Changes in version 0.4.0
+
+
+## Added highlighting of:
+
+- incomplete sudo commands
+  (a3047a912100, 2f05620b19ae)
+
+        sudo;
+        sudo -u;
+
+- command words following reserved words
+  (#207, #222, b397b12ac139 et seq, 6fbd2aa9579b et seq, 8b4adbd991b0)
+
+        if ls; then ls; else ls; fi
+        repeat 10 do ls; done
+
+    (The `ls` are now highlighted as a command.)
+
+- comments (when `INTERACTIVE_COMMENTS` is set)
+  (#163, #167, 693de99a9030)
+
+        echo Hello # comment
+
+- closing brackets of arithmetic expansion, subshells, and blocks
+  (#226, a59f442d2d34, et seq)
+
+        (( foo ))
+        ( foo )
+        { foo }
+
+- command names enabled by the `PATH_DIRS` option
+  (#228, 96ee5116b182)
+
+        # When ~/bin/foo/bar exists, is executable, ~/bin is in $PATH,
+        # and 'setopt PATH_DIRS' is in effect
+        foo/bar
+
+- parameter expansions with braces inside double quotes
+  (#186, 6e3720f39d84)
+
+        echo "${foo}"
+
+- parameter expansions in command word
+  (#101, 4fcfb15913a2)
+
+        x=/bin/ls
+        $x -l
+
+- the command separators '|&', '&!', '&|'
+
+        view file.pdf &!  ls
+
+
+## Fixed highlighting of:
+
+- precommand modifiers at non-command-word position
+  (#209, 2c9f8c8c95fa)
+
+        ls command foo
+
+- sudo commands with infix redirections
+  (#221, be006aded590, 86e924970911)
+
+        sudo -u >/tmp/foo.out user ls
+
+- subshells; anonymous functions
+  (#166, #194, 0d1bfbcbfa67, 9e178f9f3948)
+
+        (true)
+        () { true }
+
+- parameter assignment statements with no command
+  (#205, 01d7eeb3c713)
+
+        A=1;
+
+    (The semicolon used to be highlighted as a mistake)
+
+- cursor highlighter: Remove the cursor highlighting when accepting a line.
+  (#109, 4f0c293fdef0)
+
+
+## Removed features:
+
+- Removed highlighting of approximate paths (`path_approx`).
+  (#187, 98aee7f8b9a3)
+
+
+## Other changes:
+
+- main highlighter refactored to use states rather than booleans.
+  (2080a441ac49, et seq)
+
+- Fix initialization when sourcing `zsh-syntax-highlighting.zsh` via a symlink
+  (083c47b00707)
+
+- docs: Add screenshot.
+  (57624bb9f64b)
+
+- widgets wrapping: Don't add '--' when invoking widgets.
+  (c808d2187a73) [_reverted in 0.4.1_]
+
+- Refresh highlighting upon `accept-*` widgets (`accept-line` et al).
+  (59fbdda64c21)
+
+- Stop leaking match/mbegin/mend to global scope (thanks to upstream
+  `WARN_CREATE_GLOBAL` improvements).
+  (d3deffbf46a4)
+
+- 'make install': Permit setting `$(SHARE_DIR)` from the environment.
+  (e1078a8b4cf1)
+
+- driver: Tolerate KSH_ARRAYS being set in the calling context.
+  (#162, 8f19af6b319d)
+
+- 'make install': Install documentation fully and properly.
+  (#219, b1619c001390, et seq)
+
+- docs: Improve 'main' highlighter's documentation.
+  (00de155063f5, 7d4252f5f596)
+
+- docs: Moved to a new docs/ tree; assorted minor updates
+  (c575f8f37567, 5b34c23cfad5, et seq)
+
+- docs: Split README.md into INSTALL.md
+  (0b3183f6cb9a)
+
+- driver: Report `$ZSH_HIGHLIGHT_REVISION` when running from git
+  (84734ba95026)
+
+
+## Developer-visible changes:
+
+- Test harness converted to [TAP](http://testanything.org/tap-specification.html) format
+  (d99aa58aaaef, et seq)
+
+- Run each test in a separate subprocess, isolating them from each other
+  (d99aa58aaaef, et seq)
+
+- Fix test failure with nonexisting $HOME
+  (#216, b2ac98b98150)
+
+- Test output is now colorized.
+  (4d3da30f8b72, 6fe07c096109)
+
+- Document `make install`
+  (a18a7427fd2c)
+
+- tests: Allow specifying the zsh binary to use.
+  (557bb7e0c6a0)
+
+- tests: Add 'make perf' target
+  (4513eaea71d7)
+
+- tests: Run each test in a sandbox directory
+  (c01533920245)
+
+
 # Changes in version 0.3.0
 
 

docs/highlighters.md

@@ -0,0 +1,97 @@
+zsh-syntax-highlighting / highlighters
+======================================
+
+Syntax highlighting is done by pluggable highlighters:
+
+* `main` - the base highlighter, and the only one [active by default][1].
+* `brackets` - [matches brackets][2] and parenthesis.
+* `pattern` - matches [user-defined patterns][3].
+* `cursor` - matches [the cursor position][4].
+* `root` - highlights the whole command line [if the current user is root][5].
+* `line` - applied to [the whole command line][6].
+
+[1]: highlighters/main.md
+[2]: highlighters/brackets.md
+[3]: highlighters/pattern.md
+[4]: highlighters/cursor.md
+[5]: highlighters/root.md
+[6]: highlighters/line.md
+
+
+How to activate highlighters
+----------------------------
+
+To activate an highlighter, add it to the `ZSH_HIGHLIGHT_HIGHLIGHTERS` array in
+`~/.zshrc`, for example:
+
+    ZSH_HIGHLIGHT_HIGHLIGHTERS=(main brackets pattern cursor)
+
+By default, `$ZSH_HIGHLIGHT_HIGHLIGHTERS` is unset and only the `main`
+highlighter is active.
+
+
+How to tweak highlighters
+-------------------------
+
+Highlighters look up styles from the `ZSH_HIGHLIGHT_STYLES` associative array.
+Navigate into the [individual highlighters' documentation](highlighters/) to
+see what styles (keys) each highlighter defines; the syntax for values is the
+same as the syntax of "types of highlighting" of the zsh builtin
+`$zle_highlight` array, which is documented in [the `zshzle(1)` manual
+page][zshzle-Character-Highlighting].
+
+[zshzle-Character-Highlighting]: http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#Character-Highlighting
+
+Some highlighters support additional configuration parameters; see each
+highlighter's documentation for details and examples.
+
+
+How to implement a new highlighter
+----------------------------------
+
+To create your own `acme` highlighter:
+
+* Create your script at
+    `highlighters/acme/acme-highlighter.zsh`.
+
+* Implement the `_zsh_highlight_highlighter_acme_predicate` function.
+  This function must return 0 when the highlighter needs to be called and
+  non-zero otherwise, for example:
+
+        _zsh_highlight_highlighter_acme_predicate() {
+          # Call this highlighter in SVN working copies
+          [[ -d .svn ]]
+        }
+
+* Implement the `_zsh_highlight_highlighter_acme_paint` function.
+  This function does the actual syntax highlighting, by calling
+  `_zsh_highlight_add_highlight` with the start and end of the region to
+  be highlighted and the `ZSH_HIGHLIGHT_STYLES` key to use. Define the default
+  style for that key in the highlighter script outside of any function with
+  `: ${ZSH_HIGHLIGHT_STYLES[key]:=value}`, being sure to prefix
+  the key with your highlighter name and a colon. For example:
+
+        : ${ZSH_HIGHLIGHT_STYLES[acme:aurora]:=fg=green}
+
+        _zsh_highlight_highlighter_acme_paint() {
+          # Colorize the whole buffer with the 'aurora' style
+          _zsh_highlight_add_highlight 0 $#BUFFER acme:aurora
+        }
+
+* Name your own functions and global variables `_zsh_highlight_acme_*`.
+
+    - In zsh-syntax-highlighting 0.4.0 and earlier, the entrypoints 
+        `_zsh_highlight_highlighter_acme_predicate` and
+        `_zsh_highlight_highlighter_acme_paint`
+        were named
+        `_zsh_highlight_acme_highlighter_predicate` and
+        `_zsh_highlight_highlighter_acme_paint` respectively.
+
+        These names are still supported for backwards compatibility;
+        however, support for them will be removed in a a future major or minor release (v0.x.0 or v1.0.0).
+
+* Activate your highlighter in `~/.zshrc`:
+
+        ZSH_HIGHLIGHT_HIGHLIGHTERS+=(acme)
+
+* [Write tests](../tests/README.md).

docs/highlighters/brackets.md

@@ -0,0 +1,29 @@
+zsh-syntax-highlighting / highlighters / brackets
+-------------------------------------------------
+
+This is the `brackets` highlighter, that highlights brackets and parentheses, and
+matches them.
+
+
+### How to tweak it
+
+This highlighter defines the following styles:
+
+* `bracket-error` - unmatched brackets
+* `bracket-level-N` - brackets with nest level N
+* `cursor-matchingbracket` - the matching bracket, if cursor is on a bracket
+
+To override one of those styles, change its entry in `ZSH_HIGHLIGHT_STYLES`,
+for example in `~/.zshrc`:
+
+    # To define styles for nested brackets up to level 4
+    ZSH_HIGHLIGHT_STYLES[bracket-level-1]='fg=blue,bold'
+    ZSH_HIGHLIGHT_STYLES[bracket-level-2]='fg=red,bold'
+    ZSH_HIGHLIGHT_STYLES[bracket-level-3]='fg=yellow,bold'
+    ZSH_HIGHLIGHT_STYLES[bracket-level-4]='fg=magenta,bold'
+
+The syntax for values is the same as the syntax of "types of highlighting" of
+the zsh builtin `$zle_highlight` array, which is documented in [the `zshzle(1)`
+manual page][zshzle-Character-Highlighting].
+
+[zshzle-Character-Highlighting]: http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#Character-Highlighting

docs/highlighters/cursor.md

@@ -0,0 +1,22 @@
+zsh-syntax-highlighting / highlighters / cursor
+-----------------------------------------------
+
+This is the `cursor` highlighter, that highlights the cursor.
+
+
+### How to tweak it
+
+This highlighter defines the following styles:
+
+* `cursor` - the style for the current cursor position
+
+To override one of those styles, change its entry in `ZSH_HIGHLIGHT_STYLES`,
+for example in `~/.zshrc`:
+
+    ZSH_HIGHLIGHT_STYLES[cursor]='bg=blue'
+
+The syntax for values is the same as the syntax of "types of highlighting" of
+the zsh builtin `$zle_highlight` array, which is documented in [the `zshzle(1)`
+manual page][zshzle-Character-Highlighting].
+
+[zshzle-Character-Highlighting]: http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#Character-Highlighting

docs/highlighters/line.md

@@ -0,0 +1,22 @@
+zsh-syntax-highlighting / highlighters / line
+---------------------------------------------
+
+This is the `line` highlighter, that highlights the whole line.
+
+
+### How to tweak it
+
+This highlighter defines the following styles:
+
+* `line` - the style for the whole line
+
+To override one of those styles, change its entry in `ZSH_HIGHLIGHT_STYLES`,
+for example in `~/.zshrc`:
+
+    ZSH_HIGHLIGHT_STYLES[line]='bold'
+
+The syntax for values is the same as the syntax of "types of highlighting" of
+the zsh builtin `$zle_highlight` array, which is documented in [the `zshzle(1)`
+manual page][zshzle-Character-Highlighting].
+
+[zshzle-Character-Highlighting]: http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#Character-Highlighting

docs/highlighters/main.md

@@ -0,0 +1,91 @@
+zsh-syntax-highlighting / highlighters / main
+---------------------------------------------
+
+This is the `main` highlighter, that highlights:
+
+* Commands
+* Options
+* Arguments
+* Paths
+* Strings
+
+This highlighter is active by default.
+
+
+### How to tweak it
+
+This highlighter defines the following styles:
+
+* `unknown-token` - unknown tokens / errors
+* `reserved-word` - shell reserved words (`if`, `for`)
+* `alias` - aliases
+* `suffix-alias` - suffix aliases (requires zsh 5.1.1 or newer)
+* `builtin` - shell builtin commands (`shift`, `pwd`, `zstyle`)
+* `function` - function names
+* `command` - command names
+* `precommand` - precommand modifiers (e.g., `noglob`, `builtin`)
+* `commandseparator` - command separation tokens (`;`, `&&`)
+* `hashed-command` - hashed commands
+* `path` - existing filenames
+* `path_pathseparator` - path separators in filenames (`/`); if unset, `path` is used (default)
+* `path_prefix` - prefixes of existing filenames
+* `path_prefix_pathseparator` - path separators in prefixes of existing filenames (`/`); if unset, `path_prefix` is used (default)
+* `globbing` - globbing expressions (`*.txt`)
+* `history-expansion` - history expansion expressions (`!foo` and `^foo^bar`)
+* `single-hyphen-option` - single hyphen options (`-o`)
+* `double-hyphen-option` - double hyphen options (`--option`)
+* `back-quoted-argument` - backquoted expressions (`` `foo` ``)
+* `single-quoted-argument` - single quoted arguments (`` 'foo' ``)
+* `double-quoted-argument` - double quoted arguments (`` "foo" ``)
+* `dollar-quoted-argument` - dollar quoted arguments (`` $'foo' ``)
+* `dollar-double-quoted-argument` - parameter expansion inside double quotes (`$foo` inside `""`)
+* `back-double-quoted-argument` -  back double quoted arguments (`\x` inside `""`)
+* `back-dollar-quoted-argument` -  back dollar quoted arguments (`\x` inside `$''`)
+* `assign` - parameter assignments
+* `redirection` - redirection operators (`<`, `>`, etc)
+* `comment` - comments, when `setopt INTERACTIVE_COMMENTS` is in effect (`echo # foo`)
+* `arg0` - a command word other than one of those enumrated above (other than a command, precommand, alias, function, or shell builtin command).
+* `default` - everything else
+
+To override one of those styles, change its entry in `ZSH_HIGHLIGHT_STYLES`,
+for example in `~/.zshrc`:
+
+    # Declare the variable
+    typeset -A ZSH_HIGHLIGHT_STYLES
+
+    # To differentiate aliases from other command types
+    ZSH_HIGHLIGHT_STYLES[alias]='fg=magenta,bold'
+    
+    # To have paths colored instead of underlined
+    ZSH_HIGHLIGHT_STYLES[path]='fg=cyan'
+    
+    # To disable highlighting of globbing expressions
+    ZSH_HIGHLIGHT_STYLES[globbing]='none'
+
+The syntax for values is the same as the syntax of "types of highlighting" of
+the zsh builtin `$zle_highlight` array, which is documented in [the `zshzle(1)`
+manual page][zshzle-Character-Highlighting].
+
+### Useless trivia
+
+#### Forward compatibility.
+
+zsh-syntax-highlighting attempts to be forward-compatible with zsh.
+Specifically, we attempt to facilitate highlighting _command word_ types that
+had not yet been invented when this version of zsh-syntax-highlighting was
+released.
+
+A _command word_ is something like a function name, external command name, et
+cetera.  (See
+[Simple Commands & Pipelines in `zshmisc(1)`][zshmisc-Simple-Commands-And-Pipelines]
+for a formal definition.)
+
+If a new _kind_ of command word is ever added to zsh — something conceptually
+different than "function" and "alias" and "external command" — then command words
+of that (new) kind will be highlighted by the style `arg0_$kind`,
+where `$kind` is the output of `type -w` on the new kind of command word.  If that
+style is not defined, then the style `arg0` will be used instead.
+
+[zshmisc-Simple-Commands-And-Pipelines]: http://zsh.sourceforge.net/Doc/Release/Shell-Grammar.html#Simple-Commands-_0026-Pipelines
+
+[zshzle-Character-Highlighting]: http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#Character-Highlighting

docs/highlighters/pattern.md

@@ -0,0 +1,19 @@
+zsh-syntax-highlighting / highlighters / pattern
+------------------------------------------------
+
+This is the `pattern` highlighter, that highlights user-defined patterns.
+
+
+### How to tweak it
+
+To use this highlighter, associate patterns with styles in the
+`ZSH_HIGHLIGHT_PATTERNS` associative array, for example in `~/.zshrc`:
+
+    # To have commands starting with `rm -rf` in red:
+    ZSH_HIGHLIGHT_PATTERNS+=('rm -rf *' 'fg=white,bold,bg=red')
+
+The syntax for values is the same as the syntax of "types of highlighting" of
+the zsh builtin `$zle_highlight` array, which is documented in [the `zshzle(1)`
+manual page][zshzle-Character-Highlighting].
+
+[zshzle-Character-Highlighting]: http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#Character-Highlighting

docs/highlighters/root.md

@@ -0,0 +1,23 @@
+zsh-syntax-highlighting / highlighters / root
+---------------------------------------------
+
+This is the `root` highlighter, that highlights the whole line if the current
+user is root.
+
+
+### How to tweak it
+
+This highlighter defines the following styles:
+
+* `root` - the style for the whole line if the current user is root.
+
+To override one of those styles, change its entry in `ZSH_HIGHLIGHT_STYLES`,
+for example in `~/.zshrc`:
+
+    ZSH_HIGHLIGHT_STYLES[root]='bg=red'
+
+The syntax for values is the same as the syntax of "types of highlighting" of
+the zsh builtin `$zle_highlight` array, which is documented in [the `zshzle(1)`
+manual page][zshzle-Character-Highlighting].
+
+[zshzle-Character-Highlighting]: http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#Character-Highlighting

highlighters/README.md

@@ -1,50 +1,8 @@
 zsh-syntax-highlighting / highlighters
 ======================================
 
-Syntax highlighting is done by pluggable highlighters:
+Navigate into the individual highlighters' documentation to see
+what styles (`$ZSH_HIGHLIGHT_STYLES` keys) each highlighter defines.
 
-* [***main***](main) - the base highlighter, and the only one active by default.
+Refer to the [documentation on highlighters](../docs/highlighters.md) for further
-* [***brackets***](brackets) - matches brackets and parenthesis.
+information.
-* [***pattern***](pattern) - matches user-defined patterns.
-* [***cursor***](cursor) - matches the cursor position.
-* [***root***](root) - triggered if the current user is root.
-* [***line***](line) - applied to the whole command line
-
-
-How to activate highlighters
-----------------------------
-
-To activate an highlighter, add it to the `ZSH_HIGHLIGHT_HIGHLIGHTERS` array in `~/.zshrc`, for example:
-
-    ZSH_HIGHLIGHT_HIGHLIGHTERS=(main brackets pattern cursor)
-
-
-How to tweak highlighters
--------------------------
-
-Highlighters look up styles from the `ZSH_HIGHLIGHT_STYLES` array. Navigate into each highlighter directory to see what styles it defines and how to configure it.
-
-
-How to implement a new highlighter
-----------------------------------
-
-To create your own ***myhighlighter*** highlighter:
-
-* Create your script at **highlighters/*myhighlighter*/*myhighlighter*-highlighter.zsh**.
-* Implement the `_zsh_highlight_myhighlighter_highlighter_predicate` function. This function must return 0 when the highlighter needs to be called, for example:
-
-        _zsh_highlight_myhighlighter_highlighter_predicate() {
-          # Call this highlighter in SVN repositories
-          [[ -d .svn ]]
-        }
-
-* Implement the `_zsh_highlight_myhighlighter_highlighter` function. This function does the actual syntax highlighting, by modifying `region_highlight`, for example:
-
-        _zsh_highlight_myhighlighter_highlighter() {
-          # Colorize the whole buffer with blue background
-          region_highlight+=(0 $#BUFFER bg=blue)
-        }
-
-* Activate your highlighter in `~/.zshrc`:
-
-        ZSH_HIGHLIGHT_HIGHLIGHTERS+=(myhighlighter)

highlighters/brackets/README.md

@@ -1,30 +0,0 @@
-zsh-syntax-highlighting / highlighters / brackets
-=================================================
-
-This is the ***brackets*** highlighter, that highlights brackets, parenthesis and matches them.
-
-
-How to activate it
-------------------
-To activate it, add it to `ZSH_HIGHLIGHT_HIGHLIGHTERS`:
-
-    ZSH_HIGHLIGHT_HIGHLIGHTERS=( [...] brackets)
-
-
-How to tweak it
----------------
-This highlighter defines the following styles:
-
-* `bracket-error` - unmatched brackets
-* `bracket-level-N` - brackets with nest level N
-* `cursor-matchingbracket` - the matching bracket, if cursor is on a bracket
-
-To override one of those styles, change its entry in `ZSH_HIGHLIGHT_STYLES`, for example in `~/.zshrc`:
-
-    # To define styles for nested brackets up to level 4
-    ZSH_HIGHLIGHT_STYLES[bracket-level-1]='fg=blue,bold'
-    ZSH_HIGHLIGHT_STYLES[bracket-level-2]='fg=red,bold'
-    ZSH_HIGHLIGHT_STYLES[bracket-level-3]='fg=yellow,bold'
-    ZSH_HIGHLIGHT_STYLES[bracket-level-4]='fg=magenta,bold'
-
-The syntax for declaring styles is [documented here](http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#SEC135).

highlighters/brackets/README.md

@@ -0,0 +1 @@
+../../docs/highlighters/brackets.md

highlighters/brackets/brackets-highlighter.zsh

@@ -1,5 +1,5 @@
 # -------------------------------------------------------------------------------------------------
-# Copyright (c) 2010-2011 zsh-syntax-highlighting contributors
+# Copyright (c) 2010-2016 zsh-syntax-highlighting contributors
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without modification, are permitted
@@ -38,73 +38,70 @@
 : ${ZSH_HIGHLIGHT_STYLES[cursor-matchingbracket]:=standout}
 
 # Whether the brackets highlighter should be called or not.
-_zsh_highlight_brackets_highlighter_predicate()
+_zsh_highlight_highlighter_brackets_predicate()
 {
-  _zsh_highlight_cursor_moved || _zsh_highlight_buffer_modified
+  [[ $WIDGET == zle-line-finish ]] || _zsh_highlight_cursor_moved || _zsh_highlight_buffer_modified
 }
 
 # Brackets highlighting function.
-_zsh_highlight_brackets_highlighter()
+_zsh_highlight_highlighter_brackets_paint()
 {
-  local level=0 pos
+  local char style
-  local -A levelpos lastoflevel matching typepos
+  local -i bracket_color_size=${#ZSH_HIGHLIGHT_STYLES[(I)bracket-level-*]} buflen=${#BUFFER} level=0 matchingpos pos
-  region_highlight=()
+  local -A levelpos lastoflevel matching
 
   # Find all brackets and remember which one is matching
-  for (( pos = 0; $pos < ${#BUFFER}; pos++ )) ; do
+  for (( pos = 1; pos <= buflen; pos++ )) ; do
-    local char="$BUFFER[pos+1]"
+    char=$BUFFER[pos]
     case $char in
       ["([{"])
         levelpos[$pos]=$((++level))
         lastoflevel[$level]=$pos
-        _zsh_highlight_brackets_highlighter_brackettype "$char"
         ;;
       [")]}"])
-        matching[$lastoflevel[$level]]=$pos
+        matchingpos=$lastoflevel[$level]
-        matching[$pos]=$lastoflevel[$level]
         levelpos[$pos]=$((level--))
-        _zsh_highlight_brackets_highlighter_brackettype "$char"
+	if _zsh_highlight_brackets_match $matchingpos $pos; then
+          matching[$matchingpos]=$pos
+          matching[$pos]=$matchingpos
+        fi
         ;;
       ['"'\'])
         # Skip everything inside quotes
-        local quotetype=$char
+	pos=$BUFFER[(ib:pos+1:)$char]
-        while (( $pos < ${#BUFFER} )) ; do
-          (( pos++ ))
-          [[ $BUFFER[$pos+1] == $quotetype ]] && break
-        done
         ;;
     esac
   done
 
   # Now highlight all found brackets
   for pos in ${(k)levelpos}; do
-    if [[ -n $matching[$pos] ]] && [[ $typepos[$pos] == $typepos[$matching[$pos]] ]]; then
+    if (( $+matching[$pos] )); then
-      local bracket_color_size=${#ZSH_HIGHLIGHT_STYLES[(I)bracket-level-*]}
+      if (( bracket_color_size )); then
-      local bracket_color_level=bracket-level-$(( (levelpos[$pos] - 1) % bracket_color_size + 1 ))
+        style=bracket-level-$(( (levelpos[$pos] - 1) % bracket_color_size + 1 ))
-      local style=$ZSH_HIGHLIGHT_STYLES[$bracket_color_level]
-      region_highlight+=("$pos $((pos + 1)) $style")
-    else
-      local style=$ZSH_HIGHLIGHT_STYLES[bracket-error]
-      region_highlight+=("$pos $((pos + 1)) $style")
       fi
+    else
+      style=bracket-error
+    fi
+    _zsh_highlight_add_highlight $((pos - 1)) $pos $style
   done
 
-  # If cursor is on a bracket, then highlight corresponding bracket, if any
+  # If cursor is on a bracket, then highlight corresponding bracket, if any.
-  pos=$CURSOR
+  if [[ $WIDGET != zle-line-finish ]]; then
+    pos=$((CURSOR + 1))
     if [[ -n $levelpos[$pos] ]] && [[ -n $matching[$pos] ]]; then
-    local otherpos=$matching[$pos]
+      local -i otherpos=$matching[$pos]
-    local style=$ZSH_HIGHLIGHT_STYLES[cursor-matchingbracket]
+      _zsh_highlight_add_highlight $((otherpos - 1)) $otherpos cursor-matchingbracket
-    region_highlight+=("$otherpos $((otherpos + 1)) $style")
+    fi
   fi
 }
 
 # Helper function to differentiate type 
-_zsh_highlight_brackets_highlighter_brackettype()
+_zsh_highlight_brackets_match()
 {
-  case $1 in
+  case $BUFFER[$1] in
-    ["()"]) typepos[$pos]=round;;
+    \() [[ $BUFFER[$2] == \) ]];;
-    ["[]"]) typepos[$pos]=bracket;;
+    \[) [[ $BUFFER[$2] == \] ]];;
-    ["{}"]) typepos[$pos]=curly;;
+    \{) [[ $BUFFER[$2] == \} ]];;
-    *) ;;
+    *) false;;
   esac
 }

highlighters/brackets/test-data/cursor-matchingbracket-line-finish.zsh

@@ -0,0 +1,37 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+WIDGET=zle-line-finish
+
+BUFFER=': $foo[bar]'
+CURSOR=6 # cursor is zero-based
+
+expected_region_highlight=(
+  "11 11 NONE"
+)

highlighters/brackets/test-data/cursor-matchingbracket.zsh

@@ -0,0 +1,39 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+ZSH_HIGHLIGHT_STYLES[bracket-level-1]=
+ZSH_HIGHLIGHT_STYLES[bracket-level-2]=
+ZSH_HIGHLIGHT_STYLES[bracket-level-3]=
+
+BUFFER=': ((( )))'
+CURSOR=2 # cursor is zero-based
+
+expected_region_highlight=(
+  "9 9 cursor-matchingbracket"
+)

highlighters/brackets/test-data/empty-styles.zsh

@@ -0,0 +1,34 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=': (x)'
+
+expected_region_highlight=(
+  "1 5 NONE"
+)

highlighters/brackets/test-data/loop-styles.zsh

@@ -0,0 +1,51 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+ZSH_HIGHLIGHT_STYLES[bracket-level-1]=
+ZSH_HIGHLIGHT_STYLES[bracket-level-2]=
+ZSH_HIGHLIGHT_STYLES[bracket-level-3]=
+
+BUFFER=': ({[({[(x)]})]})'
+
+expected_region_highlight=(
+  "3  3  bracket-level-1"
+  "4  4  bracket-level-2"
+  "5  5  bracket-level-3"
+  "6  6  bracket-level-1"
+  "7  7  bracket-level-2"
+  "8  8  bracket-level-3"
+  "9  9  bracket-level-1"
+  "11 11 bracket-level-1"
+  "12 12 bracket-level-3"
+  "13 13 bracket-level-2"
+  "14 14 bracket-level-1"
+  "15 15 bracket-level-3"
+  "16 16 bracket-level-2"
+  "17 17 bracket-level-1"
+)

highlighters/brackets/test-data/mismatch-patentheses.zsh

@@ -27,11 +27,14 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
+ZSH_HIGHLIGHT_STYLES[bracket-level-1]=
+ZSH_HIGHLIGHT_STYLES[bracket-level-2]=
+
 BUFFER='echo ({x}]'
 
 expected_region_highlight=(
-  "6  6  $ZSH_HIGHLIGHT_STYLES[bracket-error]" # (
+  "6  6  bracket-error" # (
-  "7  7  $ZSH_HIGHLIGHT_STYLES[bracket-level-2]" # {
+  "7  7  bracket-level-2" # {
-  "9  9  $ZSH_HIGHLIGHT_STYLES[bracket-level-2]" # }
+  "9  9  bracket-level-2" # }
-  "10 10 $ZSH_HIGHLIGHT_STYLES[bracket-error]" # )
+  "10 10 bracket-error" # )
 )

highlighters/brackets/test-data/near-quotes.zsh

@@ -0,0 +1,38 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+ZSH_HIGHLIGHT_STYLES[bracket-level-1]=
+
+BUFFER=': {"{x}"}'
+
+expected_region_highlight=(
+  "3 3 bracket-level-1"
+  "4 8 NONE"
+  "9 9 bracket-level-1"
+)

highlighters/brackets/test-data/nested-parentheses.zsh

@@ -27,13 +27,17 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
+ZSH_HIGHLIGHT_STYLES[bracket-level-1]=
+ZSH_HIGHLIGHT_STYLES[bracket-level-2]=
+ZSH_HIGHLIGHT_STYLES[bracket-level-3]=
+
 BUFFER='echo $(echo ${(z)array})'
 
 expected_region_highlight=(
-  "7  7  $ZSH_HIGHLIGHT_STYLES[bracket-level-1]" # (
+  "7  7  bracket-level-1" # (
-  "14 14 $ZSH_HIGHLIGHT_STYLES[bracket-level-2]" # {
+  "14 14 bracket-level-2" # {
-  "15 15 $ZSH_HIGHLIGHT_STYLES[bracket-level-3]" # (
+  "15 15 bracket-level-3" # (
-  "17 17 $ZSH_HIGHLIGHT_STYLES[bracket-level-3]" # )
+  "17 17 bracket-level-3" # )
-  "23 23 $ZSH_HIGHLIGHT_STYLES[bracket-level-2]" # }
+  "23 23 bracket-level-2" # }
-  "24 24 $ZSH_HIGHLIGHT_STYLES[bracket-level-1]" # )
+  "24 24 bracket-level-1" # )
 )

highlighters/brackets/test-data/nested-quote.zsh

@@ -0,0 +1,36 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+ZSH_HIGHLIGHT_STYLES[bracket-level-1]=
+
+BUFFER=': "\"{x"'
+
+expected_region_highlight=(
+  "3 9 NONE 'issue #303'"
+)

highlighters/brackets/test-data/quoted-patentheses.zsh

@@ -30,5 +30,5 @@
 BUFFER='echo "foo ( bar"'
 
 expected_region_highlight=(
-"1  16 $ZSH_HIGHLIGHT_STYLES[none]" # We expect the brackets highlighter to do nothing 
+"1  16 NONE" # We expect the brackets highlighter to do nothing
 )

highlighters/brackets/test-data/simple-parentheses.zsh

@@ -27,11 +27,14 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
+ZSH_HIGHLIGHT_STYLES[bracket-level-1]=
+ZSH_HIGHLIGHT_STYLES[bracket-level-2]=
+
 BUFFER='echo ({x})'
 
 expected_region_highlight=(
-  "6  6  $ZSH_HIGHLIGHT_STYLES[bracket-level-1]" # (
+  "6  6  bracket-level-1" # (
-  "7  7  $ZSH_HIGHLIGHT_STYLES[bracket-level-2]" # {
+  "7  7  bracket-level-2" # {
-  "9  9  $ZSH_HIGHLIGHT_STYLES[bracket-level-2]" # }
+  "9  9  bracket-level-2" # }
-  "10 10 $ZSH_HIGHLIGHT_STYLES[bracket-level-1]" # )
+  "10 10 bracket-level-1" # )
 )

highlighters/brackets/test-data/unclosed-patentheses.zsh

@@ -27,10 +27,13 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
+ZSH_HIGHLIGHT_STYLES[bracket-level-1]=
+ZSH_HIGHLIGHT_STYLES[bracket-level-2]=
+
 BUFFER='echo ({x}'
 
 expected_region_highlight=(
-  "6  6  $ZSH_HIGHLIGHT_STYLES[bracket-error]" # (
+  "6  6  bracket-error" # (
-  "7  7  $ZSH_HIGHLIGHT_STYLES[bracket-level-2]" # {
+  "7  7  bracket-level-2" # {
-  "9  9  $ZSH_HIGHLIGHT_STYLES[bracket-level-2]" # }
+  "9  9  bracket-level-2" # }
 )

highlighters/brackets/test-data/unclosed-patentheses2.zsh

@@ -27,10 +27,12 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
+ZSH_HIGHLIGHT_STYLES[bracket-level-1]=
+
 BUFFER='echo {x})'
 
 expected_region_highlight=(
-  "6  6  $ZSH_HIGHLIGHT_STYLES[bracket-level-1]" # {
+  "6  6  bracket-level-1" # {
-  "8  8  $ZSH_HIGHLIGHT_STYLES[bracket-level-1]" # }
+  "8  8  bracket-level-1" # }
-  "9  9 $ZSH_HIGHLIGHT_STYLES[bracket-error]" # )
+  "9  9 bracket-error" # )
 )

highlighters/cursor/README.md

@@ -1,24 +0,0 @@
-zsh-syntax-highlighting / highlighters / cursor
-=================================================
-
-This is the ***cursor*** highlighter, that highlights the cursor.
-
-
-How to activate it
-------------------
-To activate it, add it to `ZSH_HIGHLIGHT_HIGHLIGHTERS`:
-
-    ZSH_HIGHLIGHT_HIGHLIGHTERS=( [...] cursor)
-
-
-How to tweak it
----------------
-This highlighter defines the following styles:
-
-* `cursor` - the style for the current cursor position
-
-To override one of those styles, change its entry in `ZSH_HIGHLIGHT_STYLES`, for example in `~/.zshrc`:
-
-    ZSH_HIGHLIGHT_STYLES[cursor]='bg=blue'
-
-The syntax for declaring styles is [documented here](http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#SEC135).

highlighters/cursor/README.md

@@ -0,0 +1 @@
+../../docs/highlighters/cursor.md

highlighters/cursor/cursor-highlighter.zsh

@@ -32,13 +32,16 @@
 : ${ZSH_HIGHLIGHT_STYLES[cursor]:=standout}
 
 # Whether the cursor highlighter should be called or not.
-_zsh_highlight_cursor_highlighter_predicate()
+_zsh_highlight_highlighter_cursor_predicate()
 {
-  _zsh_highlight_cursor_moved
+  # remove cursor highlighting when the line is finished
+  [[ $WIDGET == zle-line-finish ]] || _zsh_highlight_cursor_moved
 }
 
 # Cursor highlighting function.
-_zsh_highlight_cursor_highlighter()
+_zsh_highlight_highlighter_cursor_paint()
 {
-  region_highlight+=("$CURSOR $(( $CURSOR + 1 )) $ZSH_HIGHLIGHT_STYLES[cursor]")
+  [[ $WIDGET == zle-line-finish ]] && return
+  
+  _zsh_highlight_add_highlight $CURSOR $(( $CURSOR + 1 )) cursor
 }

highlighters/line/README.md

@@ -1,24 +0,0 @@
-zsh-syntax-highlighting / highlighters / line
-=================================================
-
-This is the ***line*** highlighter, that highlights the whole line.
-
-
-How to activate it
-------------------
-To activate it, add it to `ZSH_HIGHLIGHT_HIGHLIGHTERS`:
-
-    ZSH_HIGHLIGHT_HIGHLIGHTERS=( [...] line)
-
-
-How to tweak it
----------------
-This highlighter defines the following styles:
-
-* `line` - the style for the whole line
-
-To override one of those styles, change its entry in `ZSH_HIGHLIGHT_STYLES`, for example in `~/.zshrc`:
-
-    ZSH_HIGHLIGHT_STYLES[line]='bold'
-
-The syntax for declaring styles is [documented here](http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#SEC135).

highlighters/line/README.md

@@ -0,0 +1 @@
+../../docs/highlighters/line.md

highlighters/line/line-highlighter.zsh

@@ -32,13 +32,13 @@
 : ${ZSH_HIGHLIGHT_STYLES[line]:=}
 
 # Whether the root highlighter should be called or not.
-_zsh_highlight_line_highlighter_predicate()
+_zsh_highlight_highlighter_line_predicate()
 {
   _zsh_highlight_buffer_modified
 }
 
 # root highlighting function.
-_zsh_highlight_line_highlighter()
+_zsh_highlight_highlighter_line_paint()
 {
-  region_highlight+=("0 $#BUFFER $ZSH_HIGHLIGHT_STYLES[line]")
+  _zsh_highlight_add_highlight 0 $#BUFFER line
 }

highlighters/main/README.md

@@ -1,68 +0,0 @@
-zsh-syntax-highlighting / highlighters / main
-=============================================
-
-This is the ***main*** highlighter, that highlights:
-
-* Commands
-* Options
-* Arguments
-* Paths
-* Strings
-
-How to activate it
-------------------
-To activate it, add it to `ZSH_HIGHLIGHT_HIGHLIGHTERS`:
-
-    ZSH_HIGHLIGHT_HIGHLIGHTERS=( [...] main)
-
-This highlighter is active by default.
-
-
-How to tweak it
----------------
-This highlighter defines the following styles:
-
-* `unknown-token` - unknown tokens / errors
-* `reserved-word` - shell reserved words
-* `alias` - aliases
-* `suffix-alias` - suffix aliases (requires zsh 5.1.1 or newer)
-* `builtin` - shell builtin commands
-* `function` - functions
-* `command` - commands
-* `precommand` - precommands (i.e. exec, builtin, ...)
-* `commandseparator` - command separation tokens
-* `hashed-command` - hashed commands
-* `path` - paths
-* `path_prefix` - path prefixes
-* `path_approx` - approximated paths
-* `globbing` - globbing expressions (`*.txt`)
-* `history-expansion` - history expansion expressions (`!foo` and `^foo^bar`)
-* `single-hyphen-option` - single hyphen options (-o)
-* `double-hyphen-option` - double hyphen options (--option)
-* `back-quoted-argument` - backquoted expressions (`` `foo` ``)
-* `single-quoted-argument` - single quoted arguments (`` 'foo' ``)
-* `double-quoted-argument` - double quoted arguments (`` "foo" ``)
-* `dollar-quoted-argument` - dollar quoted arguments (`` $'foo' ``)
-* `dollar-double-quoted-argument` -  dollar double quoted arguments ($foo inside "")
-* `back-double-quoted-argument` -  back double quoted arguments (\x inside "")
-* `back-dollar-quoted-argument` -  back dollar quoted arguments (\x inside $'')
-* `assign` - variable assignments
-* `redirection` - redirection operators (`<`, `>`, etc)
-* `default` - parts of the buffer that do not match anything
-
-To override one of those styles, change its entry in `ZSH_HIGHLIGHT_STYLES`, for example in `~/.zshrc`:
-
-    # Declare the variable
-    typeset -A ZSH_HIGHLIGHT_STYLES
-
-    # To differentiate aliases from other command types
-    ZSH_HIGHLIGHT_STYLES[alias]='fg=magenta,bold'
-    
-    # To have paths colored instead of underlined
-    ZSH_HIGHLIGHT_STYLES[path]='fg=cyan'
-    
-    # To disable highlighting of globbing expressions
-    ZSH_HIGHLIGHT_STYLES[globbing]='none'
-
-The syntax for declaring styles is documented in [the `zshzle(1)` manual
-page](http://zsh.sourceforge.net/Doc/Release/Zsh-Line-Editor.html#SEC135).

highlighters/main/README.md

@@ -0,0 +1 @@
+../../docs/highlighters/main.md

highlighters/main/main-highlighter.zsh

@@ -1,5 +1,5 @@
 # -------------------------------------------------------------------------------------------------
-# Copyright (c) 2010-2015 zsh-syntax-highlighting contributors
+# Copyright (c) 2010-2016 zsh-syntax-highlighting contributors
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without modification, are permitted
@@ -32,17 +32,12 @@
 : ${ZSH_HIGHLIGHT_STYLES[default]:=none}
 : ${ZSH_HIGHLIGHT_STYLES[unknown-token]:=fg=red,bold}
 : ${ZSH_HIGHLIGHT_STYLES[reserved-word]:=fg=yellow}
-: ${ZSH_HIGHLIGHT_STYLES[alias]:=fg=green}
 : ${ZSH_HIGHLIGHT_STYLES[suffix-alias]:=fg=green,underline}
-: ${ZSH_HIGHLIGHT_STYLES[builtin]:=fg=green}
-: ${ZSH_HIGHLIGHT_STYLES[function]:=fg=green}
-: ${ZSH_HIGHLIGHT_STYLES[command]:=fg=green}
 : ${ZSH_HIGHLIGHT_STYLES[precommand]:=fg=green,underline}
 : ${ZSH_HIGHLIGHT_STYLES[commandseparator]:=none}
-: ${ZSH_HIGHLIGHT_STYLES[hashed-command]:=fg=green}
 : ${ZSH_HIGHLIGHT_STYLES[path]:=underline}
-: ${ZSH_HIGHLIGHT_STYLES[path_prefix]:=underline}
+: ${ZSH_HIGHLIGHT_STYLES[path_pathseparator]:=}
-: ${ZSH_HIGHLIGHT_STYLES[path_approx]:=fg=yellow,underline}
+: ${ZSH_HIGHLIGHT_STYLES[path_prefix_pathseparator]:=}
 : ${ZSH_HIGHLIGHT_STYLES[globbing]:=fg=blue}
 : ${ZSH_HIGHLIGHT_STYLES[history-expansion]:=fg=blue}
 : ${ZSH_HIGHLIGHT_STYLES[single-hyphen-option]:=none}
@@ -56,17 +51,47 @@
 : ${ZSH_HIGHLIGHT_STYLES[back-dollar-quoted-argument]:=fg=cyan}
 : ${ZSH_HIGHLIGHT_STYLES[assign]:=none}
 : ${ZSH_HIGHLIGHT_STYLES[redirection]:=none}
+: ${ZSH_HIGHLIGHT_STYLES[comment]:=fg=black,bold}
+: ${ZSH_HIGHLIGHT_STYLES[arg0]:=fg=green}
 
 # Whether the highlighter should be called or not.
-_zsh_highlight_main_highlighter_predicate()
+_zsh_highlight_highlighter_main_predicate()
 {
-  _zsh_highlight_buffer_modified
+  # may need to remove path_prefix highlighting when the line ends
+  [[ $WIDGET == zle-line-finish ]] || _zsh_highlight_buffer_modified
 }
 
 # Helper to deal with tokens crossing line boundaries.
 _zsh_highlight_main_add_region_highlight() {
   integer start=$1 end=$2
-  local style=$3
+  shift 2
+
+  if (( $+argv[2] )); then
+    # Caller specified inheritance explicitly.
+  else
+    # Automate inheritance.
+    typeset -A fallback_of; fallback_of=(
+        alias arg0
+        suffix-alias arg0
+        builtin arg0
+        function arg0
+        command arg0
+        precommand arg0
+        hashed-command arg0
+        
+        path_prefix path
+        # The path separator fallback won't ever be used, due to the optimisation
+        # in _zsh_highlight_main_highlighter_highlight_path_separators().
+        path_pathseparator path
+        path_prefix_pathseparator path_prefix
+    )
+    local needle=$1 value
+    while [[ -n ${value::=$fallback_of[$needle]} ]]; do
+      unset "fallback_of[$needle]" # paranoia against infinite loops
+      argv+=($value)
+      needle=$value
+    done
+  fi
 
   # The calculation was relative to $PREBUFFER$BUFFER, but region_highlight is
   # relative to $BUFFER.
@@ -75,45 +100,260 @@ _zsh_highlight_main_add_region_highlight() {
 
   (( end < 0 )) && return # having end<0 would be a bug
   (( start < 0 )) && start=0 # having start<0 is normal with e.g. multiline strings
-  region_highlight+=("$start $end $style")
+  _zsh_highlight_add_highlight $start $end "$@"
+}
+
+# Get the type of a command.
+#
+# Uses the zsh/parameter module if available to avoid forks, and a
+# wrapper around 'type -w' as fallback.
+#
+# Takes a single argument.
+#
+# The result will be stored in REPLY.
+_zsh_highlight_main__type() {
+  if (( $+_zsh_highlight_main__command_type_cache )); then
+    REPLY=$_zsh_highlight_main__command_type_cache[(e)$1]
+    if [[ -n "$REPLY" ]]; then
+      return
+    fi
+  fi
+  if (( $#options_to_set )); then
+    setopt localoptions $options_to_set;
+  fi
+  unset REPLY
+  if zmodload -e zsh/parameter; then
+    if (( $+aliases[(e)$1] )); then
+      REPLY=alias
+    elif (( $+saliases[(e)${1##*.}] )); then
+      REPLY='suffix alias'
+    elif (( $reswords[(Ie)$1] )); then
+      REPLY=reserved
+    elif (( $+functions[(e)$1] )); then
+      REPLY=function
+    elif (( $+builtins[(e)$1] )); then
+      REPLY=builtin
+    elif (( $+commands[(e)$1] )); then
+      REPLY=command
+    # zsh 5.2 and older have a bug whereby running 'type -w ./sudo' implicitly
+    # runs 'hash ./sudo=/usr/local/bin/./sudo' (assuming /usr/local/bin/sudo
+    # exists and is in $PATH).  Avoid triggering the bug, at the expense of
+    # falling through to the $() below, incurring a fork.  (Issue #354.)
+    #
+    # The first disjunct mimics the isrelative() C call from the zsh bug.
+    elif {  [[ $1 != */* ]] || is-at-least 5.3 } &&
+         ! builtin type -w -- $1 >/dev/null 2>&1; then
+      REPLY=none
+    fi
+  fi
+  if ! (( $+REPLY )); then
+    REPLY="${$(LC_ALL=C builtin type -w -- $1 2>/dev/null)#*: }"
+  fi
+  if (( $+_zsh_highlight_main__command_type_cache )); then
+    _zsh_highlight_main__command_type_cache[(e)$1]=$REPLY
+  fi
+}
+
+# Check whether the first argument is a redirection operator token.
+# Report result via the exit code.
+_zsh_highlight_main__is_redirection() {
+  # A redirection operator token:
+  # - starts with an optional single-digit number;
+  # - then, has a '<' or '>' character;
+  # - is not a process substitution [<(...) or >(...)].
+  [[ $1 == (<0-9>|)(\<|\>)* ]] && [[ $1 != (\<|\>)$'\x28'* ]]
+}
+
+# Resolve alias.
+#
+# Takes a single argument.
+#
+# The result will be stored in REPLY.
+_zsh_highlight_main__resolve_alias() {
+  if zmodload -e zsh/parameter; then
+    REPLY=${aliases[$arg]}
+  else
+    REPLY="${"$(alias -- $arg)"#*=}"
+  fi
+}
+
+# Check that the top of $braces_stack has the expected value.  If it does, set
+# the style according to $2; otherwise, set style=unknown-token.
+#
+# $1: character expected to be at the top of $braces_stack
+# $2: assignment to execute it if matches
+_zsh_highlight_main__stack_pop() {
+  if [[ $braces_stack[1] == $1 ]]; then
+    braces_stack=${braces_stack:1}
+    eval "$2"
+  else
+    style=unknown-token
+  fi
 }
 
 # Main syntax highlighting function.
-_zsh_highlight_main_highlighter()
+_zsh_highlight_highlighter_main_paint()
 {
+  ## Before we even 'emulate -L', we must test a few options that would reset.
+  if [[ -o interactive_comments ]]; then
+    local interactive_comments= # set to empty
+  fi
+  if [[ -o ignore_braces ]] || eval '[[ -o ignore_close_braces ]] 2>/dev/null'; then
+    local right_brace_is_recognised_everywhere=false
+  else
+    local right_brace_is_recognised_everywhere=true
+  fi
+  if [[ -o path_dirs ]]; then
+    integer path_dirs_was_set=1
+  else
+    integer path_dirs_was_set=0
+  fi
+  if [[ -o multi_func_def ]]; then
+    integer multi_func_def=1
+  else
+    integer multi_func_def=0
+  fi
   emulate -L zsh
   setopt localoptions extendedglob bareglobqual
-  local start_pos=0 end_pos highlight_glob=true new_expression=true arg style sudo=false sudo_arg=false
+
-  local redirection=false # true when we've seen a redirection operator before seeing the command word
+  # At the PS3 prompt and in vared, highlight nothing.
+  #
+  # (We can't check this in _zsh_highlight_highlighter_main_predicate because
+  # if the predicate returns false, the previous value of region_highlight
+  # would be reused.)
+  if [[ $CONTEXT == (select|vared) ]]; then
+    return
+  fi
+
+  ## Variable declarations and initializations
+  local start_pos=0 end_pos highlight_glob=true arg style
+  local in_array_assignment=false # true between 'a=(' and the matching ')'
   typeset -a ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR
   typeset -a ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS
-  typeset -a ZSH_HIGHLIGHT_TOKENS_FOLLOWED_BY_COMMANDS
+  typeset -a ZSH_HIGHLIGHT_TOKENS_CONTROL_FLOW
+  local -a options_to_set # used in callees
   local buf="$PREBUFFER$BUFFER"
-  region_highlight=()
+  integer len="${#buf}"
+  integer pure_buf_len=$(( len - ${#PREBUFFER} ))   # == $#BUFFER, used e.g. in *_check_path
+
+  # "R" for round
+  # "Q" for square
+  # "Y" for curly
+  # "D" for do/done
+  # "$" for 'end' (matches 'foreach' always; also used with cshjunkiequotes in repeat/while)
+  local braces_stack
+
+  if (( path_dirs_was_set )); then
+    options_to_set+=( PATH_DIRS )
+  fi
+  unset path_dirs_was_set
 
   ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR=(
     '|' '||' ';' '&' '&&'
+    '|&'
+    '&!' '&|'
+    # ### 'case' syntax, but followed by a pattern, not by a command
+    # ';;' ';&' ';|'
   )
   ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS=(
     'builtin' 'command' 'exec' 'nocorrect' 'noglob'
-  )
+    'pkexec' # immune to #121 because it's usually not passed --option flags
-  # Tokens that are always immediately followed by a command.
-  ZSH_HIGHLIGHT_TOKENS_FOLLOWED_BY_COMMANDS=(
-    $ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR $ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS
   )
 
-  for arg in ${(z)buf}; do
+  # Tokens that, at (naively-determined) "command position", are followed by
-    # substr_color is set to 1 to disable adding an entry to region_highlight
+  # a de jure command position.  All of these are reserved words.
+  ZSH_HIGHLIGHT_TOKENS_CONTROL_FLOW=(
+    $'\x7b' # block
+    $'\x28' # subshell
+    '()' # anonymous function
+    'while'
+    'until'
+    'if'
+    'then'
+    'elif'
+    'else'
+    'do'
+    'time'
+    'coproc'
+    '!' # reserved word; unrelated to $histchars[1]
+  )
+
+  local -a match mbegin mend
+
+  # State machine
+  #
+  # The states are:
+  # - :start:      Command word
+  # - :sudo_opt:   A leading-dash option to sudo (such as "-u" or "-i")
+  # - :sudo_arg:   The argument to a sudo leading-dash option that takes one,
+  #                when given as a separate word; i.e., "foo" in "-u foo" (two
+  #                words) but not in "-ufoo" (one word).
+  # - :regular:    "Not a command word", and command delimiters are permitted.
+  #                Mainly used to detect premature termination of commands.
+  # - :always:     The word 'always' in the «{ foo } always { bar }» syntax.
+  #
+  # When the kind of a word is not yet known, $this_word / $next_word may contain
+  # multiple states.  For example, after "sudo -i", the next word may be either
+  # another --flag or a command name, hence the state would include both :start:
+  # and :sudo_opt:.
+  #
+  # The tokens are always added with both leading and trailing colons to serve as
+  # word delimiters (an improvised array); [[ $x == *:foo:* ]] and x=${x//:foo:/} 
+  # will DTRT regardless of how many elements or repetitions $x has..
+  #
+  # Handling of redirections: upon seeing a redirection token, we must stall
+  # the current state --- that is, the value of $this_word --- for two iterations
+  # (one for the redirection operator, one for the word following it representing
+  # the redirection target).  Therefore, we set $in_redirection to 2 upon seeing a
+  # redirection operator, decrement it each iteration, and stall the current state
+  # when it is non-zero.  Thus, upon reaching the next word (the one that follows
+  # the redirection operator and target), $this_word will still contain values
+  # appropriate for the word immediately following the word that preceded the
+  # redirection operator.
+  #
+  # The "the previous word was a redirection operator" state is not communicated
+  # to the next iteration via $next_word/$this_word as usual, but via
+  # $in_redirection.  The value of $next_word from the iteration that processed
+  # the operator is discarded.
+  #
+  local this_word=':start:' next_word
+  integer in_redirection
+  # Processing buffer
+  local proc_buf="$buf"
+  for arg in ${interactive_comments-${(z)buf}} \
+             ${interactive_comments+${(zZ+c+)buf}}; do
+    # Initialize $next_word.
+    if (( in_redirection )); then
+      (( --in_redirection ))
+    fi
+    if (( in_redirection == 0 )); then
+      # Initialize $next_word to its default value.
+      next_word=':regular:'
+    else
+      # Stall $next_word.
+    fi
+
+    # Initialize per-"simple command" [zshmisc(1)] variables:
+    #
+    #   $already_added       (see next paragraph)
+    #   $style               how to highlight $arg
+    #   $in_array_assignment boolean flag for "between '(' and ')' of array assignment"
+    #   $highlight_glob      boolean flag for "'noglob' is in effect"
+    #
+    # $already_added is set to 1 to disable adding an entry to region_highlight
     # for this iteration.  Currently, that is done for "" and $'' strings,
     # which add the entry early so escape sequences within the string override
     # the string's color.
-    integer substr_color=0
+    integer already_added=0
-    local style_override=""
+    style=unknown-token
-    if $new_expression && [[ $arg = 'noglob' ]]; then
+    if [[ $this_word == *':start:'* ]]; then
+      in_array_assignment=false
+      if [[ $arg == 'noglob' ]]; then
         highlight_glob=false
       fi
+    fi
 
-    # advance $start_pos, skipping over whitespace in $buf.
+    # Compute the new $start_pos and $end_pos, skipping over whitespace in $buf.
     if [[ $arg == ';' ]] ; then
       # We're looking for either a semicolon or a newline, whichever comes
       # first.  Both of these are rendered as a ";" (SEPER) by the ${(z)..}
@@ -124,66 +364,210 @@ _zsh_highlight_main_highlighter()
       # indistinguishable from 'echo foo echo bar' (one command with three
       # words for arguments).
       local needle=$'[;\n]'
-      integer offset=${${buf[start_pos+1,-1]}[(i)$needle]}
+      integer offset=$(( ${proc_buf[(i)$needle]} - 1 ))
-      (( start_pos += offset - 1 ))
+      (( start_pos += offset ))
       (( end_pos = start_pos + $#arg ))
     else
-      ((start_pos+=${#buf[$start_pos+1,-1]}-${#${buf[$start_pos+1,-1]##([[:space:]]|\\[[:space:]])#}}))
+      # The line was:
+      #
+      # integer offset=$(((len-start_pos)-${#${proc_buf##([[:space:]]|\\[[:space:]])#}}))
+      #
+      # - len-start_pos is length of current proc_buf; basically: initial length minus where
+      #   we are, and proc_buf is chopped to the "where we are" (compare the "previous value
+      #   of start_pos" below, and the len-(start_pos-offset) = len-start_pos+offset)
+      # - what's after main minus sign is: length of proc_buf without spaces at the beginning
+      # - so what the line actually did, was computing length of the spaces!
+      # - this can be done via (#b) flag, like below
+      if [[ "$proc_buf" = (#b)(#s)(([[:space:]]|\\[[:space:]])##)* ]]; then
+          # The first, outer parenthesis
+          integer offset="${#match[1]}"
+      else
+          integer offset=0
+      fi
+      ((start_pos+=offset))
       ((end_pos=$start_pos+${#arg}))
     fi
 
+    # Compute the new $proc_buf. We advance it
+    # (chop off characters from the beginning)
+    # beyond what end_pos points to, by skipping
+    # as many characters as end_pos was advanced.
+    #
+    # end_pos was advanced by $offset (via start_pos)
+    # and by $#arg. Note the `start_pos=$end_pos`
+    # below.
+    #
+    # As for the [,len]. We could use [,len-start_pos+offset]
+    # here, but to make it easier on eyes, we use len and
+    # rely on the fact that Zsh simply handles that. The
+    # length of proc_buf is len-start_pos+offset because
+    # we're chopping it to match current start_pos, so its
+    # length matches the previous value of start_pos.
+    #
+    # Why [,-1] is slower than [,length] isn't clear.
+    proc_buf="${proc_buf[offset + $#arg + 1,len]}"
+
+    # Handle the INTERACTIVE_COMMENTS option.
+    #
+    # We use the (Z+c+) flag so the entire comment is presented as one token in $arg.
+    if [[ -n ${interactive_comments+'set'} && $arg[1] == $histchars[3] ]]; then
+      if [[ $this_word == *(':regular:'|':start:')* ]]; then
+        style=comment
+      else
+        style=unknown-token # prematurely terminated
+      fi
+      _zsh_highlight_main_add_region_highlight $start_pos $end_pos $style
+      already_added=1
+      continue
+    fi
+
+    # Analyse the current word.
+    if _zsh_highlight_main__is_redirection $arg ; then
+      # A '<' or '>', possibly followed by a digit
+      in_redirection=2
+    fi
+
+    # Special-case the first word after 'sudo'.
+    if (( ! in_redirection )); then
+      if [[ $this_word == *':sudo_opt:'* ]] && [[ $arg != -* ]]; then
+        this_word=${this_word//:sudo_opt:/}
+      fi
+    fi
+
     # Parse the sudo command line
-    if $sudo; then
+    if (( ! in_redirection )); then
+      if [[ $this_word == *':sudo_opt:'* ]]; then
         case "$arg" in
           # Flag that requires an argument
-        '-'[Cgprtu]) sudo_arg=true;;
+          '-'[Cgprtu]) this_word=${this_word//:start:/};
+                       next_word=':sudo_arg:';;
           # This prevents misbehavior with sudo -u -otherargument
-        '-'*)        sudo_arg=false;;
+          '-'*)        this_word=${this_word//:start:/};
-        *)           if $sudo_arg; then
+                       next_word+=':start:';
-                       sudo_arg=false
+                       next_word+=':sudo_opt:';;
-                     else
+          *)           ;;
-                       sudo=false
-                       new_expression=true; highlight_glob=true
-                     fi
-                     ;;
         esac
+      elif [[ $this_word == *':sudo_arg:'* ]]; then
+        next_word+=':sudo_opt:'
+        next_word+=':start:'
       fi
-    if $new_expression && ! $redirection; then # $arg is the command word
+   fi
-      new_expression=false
+
+   # The Great Fork: is this a command word?  Is this a non-command word?
+   if [[ $this_word == *':always:'* && $arg == 'always' ]]; then
+     # try-always construct
+     style=reserved-word # de facto a reserved word, although not de jure
+     next_word=':start:'
+   elif [[ $this_word == *':start:'* ]] && (( in_redirection == 0 )); then # $arg is the command word
      if [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS:#"$arg"} ]]; then
-      style=$ZSH_HIGHLIGHT_STYLES[precommand]
+      style=precommand
      elif [[ "$arg" = "sudo" ]]; then
-      style=$ZSH_HIGHLIGHT_STYLES[precommand]
+      style=precommand
-      sudo=true
+      next_word=${next_word//:regular:/}
+      next_word+=':sudo_opt:'
+      next_word+=':start:'
      else
       _zsh_highlight_main_highlighter_expand_path $arg
       local expanded_arg="$REPLY"
-      local res="$(LC_ALL=C builtin type -w -- ${expanded_arg} 2>/dev/null)"
+      _zsh_highlight_main__type ${expanded_arg}
+      local res="$REPLY"
+      () {
+        # Special-case: command word is '$foo', like that, without braces or anything.
+        #
+        # That's not entirely correct --- if the parameter's value happens to be a reserved
+        # word, the parameter expansion will be highlighted as a reserved word --- but that
+        # incorrectness is outweighed by the usability improvement of permitting the use of
+        # parameters that refer to commands, functions, and builtins.
+        local -a match mbegin mend
+        local MATCH; integer MBEGIN MEND
+        if [[ $res == none ]] && (( ${+parameters} )) &&
+           [[ ${arg[1]} == \$ ]] && [[ ${arg:1} =~ ^([A-Za-z_][A-Za-z0-9_]*|[0-9]+)$ ]] &&
+           (( ${+parameters[${MATCH}]} ))
+           then
+          _zsh_highlight_main__type ${(P)MATCH}
+          res=$REPLY
+        fi
+      }
       case $res in
-        *': reserved')  style=$ZSH_HIGHLIGHT_STYLES[reserved-word];;
+        reserved)       # reserved word
-        *': suffix alias')
+                        style=reserved-word
-                        style=$ZSH_HIGHLIGHT_STYLES[suffix-alias]
+                        case $arg in
+                          ($'\x7b')
+                            braces_stack='Y'"$braces_stack"
                             ;;
-        *': alias')     style=$ZSH_HIGHLIGHT_STYLES[alias]
+                          ($'\x7d')
-                        local aliased_command="${"$(alias -- $arg)"#*=}"
+                            # We're at command word, so no need to check $right_brace_is_recognised_everywhere
-                        [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_FOLLOWED_BY_COMMANDS:#"$aliased_command"} && -z ${(M)ZSH_HIGHLIGHT_TOKENS_FOLLOWED_BY_COMMANDS:#"$arg"} ]] && ZSH_HIGHLIGHT_TOKENS_FOLLOWED_BY_COMMANDS+=($arg)
+                            _zsh_highlight_main__stack_pop 'Y' style=reserved-word
+                            if [[ $style == reserved-word ]]; then
+                              next_word+=':always:'
+                            fi
                             ;;
-        *': builtin')   style=$ZSH_HIGHLIGHT_STYLES[builtin];;
+                          ('do')
-        *': function')  style=$ZSH_HIGHLIGHT_STYLES[function];;
+                            braces_stack='D'"$braces_stack"
-        *': command')   style=$ZSH_HIGHLIGHT_STYLES[command];;
+                            ;;
-        *': hashed')    style=$ZSH_HIGHLIGHT_STYLES[hashed-command];;
+                          ('done')
-        *)              if _zsh_highlight_main_highlighter_check_assign; then
+                            _zsh_highlight_main__stack_pop 'D' style=reserved-word
-                          style=$ZSH_HIGHLIGHT_STYLES[assign]
+                            ;;
-                          if [[ $arg[-1] != '(' ]]; then
+                          ('foreach')
+                            braces_stack='$'"$braces_stack"
+                            ;;
+                          ('end')
+                            _zsh_highlight_main__stack_pop '$' style=reserved-word
+                            ;;
+                        esac
+                        ;;
+        'suffix alias') style=suffix-alias;;
+        alias)          () {
+                          integer insane_alias
+                          case $arg in 
+                            # Issue #263: aliases with '=' on their LHS.
+                            #
+                            # There are three cases:
+                            #
+                            # - Unsupported, breaks 'alias -L' output, but invokable:
+                            ('='*) :;;
+                            # - Unsupported, not invokable:
+                            (*'='*) insane_alias=1;;
+                            # - The common case:
+                            (*) :;;
+                          esac
+                          if (( insane_alias )); then
+                            style=unknown-token
+                          else
+                            style=alias
+                            _zsh_highlight_main__resolve_alias $arg
+                            local alias_target="$REPLY"
+                            [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS:#"$alias_target"} && -z ${(M)ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS:#"$arg"} ]] && ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS+=($arg)
+                          fi
+                        }
+                        ;;
+        builtin)        style=builtin;;
+        function)       style=function;;
+        command)        style=command;;
+        hashed)         style=hashed-command;;
+        none)           if _zsh_highlight_main_highlighter_check_assign; then
+                          style=assign
+                          if [[ $arg[-1] == '(' ]]; then
+                            in_array_assignment=true
+                          else
                             # assignment to a scalar parameter.
                             # (For array assignments, the command doesn't start until the ")" token.)
-                            new_expression=true; highlight_glob=true
+                            next_word+=':start:'
                           fi
-                        elif [[ $arg[0,1] == $histchars[0,1] || $arg[0,1] == $histchars[2,2] ]]; then
+                        elif [[ $arg[0,1] = $histchars[0,1] ]] && (( $#arg[0,2] == 2 )); then
-                          style=$ZSH_HIGHLIGHT_STYLES[history-expansion]
+                          style=history-expansion
-                        elif [[ $arg[1] == '<' || $arg[1] == '>' ]]; then
+                        elif [[ $arg[0,1] == $histchars[2,2] ]]; then
-                          style=$ZSH_HIGHLIGHT_STYLES[redirection]
+                          style=history-expansion
-                          redirection=true
+                        elif [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR:#"$arg"} ]]; then
+                          if [[ $this_word == *':regular:'* ]]; then
+                            # This highlights empty commands (semicolon follows nothing) as an error.
+                            # Zsh accepts them, though.
+                            style=commandseparator
+                          else
+                            style=unknown-token
+                          fi
+                        elif (( in_redirection == 2 )); then
+                          style=redirection
                         elif [[ $arg[1,2] == '((' ]]; then
                           # Arithmetic evaluation.
                           #
@@ -194,64 +578,145 @@ _zsh_highlight_main_highlighter()
                           #
                           # We highlight just the opening parentheses, as a reserved word; this
                           # is how [[ ... ]] is highlighted, too.
-                          style=$ZSH_HIGHLIGHT_STYLES[reserved-word]
+                          style=reserved-word
                           _zsh_highlight_main_add_region_highlight $start_pos $((start_pos + 2)) $style
-                          substr_color=1
+                          already_added=1
+                          if [[ $arg[-2,-1] == '))' ]]; then
+                            _zsh_highlight_main_add_region_highlight $((end_pos - 2)) $end_pos $style
+                            already_added=1
+                          fi
+                        elif [[ $arg == '()' ]]; then
+                          # anonymous function
+                          style=reserved-word
+                        elif [[ $arg == $'\x28' ]]; then
+                          # subshell
+                          style=reserved-word
+                          braces_stack='R'"$braces_stack"
+                        elif [[ $arg == $'\x29' ]]; then
+                          # end of subshell
+                          _zsh_highlight_main__stack_pop 'R' style=reserved-word
                         else
                           if _zsh_highlight_main_highlighter_check_path; then
-                            style=$ZSH_HIGHLIGHT_STYLES[path]
+                            style=$REPLY
                           else
-                            style=$ZSH_HIGHLIGHT_STYLES[unknown-token]
+                            style=unknown-token
                           fi
                         fi
                         ;;
+        *)              _zsh_highlight_main_add_region_highlight $start_pos $end_pos arg0_$res arg0
+                        already_added=1
+                        ;;
       esac
      fi
-    else # $arg is the file target of a prefix redirection, or a non-command word
-      if $redirection; then
-        redirection=false
-        new_expression=true; highlight_glob=true
    fi
+   if (( ! already_added )) && [[ $style == unknown-token ]] && # not handled by the 'command word' codepath
+      { (( in_redirection )) || [[ $this_word == *':regular:'* ]] || [[ $this_word == *':sudo_opt:'* ]] || [[ $this_word == *':sudo_arg:'* ]] }
+   then # $arg is a non-command word
       case $arg in
-        '--'*)   style=$ZSH_HIGHLIGHT_STYLES[double-hyphen-option];;
+        $'\x29') # subshell or end of array assignment
-        '-'*)    style=$ZSH_HIGHLIGHT_STYLES[single-hyphen-option];;
+                 if $in_array_assignment; then
-        "'"*)    style=$ZSH_HIGHLIGHT_STYLES[single-quoted-argument];;
+                   style=assign
-        '"'*)    style=$ZSH_HIGHLIGHT_STYLES[double-quoted-argument]
+                   in_array_assignment=false
+                   next_word+=':start:'
+                 else
+                   _zsh_highlight_main__stack_pop 'R' style=reserved-word
+                 fi;;
+        $'\x28\x29') # possibly a function definition
+                 if (( multi_func_def )) || false # TODO: or if the previous word was a command word
+                 then
+                   next_word+=':start:'
+                 fi
+                 style=reserved-word
+                 ;;
+        $'\x7d') # right brace
+                 #
+                 # Parsing rule: # {
+                 #
+                 #     Additionally, `tt(})' is recognized in any position if neither the
+                 #     tt(IGNORE_BRACES) option nor the tt(IGNORE_CLOSE_BRACES) option is set."""
+                 if $right_brace_is_recognised_everywhere; then
+                   _zsh_highlight_main__stack_pop 'Y' style=reserved-word
+                   if [[ $style == reserved-word ]]; then
+                     next_word+=':always:'
+                   fi
+                 else
+                   # Fall through to the catchall case at the end.
+                 fi
+                 ;|
+        '--'*)   style=double-hyphen-option;;
+        '-'*)    style=single-hyphen-option;;
+        "'"*)    style=single-quoted-argument;;
+        '"'*)    style=double-quoted-argument
                  _zsh_highlight_main_add_region_highlight $start_pos $end_pos $style
                  _zsh_highlight_main_highlighter_highlight_string
-                 substr_color=1
+                 already_added=1
                  ;;
-        \$\'*)   style=$ZSH_HIGHLIGHT_STYLES[dollar-quoted-argument]
+        \$\'*)   style=dollar-quoted-argument
                  _zsh_highlight_main_add_region_highlight $start_pos $end_pos $style
                  _zsh_highlight_main_highlighter_highlight_dollar_string
-                 substr_color=1
+                 already_added=1
                  ;;
-        '`'*)    style=$ZSH_HIGHLIGHT_STYLES[back-quoted-argument];;
+        '`'*)    style=back-quoted-argument;;
         [*?]*|*[^\\][*?]*)
-                 $highlight_glob && style=$ZSH_HIGHLIGHT_STYLES[globbing] || style=$ZSH_HIGHLIGHT_STYLES[default];;
+                 $highlight_glob && style=globbing || style=default;;
         *)       if false; then
-                 elif [[ $arg[0,1] = $histchars[0,1] ]]; then
+                 elif [[ $arg = $'\x7d' ]] && $right_brace_is_recognised_everywhere; then
-                   style=$ZSH_HIGHLIGHT_STYLES[history-expansion]
+                   # was handled by the $'\x7d' case above
+                 elif [[ $arg[0,1] = $histchars[0,1] ]] && (( $#arg[0,2] == 2 )); then
+                   style=history-expansion
                  elif [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR:#"$arg"} ]]; then
-                   style=$ZSH_HIGHLIGHT_STYLES[commandseparator]
+                   if [[ $this_word == *':regular:'* ]]; then
-                 elif [[ $arg[1] == '<' || $arg[1] == '>' ]]; then
+                     style=commandseparator
-                   style=$ZSH_HIGHLIGHT_STYLES[redirection]
+                   else
+                     style=unknown-token
+                   fi
+                 elif (( in_redirection == 2 )); then
+                   style=redirection
                  else
                    if _zsh_highlight_main_highlighter_check_path; then
-                     style=$ZSH_HIGHLIGHT_STYLES[path]
+                     style=$REPLY
                    else
-                     style=$ZSH_HIGHLIGHT_STYLES[default]
+                     style=default
                    fi
                  fi
                  ;;
       esac
     fi
-    # if a style_override was set (eg in _zsh_highlight_main_highlighter_check_path), use it
+    if ! (( already_added )); then
-    [[ -n $style_override ]] && style=$ZSH_HIGHLIGHT_STYLES[$style_override]
+      _zsh_highlight_main_add_region_highlight $start_pos $end_pos $style
-    [[ $substr_color = 0 ]] && _zsh_highlight_main_add_region_highlight $start_pos $end_pos $style
+      [[ $style == path || $style == path_prefix ]] && _zsh_highlight_main_highlighter_highlight_path_separators
-    [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_FOLLOWED_BY_COMMANDS:#"$arg"} ]] && new_expression=true
+    fi
-    [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR:#"$arg"} ]] && highlight_glob=true
+    if [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR:#"$arg"} ]]; then
+      if [[ $arg == ';' ]] && $in_array_assignment; then
+        # literal newline inside an array assignment
+        next_word=':regular:'
+      else
+        next_word=':start:'
+        highlight_glob=true
+      fi
+    elif
+       [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_CONTROL_FLOW:#"$arg"} && $this_word == *':start:'* ]] ||
+       [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS:#"$arg"} && $this_word == *':start:'* ]]; then
+      next_word=':start:'
+    elif [[ $arg == "repeat" && $this_word == *':start:'* ]]; then
+      # skip the repeat-count word
+      in_redirection=2
+      # The redirection mechanism assumes $this_word describes the word
+      # following the redirection.  Make it so.
+      #
+      # That word can be a command word with shortloops (`repeat 2 ls`)
+      # or a command separator (`repeat 2; ls` or `repeat 2; do ls; done`).
+      #
+      # The repeat-count word will be handled like a redirection target.
+      this_word=':start::regular:'
+    fi
     start_pos=$end_pos
+    if (( in_redirection == 0 )); then
+      # This is the default/common codepath.
+      this_word=$next_word
+    else
+      # Stall $this_word.
+    fi
   done
 }
 
@@ -259,16 +724,34 @@ _zsh_highlight_main_highlighter()
 _zsh_highlight_main_highlighter_check_assign()
 {
     setopt localoptions extended_glob
-    [[ $arg == [[:alpha:]_][[:alnum:]_]#(|\[*\])(|[+])=* ]]
+    [[ $arg == [[:alpha:]_][[:alnum:]_]#(|\[*\])(|[+])=* ]] ||
+      [[ $arg == [0-9]##(|[+])=* ]]
+}
+
+_zsh_highlight_main_highlighter_highlight_path_separators()
+{
+  local pos style_pathsep
+  style_pathsep=${style}_pathseparator
+  [[ -z "$ZSH_HIGHLIGHT_STYLES[$style_pathsep]" || "$ZSH_HIGHLIGHT_STYLES[$style]" == "$ZSH_HIGHLIGHT_STYLES[$style_pathsep]" ]] && return 0
+  for (( pos = start_pos; $pos <= end_pos; pos++ )) ; do
+    if [[ $BUFFER[pos] == / ]]; then
+      _zsh_highlight_main_add_region_highlight $((pos - 1)) $pos $style_pathsep
+    fi
+  done
 }
 
 # Check if $arg is a path.
+# If yes, return 0 and in $REPLY the style to use.
+# Else, return non-zero (and the contents of $REPLY is undefined).
 _zsh_highlight_main_highlighter_check_path()
 {
   _zsh_highlight_main_highlighter_expand_path $arg;
   local expanded_path="$REPLY"
 
+  REPLY=path
+
   [[ -z $expanded_path ]] && return 1
+  [[ -L $expanded_path ]] && return 0
   [[ -e $expanded_path ]] && return 0
 
   # Search the path in CDPATH
@@ -278,16 +761,14 @@ _zsh_highlight_main_highlighter_check_path()
   done
 
   # If dirname($arg) doesn't exist, neither does $arg.
-  [[ ! -e ${expanded_path:h} ]] && return 1
+  [[ ! -d ${expanded_path:h} ]] && return 1
 
   # If this word ends the buffer, check if it's the prefix of a valid path.
-  if [[ ${BUFFER[1]} != "-" && ${#BUFFER} == $end_pos ]]; then
+  if [[ ${BUFFER[1]} != "-" && $pure_buf_len == $end_pos ]] &&
+     [[ $WIDGET != zle-line-finish ]]; then
     local -a tmp
     tmp=( ${expanded_path}*(N) )
-    (( $#tmp > 0 )) && style_override=path_prefix && return 0
+    (( $#tmp > 0 )) && REPLY=path_prefix && return 0
-    # or maybe an approximate path?
-    tmp=( (#a1)${expanded_path}*(N) )
-    (( $#tmp > 0 )) && style_override=path_approx && return 0
   fi
 
   # It's not a path.
@@ -298,23 +779,38 @@ _zsh_highlight_main_highlighter_check_path()
 _zsh_highlight_main_highlighter_highlight_string()
 {
   setopt localoptions noksharrays
+  local -a match mbegin mend
+  local MATCH; integer MBEGIN MEND
   local i j k style
   # Starting quote is at 1, so start parsing at offset 2 in the string.
   for (( i = 2 ; i < end_pos - start_pos ; i += 1 )) ; do
     (( j = i + start_pos - 1 ))
     (( k = j + 1 ))
     case "$arg[$i]" in
-      '$' ) style=$ZSH_HIGHLIGHT_STYLES[dollar-double-quoted-argument]
+      '$' ) style=dollar-double-quoted-argument
             # Look for an alphanumeric parameter name.
             if [[ ${arg:$i} =~ ^([A-Za-z_][A-Za-z0-9_]*|[0-9]+) ]] ; then
               (( k += $#MATCH )) # highlight the parameter name
               (( i += $#MATCH )) # skip past it
+            elif [[ ${arg:$i} =~ ^[{]([A-Za-z_][A-Za-z0-9_]*|[0-9]+)[}] ]] ; then
+              (( k += $#MATCH )) # highlight the parameter name and braces
+              (( i += $#MATCH )) # skip past it
+            elif [[ $arg[i+1] == '$' ]]; then
+              # $$ - pid
+              (( k += 1 )) # highlight both dollar signs
+              (( i += 1 )) # don't consider the second one as introducing another parameter expansion
+            elif [[ $arg[i+1] == [-#*@?] ]]; then
+              # $#, $*, $@, $?, $- - like $$ above
+              (( k += 1 )) # highlight both dollar signs
+              (( i += 1 )) # don't consider the second one as introducing another parameter expansion
+            elif [[ $arg[i+1] == $'\x28' ]]; then
+              # Highlight just the '$'.
             else
               continue
             fi
             ;;
-      "\\") style=$ZSH_HIGHLIGHT_STYLES[back-double-quoted-argument]
+      "\\") style=back-double-quoted-argument
-            if [[ \\\`\"\$ == *$arg[$i+1]* ]]; then
+            if [[ \\\`\"\$${histchars[1]} == *$arg[$i+1]* ]]; then
               (( k += 1 )) # Color following char too.
               (( i += 1 )) # Skip parsing the escaped char.
             else
@@ -332,6 +828,8 @@ _zsh_highlight_main_highlighter_highlight_string()
 _zsh_highlight_main_highlighter_highlight_dollar_string()
 {
   setopt localoptions noksharrays
+  local -a match mbegin mend
+  local MATCH; integer MBEGIN MEND
   local i j k style
   local AA
   integer c
@@ -340,7 +838,7 @@ _zsh_highlight_main_highlighter_highlight_dollar_string()
     (( j = i + start_pos - 1 ))
     (( k = j + 1 ))
     case "$arg[$i]" in
-      "\\") style=$ZSH_HIGHLIGHT_STYLES[back-dollar-quoted-argument]
+      "\\") style=back-dollar-quoted-argument
             for (( c = i + 1 ; c <= end_pos - start_pos ; c += 1 )); do
               [[ "$arg[$c]" != ([0-9xXuUa-fA-F]) ]] && break
             done
@@ -356,7 +854,7 @@ _zsh_highlight_main_highlighter_highlight_dollar_string()
             else
               if (( $#arg > $i+1 )) && [[ $arg[$i+1] == [xXuU] ]]; then
                 # \x not followed by hex digits is probably an error
-                style=$ZSH_HIGHLIGHT_STYLES[unknown-token]
+                style=unknown-token
               fi
               (( k += 1 )) # Color following char too.
               (( i += 1 )) # Skip parsing the escaped char.
@@ -375,10 +873,28 @@ _zsh_highlight_main_highlighter_highlight_dollar_string()
 # Does not perform filename generation (globbing).
 _zsh_highlight_main_highlighter_expand_path()
 {
-  (( $# == 1 )) || echo "zsh-syntax-highlighting: BUG: _zsh_highlight_main_highlighter_expand_path: called without argument" >&2
+  (( $# == 1 )) || print -r -- >&2 "zsh-syntax-highlighting: BUG: _zsh_highlight_main_highlighter_expand_path: called without argument"
 
   # The $~1 syntax normally performs filename generation, but not when it's on the right-hand side of ${x:=y}.
   setopt localoptions nonomatch
   unset REPLY
   : ${REPLY:=${(Q)~1}}
 }
+
+# -------------------------------------------------------------------------------------------------
+# Main highlighter initialization
+# -------------------------------------------------------------------------------------------------
+
+_zsh_highlight_main__precmd_hook() {
+  _zsh_highlight_main__command_type_cache=()
+}
+
+autoload -U add-zsh-hook
+if add-zsh-hook precmd _zsh_highlight_main__precmd_hook 2>/dev/null; then
+  # Initialize command type cache
+  typeset -gA _zsh_highlight_main__command_type_cache
+else
+  print -r -- >&2 'zsh-syntax-highlighting: Failed to load add-zsh-hook. Some speed optimizations will not be used.'
+  # Make sure the cache is unset
+  unset _zsh_highlight_main__command_type_cache
+fi

highlighters/main/test-data/alias-assignment1.zsh

@@ -0,0 +1,37 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+# Issue #263 (more-pathological case): aliases[x=y]=z works; the ${(z)} splitter considers
+# that a single word; but it's not looked up as an alias.  Hence, highlight it as an error.
+aliases[x=y]='lorem ipsum dolor sit amet'
+BUFFER='x=y ls'
+
+expected_region_highlight=(
+  "1 3 unknown-token" # x=y
+)

highlighters/main/test-data/alias.zsh

@@ -31,7 +31,6 @@ alias alias1="unused expansion"
 alias -s alias2="echo"
 alias1() {} # to check that it's highlighted as an alias, not as a function
 
-ZSH_HIGHLIGHT_STYLES[alias]=$unused_highlight
 BUFFER='x.alias2; alias1'
 
 # Set expected_region_highlight as a function of zsh version.
@@ -43,9 +42,9 @@ BUFFER='x.alias2; alias1'
 expected_region_highlight=()
 if [[ "$(type -w x.alias2)" == *suffix* ]]; then
   expected_region_highlight+=(
-    "1 8 $ZSH_HIGHLIGHT_STYLES[suffix-alias]" # x.alias2
+    "1 8 suffix-alias" # x.alias2
   )
 fi
 expected_region_highlight+=(
-  "11 16 $ZSH_HIGHLIGHT_STYLES[alias]" # alias1
+  "11 16 alias" # alias1
 )

highlighters/main/test-data/always1.zsh

@@ -0,0 +1,41 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='{ ls } always { pwd }'
+
+expected_region_highlight=(
+  '1 1 reserved-word' # {
+  '3 4 command' # ls
+  '6 6 reserved-word' # }
+  '8 13 reserved-word' # always
+  '15 15 reserved-word' # {
+  '17 19 builtin' # pwd
+  '21 21 reserved-word' # }
+)

highlighters/main/test-data/always2.zsh

@@ -0,0 +1,43 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=$'{\nls\n} always { pwd }'
+
+expected_region_highlight=(
+  '1 1 reserved-word' # {
+  '2 2 unknown-token' # \n
+  '3 4 command' # ls
+  '5 5 commandseparator' # \n
+  '6 6 reserved-word' # }
+  '8 13 reserved-word' # always
+  '15 15 reserved-word' # {
+  '17 19 builtin' # pwd
+  '21 21 reserved-word' # }
+)

highlighters/main/test-data/always3.zsh

@@ -0,0 +1,43 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+setopt ignorebraces
+BUFFER='echo { foo } always { bar }'
+
+expected_region_highlight=(
+  '1 4 builtin' # echo
+  '6 6 default' # {
+  '8 10 default' # foo
+  '12 12 default' # }
+  '14 19 default' # always
+  '21 21 default' # {
+  '23 25 default' # bar
+  '27 27 default' # }
+)

highlighters/main/test-data/anonymous-function.zsh

@@ -0,0 +1,41 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='() echo hello; () { echo world } "argument"'
+
+expected_region_highlight=(
+  "1 2 reserved-word" # ()
+  "4 7 builtin" # echo
+  "9 13 default" # hello
+  "14 14 commandseparator" # ;
+  "16 17 reserved-word" # ()
+  "19 19 reserved-word" # {
+  "21 24 builtin" # echo
+  "34 43 double-quoted-argument" # "argument"
+)

highlighters/main/test-data/arithmetic-evaluation.zsh

@@ -35,5 +35,6 @@
 BUFFER='(( x == 42 ))'
 
 expected_region_highlight=(
-  "1 2 $ZSH_HIGHLIGHT_STYLES[reserved-word]" # ((
+  "1 2 reserved-word" # ((
+  "12 13 reserved-word" # ))
 )

highlighters/main/test-data/assign-append.zsh

@@ -27,9 +27,8 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
-ZSH_HIGHLIGHT_STYLES[assign]=$unused_highlight
 BUFFER='a+=(lorem ipsum)'
 
 expected_region_highlight=(
-  "1 4 $ZSH_HIGHLIGHT_STYLES[assign]" # a+=(
+  "1 4 assign" # a+=(
 )

highlighters/main/test-data/assign-argv.zsh

@@ -0,0 +1,35 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='42=foo 43+=bar'
+
+expected_region_highlight=(
+  "1 6 assign" # 42=foo
+  "8 14 assign" # 43+=bar
+)

highlighters/main/test-data/assign-array.zsh

@@ -0,0 +1,37 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='(A=(hello world))'
+
+expected_region_highlight=(
+  "1 1 reserved-word" # (
+  "2 4 assign" # A=(
+  "16 16 assign" # )
+  "17 17 reserved-word" # )
+)

highlighters/main/test-data/assign-array2.zsh

@@ -0,0 +1,36 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='A=(hello world) ls'
+
+expected_region_highlight=(
+  "1 3 assign" # A=(
+  "15 15 assign" # )
+  "17 18 command" # ls
+)

highlighters/main/test-data/assign-array3.zsh

@@ -0,0 +1,36 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='A=(hello world) b=42'
+
+expected_region_highlight=(
+  "1 3 assign" # A=(
+  "15 15 assign" # )
+  "17 20 assign" # b=42
+)

highlighters/main/test-data/assign-semicolon.zsh

@@ -0,0 +1,35 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='A=1; echo hello world'
+
+expected_region_highlight=(
+  "4 4 commandseparator" # ;
+  "6 9 builtin" # echo
+)

highlighters/main/test-data/assign-subshell.zsh

@@ -0,0 +1,36 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='(A=1)'
+
+expected_region_highlight=(
+  "1 1 reserved-word" # (
+  "2 4 assign" # A=1
+  "5 5 reserved-word" # )
+)

highlighters/main/test-data/assign.zsh

@@ -27,10 +27,12 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
-ZSH_HIGHLIGHT_STYLES[assign]=$unused_highlight
 BUFFER='A=1 b=("foo" bar)'
 
 expected_region_highlight=(
-  "1 3 $ZSH_HIGHLIGHT_STYLES[assign]" # A=1
+  "1 3 assign" # A=1
-  "8 12 $ZSH_HIGHLIGHT_STYLES[double-quoted-argument]" # "foo"
+  "5 7 assign" # b=(
+  "8 12 double-quoted-argument" # "foo"
+  "14 16 default" # bar
+  "17 17 assign" # )
 )

highlighters/main/test-data/back-quoted-argument.zsh

@@ -27,9 +27,8 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
-ZSH_HIGHLIGHT_STYLES[back-quoted-argument]=$unused_highlight
 BUFFER='echo `echo 42`'
 
 expected_region_highlight=(
-  "6 14 $ZSH_HIGHLIGHT_STYLES[back-quoted-argument]"
+  "6 14 back-quoted-argument"
 )

highlighters/main/test-data/backslash-continuation.zsh

@@ -31,5 +31,5 @@ PREBUFFER=$'echo \\\n'
 BUFFER='noglob'
 
 expected_region_highlight=(
-  "1 6 none" # 'noglob' highlighted as a string, not as a precomand
+  "1 6 default" # 'noglob' highlighted as a string, not as a precomand
 )

highlighters/main/test-data/braces1.zsh

@@ -0,0 +1,41 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=$'() { echo }\n}'
+# no special setopts
+
+expected_region_highlight=(
+  '1 2 reserved-word' # ()
+  '4 4 reserved-word' # {
+  '6 9 builtin' # echo
+  '11 11 reserved-word' # }
+  '12 12 commandseparator' # \n
+  '13 13 unknown-token' # }
+)

highlighters/main/test-data/braces2.zsh

@@ -0,0 +1,41 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=$'() { echo }\n}'
+setopt ignorebraces
+
+expected_region_highlight=(
+  '1 2 reserved-word' # ()
+  '4 4 reserved-word' # {
+  '6 9 builtin' # echo
+  '11 11 default' # }
+  '12 12 commandseparator' # \n
+  '13 13 reserved-word' # }
+)

highlighters/main/test-data/brackets-mismatch1.zsh

@@ -0,0 +1,39 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='() { echo foo )'
+
+expected_region_highlight=(
+  '1 2 reserved-word' # ()
+  '4 4 reserved-word' # {
+  '6 9 builtin' # echo
+  '11 13 default' # foo
+  '15 15 unknown-token' # )
+)

highlighters/main/test-data/brackets-mismatch2.zsh

@@ -0,0 +1,39 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='() ( echo foo }'
+
+expected_region_highlight=(
+  '1 2 reserved-word' # ()
+  '4 4 reserved-word' # (
+  '6 9 builtin' # echo
+  '11 13 default' # foo
+  '15 15 unknown-token' # }
+)

highlighters/main/test-data/brackets-mismatch3.zsh

@@ -0,0 +1,36 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='echo )'
+
+expected_region_highlight=(
+  '1 4 builtin' # echo
+  '6 6 unknown-token' # )
+)

highlighters/main/test-data/brackets-mismatch4.zsh

@@ -0,0 +1,36 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='echo }'
+
+expected_region_highlight=(
+  '1 4 builtin' # echo
+  '6 6 unknown-token' # }
+)

highlighters/main/test-data/brackets-mismatch5.zsh

@@ -0,0 +1,37 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='echo { }'
+
+expected_region_highlight=(
+  '1 4 builtin' # echo
+  '6 6 default' # {
+  '8 8 unknown-token' # }
+)

highlighters/main/test-data/brackets-mismatch6.zsh

@@ -0,0 +1,40 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='(repeat 1; do)'
+
+expected_region_highlight=(
+  '1 1 reserved-word' # (
+  '2 7 reserved-word' # repeat
+  '9 9 default' # 1
+  '10 10 commandseparator' # ;
+  '12 13 reserved-word' # do
+  '14 14 unknown-token' # )
+)

highlighters/main/test-data/brackets-mismatch7.zsh

@@ -0,0 +1,44 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2012 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='for n in *; do echo $n; end'
+
+expected_region_highlight=(
+  '1 3 reserved-word' # for
+  '5 5 default' # n
+  '7 8 default' # in
+  '10 10 globbing' # *
+  '11 11 commandseparator' # ;
+  '13 14 reserved-word' # do
+  '16 19 builtin' # echo
+  '21 22 default' # $n
+  '23 23 commandseparator' # ;
+  '25 27 unknown-token' # end
+)

highlighters/main/test-data/brackets-mismatch8.zsh

@@ -0,0 +1,38 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='(ls&)'
+
+expected_region_highlight=(
+  '1 1 reserved-word' # (
+  '2 3 command' # ls
+  '4 4 commandseparator' # &
+  '5 5 reserved-word' # )
+)

highlighters/main/test-data/commandseparator.zsh

@@ -27,11 +27,11 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
-ZSH_HIGHLIGHT_STYLES[commandseparator]=$unused_highlight
+BUFFER=':; pwd &! ls'
-
-BUFFER=':; pwd'
 
 expected_region_highlight=(
-  "2 2 $ZSH_HIGHLIGHT_STYLES[commandseparator]" # ;
+  "2 2 commandseparator" # ;
-  "4 6 $ZSH_HIGHLIGHT_STYLES[builtin]" # pwd
+  "4 6 builtin" # pwd
+  "8 9 commandseparator" # &!
+  "11 12 command" # ls
 )

highlighters/main/test-data/comment-leading.zsh

@@ -0,0 +1,37 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+setopt interactive_comments
+
+BUFFER='# echo foo'
+
+expected_region_highlight=(
+  "1 1 comment" # #
+  "2 10 comment" # " echo foo"
+)

highlighters/main/test-data/comment-off.zsh

@@ -0,0 +1,38 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+unsetopt interactive_comments
+
+BUFFER='# echo foo'
+
+expected_region_highlight=(
+  "1 1 unknown-token" # #
+  "3 6 default" # " echo foo"
+  "8 10 default" # " echo foo"
+)

highlighters/main/test-data/comments.zsh

@@ -0,0 +1,38 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+setopt interactive_comments
+
+BUFFER='echo "foo #bar" #baz # quux'
+
+expected_region_highlight=(
+  "1 4 builtin" # echo
+  "6 15 double-quoted-argument" # "foo #bar"
+  "17 27 comment" # #baz # quux
+)

highlighters/main/test-data/commmand-parameter.zsh

@@ -0,0 +1,36 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+x=/usr/bin/env
+BUFFER='$x "argument"'
+
+expected_region_highlight=(
+  "1 2 command" # $x
+  "4 13 double-quoted-argument" # "argument"
+)

highlighters/main/test-data/control-flow.zsh

@@ -0,0 +1,50 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='while if echo Hello; then ls /; else ls; fi; do stat "x"; done; repeat 10 ls'
+
+expected_region_highlight+=(
+  "1 5 reserved-word" # while
+  "7 8 reserved-word" # if
+  "10 13 builtin" # echo
+  "15 19 default" # Hello
+  "22 25 reserved-word" # then
+  "27 28 command" # ls
+  "30 30 path" # /
+  "31 31 commandseparator" # ;
+  "33 36 reserved-word" # else
+  "38 39 command" # ls
+  "42 43 reserved-word" # fi
+  "46 47 reserved-word" # do
+  "49 52 command" # stat
+  "54 56 double-quoted-argument" # "x"
+  "59 62 reserved-word" # done
+  "65 70 reserved-word" # repeat
+  "75 76 command" # ls
+)

highlighters/main/test-data/control-flow2.zsh

@@ -0,0 +1,40 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='repeat 42; do ls; done'
+
+expected_region_highlight+=(
+  "1 6 reserved-word" # repeat
+  "8 9 default" # 42
+  "10 10 commandseparator" # ;
+  "12 13 reserved-word" # do
+  "15 16 command" # ls
+  "17 17 commandseparator" # ;
+  "19 22 reserved-word" # done
+)

highlighters/main/test-data/control-flow3.zsh

@@ -0,0 +1,39 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='repeat 42; ls; pwd'
+
+expected_region_highlight+=(
+  "1 6 reserved-word" # repeat
+  "8 9 default" # 42
+  "10 10 commandseparator" # ;
+  "12 13 command" # ls
+  "14 14 commandseparator" # ;
+  "16 18 builtin" # pwd
+)

highlighters/main/test-data/dollar-dollar.zsh

@@ -0,0 +1,39 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=': "$$ $$foo"'
+
+expected_region_highlight=(
+  '1 1 builtin' # :
+  '3 3 double-quoted-argument' # "
+  '4 5 dollar-double-quoted-argument' # $$
+  '7 8 dollar-double-quoted-argument' # $$
+  '9 12 double-quoted-argument' # foo"
+)

highlighters/main/test-data/dollar-noise.zsh

@@ -0,0 +1,46 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=': "$- $# $* $@ $?"'
+
+expected_region_highlight=(
+  '1 1 builtin' # :
+  '3 3 double-quoted-argument' # "
+  '4 5 dollar-double-quoted-argument' # $-
+  '6 6 double-quoted-argument' # <space>
+  '7 8 dollar-double-quoted-argument' # $#
+  '9 9 double-quoted-argument' # <space>
+  '10 11 dollar-double-quoted-argument' # $*
+  '12 12 double-quoted-argument' # <space>
+  '13 14 dollar-double-quoted-argument' # $@
+  '15 15 double-quoted-argument' # <space>
+  '16 17 dollar-double-quoted-argument' # $?
+  '18 18 double-quoted-argument' # "
+)

highlighters/main/test-data/dollar-paren.zsh

@@ -0,0 +1,38 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=': "$(:)"'
+
+expected_region_highlight=(
+  '1 1 builtin' # :
+  '3 3 double-quoted-argument' # "
+  '4 4 dollar-double-quoted-argument' # $
+  '5 8 double-quoted-argument' # (:)"
+)

highlighters/main/test-data/dollar-quoted.zsh

@@ -27,10 +27,9 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
-ZSH_HIGHLIGHT_STYLES[dollar-quoted-argument]=$unused_highlight
 BUFFER=": \$'*' 'foo'"
 
 expected_region_highlight=(
-  "3 6 $ZSH_HIGHLIGHT_STYLES[dollar-quoted-argument]" # $'*' - not a glob
+  "3 6 dollar-quoted-argument" # $'*' - not a glob
-  "8 12 $ZSH_HIGHLIGHT_STYLES[single-quoted-argument]" # 'foo'
+  "8 12 single-quoted-argument" # 'foo'
 )

highlighters/main/test-data/dollar-quoted2.zsh

@@ -27,15 +27,14 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
-ZSH_HIGHLIGHT_STYLES[back-dollar-quoted-argument]=$unused_highlight
 BUFFER=": \$'foo\xbar\udeadbeef\uzzzz'"
 
 expected_region_highlight=(
-  "3 7 $ZSH_HIGHLIGHT_STYLES[dollar-quoted-argument]" # $'foo
+  "3 7 dollar-quoted-argument" # $'foo
-  "8 11 $ZSH_HIGHLIGHT_STYLES[back-dollar-quoted-argument]" # \xba
+  "8 11 back-dollar-quoted-argument" # \xba
-  "12 12 $ZSH_HIGHLIGHT_STYLES[dollar-quoted-argument]" # r
+  "12 12 dollar-quoted-argument" # r
-  "13 18 $ZSH_HIGHLIGHT_STYLES[back-dollar-quoted-argument]" # \dead
+  "13 18 back-dollar-quoted-argument" # \dead
-  "19 22 $ZSH_HIGHLIGHT_STYLES[dollar-quoted-argument]" # beef
+  "19 22 dollar-quoted-argument" # beef
-  "23 24 $ZSH_HIGHLIGHT_STYLES[unknown-token]" # \u
+  "23 24 unknown-token" # \u
-  "25 29 $ZSH_HIGHLIGHT_STYLES[dollar-quoted-argument]" # zzzz'
+  "25 29 dollar-quoted-argument" # zzzz'
 )

highlighters/main/test-data/dollar-quoted3.zsh

@@ -28,11 +28,10 @@
 # -------------------------------------------------------------------------------------------------
 
 # Similar to double-quoted2.zsh
-ZSH_HIGHLIGHT_STYLES[back-dollar-quoted-argument]=$unused_highlight
 # This test checks that the '1' gets highlighted correctly.  Do not append to the BUFFER.
 BUFFER=": \$'\xa1"
 
 expected_region_highlight=(
-  "3 4 $ZSH_HIGHLIGHT_STYLES[dollar-quoted-argument]" # $'
+  "3 4 dollar-quoted-argument" # $'
-  "5 8 $ZSH_HIGHLIGHT_STYLES[back-dollar-quoted-argument]" # \xa1
+  "5 8 back-dollar-quoted-argument" # \xa1
 )

highlighters/main/test-data/double-hyphen-option.zsh

@@ -27,9 +27,8 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
-ZSH_HIGHLIGHT_STYLES[double-hyphen-option]=$unused_highlight
 BUFFER='hello --world'
 
 expected_region_highlight=(
-  "7 13 $ZSH_HIGHLIGHT_STYLES[double-hyphen-option]" # --world
+  "7 13 double-hyphen-option" # --world
 )

highlighters/main/test-data/double-quoted.zsh

@@ -31,15 +31,15 @@ BUFFER=': "foo$bar:\`:\":\$:'
 BUFFER+=\\\\:\"
 
 expected_region_highlight=(
-  "3 6 $ZSH_HIGHLIGHT_STYLES[double-quoted-argument]" # "foo
+  "3 6 double-quoted-argument" # "foo
-  "7 10 $ZSH_HIGHLIGHT_STYLES[dollar-double-quoted-argument]" # $bar
+  "7 10 dollar-double-quoted-argument" # $bar
-  "11 11 $ZSH_HIGHLIGHT_STYLES[double-quoted-argument]" # :
+  "11 11 double-quoted-argument" # :
-  "12 13 $ZSH_HIGHLIGHT_STYLES[back-double-quoted-argument]" # \`
+  "12 13 back-double-quoted-argument" # \`
-  "14 14 $ZSH_HIGHLIGHT_STYLES[double-quoted-argument]" # :
+  "14 14 double-quoted-argument" # :
-  "15 16 $ZSH_HIGHLIGHT_STYLES[back-double-quoted-argument]" # \$
+  "15 16 back-double-quoted-argument" # \$
-  "17 17 $ZSH_HIGHLIGHT_STYLES[double-quoted-argument]" # :
+  "17 17 double-quoted-argument" # :
-  "18 19 $ZSH_HIGHLIGHT_STYLES[back-double-quoted-argument]" # \"
+  "18 19 back-double-quoted-argument" # \"
-  "20 20 $ZSH_HIGHLIGHT_STYLES[double-quoted-argument]" # :
+  "20 20 double-quoted-argument" # :
-  "21 22 $ZSH_HIGHLIGHT_STYLES[back-double-quoted-argument]" # \\
+  "21 22 back-double-quoted-argument" # \\
-  "23 24 $ZSH_HIGHLIGHT_STYLES[double-quoted-argument]" # :"
+  "23 24 double-quoted-argument" # :"
 )

highlighters/main/test-data/double-quoted2.zsh

@@ -1,5 +1,5 @@
 # -------------------------------------------------------------------------------------------------
-# Copyright (c) 2010-2011 zsh-syntax-highlighting contributors
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without modification, are permitted
@@ -32,6 +32,6 @@
 BUFFER=': "foo$bar'
 
 expected_region_highlight=(
-  "3 6 $ZSH_HIGHLIGHT_STYLES[double-quoted-argument]" # "foo
+  "3 6 double-quoted-argument" # "foo
-  "7 10 $ZSH_HIGHLIGHT_STYLES[dollar-double-quoted-argument]" # $bar
+  "7 10 dollar-double-quoted-argument" # $bar
 )

highlighters/main/test-data/double-quoted3.zsh

@@ -1,5 +1,5 @@
 # -------------------------------------------------------------------------------------------------
-# Copyright (c) 2010-2011 zsh-syntax-highlighting contributors
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without modification, are permitted
@@ -31,9 +31,9 @@ BUFFER=': "$" "$42foo"'
 BUFFER+=\ \"\\\'\\x\"
 
 expected_region_highlight=(
-  "3 5 $ZSH_HIGHLIGHT_STYLES[double-quoted-argument]" # "$"
+  "3 5 double-quoted-argument" # "$"
-  "7 7 $ZSH_HIGHLIGHT_STYLES[double-quoted-argument]" # "
+  "7 7 double-quoted-argument" # "
-  "8 10 $ZSH_HIGHLIGHT_STYLES[dollar-double-quoted-argument]" # $42
+  "8 10 dollar-double-quoted-argument" # $42
-  "11 14 $ZSH_HIGHLIGHT_STYLES[double-quoted-argument]" # foo"
+  "11 14 double-quoted-argument" # foo"
-  "16 21 $ZSH_HIGHLIGHT_STYLES[double-quoted-argument]" # "\'\x" - \' and \x are not escape sequences
+  "16 21 double-quoted-argument" # "\'\x" - \' and \x are not escape sequences
 )

highlighters/main/test-data/double-quoted4.zsh

@@ -0,0 +1,36 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=': "${foo}bar"'
+
+expected_region_highlight=(
+  "3 3 double-quoted-argument" # "
+  "4 9 dollar-double-quoted-argument" # ${foo}
+  "10 13 double-quoted-argument" # bar"
+)

highlighters/main/test-data/empty-command.zsh

@@ -27,10 +27,9 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
-# without the trailing space, it's highlighted as path_approx (issue #187, issue #204)
+BUFFER='echo; ;'
-BUFFER='echo; ; '
 
 expected_region_highlight=(
-  "5 5 $ZSH_HIGHLIGHT_STYLES[commandseparator]" # ;
+  "5 5 commandseparator" # ;
-  "7 7 $ZSH_HIGHLIGHT_STYLES[unknown-token]" # ;
+  "7 7 unknown-token" # ;
 )

highlighters/main/test-data/empty-command2.zsh

@@ -0,0 +1,38 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+# Same test data and expectations as empty-command.zsh; the only difference is:
+touch ';'
+
+BUFFER='echo; ;'
+
+expected_region_highlight=(
+  "5 5 commandseparator" # ;
+  "7 7 unknown-token" # ;
+)

highlighters/main/test-data/exec-redirection1.zsh

@@ -0,0 +1,38 @@
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2015 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='exec {foo}>&/tmp ls'
+
+expected_region_highlight=(
+  "1 4 precommand" # exec
+  "6 10 redirection 'issue #238'" # {foo}
+  "11 12 redirection" # >&
+  "13 16 path" # /tmp
+  "18 19 command 'issue #238'" # ls
+)

highlighters/main/test-data/function-named1.zsh

@@ -0,0 +1,44 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='f() pwd; f() { balanced braces }'
+
+expected_region_highlight=(
+  '1 1 TBD "issue #223"' # f
+  '2 3 reserved-word' # ()
+  '5 7 builtin' # pwd
+  '8 8 commandseparator' # ;
+  '10 10 TBD "issue #223"' # f
+  '11 12 reserved-word' # ()
+  '14 14 reserved-word' # {
+  '16 23 unknown-token' # balanced
+  '25 30 default' # braces
+  '32 32 reserved-word' # }
+)

highlighters/main/test-data/function-named2.zsh

@@ -0,0 +1,39 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='f g h () pwd'
+
+expected_region_highlight=(
+  '1 1 TBD "issue #223"' # f
+  '3 3 TBD "issue #223"' # g
+  '5 5 TBD "issue #223"' # h
+  '7 8 reserved-word' # ()
+  '10 12 builtin' # pwd
+)

highlighters/main/test-data/function.zsh

@@ -27,7 +27,6 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
-ZSH_HIGHLIGHT_STYLES[function]=$unused_highlight
 cd() {
   builtin cd "$@"
 }
@@ -36,11 +35,9 @@ ls() {
 }
 BUFFER='cd;ls'
 
-# Use $unused_highlight to see that function highlighting has precedence over command and builtin
-
 expected_region_highlight=(
-  "1 2 $ZSH_HIGHLIGHT_STYLES[function]" # cd
+  "1 2 function" # cd
-  "4 5 $ZSH_HIGHLIGHT_STYLES[function]" # ls
+  "4 5 function" # ls
 )
 
 # don't 'unfunction cd ls', since cd() and ls() should still be a functions

highlighters/main/test-data/glob.zsh

@@ -30,9 +30,9 @@
 BUFFER=': foo* bar? *baz qux\?'
 
 expected_region_highlight=(
-  "1 1 $ZSH_HIGHLIGHT_STYLES[builtin]" # :
+  "1 1 builtin" # :
-  "3 6 $ZSH_HIGHLIGHT_STYLES[globbing]" # foo*
+  "3 6 globbing" # foo*
-  "8 11 $ZSH_HIGHLIGHT_STYLES[globbing]" # bar?
+  "8 11 globbing" # bar?
-  "13 16 $ZSH_HIGHLIGHT_STYLES[globbing]" # *baz
+  "13 16 globbing" # *baz
-  "18 22 $ZSH_HIGHLIGHT_STYLES[default]" # qux\?
+  "18 22 default" # qux\?
 )

highlighters/main/test-data/hashed-command.zsh

@@ -28,9 +28,8 @@
 # -------------------------------------------------------------------------------------------------
 
 hash zsh_syntax_highlighting_hash=/doesnotexist
-ZSH_HIGHLIGHT_STYLES[hashed-command]=$unused_highlight
 BUFFER='zsh_syntax_highlighting_hash'
 
 expected_region_highlight=(
-  "1 28 $ZSH_HIGHLIGHT_STYLES[hashed-command]"
+  "1 28 hashed-command 'zsh/parameter cannot distinguish between hashed and command'"
 )

highlighters/main/test-data/history-double-quoted-escaped.zsh

@@ -0,0 +1,38 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER='echo "Hello\!"'
+
+expected_region_highlight=(
+  '1 4 builtin' # echo
+  '6 11 double-quoted-argument' # "Hello
+  '12 13 back-double-quoted-argument' # \!
+  '14 14 double-quoted-argument' # "Hello\!"
+)

highlighters/main/test-data/history-expansion.zsh

@@ -27,10 +27,12 @@
 # vim: ft=zsh sw=2 ts=2 et
 # -------------------------------------------------------------------------------------------------
 
-BUFFER='!foo bar !baz'
+BUFFER='!foo bar !baz ! ; !'
 
 expected_region_highlight=(
-  "1 4 $ZSH_HIGHLIGHT_STYLES[history-expansion]" # !foo
+  "1 4 history-expansion" # !foo
-  "6 8 $ZSH_HIGHLIGHT_STYLES[default]" # bar
+  "6 8 default" # bar
-  "10 13 $ZSH_HIGHLIGHT_STYLES[history-expansion]" # !baz
+  "10 13 history-expansion" # !baz
+  "15 15 default" # ! (before the semicolon)
+  "19 19 reserved-word" # ! (after the semicolon)
 )

highlighters/main/test-data/history-expansion2.zsh

@@ -30,5 +30,5 @@
 BUFFER='^foo^bar'
 
 expected_region_highlight=(
-  "1 8 $ZSH_HIGHLIGHT_STYLES[history-expansion]" # ^foo^bar
+  "1 8 history-expansion" # ^foo^bar
 )

highlighters/main/test-data/inheritance.zsh

@@ -0,0 +1,40 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+_zsh_highlight_add_highlight()
+{
+  region_highlight+=("$1 $2 ${(j.,.)argv[3,-1]}")
+}
+
+BUFFER='type'
+
+expected_region_highlight=(
+  '1 4 builtin,arg0' # type
+)

highlighters/main/test-data/multiline-array-assignment1.zsh

@@ -0,0 +1,39 @@
+#!/usr/bin/env zsh
+# -------------------------------------------------------------------------------------------------
+# Copyright (c) 2016 zsh-syntax-highlighting contributors
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification, are permitted
+# provided that the following conditions are met:
+#
+#  * Redistributions of source code must retain the above copyright notice, this list of conditions
+#    and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright notice, this list of
+#    conditions and the following disclaimer in the documentation and/or other materials provided
+#    with the distribution.
+#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
+#    may be used to endorse or promote products derived from this software without specific prior
+#    written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
+# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# -------------------------------------------------------------------------------------------------
+# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
+# vim: ft=zsh sw=2 ts=2 et
+# -------------------------------------------------------------------------------------------------
+
+BUFFER=$'foo=(\nbar) env'
+
+expected_region_highlight=(
+  '1 5 assign' # foo=(
+  '6 6 commandseparator' # \n
+  '7 9 default' # bar
+  '10 10 assign' # )
+  '12 14 command' # env
+)

highlighters/main/test-data/multiline-string.zsh

@@ -31,6 +31,6 @@ PREBUFFER=$'echo "foo1\n'
 BUFFER='foo2" ./'
 
 expected_region_highlight=(
-  "1 5 $ZSH_HIGHLIGHT_STYLES[double-quoted-argument]" # 'foo2"'
+  "1 5 double-quoted-argument" # 'foo2"'
-  "7 8 $ZSH_HIGHLIGHT_STYLES[path]" # './'
+  "7 8 path" # './'
 )