test harness: Run each test as a single subshell.

That allows tests to be completely independent of each other, so tests that
change global state — such as modify environment variables, define functions or
aliases, or hash commands — will not affect other tests, without needing an
explicit cleanup step.

This enables testing path-tilde-home with and without $HOME set, which is part
of issue #216.

While at it, convert the test harness to TAP.  This fixes issue #180 by adding
support for "not ok 42 # TODO" output.

This commit assumes that 'grep' supports POSIX-compliant -q and -v flags.

Patch-by: Matthew Martin <phy1729@gmail.com>
This commit is contained in:
Matthew Martin
2015-10-23 03:27:05 +00:00
committed by Daniel Shahaf
parent 083c47b007
commit d99aa58aaa
2 changed files with 57 additions and 66 deletions

View File

@@ -6,9 +6,10 @@ Utility scripts for testing zsh-syntax-highlighting highlighters.
The tests expect the highlighter directory to contain a `test-data` directory with test data files. See the [main highlighter](../highlighters/main/test-data) for examples.
Each test should define the array parameter `$expected_region_highlight`.
The value of that parameter is a list of `"$i $j $style"` strings.
The value of that parameter is a list of `"$i $j $style [$todo]"` strings.
Each string specifies the highlighting that `$BUFFER[$i,$j]` should have;
that is, `$i` and `$j` specify a range, 1-indexed, inclusive of both endpoints.
If `$todo` exists, the test point is marked as TODO (the failure of that test point will not fail the test), and `$todo` is used as the explanation.
_Note_: `$region_highlight` uses the same `"$i $j $style"` syntax but interprets the indexes differently.
@@ -19,6 +20,12 @@ highlighting test
zsh test-highlighting.zsh <HIGHLIGHTER NAME>
All tests may be run with
make test
which will run all highlighting tests and report results in [TAP](http://testanything.org/) format.
performance test
----------------