doc: test/README.md: migrate wiki info (#7552)

test/README.md

@@ -1,10 +1,224 @@
 Tests
 =====
 
-Tests are run by `/cmake/RunTests.cmake` file, using busted.
+Tests are run by `/cmake/RunTests.cmake` file, using `busted`.
 
 For some failures, `.nvimlog` (or `$NVIM_LOG_FILE`) may provide insight.
 
+---
+
+- [Running tests](#running-tests)
+- [Unit tests](#unit-tests)
+- [Lint](#lint)
+- [Environment variables](#environment-variables)
+
+---
+
+Running tests
+-------------
+
+Neovim uses third-party tooling to execute tests. So be sure, from the
+repository directory, to build the tools before testing:
+
+    make cmake
+
+## Executing Tests
+
+To run all _non-legacy_ (unit + functional) tests:
+
+    make test
+
+To run only _unit_ tests:
+
+    make unittest
+
+To run only _functional_ tests:
+
+    make functionaltest
+
+---
+
+## Filter Tests
+
+### Filter by name
+
+Another filter method is by setting a pattern of test name to `TEST_FILTER`.
+
+``` lua
+it('foo api',function()
+  ...
+end)
+it('bar api',function()
+  ...
+end)
+```
+
+To run only test with filter name:
+
+    TEST_TAG='foo.*api' make functionaltest
+
+### Filter by file
+
+To run a *specific* unit test:
+
+    TEST_FILE=test/unit/foo.lua make unittest
+
+To run a *specific* functional test:
+
+    TEST_FILE=test/functional/foo.lua make functionaltest
+
+To *repeat* a test many times:
+
+    .deps/usr/bin/busted --filter 'foo' --repeat 1000 test/functional/ui/foo_spec.lua
+
+### Filter by tag
+
+Tests can be "tagged" by adding `#` before a token in the test description.
+
+``` lua
+it('#foo bar baz', function()
+  ...
+end)
+it('#foo another test', function()
+  ...
+end)
+```
+
+To run only the tagged tests:
+
+    TEST_TAG=foo make functionaltest
+
+**NOTES**:
+* Tags are mainly used for testing issues (ex: `#1234`), so use the following
+  method.
+* `TEST_FILE` is not a pattern string like `TEST_TAG` or `TEST_FILTER`. The
+  given value to `TEST_FILE` must be a path to an existing file.
+* Both `TEST_TAG` and `TEST_FILTER` filter tests by the strings from either
+  `it()` or `describe()` functions.
+
+---
+
+### Legacy
+
+To run all legacy (Vim) integration tests:
+
+    make oldtest
+
+To run a *single* legacy test,  run `make` with `TEST_FILE=test_name.res`. E.g.
+to run `test_syntax.vim`:
+
+    TEST_FILE=test_syntax.res make oldtest
+
+-  The `.res` extension (instead of `.vim`) is required.
+- Specify only the test file name, not the full path.
+
+### Functional tests
+
+`$GDB` can be set to [run tests under
+gdbserver](https://github.com/neovim/neovim/pull/1527). If `$VALGRIND` is also
+set, it will add the `--vgdb=yes` option to valgrind instead of
+starting gdbserver directly.
+
+Unit tests
+----------
+
+Tests are broadly divided into *unit tests*
+([test/unit](https://github.com/neovim/neovim/tree/master/test/unit) directory)
+and *functional tests*
+([test/functional](https://github.com/neovim/neovim/tree/master/test/functional)
+directory). Use any of the existing tests as a template to start writing new
+tests.
+
+- _Unit_ testing is achieved by compiling the tests as a shared library which is
+  loaded and called by LuaJit [FFI](http://luajit.org/ext_ffi.html).
+- _Functional_ tests are driven by RPC, so they do not require LuaJit (as
+  opposed to Lua).
+
+You can learn the [key concepts of Lua in 15
+minutes](http://learnxinyminutes.com/docs/lua/).
+
+## Guidelines for writing tests
+
+- Consider [BDD](http://en.wikipedia.org/wiki/Behavior-driven_development)
+  guidelines for organization and readability of tests. Describe what you're
+  testing (and the environment if applicable) and create specs that assert its
+  behavior.
+- For testing static functions or functions that have side effects visible only
+  in module-global variables, create accessors for the modified variables. For
+  example, say you are testing a function in misc1.c that modifies a static
+  variable, create a file `test/c-helpers/misc1.c` and add a function that
+  retrieves the value after the function call. Files under `test/c-helpers` will
+  only be compiled when building the test shared library.
+- Luajit needs to know about type and constant declarations used in function
+  prototypes. The
+  [helpers.lua](https://github.com/neovim/neovim/blob/master/test/unit/helpers.lua)
+  file automatically parses `types.h`, so types used in the tested functions
+  must be moved to it to avoid having to rewrite the declarations in the test
+  files (even though this is how it's currently done currently in the misc1/fs
+  modules, but contributors are encouraged to refactor the declarations).
+  - Macro constants must be rewritten as enums so they can be "visible" to the
+    tests automatically.
+- Busted supports various "output providers". The
+  **[gtest](https://github.com/Olivine-Labs/busted/pull/394) output provider**
+  shows verbose details that can be useful to diagnose hung tests. Either modify
+  the Makefile or compile with `make
+  CMAKE_EXTRA_FLAGS=-DBUSTED_OUTPUT_TYPE=gtest` to enable it.
+- **Use busted's `pending()` feature** to skip tests
+  ([example](https://github.com/neovim/neovim/commit/5c1dc0fbe7388528875aff9d7b5055ad718014de#diff-bf80b24c724b0004e8418102f68b0679R18)).
+  Do not silently skip the test with `if-else`. If a functional test depends on
+  some external factor (e.g. the existence of `md5sum` on `$PATH`), *and* you
+  can't mock or fake the dependency, then skip the test via `pending()` if the
+  external factor is missing. This ensures that the *total* test-count (success
+  + fail + error + pending) is the same in all environments.
+    - *Note:* `pending()` is ignored if it is missing an argument _unless_ it is
+      [contained in an `it()`
+      block](https://github.com/neovim/neovim/blob/d21690a66e7eb5ebef18046c7a79ef898966d786/test/functional/ex_cmds/grep_spec.lua#L11).
+      Provide empty function argument if the `pending()` call is outside of
+      `it()`
+      ([example](https://github.com/neovim/neovim/commit/5c1dc0fbe7388528875aff9d7b5055ad718014de#diff-bf80b24c724b0004e8418102f68b0679R18)).
+- Use `make testlint` for using the shipped luacheck program ([supported by
+  syntastic](https://github.com/scrooloose/syntastic/blob/d6b96c079be137c83009827b543a83aa113cc011/doc/syntastic-checkers.txt#L3546))
+  to lint all tests.
+
+### Where tests go
+
+- _Unit tests_
+  ([test/unit](https://github.com/neovim/neovim/tree/master/test/unit))  should
+  match 1-to-1 with the structure of `src/nvim/`, because they are testing
+  functions directly. E.g. unit-tests for `src/nvim/undo.c` should live in
+  `test/unit/undo_spec.lua`.
+- _Functional tests_
+  ([test/functional](https://github.com/neovim/neovim/tree/master/test/functional))
+  are higher-level (plugins and user input) than unit tests; they are organized
+  by concept. 
+    - Try to find an existing `test/functional/*/*_spec.lua` group that makes
+      sense, before creating a new one.
+
+## Checklist for migrating legacy tests
+
+**Note:** Only "old style" (`src/testdir/*.in`) legacy tests should be
+converted. Please _do not_ convert "new style" Vim tests (`src/testdir/*.vim`).
+The "new style" Vim tests are faster than the old ones, and converting them
+takes time and effort better spent elsewhere.
+
+- Remove the test from the Makefile (`src/nvim/testdir/Makefile`).
+- Remove the associated `test.in`, `test.out`, and `test.ok` files from
+  `src/nvim/testdir/`.
+- Make sure the lua test ends in `_spec.lua`.
+- Make sure the test count increases accordingly in the build log.
+- Make sure the new test contains the same control characters (`^]`, ...) as the
+  old test.
+  - Instead of the actual control characters, use an equivalent textual
+    representation (e.g. `<esc>` instead of `^]`). The
+    `scripts/legacy2luatest.pl` script does some of these conversions
+    automatically.
+
+## Tips
+
+- Really long `source([=[...]=])` blocks may break syntax highlighting. Try
+  `:syntax sync fromstart` to fix it.
+
+
 Lint
 ----