IntenseFinance/neovim
3
0
Fork 0
mirror of https://github.com/neovim/neovim.git synced 2025-02-25 18:55:25 -06:00
Code Issues Packages Projects Releases Wiki Activity

doc: eliminate msgpack_rpc.txt [ci skip]

Browse Source 
- Migrate msgpack_rpc.txt into api.txt, develop.txt.
- fix #10740: Remove warning about "avoid hardcoding the type codes".
This commit is contained in:
Justin M. Keyes 2019-08-26 01:00:58 +02:00
parent 81c3fa6c9d
commit 4b2a2c332c
6 changed files with 179 additions and 264 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

152
runtime/doc/api.txt
View File

@ -13,6 +13,76 @@ Applications can also embed libnvim to work with the C API directly.
Type |gO| to see the table of contents.
==============================================================================
API Usage *api-rpc* *RPC* *rpc*
*msgpack-rpc*
RPC is the typical way to control Nvim programmatically. Nvim implements the
MessagePack-RPC protocol:
https://github.com/msgpack/msgpack/blob/0b8f5ac/spec.md
Many clients use the API: user interfaces (GUIs), remote plugins, scripts like
"nvr" (https://github.com/mhinz/neovim-remote). Even Nvim itself can control
other Nvim instances. API clients can:
- Call any API function
- Listen for events
- Receive remote calls from Nvim
The RPC API is like a more powerful version of Vim's "clientserver" feature.
CONNECTING *rpc-connecting*
See |channel-intro| for various ways to open a channel. Channel-opening
functions take an `rpc` key in the options dictionary. RPC channels can also
be opened by other processes connecting to TCP/IP sockets or named pipes
listened to by Nvim.
Nvim creates a default RPC socket at |startup|, given by |v:servername|. To
start with a TCP/IP socket instead, use |--listen| with a TCP-style address: >
nvim --listen 127.0.0.1:6666
More endpoints can be started with |serverstart()|.
Note that localhost TCP sockets are generally less secure than named pipes,
and can lead to vunerabilities like remote code execution.
Connecting to the socket is the easiest way a programmer can test the API,
which can be done through any msgpack-rpc client library or full-featured
|api-client|. Here's a Ruby script that prints "hello world!" in the current
Nvim instance:
>
#!/usr/bin/env ruby
# Requires msgpack-rpc: gem install msgpack-rpc
#
# To run this script, execute it from a running Nvim instance (notice the
# trailing '&' which is required since Nvim won't process events while
# running a blocking command):
#
# :!./hello.rb &
#
# Or from another shell by setting NVIM_LISTEN_ADDRESS:
# $ NVIM_LISTEN_ADDRESS=[address] ./hello.rb
require 'msgpack/rpc'
require 'msgpack/rpc/transport/unix'
nvim = MessagePack::RPC::Client.new(MessagePack::RPC::UNIXTransport.new, ENV['NVIM_LISTEN_ADDRESS'])
result = nvim.call(:nvim_command, 'echo "hello world!"')
<
A better way is to use the Python REPL with the `neovim` package, where API
functions can be called interactively:
>
>>> from pynvim import attach
>>> nvim = attach('socket', path='[address]')
>>> nvim.command('echo "hello world!"')
<
You can also embed Nvim via |jobstart()|, and communicate using |rpcrequest()|
and |rpcnotify()|:
>
let nvim = jobstart(['nvim', '--embed'], {'rpc': v:true})
echo rpcrequest(nvim, 'nvim_eval', '"Hello " . "world!"')
call jobstop(nvim)
==============================================================================
API Definitions *api-definitions*
@ -20,20 +90,37 @@ API Definitions *api-definitions*
The Nvim C API defines custom types for all function parameters. Some are just
typedefs around C99 standard types, others are Nvim-defined data structures.
Boolean -> bool
Integer (signed 64-bit integer) -> int64_t
Float (IEEE 754 double precision) -> double
String -> {char* data, size_t size} struct
Basic types ~
API Type C type
------------------------------------------------------------------------
Nil
Boolean bool
Integer (signed 64-bit integer) int64_t
Float (IEEE 754 double precision) double
String {char* data, size_t size} struct
Array
Dictionary
Dictionary (msgpack: map)
Object
The following handle types are defined as integer typedefs, but are
discriminated as separate types in an Object:
Note: empty Array is accepted as a valid argument for Dictionary parameter.
Special types (msgpack EXT) ~
These are integer typedefs discriminated as separate Object subtypes. They
can be treated as opaque integers, but are mutually incompatible: Buffer may
be passed as an integer but not as Window or Tabpage.
The EXT object data is the (integer) object handle. The EXT type codes given
in the |api-metadata| `types` key are stable: they will not change and are
thus forward-compatible.
EXT Type C type Data
------------------------------------------------------------------------
Buffer enum value kObjectTypeBuffer |bufnr()|
Window enum value kObjectTypeWindow |window-ID|
Tabpage enum value kObjectTypeTabpage internal handle
Buffer -> enum value kObjectTypeBuffer
Window -> enum value kObjectTypeWindow
Tabpage -> enum value kObjectTypeTabpage
*api-indexing*
Most of the API uses 0-based indices, and ranges are end-exclusive. For the
@ -49,22 +136,59 @@ lines, 0-based columns):
==============================================================================
API metadata *api-metadata*
Nvim exposes API metadata as a Dictionary. Some items are described below:
The Nvim C API is automatically exposed to RPC by the build system, which
parses headers in src/nvim/api/* and generates dispatch-functions mapping RPC
API method names to public C API functions, converting/validating arguments
and return values.
Nvim exposes its API metadata as a Dictionary with these items:
version Nvim version, API level/compatibility
version.api_level API version integer *api-level*
version.api_compatible API is backwards-compatible with this level
version.api_prerelease Declares the API as unstable/unreleased >
(version.api_prerelease && fn.since == version.api_level)
functions API function signatures
ui_events UI event signatures |ui|
functions API function signatures, containing |api-types| info
describing the return value and parameters.
ui_events |UI| event signatures
ui_options Supported |ui-option|s
{fn}.since API level where function {fn} was introduced
{fn}.deprecated_since API level where function {fn} was deprecated
types Custom handle types defined by Nvim
error_types Possible error types returned by API functions
External programs ("clients") can use the metadata to discover the |rpc-api|.
About the `functions` map:
- Container types may be decorated with type/size constraints, e.g.
ArrayOf(Buffer) or ArrayOf(Integer, 2).
- Functions considered to be methods that operate on instances of Nvim
special types (msgpack EXT) have the "method=true" flag. The receiver type
is that of the first argument. Method names are prefixed with `nvim_` plus
a type name, e.g. `nvim_buf_get_lines` is the `get_lines` method of
a Buffer instance. |dev-api|
- Global functions have the "method=false" flag and are prefixed with just
`nvim_`, e.g. `nvim_get_buffers`.
*api-mapping*
External programs (clients) can use the metadata to discover the API, using
any of these approaches:
1. Connect to a running Nvim instance and call |nvim_get_api_info()| via
msgpack-rpc. This is best for clients written in dynamic languages which
can define functions at runtime.
2. Start Nvim with the |--api-info| option. Useful for clients written in
statically-compiled languages.
3. Use the |api_info()| Vimscript function.
Example: To get a human-readable list of API functions: >
:new|put =map(filter(api_info().functions, '!has_key(v:val,''deprecated_since'')'), 'v:val.name')
<
Example: To get a formatted dump of the API using python (requires the
"pyyaml" and "msgpack-python" modules): >
nvim --api-info | python -c 'import msgpack, sys, yaml; print yaml.dump(msgpack.unpackb(sys.stdin.read()))'
<
==============================================================================
API contract *api-contract*

2
runtime/doc/channel.txt
View File

@ -30,7 +30,7 @@ There are several ways to open a channel:
Channels support multiple modes or protocols. In the most basic
mode of operation, raw bytes are read and written to the channel.
The |rpc| protocol, based on the msgpack-rpc standard, enables nvim and the
The |RPC| protocol, based on the msgpack-rpc standard, enables nvim and the
process at the other end to send remote calls and events to each other.
The builtin |terminal-emulator| is also implemented on top of PTY channels.

36
runtime/doc/develop.txt
View File

@ -185,11 +185,25 @@ Example:
API-CLIENT *dev-api-client*
*api-client*
API clients wrap the Nvim |API| to provide idiomatic "SDKs" for their
respective platforms (see |jargon|). You can build a new API client for your
favorite platform or programming language.
List of API clients:
https://github.com/neovim/neovim/wiki/Related-projects#api-clients
*pynvim*
The Python client is the reference implementation for API clients.
https://github.com/neovim/pynvim
Standard Features ~
- API clients exist to hide msgpack-rpc details. The wrappers can be
automatically generated by reading the |api-metadata| from Nvim. |api-mapping|
- Clients should call |nvim_set_client_info()| after connecting, so users and
plugins can detect the client by handling the |ChanInfo| event. This
avoids the need for special variables or other client hints.
plugins can detect the client by handling the |ChanInfo| event. This avoids
the need for special variables or other client hints.
- Clients should handle |nvim_error_event| notifications, which will be sent
if an async request to nvim was rejected or caused an error.
@ -209,15 +223,29 @@ Examples of API-client package names:
BAD: python-client
BAD: neovim
Implementation ~
API client implementation guidelines ~
For C/C++ projects, consider libmpack instead of the msgpack.org library.
- Separate the transport layer from the rest of the library. |rpc-connecting|
- Use a MessagePack library that implements at least version 5 of the
MessagePack spec, which supports the BIN and EXT types used by Nvim.
- Use a single-threaded event loop library/pattern.
- Use a fiber/coroutine library for the language being used for implementing
a client. These greatly simplify concurrency and allow the library to
expose a blocking API on top of a non-blocking event loop without the
complexity that comes with preemptive multitasking.
- Don't assume anything about the order of responses to RPC requests.
- Clients should expect requests, which must be handled immediately because
Nvim is blocked while waiting for the client response.
- Clients should expect notifications, but these can be handled "ASAP" (rather
than immediately) because they won't block Nvim.
- For C/C++ projects, consider libmpack instead of the msgpack.org library.
https://github.com/libmpack/libmpack/
libmpack is small (no dependencies, can inline into your C/C++ project) and
efficient (no allocations). It also implements msgpack-RPC, the protocol
required by Nvim.
https://github.com/msgpack-rpc/msgpack-rpc
EXTERNAL UI *dev-ui*
External UIs should be aware of the |api-contract|. In particular, future

239
runtime/doc/msgpack_rpc.txt
View File

@ -3,242 +3,5 @@
NVIM REFERENCE MANUAL by Thiago de Arruda
RPC API for Nvim *RPC* *rpc* *msgpack-rpc*
Type |gO| to see the table of contents.
==============================================================================
1. Introduction *rpc-intro*
The primary way to control Nvim programmatically is the RPC API, which speaks
MessagePack-RPC ("msgpack-rpc"), a messaging protocol that uses the
MessagePack serialization format:
https://github.com/msgpack/msgpack/blob/0b8f5ac/spec.md
All kinds of Nvim "clients" use the RPC API: user interfaces (GUIs), remote
plugins, scripts like "nvr" (https://github.com/mhinz/neovim-remote), and even
`nvim` itself can control other `nvim` instances. By connecting to the RPC API
programs can:
- Call any API function
- Listen for events
- Receive remote calls from Nvim
The RPC API is like a more powerful version of Vim's `clientserver` feature.
==============================================================================
2. API mapping *rpc-api*
The Nvim C |API| is automatically exposed to the RPC API by the build system,
which parses headers at src/nvim/api/*. A dispatch function is generated which
matches RPC API method names with public API functions, converting/validating
arguments and return values back to msgpack.
Client libraries (|api-client|s) normally provide wrappers that hide
msgpack-rpc details from application developers. The wrappers can be
automatically generated by reading bundled API metadata from a compiled Nvim
instance.
There are three ways to obtain API metadata:
1. Connect to a running Nvim instance and call `nvim_get_api_info` via
msgpack-rpc. This is best for clients written in dynamic languages which
can define functions at runtime.
2. Start Nvim with the |--api-info| option. Useful for clients written in
statically-compiled languages.
3. Use the |api_info()| vimscript function.
To get a human-readable list of API functions: >
:new|put =map(filter(api_info().functions, '!has_key(v:val,''deprecated_since'')'), 'v:val.name')
<
To get a formatted dump of the API using python (requires the `pyyaml` and
`msgpack-python` packages): >
nvim --api-info | python -c 'import msgpack, sys, yaml; print yaml.dump(msgpack.unpackb(sys.stdin.read()))'
<
==============================================================================
3. Connecting *rpc-connecting*
See |channel-intro|, for various ways to open a channel. Most of the channel
opening functions take an `rpc` key in the options dictionary, to enable RPC.
Additionally, RPC channels can be opened by other processes connecting to
TCP/IP sockets or named pipes listened to by nvim.
Nvim creates a default RPC socket at |startup|, given by |v:servername|. To
start with a TCP/IP socket instead, use |--listen| with a TCP-style address: >
nvim --listen 127.0.0.1:6666
Additional sockets and named pipes can be started with |serverstart()|.
Note that localhost TCP sockets are generally less secure than named pipes,
and can lead to vunerabilities like remote code execution.
Connecting to the socket is the easiest way a programmer can test the API,
which can be done through any msgpack-rpc client library or full-featured
|api-client|. Here's a Ruby script that prints 'hello world!' in the current
Nvim instance:
>
#!/usr/bin/env ruby
# Requires msgpack-rpc: gem install msgpack-rpc
#
# To run this script, execute it from a running Nvim instance (notice the
# trailing '&' which is required since Nvim won't process events while
# running a blocking command):
#
# :!./hello.rb &
#
# Or from another shell by setting NVIM_LISTEN_ADDRESS:
# $ NVIM_LISTEN_ADDRESS=[address] ./hello.rb
require 'msgpack/rpc'
require 'msgpack/rpc/transport/unix'
nvim = MessagePack::RPC::Client.new(MessagePack::RPC::UNIXTransport.new, ENV['NVIM_LISTEN_ADDRESS'])
result = nvim.call(:nvim_command, 'echo "hello world!"')
<
A better way is to use the Python REPL with the `neovim` package, where API
functions can be called interactively:
>
>>> from neovim import attach
>>> nvim = attach('socket', path='[address]')
>>> nvim.command('echo "hello world!"')
<
You can also embed an Nvim instance via |jobstart()|, and communicate using
|rpcrequest()| and |rpcnotify()|:
>
let nvim = jobstart(['nvim', '--embed'], {'rpc': v:true})
echo rpcrequest(nvim, 'nvim_eval', '"Hello " . "world!"')
call jobstop(nvim)
<
==============================================================================
4. Implementing API clients *rpc-api-client* *api-client*
API clients wrap the Nvim API to provide idiomatic "SDKs" for their respective
platforms (see |jargon|). You can build a new API client for your favorite
platform or programming language.
API clients are listed here:
https://github.com/neovim/neovim/wiki/Related-projects#api-clients
*pynvim*
The Python client is the reference implementation for API clients. It is
always up-to-date with the Nvim API, so its source code and test suite are
authoritative references.
https://github.com/neovim/pynvim
API client implementation guidelines ~
- Separate the transport layer from the rest of the library. See
|rpc-connecting| for details on how clients can connect to Nvim.
- Use a MessagePack library that implements at least version 5 of the
MessagePack spec, which supports the `bin` and `ext` types used by Nvim.
- Read API metadata in order to create client-side wrappers for all
msgpack-rpc methods.
- Use a single-threaded event loop library/pattern.
- Use a fiber/coroutine library for the language being used for implementing
a client. These greatly simplify concurrency and allow the library to
expose a blocking API on top of a non-blocking event loop without the
complexity that comes with preemptive multitasking.
- Don't assume anything about the order that responses to msgpack-rpc
requests will arrive.
- Clients should expect msgpack-rpc requests, which need to be handled
immediately because Nvim is blocked while waiting for the client response.
- Clients should expect to receive msgpack-rpc notifications, but these
don't need to be handled immediately because they won't block Nvim
(although they should probably be handled immediately anyway).
Note: Most of the complexity could be handled by a msgpack-rpc library that
supports server to client requests and notifications, but it's not clear if
this is part of the msgpack-rpc spec. At least the Ruby msgpack-rpc library
does not seem to support it:
https://github.com/msgpack-rpc/msgpack-rpc-ruby/blob/master/lib/msgpack/rpc/transport/tcp.rb#L150-L158
API metadata object ~
API clients exist to hide msgpack-rpc details. The API metadata object
contains information that makes this task easier (see also |rpc-types|):
- The "version" key contains the Nvim version, API level, and API
backwards-compatibility level.
- The "functions" key contains a list of metadata objects for individual
functions.
- Each function metadata object has |rpc-types| information about the return
value and parameters. These can be used for generating strongly-typed APIs
in static languages.
- Container types may be decorated with type/size constraints, e.g.
ArrayOf(Buffer) or ArrayOf(Integer, 2). This can be useful to generate
even more strongly-typed APIs.
- Functions that are considered to be methods that operate on instances of
Nvim special types (msgpack EXT) will have the `"method"` attribute set to
`true`. The receiver type is the type of the first argument. The method
names are prefixed with `nvim_` plus a shortened type name, e.g.
`nvim_buf_get_lines` represents the `get_lines` method of a Buffer instance.
- Global functions have `"method"` set to `false` and are prefixed with just
`nvim_`, e.g. `nvim_get_buffers`.
So for an object-oriented language, an API client contains the classes
representing Nvim special types, and the methods of each class could be
defined by stripping the prefix for the type as defined in the `types` metadata
(this will always be the first two "_"-separated parts of the function name).
There could also be a singleton Vim class with methods where the `nvim_`
prefix is stripped off.
==============================================================================
5. Types *rpc-types*
The Nvim C API uses custom types for all functions. |api-types|
At the RPC layer, types form two groups:
- Basic types that map natively to msgpack (and have a default
representation in most msgpack-supported programming languages)
- Special Nvim types that map to msgpack EXT with custom type codes.
Basic types ~
Nil -> msgpack nil
Boolean -> msgpack boolean
Integer (signed 64-bit integer) -> msgpack integer
Float (IEEE 754 double precision) -> msgpack float
String -> msgpack string
Array -> msgpack array
Dictionary -> msgpack map
Note: in function calls, empty Array is accepted for Dictionary parameter.
Special types (msgpack EXT) ~
Buffer -> enum value kObjectTypeBuffer
Window -> enum value kObjectTypeWindow
Tabpage -> enum value kObjectTypeTabpage
API functions expecting one of the special EXT types may be passed an integer
instead, but not another EXT type. E.g. Buffer may be passed as an integer but
not as a Window or Tabpage. The EXT object data is the object id encoded as
a msgpack integer: For buffers this is the |bufnr()| and for windows the
|window-ID|. For tabpages the id is an internal handle, not the tabpage
number.
To determine the type codes of the special EXT types, inspect the `types` key
of the |api-metadata| at runtime. Example JSON representation: >
"types": {
"Buffer": {
"id": 0,
"prefix": "nvim_buf_"
},
"Window": {
"id": 1,
"prefix": "nvim_win_"
},
"Tabpage": {
"id": 2,
"prefix": "nvim_tabpage_"
}
}
Even for statically compiled clients it is good practice to avoid hardcoding
the type codes, because a client may be built against one Nvim version but
connect to another with different type codes.
vim:tw=78:ts=8:ft=help:norl:
This document was merged into |api.txt| and |develop.txt|.

2
runtime/doc/starting.txt
View File

@ -350,7 +350,7 @@ argument.
*--embed*
--embed Use stdin/stdout as a msgpack-RPC channel, so applications can
embed and control Nvim via the |rpc-api|.
embed and control Nvim via the RPC |API|.
Waits for the client ("embedder") to call |nvim_ui_attach()|
before sourcing startup files and reading buffers, so that UIs

2
runtime/doc/ui.txt
View File

@ -4,7 +4,7 @@
NVIM REFERENCE MANUAL
Nvim UI protocol *ui*
Nvim UI protocol *UI* *ui*
Type |gO| to see the table of contents.