update documentation (use links in README, lua, note about __init__)

bfredl · bfredl · commit a437f976131d · 2018-04-29T09:46:55.000+02:00
diff --git a/README.md b/README.md
@@ -1,10 +1,11 @@
-### Python client to [Neovim](https://github.com/neovim/neovim)
+### Pynvim: Python client to [Neovim](https://github.com/neovim/neovim)
 
 [![Build Status](https://travis-ci.org/neovim/python-client.svg?branch=master)](https://travis-ci.org/neovim/python-client)
+[![Documentation Status](https://readthedocs.org/projects/pynvim/badge/?version=latest)](http://pynvim.readthedocs.io/en/latest/?badge=latest)
 [![Scrutinizer Code Quality](https://scrutinizer-ci.com/g/neovim/python-client/badges/quality-score.png?b=master)](https://scrutinizer-ci.com/g/neovim/python-client/?branch=master)
 [![Code Coverage](https://scrutinizer-ci.com/g/neovim/python-client/badges/coverage.png?b=master)](https://scrutinizer-ci.com/g/neovim/python-client/?branch=master)
 
-Implements support for python plugins in Nvim. Also works as a library for
+Pynvim implements support for python plugins in Nvim. It also works as a library for
 connecting to and scripting Nvim processes through its msgpack-rpc API.
 
 #### Installation
@@ -36,96 +37,23 @@ pip3 install .
 
 #### Python Plugin API
 
-Neovim has a new mechanism for defining plugins, as well as a number of
-extensions to the python API. The API extensions are accessible no matter if the
-traditional `:python` interface or the new mechanism is used, as discussed
-below.
-
-* `vim.funcs` exposes vimscript functions (both builtin and global user defined
-  functions) as a python namespace. For instance to set the value of a register
-  ```
-  vim.funcs.setreg('0', ["some", "text"], 'l')
-  ```
-
-* `vim.api` exposes nvim API methods. For instance to call `nvim_strwidth`,
-  ```
-  result = vim.api.strwidth("some text")
-  ```
-  Note the initial `nvim_` is not included. Also, object methods can be called
-  directly on their object,
-  ```
-  buf = vim.current.buffer
-  len = buf.api.line_count()
-  ```
-  calls `nvim_buf_line_count`. Alternatively msgpack requests can be invoked
-  directly,
-  ```
-  result = vim.request("nvim_strwith", "some text")
-  len = vim.request("nvim_buf_line_count", buf)
-  ```
-
-* The API is not thread-safe in general. However, `vim.async_call` allows a
-  spawned thread to schedule code to be executed on the main thread. This method
-  could also be called from `:python` or a synchronous request handler, to defer
-  some execution that shouldn't block nvim.
-  ```
-  :python vim.async_call(myfunc, args...)
-
-  ```
-  Note that this code will still block the plugin host if it does long-running
-  computations. Intensive computations should be done in a separate thread (or
-  process), and `vim.async_call` can be used to send results back to nvim.
-
-* Some methods accept an `async_` keyword argument: `vim.eval`, `vim.command`,
-  `vim.request` as well as the `vim.funcs` and `vim.api` wrappers.  When
-  `async_=True` is passed the client will not wait for nvim to complete the
-  request (which also means that the return value is unavailable).
-
-#### Remote (new-style) plugins
-
-Neovim allows python3 plugins to be defined by placing python files or packages
-in `rplugin/python3/` (in a runtimepath folder). Python2 rplugins are also
-supported and placed in `rplugin/python/`, but are considered deprecated.
-Further added library features will only be available on python3. Rplugins follow
-the structure of this example:
+Pynvim supports python _remote plugins_ (via the language-agnostic Nvim rplugin
+interface), as well as _Vim plugins_ (via the `:python[3]` interface). Thus when
+pynvim is installed Neovim will report support for the `+python[3]` Vim feature.
 
-```python
-import neovim
-
-@neovim.plugin
-class TestPlugin(object):
-
-    def __init__(self, nvim):
-        self.nvim = nvim
-
-    @neovim.function("TestFunction", sync=True)
-    def testfunction(self, args):
-        return 3
-
-    @neovim.command("TestCommand", range='', nargs='*')
-    def testcommand(self, args, range):
-        self.nvim.current.line = ('Command with args: {}, range: {}'
-                                  .format(args, range))
-
-    @neovim.autocmd('BufEnter', pattern='*.py', eval='expand("<afile>")', sync=True)
-    def on_bufenter(self, filename):
-        self.nvim.out_write("testplugin is in " + filename + "\n")
-```
+The rplugin interface allows plugins to handle vimL function calls as well as
+defining commands and autocommands, and such plugins can operate asynchronously
+without blocking nvim.  For details on the new rplugin interface, 
+see the [Remote Plugin](http://pynvim.readthedocs.io/en/latest/usage/remote-plugins.html) documentation.
 
-If `sync=True` is supplied nvim will wait for the handler to finish (this is
-required for function return values), but by default handlers are executed
-asynchronously.
+Pynvim defines some extensions over the vim python API:
 
-Normally async handlers (`sync=False`, the default) are blocked while a
-synchronous handler is running. This ensures that async handlers can call
-requests without nvim confusing these requests with requests from a synchronous
-handler. To execute an asynchronous handler even when other handlers are
-running, add `allow_nested=True` to the decorator. The handler must then not
-make synchronous nvim requests, but it can make asynchronous requests, i e
-passing `async_=True`.
+* Builtin and plugin vimL functions are available as `nvim.funcs`
+* API functions are available as `vim.api` and for objects such as `buffer.api`
+* Lua functions can be defined using `vim.exec_lua` and called with `vim.lua`
+* Support for thread-safety and async requests.
 
-You need to run `:UpdateRemotePlugins` in nvim for changes in the specifications
-to have effect. For details see `:help remote-plugin` in nvim.
+See the [Python Plugin API](http://pynvim.readthedocs.io/en/latest/usage/python-plugin-api.html) documentation for usage of this new functionality.
 
 #### Development
 
@@ -134,53 +62,8 @@ If you change the code, you need to run
 pip2 install .
 pip3 install .
 ```
-for the changes to have effect. Alternatively you could execute neovim
-with the `$PYTHONPATH` environment variable
-```
-PYTHONPATH=/path/to/python-client nvim
-```
-But note this is not completely reliable as installed packages can appear before
-`$PYTHONPATH` in the python search path.
-
-You need to rerun this command if you have changed the code, in order for nvim
-to use it for the plugin host.
-
-To run the tests execute
-
-```sh
-pytest
-```
-
-This will run the tests in an embedded instance of nvim.
-If you want to test a different version than `nvim` in `$PATH` use
-```sh
-NVIM_CHILD_ARGV='["/path/to/nvim", "-u", "NONE", "--embed"]' pytest
-```
-
-Alternatively, if you want to see the state of nvim, you could use
-
-```sh
-export NVIM_LISTEN_ADDRESS=/tmp/nvimtest
-xterm -e "nvim -u NONE"&
-pytest
-```
-
-But note you need to restart nvim every time you run the tests! Substitute your
-favorite terminal emulator for `xterm`.
-
-#### Troubleshooting
-
-You can run the plugin host in nvim with logging enabled to debug errors:
-```
-NVIM_PYTHON_LOG_FILE=logfile NVIM_PYTHON_LOG_LEVEL=DEBUG nvim
-```
-As more than one python host process might be started, the log filenames take
-the pattern `logfile_pyX_KIND` where `X` is the major python version (2 or 3)
-and `KIND` is either "rplugin" or "script" (for the `:python[3]`
-script interface).
-
-If the host cannot start at all, the error could be found in `~/.nvimlog` if
-`nvim` was compiled with logging.
+for the changes to have effect. For instructions of testing and troubleshooting,
+see the [development](http://pynvim.readthedocs.io/en/latest/development.html) documentation.
 
 #### Usage through the python REPL
 
diff --git a/docs/development.rst b/docs/development.rst
@@ -19,18 +19,18 @@ in order for Neovim to use it for the plugin host.
 
 To run the tests execute::
 
-    nosetests
+    pytest
 
 This will run the tests in an embedded instance of Neovim.
 If you want to test a different version than ``nvim`` in ``$PATH`` use::
 
-    NVIM_CHILD_ARGV='["/path/to/nvim", "-u", "NONE", "--embed"]' nosetests
+    NVIM_CHILD_ARGV='["/path/to/nvim", "-u", "NONE", "--embed"]' pytest
 
 Alternatively, if you want to see the state of nvim, you could use::
 
     export NVIM_LISTEN_ADDRESS=/tmp/nvimtest
     xterm -e "nvim -u NONE"&
-    nosetests
+    pytest
 
 But note you need to restart Neovim every time you run the tests!
 Substitute your favorite terminal emulator for ``xterm``.
diff --git a/docs/usage/python-plugin-api.rst b/docs/usage/python-plugin-api.rst
@@ -6,17 +6,7 @@ as well as a number of extensions to the python API.
 The API extensions are accessible no matter if the traditional ``:python`` interface or the new mechanism is used,
 as discussed on :doc:`remote-plugins`.
 
-``vim.funcs``
--------------
-
-Exposes vimscript functions (both builtin and global user defined functions) as a python namespace.
-For instance to set the value of a register:
-
-.. code-block:: python
-
-   vim.funcs.setreg('0', ["some", "text"], 'l')
-
-``vim.api``
+Nvim API methods: ``vim.api``
 -----------
 
 Exposes Neovim API methods.
@@ -32,15 +22,83 @@ Also, object methods can be called directly on their object:
 .. code-block:: python
 
    buf = vim.current.buffer
-   len = buf.api.line_count()
+   length = buf.api.line_count()
 
 calls ``nvim_buf_line_count``.
 Alternatively msgpack requests can be invoked directly:
 
 .. code-block:: python
 
    result = vim.request("nvim_strwith", "some text")
-   len = vim.request("nvim_buf_line_count", buf)
+   length = vim.request("nvim_buf_line_count", buf)
+
+Both ``vim.api`` and ``vim.request`` can take an ``async_=True`` keyword argument
+to instead send a msgpack notification. Nvim will execute the API method the
+same way, but python will not wait for it to finish, so the return value is
+unavailable.
+
+Vimscript functions: ``vim.funcs``
+-------------
+
+Exposes vimscript functions (both builtin and global user defined functions) as a python namespace.
+For instance to set the value of a register:
+
+.. code-block:: python
+
+   vim.funcs.setreg('0', ["some", "text"], 'l')
+
+These functions can also take the ``async_=True`` keyword argument, just like API
+methods.
+
+Lua integration
+-----------
+
+Python plugins can define and invoke lua code in Nvim's in-process lua
+interpreter. This is especially useful in asynchronous contexts, where an async
+event handler can schedule a complex operation with many api calls to be
+executed by nvim without interleaved processing of user input or other event
+sources (unless requested).
+
+The recommended usage is the following pattern. First use ``vim.exec_lua(code)``
+to define a module with lua functions:
+
+.. code-block:: python
+
+   vim.exec_lua("""
+      local a = vim.api
+      local function add(a,b)
+          return a+b
+      end
+
+      local function buffer_ticks()
+         local ticks = {}
+         for _, buf in ipairs(a.nvim_list_bufs()) do
+             ticks[#ticks+1] = a.nvim_buf_get_changedtick(buf)
+         end
+         return ticks
+      end
+
+      _testplugin = {add=add, buffer_ticks=buffer_ticks}
+   """)
+
+Alternatively, place the code in ``/lua/testplugin.lua`` under your plugin repo
+root, and use ``vim.exec_lua("_testplugin = require('testplugin')")``.
+In both cases, replace ``testplugin`` with a unique string based on your plugin
+name.
+
+Then, the module can be acessed as ``vim.lua._testplugin``.
+
+.. code-block:: python
+
+    mod = vim.lua._testplugin
+    mod.add(2,3) # => 5
+    mod.buffer_ticks() # => list of ticks
+
+These functions can also take the ``async_=True`` keyword argument, just like API
+methods.
+
+It is also possible to pass arguments directly to a code block. Using
+``vim.exec_lua(code, args...)``, the arguments will be available in lua as ``...``.
 
 Async calls
 -----------
@@ -58,7 +116,8 @@ Note that this code will still block the plugin host if it does long-running com
 Intensive computations should be done in a separate thread (or process),
 and ``vim.async_call`` can be used to send results back to Neovim.
 
-Some methods accept an ``async_`` keyword argument:
-``vim.eval``, ``vim.command``, ``vim.request`` as well as the ``vim.funcs`` and ``vim.api`` wrappers.
-When ``async_=True`` is passed the client will not wait for Neovim to complete the request
-(which also means that the return value is unavailable).
+Some methods accept an ``async_`` keyword argument: ``vim.eval``,
+``vim.command``, ``vim.request`` as well as the ``vim.funcs``, ``vim.api` and
+``vim.lua``` wrappers.  When ``async_=True`` is passed the client will not wait
+for Neovim to complete the request (which also means that the return value is
+unavailable).
diff --git a/docs/usage/remote-plugins.rst b/docs/usage/remote-plugins.rst
@@ -41,9 +41,18 @@ are blocked while a synchronous handler is running.
 This ensures that async handlers can call requests without Neovim confusing these requests with requests from a synchronous handler.
 To execute an asynchronous handler even when other handlers are running,
 add ``allow_nested=True`` to the decorator.
-The handler must then not make synchronous Neovim requests,
+This handler must then not make synchronous Neovim requests,
 but it can make asynchronous requests, i.e. passing ``async_=True``.
 
+.. note::
+
+    Plugins must not invoke API methods in ``__init__`` or global module scope
+    (or really do anything with non-trivial side-effects). A well-behaved rplugin
+    will not start executing until its functionality is requested by the user.
+    Initialize the plugin the first time the user invokes a command, or use an
+    appropriate autocommand, if it e.g. makes sense to automatically start the
+    plugin for a given filetype.
+
 You need to run ``:UpdateRemotePlugins`` in Neovim for changes in the specifications to have effect.
 For details see ``:help remote-plugin`` in Neovim.
 
