Skip to content

Commit 2ec1207

Browse files
doorganmhanberg
andauthored
chore: update README.md and docs (#46)
* chore: update README.md and docs * chore: cleanup CHANGELOG.md * chore: add section for the development server * fix: change wording about what the project is Co-authored-by: Mitchell Hanberg <mitch@mitchellhanberg.com> --------- Co-authored-by: Mitchell Hanberg <mitch@mitchellhanberg.com>
1 parent 15581e7 commit 2ec1207

File tree

6 files changed

+164
-804
lines changed

6 files changed

+164
-804
lines changed

CHANGELOG.md

Lines changed: 0 additions & 479 deletions
Large diffs are not rendered by default.

README.md

Lines changed: 20 additions & 190 deletions
Original file line numberDiff line numberDiff line change
@@ -1,212 +1,42 @@
1-
[![Discord](https://img.shields.io/badge/Discord-5865F3?style=flat&logo=discord&logoColor=white&link=https://discord.gg/FvdkuVyted)](https://discord.gg/FvdkuVyted)
2-
![GitHub Workflow Status (with event)](https://img.shields.io/github/actions/workflow/status/lexical-lsp/lexical/elixir.yml)
3-
4-
<img alt="Lexical logo: Lexi the lynx" src="assets/lexi-logo.svg" width="150" height="200" align="right"/>
5-
61
# Expert
72

8-
Expert is a next-generation language server for the Elixir programming language.
9-
10-
<br><br><br>
11-
12-
### Note: Development on Lexical will slow while we transition to the new combined effort, Expert.
13-
14-
## Features
15-
16-
* Context aware code completion
17-
* As-you-type compilation
18-
* Advanced error highlighting
19-
* Code actions
20-
* Code Formatting
21-
* Go To Definition
22-
* Completely isolated build environment
23-
24-
## What makes Lexical different?
25-
There are a couple things that lexical does differently than other language servers. Let's look at what separates it from
26-
the pack.
27-
28-
#### Architecture
29-
30-
When lexical starts, it boots an erlang virtual machine that runs the language server and its code. It then boots a
31-
separate virtual machine that runs your project code and connects the two via distribution. This provides the following benefits:
32-
33-
* None of lexical's dependencies will conflict with your project. This means that lexical can make use of dependencies to make developing in it easier without having to "vendor" them. It also means that you can use lexical to work on your project, **even if lexical depends on your project**.
34-
* Your project can depend on a different version of elixir and erlang than lexical itself. This means that lexical can make use of the latest versions of elixir and erlang while still supporting projects that run on older versions.
35-
* The build environment for your project is only aware of your project, which enables as-you-type compilation and error reporting.
36-
* In the future, there is a possibility of having the lexical vm instance control multiple projects
37-
38-
#### Ease of contribution
39-
40-
Lexical has been designed to be easy to contribute to and work on. It features:
41-
42-
* A consistent data model that represents the Language Server Protocol and `mix` tasks to generate new Language Server models.
43-
* A clearly defined separation between the language server and project code
44-
* A set of utilities that deal with manipulating code
45-
* A set of unit tests and test cases that make testing new features easy.
46-
47-
#### Focus on developer productivity
48-
49-
Lexical is also built with an eye on increasing developer productivity, and approaches some common features a little bit
50-
differently. For example, Lexical's code completion is _context aware_, which means that if you type `alias MyModule.|`
51-
you will only receive completions for modules and not the names of functions in `MyModule`. This awareness will extend
52-
to other areas, which means:
53-
54-
* You won't see completions for random functions and types in strings. In fact, when extended to string literals, Lexical will only show you completions if you're inside of an interpolation (`"hello there #{na|}'`).
55-
* If you're inside of a struct reference (`%StructModule.|`), you will only see modules listed that define structs, or are the parents of modules that define structs.
56-
57-
Because of this focus, Lexical aims to deliver depth of features rather than breadth of them. We'll likely spend
58-
more time making sure each thing we add works and feels _just right_ rather than adding a whole slew of features
59-
that people mostly won't use.
60-
61-
#### As you type compilation
62-
Because your project is run in a separate virtual machine, we can compile the code that you're working on as you
63-
type. This means you see errors _immediately_ rather than having to wait for a save. The result is you see and
64-
fix typos, warnings, unused variables and a whole host of errors when they occur, which makes your code better,
65-
faster.
3+
Expert is the official language server implementation for the Elixir programming language.
664

675
## Installation
686

69-
Follow the [Detailed Installation Instructions](pages/installation.md)
70-
71-
```
72-
mix package
73-
```
74-
75-
Lexical will now be available in `_build/dev/package/lexical`
76-
77-
If you would like to change the output directory, you can do so with the `--path` option
78-
79-
```
80-
mix package --path /path/to/lexical
81-
```
82-
83-
Lexical will be available in `/path/to/lexical`.
84-
85-
## Development
86-
87-
Lexical is intended to run on any version of Erlang 25+ and Elixir
88-
1.15+. Before beginning development, you should install Erlang
89-
`25.0` and Elixir `1.15.8` and use those versions when you're
90-
building code.
91-
92-
You should also look at the [complete compatibility
93-
matrix](pages/installation.md#caveats) do see which versions are
94-
supported.
7+
You can download Expert from the [releases page](https://github.com/elixir-lang/expert/releases) for your
8+
operating system and architecture.
959

96-
You're going to need a local instance in order to develop lexical, so follow the [Detailed Installation Instructions](pages/installation.md) first.
10+
For editor specific installation instructions, please refer to the [Installation Instructions](pages/installation.md)
9711

98-
Then, install the git hooks with
99-
100-
```
101-
mix hooks
102-
```
103-
104-
These are pre-commit hooks that will check for correct formatting and run credo for you.
105-
106-
After this, you're ready to put together a pull request for Lexical!
107-
108-
#### Benchmarks
109-
110-
The `remote_control` project has a set of benchmarks that measure the speed of various internal functions and data structures. In order to use them, you first need to install [git large file storage](https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage), and then run `git pull`. Benchmarks are stored in the `benchmarks` subdirectory, and can be run via
111-
112-
```
113-
mix benchmark /benchmarks/<benchmark_file>.exs
114-
```
12+
### Nightly Builds
11513

116-
### Logging
14+
If you want to try out the latest features, you can download a nightly build.
11715

118-
When lexical starts up, it creates a `.lexical` directory in the root
119-
directory of a project. Inside that directory are two log files,
120-
`lexical.log` and `project.log`. The `.lexical.log` log file contains
121-
logging and OTP messages from the language server, while the
122-
`project.log` file contains logging and OTP messages from the
123-
project's node. While developing lexical, it's helpful to open up a
124-
terminal and tail both of these log files so you can see any errors
125-
and messages that lexical emits. To do that, run the following in a
126-
terminal while in the project's root directory:
16+
Using the GH CLI, you can run the following command to download the latest nightly build:
12717

12818
```shell
129-
tail -f .lexical/*.log
19+
gh release download nightly --pattern 'expert_linux_amd64'
13020
```
13121

132-
Note: These log files roll over when they reach 1 megabyte, so after a
133-
time, it will be necessary to re-run the above command.
134-
135-
### Debugging
22+
Then point your editor to the downloaded binary.
13623

137-
Lexical supports a debug shell, which will connect a remote shell to a
138-
currently-running language server process. To use it, `cd` into your
139-
lexical installation directory and run
24+
### Building from source
14025

141-
```
142-
./bin/debug_shell.sh <name of project>
143-
```
26+
To build Expert from source, you need Zig `1.14.1` installed on your system.
14427

145-
For example, if I would like to run the debug server for a server
146-
running in your `lexical` project, run:
147-
148-
```
149-
./bin/debug_shell.sh lexical
150-
```
28+
Then you can run the following command:
15129

152-
...and you will be connected to a remote IEx session _inside_ my
153-
language server. This allows you to investigate processes, make
154-
changes to the running code, or run `:observer`.
155-
156-
While in the debugging shell, all the functions in
157-
`Expert.IEx.Helpers` are imported for you, and some common
158-
modules, like `Lexical.Project` and `Lexical.Document` are
159-
aliased.
160-
161-
You can also start the lexical server in interactive mode via
162-
`./bin/start_lexical.sh iex`. Combining this with the helpers that are
163-
imported will allow you to run projects and do completions entirely in
164-
the shell.
165-
166-
*Note*: The helpers assume that all of your projects are in folders that are siblings with your lexical project.
167-
168-
Consider the example shell session:
169-
170-
```
171-
./bin/start_lexical.sh iex
172-
iex(1)> start_project :other
173-
# the project in the ../other directory is started
174-
compile_project(:other)
175-
# the other project is compiled
176-
iex(2)> complete :other, "defmo|"
177-
[
178-
#GenLSP.Structures.CompletionItem<[
179-
detail: "",
180-
insert_text: "defmacro ${1:name}($2) do\n $0\nend\n",
181-
insert_text_format: :snippet,
182-
kind: :class,
183-
label: "defmacro (Define a macro)",
184-
sort_text: "093_defmacro (Define a macro)"
185-
]>,
186-
#GenLSP.Structures.CompletionItem<[
187-
detail: "",
188-
insert_text: "defmacrop ${1:name}($2) do\n $0\nend\n",
189-
insert_text_format: :snippet,
190-
kind: :class,
191-
label: "defmacrop (Define a private macro)",
192-
sort_text: "094_defmacrop (Define a private macro)"
193-
]>,
194-
#GenLSP.Structures.CompletionItem<[
195-
detail: "",
196-
insert_text: "defmodule ${1:module name} do\n $0\nend\n",
197-
insert_text_format: :snippet,
198-
kind: :class,
199-
label: "defmodule (Define a module)",
200-
sort_text: "092_defmodule (Define a module)"
201-
]>
202-
]
30+
```sh
31+
just release-local
20332
```
20433

205-
The same kind of support is available when you run `iex -S mix` in the
206-
lexical directory, and is helpful for narrowing down issues without
207-
disturbing your editor flow.
34+
This will build the Expert binary and place it in the `apps/expert/burrito_out` directory. You can then point your
35+
editor to this binary.
20836

20937
### Other resources
21038

211-
* [Architecture](pages/architecture.md)
212-
* [Glossary](pages/glossary.md)
39+
- [Architecture](pages/architecture.md)
40+
- [Development Guide](pages/development.md)
41+
- [Glossary](pages/glossary.md)
42+
- [Installation Instructions](pages/installation.md)

pages/architecture.md

Lines changed: 11 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -2,32 +2,27 @@
22

33
## Project Structure
44

5-
Lexical is designed to keep your application isolated from lexical's code. Because of this, lexical is structured as an umbrella app, with the following sub-apps:
5+
Expert is designed to keep your application isolated from Expert's code. Because of this, Expert is structured as an umbrella app, with the following sub-apps:
66

7-
* `common`: Contains all code common to the other applications.
8-
* `proto`: Used by `protocol` to generate the Elixir representation of LSP data structures.
9-
* `protocol`: Code related to speaking the language server protocol.
10-
* `remote_control`: The application that's injected into a project's code, which
11-
gives lexical an API to do things in the context of your app.
12-
* `server` The language server itself.
7+
* `forge`: Contains all code common to the other applications.
8+
* `engine`: The application that's injected into a project's code, which
9+
gives expert an API to do things in the context of your app.
10+
* `expert` The language server itself.
1311

14-
Lexical is an umbrella app so we can control how many dependencies the remote control app has. By separating lexical into sub-applications, each is built as a separate archive, and we can pick and choose which of these applications (and their dependencies) are injected into the project's VM, thus reducing how much contamination the project sees. If lexical was a standard application, adding dependencies to lexical would cause those dependencies to appear in the project's VM, which might cause build issues, version conflicts in mix or other inconsistencies.
15-
16-
Since the `remote_control` app only depends on `common`, `path_glob` and `elixir_sense`, only those applications pollute the project's vm. Keeping `remote_control`'s dependencies to a minimum is a design goal of this architecture.
12+
By separating expert into sub-applications, each is built as a separate archive, and we can pick and choose which of these applications (and their dependencies) are injected into the project's VM, thus reducing how much contamination the project sees. If Expert was a standard application, adding dependencies to Expert would cause those dependencies to appear in the project's VM, which might cause build issues, version conflicts in mix or other inconsistencies.
1713

14+
Since the `engine` app only depends on `forge`, `path_glob` and `elixir_sense`, only those applications pollute the project's vm. Keeping `engine`'s dependencies to a minimum is a design goal of this architecture.
1815

1916
## Language Server
2017

21-
The language server (the `server` app) is the entry point to Lexical. When started by the `start_lexical.sh` command, it sets up a [transport](https://github.com/lexical-lsp/lexical/blob/main/apps/expert/lib/lexical/server/transport.ex) that [reads JsonRPC from standard input and writes responses to standard output](https://github.com/lexical-lsp/lexical/blob/main/apps/expert/lib/lexical/server/transport/std_io.ex).
18+
The language server (the `expert` app) is the entry point to Expert. When started, it sets up a transport via GenLSP that reads JsonRPC and responds to it. The default transport is Standard IO, but it can be configured to use TCP.
2219

23-
When a message is received, it is parsed into either a [LSP Request](https://github.com/lexical-lsp/lexical/blob/main/apps/protocol/lib/lexical/protocol/requests.ex) or a [LSP Notification](https://github.com/lexical-lsp/lexical/blob/main/apps/protocol/lib/lexical/protocol/notifications.ex) and then it's handed to the [language server](https://github.com/lexical-lsp/lexical/blob/main/apps/expert/lib/lexical/server.ex) to process.
20+
When a message is received, it is parsed into either a LSP Request or a LSP Notification and then it's handed to the [language server](https://github.com/elixir-lang/expert/blob/main/apps/expert/lib/expert.ex) to process.
2421

25-
The only messages the [lexical server process](https://github.com/lexical-lsp/lexical/blob/main/apps/expert/lib/lexical/server.ex) handles directly are those related to the lifecycle of the language server itself:
22+
The only messages the Expert server process handles directly are those related to the lifecycle of the language server itself:
2623

2724
- Synchronizing document states.
2825
- Processing LSP configuration changes.
2926
- Performing initialization and shutdown.
3027

31-
All other messages are delegated to a _Provider Handler_. This delegation is accomplished by the server process adding the request to the [provider queue](https://github.com/lexical-lsp/lexical/blob/main/apps/expert/lib/lexical/server/provider/queue.ex). The provider queue asks the `Expert.Provider.Handlers.for_request/1` function which handler is configured to handle the request, creates a task for the handler and starts it.
32-
33-
A _Provider Handler_ is just a module that defines a function of arity 2 that takes the request to handle and a `%Expert.Configuration{}`. These functions can reply to the request, ignore it, or do some other action.
28+
All other messages are delegated to a _Provider Handler_. A _Provider Handler_ is a module that defines a function of arity 2 that takes the request to handle and a `%Expert.Configuration{}`. These functions can reply to the request, ignore it, or do some other action.

pages/development.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# Development
2+
3+
## Logging
4+
5+
When expert starts up, it creates a `.expert` directory in the root
6+
directory of a project. Inside that directory are two log files,
7+
`expert.log` and `project.log`. The `.expert.log` log file contains
8+
logging and OTP messages from the language server, while the
9+
`project.log` file contains logging and OTP messages from the
10+
project's node. While developing expert, it's helpful to open up a
11+
terminal and tail both of these log files so you can see any errors
12+
and messages that expert emits. To do that, run the following in a
13+
terminal while in the project's root directory:
14+
15+
```sh
16+
tail -f .expert/*.log
17+
```
18+
19+
Note: These log files roll over when they reach 1 megabyte, so after a
20+
time, it will be necessary to re-run the above command.
21+
22+
## Development server
23+
24+
To start a development server with an interactive shell, you can run the
25+
following command:
26+
27+
```sh
28+
just start
29+
```
30+
31+
This will launch an IEx session, and it will start Expert listening
32+
in the TCP port `9000`.
33+
34+
You will need to configure your editor to connect to the Expert LSP
35+
via TCP at that port. After that, opening you project in your editor
36+
will connect it to the running dev server, and it will terminate it
37+
when you close the editor.
38+
39+
In this dev server you can run `:observer.start()`, and call any
40+
function from Expert to inspect the state of the server, or run
41+
arbitrary code.
42+
43+
Since Expert needs namespacing to work, modules from the `forge`
44+
application will be namespaced as `XPForge`; the same applies for
45+
any module that is shared between the `expert` and `engine`
46+
applications.
47+
48+
## Debugging
49+
50+
Expert supports a debug shell, which will connect a remote shell to a
51+
currently-running language server process. To use it, `cd` into your expert
52+
installation directory and run
53+
54+
```sh
55+
./apps/expert/bin/debug_server.sh <name of project>
56+
```
57+
58+
For example, if I would like to run the debug server for a server running in
59+
your `my_project` project, run:
60+
61+
```sh
62+
./apps/expert/bin/debug_server.sh my_project
63+
```
64+
65+
...and you will be connected to a remote IEx session _inside_ the language
66+
server for `my_project`. This allows you to investigate processes, make changes
67+
to the running code, or run `:observer`.

0 commit comments

Comments
 (0)