Skip to content

Commit 4822fe8

Browse files
bladyjokerbladyjoker
authored
Merge pull request #17 from mlabs-haskell/bladyjoker/documentation
Catalyst Milestone 1 Work PR
2 parents f325ac2 + 84bc01e commit 4822fe8

File tree

18 files changed

+1766
-160
lines changed

18 files changed

+1766
-160
lines changed

‎.markdownlint.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,5 @@ default: true
22
MD013: false
33
MD024: false
44
MD033: false
5+
MD041: false
6+
MD022: false

‎README.md‎

Lines changed: 47 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,52 @@
22

33
## Introduction
44

5+
_LambdaBuffers_ is a schema language (similar to ProtoBuffers, ADL, ASN.1, JSON
6+
Schema, etc.) and associated code generation toolkit. The goal of this project
7+
is to provide developers tools to define algebraic data types in a
8+
language-agnostic format such that shared data types can be declared in one
9+
place while maintaining compatibility across a plethora of supported languages.
10+
11+
Users may refer to the [comparison matrix](./docs/comparison-matrix.md) for an
12+
in-depth comparison of LambdaBuffers' features against the feature-set of other
13+
popular schema-languages.
14+
15+
At a glance, you may wish to choose LambdaBuffers instead of one of its
16+
competitors if your project requires:
17+
18+
1. _Parameterized Data Types_ (aka. type functions): Unlike ProtoBuffers or
19+
JSON Schema, LambdaBuffers allows users to define algebraic data types which
20+
take type variable arguments. If your project's domain is most accurately
21+
represented by parameterized data types, LamdaBuffers may be a good choice
22+
for your needs.
23+
24+
2. _Opaque Types_: Almost every competing schema language provides users a
25+
fixed set of builtin or primitive types, which are handled in a special
26+
manner by the code generation and cannot be extended. LambdaBuffers, by
27+
contrast, allows users to add their own builtin types and extend the
28+
existing code generation framework to handle those builtins in a manner
29+
intended by the users. There are no _special_ primitive types in
30+
LambdaBuffers; a user-defined primitive type is defined in exactly the same
31+
way (i.e. as an `opaque` type) as a LambdaBuffers "builtin".
32+
33+
3. _Typeclass Support_: While nearly every schema language supports generating
34+
type definitions in supported target languages, to our knowledge no schema
35+
language supports generating commonly used functions that operate on those
36+
types. Unlike other schema languages, LambdaBuffers supports code generation
37+
for _typeclass instances_ (or the equivalent in languages that lack support
38+
for typeclasses) to reduce the amount of boilerplate required to
39+
productively make use of the generated types. While LambdaBuffers is still a
40+
work-in-progress, we expect that, upon completion, an extensive test suite
41+
will provide a high degree of assurance that the instances/methods generated
42+
by the LamdaBuffers code generator behave identically.
43+
544
## Documentation
645

7-
- [Compiler](./docs/compiler.md)
46+
- [Design](./docs/design.md),
47+
- [Compiler](./docs/compiler.md),
48+
- [Codegen](./docs/codegen.md),
49+
- [Comparison matrix](./docs/comparison-matrix.md),
50+
- [User feedback](.docs/feedback)
851

952
## Getting started
1053

@@ -48,7 +91,9 @@ trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDS
4891

4992
### Building and development
5093

51-
To facilitate seamlessly moving between directories and associated Nix development shells we use [direnv](https://direnv.net) and [nix-direnv](https://github.com/nix-community/nix-direnv):
94+
To facilitate seamlessly moving between directories and associated Nix
95+
development shells we use [direnv](https://direnv.net) and
96+
[nix-direnv](https://github.com/nix-community/nix-direnv):
5297

5398
To install both using `nixpkgs`:
5499

‎_typos.toml‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,6 @@
11
[default.extend-words]
2-
substituters = "substituters"
2+
substituters = "substituters"
3+
4+
[type.pdf]
5+
extend-glob = ["*.pdf"]
6+
check-file = false

‎docs/codegen.md‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# LambdaBuffers Codegen
2+
3+
NOTE: The implementation of the code generation framework is still in early
4+
stages and will likely undergo substantial changes as development
5+
continues. This document serves to outline general principles that any
6+
implementation of the LambdaBuffers code generation framework ought to adhere
7+
to.
8+
9+
## Requirements
10+
11+
1. Modular & reusable components
12+
2. Ergonomic interface
13+
3. Extensible to new opaque types
14+
4. Extensible to new type classes
15+
16+
### Modular & Reusable Components
17+
18+
Because the code generation modules for each target language will almost
19+
certainly constitute the bulk of the final LambdaBuffers codebase, it is
20+
essential that components of the code generation framework be as modular and
21+
reusable as is practicable.
22+
23+
Although each target language has its own distinct syntax and semantics, many
24+
syntactic forms exist in multiple target languages. For example, Haskell and
25+
PureScript both use a comma-separated set enclosed in square brackets
26+
(e.g. `[1,2,3]`) to represent a `List _/[_]` value, while Rust uses a similar
27+
form (in conjunction with a macro, e.g. `vec![1,2,3]`) to represent a
28+
`Vector<_>` value. To reduce redundant and superfluous code, common syntactic
29+
patterns such at this should be abstracted out into a set of functions that can
30+
be reused across languages.
31+
32+
### Ergonomic Interface
33+
34+
While the LambdaBuffers team will support a finite set of specific target
35+
languages, adding support for an additional language should be as painless as
36+
possible (ceteris paribus) to encourage and facilitate open source contributions
37+
by third parties. A basic set of tools which can be used to write code
38+
generation modules for any target language should be developed, and all code
39+
generation modules written by the LambdaBuffers team should employ those tools
40+
(in order to provide a robust set of examples for future contributors, among
41+
other benefits).
42+
43+
### Extensible to New Opaque Types
44+
45+
Users and contributors should be able to easily extend the default set of
46+
supported opaque types to support additional opaque types. In the context of
47+
code generation, this means: Users should have the ability to specify the target
48+
type for a given opaque type in the target language (including the package or
49+
module that contains the target type if the target type is not part of the
50+
language's standard library).
51+
52+
Because type class instances must be derived structurally, and because an opaque
53+
type is by definition a type with no visible internal structure, users should be
54+
provided with an ergonomic interface for noting the presence of a type class
55+
instance for an opaque type's target type in a particular language (if the
56+
instance exists in the standard library), or for referring to the module where
57+
such an instance is located (if the instance is defined in a library or by the
58+
user in one of their own modules).
59+
60+
### Extensible to New Type Classes
61+
62+
Users and contributors should be able to easily extend the default set of
63+
supported type classes to support additional type classes and facilitate the
64+
derivation of instances for newly defined classes.
65+
66+
In practice, the type class code generation machinery must consist of two
67+
distinct parts: First, a set of deriving rules which correspond to instances
68+
that already exist (in some module or in a language's standard
69+
library/prelude). Second, code generation functions that operate on user-defined
70+
types which satisfy those rules.
71+
72+
Each of these parts should be ergonomic, well-documented, and the implementation
73+
of the default set of supported type classes ought to be thoroughly commented in
74+
order that users have a diverse set of real examples to work from.

‎docs/command-line-interface.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# LambdaBuffers Command Line Interface
2+
3+
LambdaBuffers consists of three runtime command line interface components:
4+
5+
- [lambda-buffers-frontend-cli](lambda-buffers-frontend-cli)
6+
- [lambda-buffers-compiler-cli](lambda-buffers-compiler-cli)
7+
- [lambda-buffers-codegen-cli](todo-link)
8+
9+
The *Frontend* CLI orchestrates work between the user and the *Compiler* and
10+
*Codegen* components.
11+
12+
It's desirable to have both the *Compiler* CLI and the *Codegen* CLI subject to
13+
a strict API with a specified set of flags to enable CLI implementation from
14+
various sources. This would be especially helpful with *Codegen* modules that
15+
bring about support for new targets, opaque types and typeclasses.
16+
17+
<!-- TODO(bladyjoker): Complete this chapter -->

‎docs/comparison-matrix.md‎

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
<!-- markdownlint-disable-file -->
2+
3+
# Comparison Matrix
4+
5+
Legend:
6+
7+
- 🟢 Available (grading added in some cases spanning: Bad, Average, Good, Excellent)
8+
- 🟡 In development
9+
- 🔵 Potential future feature
10+
- 🔴 Not currently available
11+
- ❔ Not clear
12+
13+
| **Feature** | **Proto Buffers** | **ADL** | **JSON Schema** | **Lambda Buffers** | **CDDL** | **ASN.1** |
14+
|--------------------------------------------------------|-------------------|--------------|-----------------|--------------------|----------|--------------|
15+
| Sum types | 🟢 | 🟢 | 🔴 | 🟢 | 🟢 | 🟢 |
16+
| Record types | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
17+
| Product types | 🔴 | 🔴 | 🔴 | 🟢 | ❔ | 🔴 |
18+
| Recursive types | 🟢 | 🟢 | 🔴 | 🟢 | 🟢 | ❔ |
19+
| Parameterized types (generic types) | 🔴 | 🟢 | 🔴 | 🟢 | 🟢 | 🔴 |
20+
| Type annotations/constraints | 🟢 | 🟢 | 🟢 | 🔵 | 🟢 | 🟢 |
21+
| Add new builtin types | 🔴 | 🟢 | 🔴 | 🟢 | 🔴 | 🔴 |
22+
| Add new type semantics (e.g. different encodings) | 🟢 | 🟢 | 🔴 | 🟢 | 🔴 | 🟢 |
23+
| Manage type semantics (at language level) | 🔴 | 🔴 | 🔴 | 🟢 | 🔴 | 🔴 |
24+
| Codegen support | 🟢 (Excellent) | 🟢 (Average) | 🟢 (Excellent) | 🟡 | 🟢 (Bad) | 🟢 (Average) |
25+
| DevOps tooling - build system integration | 🟢 | 🔴 | ❔ | 🟡 | 🔴 | 🔴 |
26+
| Documentation tooling | 🟢 | 🔴 | 🟢 | 🔵 | 🔴 | ❔ |
27+
| Formatting, linting, and development environment tools | 🟢 | 🔴 | 🟢 | 🟢 | 🔴 | 🔴 |
28+
| Language checker API | 🟢 | 🔴 | 🟢 | 🟢 | 🔴 | 🔴 |
29+
| Codegen API | 🟢 | 🟢 | 🔴 | 🟢 | 🔴 | 🔴 |
30+
| Language specification | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
31+
| Backwards compatibility strategy | 🟢 | 🔴 | 🔴 | 🔴 | 🔴 | 🔴 |
32+
33+
## Descriptions
34+
35+
### Sum Types
36+
37+
Types of the form `Time = Present | Past | Future`, which allow a type do be
38+
constructed by one of many variants. Think Rust's `enums`.
39+
40+
### Product Types
41+
42+
Types of the form `Person = MkPerson Age Name`, where `MkPerson` is of Kind
43+
`Type->Type->Type`. Product types combine multiple elements into one data type
44+
without tagging the elements.
45+
46+
### Record Types
47+
48+
Types of the form `Person = MkPerson { age :: Age, name :: Name }`. Record types
49+
are similar to `structs` in most programming languages.
50+
51+
### Recursive Types
52+
53+
Recursive types are defined by the presence of the LHS type in its RHS
54+
definition. A classic example is:
55+
56+
```text
57+
List a = Nil | Cons a (List a)
58+
^^^^^^ ^^^^^^
59+
```
60+
61+
### Parameterized Types (Generics)
62+
63+
Type functions allow for the introduction of type variables in the LHS definition
64+
of the term - creating a parametrised type definition. The classic example is
65+
`Maybe a` which is the equivalento of `Option <A>` in rust:
66+
67+
```text
68+
Maybe a = Nothing | Just a
69+
```
70+
71+
Using the above type definition we can now define another type that uses `Maybe`
72+
and instantiates it to use `Integer`
73+
74+
```text
75+
Time_Saved_via_LambdaBuffers = Maybe Integer
76+
```
77+
78+
### Type Annotations / Constraints
79+
80+
There exists a system of constraining or further annotating types - enriching
81+
the type's specification.
82+
83+
### Add New Built-in Types
84+
85+
Refer to [design document](design.md#extensible-to-new-types).
86+
87+
### Add New Type Semantics
88+
89+
Refer to the [design document](design.md#extensible-to-new-semantics).
90+
91+
### Manage Type Semantics (at Language Level)
92+
93+
Refer to the [design document](design.md#expressive-semantics-annotation)..
94+
95+
### Codegen Support
96+
97+
Codegen support relates to the language being able to generate types for other
98+
programming languages.
99+
100+
### DevOps Tooling - Build System Integration
101+
102+
The framework/language provides a seamless way of integrating with normal build
103+
tools and systems (eg. Bazel, Nix, etc.).
104+
105+
### Documentation Tooling
106+
107+
The language can generate human readable documentation in an easy to share and
108+
view format. For example HTML, or Markdown.
109+
110+
### Formatting, Linting, and Development Environment Tools
111+
112+
Tools that allow formatting, linting, and automating standardisation of the
113+
language specific files.
114+
115+
### Language Checker API
116+
117+
The language checker component exposes an API to interface with itself in a
118+
language agnostic manner.
119+
120+
### Codegen API
121+
122+
The language codegen component exposes an API to interface with itself in a
123+
language agnostic manner.
124+
125+
### Language Specification
126+
127+
There exists a well defined language specification document.
128+
129+
### Backwards Compatibility Strategy
130+
131+
The language makes certain backwards compatibility guarantees between versions of
132+
the same type definition.
133+
134+
## References
135+
136+
- https://json-schema.org/implementations.html
137+
- https://www.rfc-editor.org/rfc/rfc8610
138+
- https://github.com/timbod7/adl
139+
- https://www.itu.int/en/ITU-T/asn1/Pages/introduction.aspx
140+
- https://protobuf.dev/
141+
- https://github.com/dcSpark/cddl-codegen

0 commit comments

Comments
 (0)