Skip to content

Commit 756d2e3

Browse files
NoarkhhNoarkhh
committed
Finish first version of the guide
1 parent 5fa0b6a commit 756d2e3

File tree

1 file changed

+177
-6
lines changed

1 file changed

+177
-6
lines changed

‎guides/useful_concepts/pads.md‎

Lines changed: 177 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,181 @@
11
# Everything about pads
22

3-
When developing some intuition about the structure of pipelines the pads are
3+
When developing intuition about the structure of pipelines pads are
44
something that can't be ignored. If you think about elements and bins as some
5-
sort of containers in which some processing happens, then pads are the parts
6-
that with these tanks are connected with. A pad of one element can only connect
7-
to a single pad of other element and only once two pads are connected the media
8-
can flow through them.
5+
sort of containers or boxes in which processing happens, then pads are the parts
6+
that with these containers are connected with. There are some constraints
7+
regarding pads:
98

10-
`t:Membrane.Pad.element_spec/0`
9+
* A pad of one element can only connect to a single pad of other element and
10+
only once two pads are connected communication through them can happen.
11+
* One pad needs to be an input pad, and the other an output pad.
12+
* The accepted stream formats of the pads need to match.
13+
14+
When looking at the insides of elements, the pads are their main way to
15+
communicate with other elements in the pipeline. When an element receives
16+
something from another one (e.g. a buffer or stream format), it receives it
17+
from a pad. The reference to this pad is then also available as an argument
18+
of the callback that handles the received thing. For example an invocation
19+
of the [following callback](`c:Membrane.Element.WithInputPads.handle_buffer/4`)
20+
would mean that a buffer `buffer` has arrived on a pad `some_pad`:
21+
22+
```elixir
23+
@impl true
24+
def handle_buffer(some_pad, buffer, context, state) do
25+
... ^^^^^^^^
26+
end
27+
```
28+
29+
When an element wants to send something to another element in the
30+
pipeline, most likely it should send it on a pad that's connected to it. It
31+
can do that by using the pad reference in actions that send things, for example
32+
returning the following [buffer action](`t:Membrane.Element.Action.buffer/0`)
33+
from a callback would mean that a buffer `buffer` will be sent on a pad `some_pad`:
34+
35+
```elixir
36+
@impl true
37+
def some_callback(...) do
38+
...
39+
{[buffer: {some_pad, buffer}], state}
40+
end ^^^^^^^^
41+
```
42+
43+
## Defining pads
44+
45+
To define what pads an element will have and how they'll behave we use
46+
[`def_input_pad/2`](`Membrane.Element.WithInputPads.def_input_pad/2`)
47+
and [`def_output_pad/2`](`Membrane.Element.WithOutputPads.def_output_pad/2`) macros.
48+
Input pads can only be defined for Sinks, Filters and Endpoints, and output
49+
pads can only be defined for Sources, Filters and Endpoints.
50+
The first argument for these macros is a name, which then will be used to
51+
identify the pads. The second argument is a `t:Membrane.Pad.element_spec/0`
52+
keyword list, which is used to define how this pad will work. An option we'll
53+
now focus on is [`availability`](`t:Membrane.Pad.availability/0`), which
54+
determines if the pad is _static_ or _dynamic_.
55+
56+
### Static pads
57+
58+
Static pads are pretty straightforward - when a static pad is defined there
59+
will always be exactly one instance of this pad and it's referenced by it's
60+
name.
61+
62+
#### File Source Example
63+
64+
Example of an element with only static pads is a [File Source](https://hexdocs.pm/membrane_file_plugin/Membrane.File.Source.html).
65+
This element reads contents of a file and sends them in batches through a static
66+
output pad. The content of the buffers sent by this element is unknown - the file
67+
that's being read can contain anything - so this pad has `:accepted_format` set to
68+
`%RemoteStream{type: :bytestream}`. That means that any stream format that
69+
matches on this struct can be sent on the output pad and this fact has to be
70+
accounted for when connecting an element after the source.
71+
72+
A pipeline spec with a file source passing buffers to a MP4 demuxer could look
73+
like this:
74+
75+
```elixir
76+
@impl true
77+
def handle_init(_context, state) do
78+
spec =
79+
child(:source, %Membrane.File.Source{location: "my_file.mp4"})
80+
|> via_out(:output)
81+
|> via_in(:input)
82+
|> child(:mp4_demuxer, Membrane.MP4.ISOM.Demuxer)
83+
84+
{[spec: spec], state}
85+
end
86+
```
87+
88+
This spec will connect a pad named `:output` of the source to a pad named
89+
`:input` of the demuxer. However this can be shortened - if an output pad is
90+
called `:output` or an input pad is called `:input`, their respective
91+
[`via_in/3`](`Membrane.ChildrenSpec.via_in/3`) and
92+
[`via_out/3`](`Membrane.ChildrenSpec.via_out/3`) calls can be omitted and
93+
Membrane will automatically recognize and connect them:
94+
95+
```elixir
96+
@impl true
97+
def handle_init(_context, state) do
98+
spec =
99+
child(:source, %Membrane.File.Source{location: "my_file.mp4"})
100+
|> child(:mp4_demuxer, Membrane.MP4.ISOM.Demuxer)
101+
102+
{[spec: spec], state}
103+
end
104+
```
105+
106+
### Dynamic pads
107+
108+
Dynamic pads are a bit more complex. They're used when the amount of pads of
109+
given type is variable - dependent on the processed stream or external factors.
110+
The creation of these pads is controlled by the parent of the element - if a
111+
[`:spec`](`t:Membrane.Pipeline.Action.spec/0`) action linking the dynamic pad is
112+
being executed, then the pad is created dynamically and the element needs to
113+
handle this, in most cases with
114+
[`handle_pad_added/3`](`c:Membrane.Element.Base.handle_pad_added/3`).
115+
116+
Another thing that's different are the pad references. The pad's name can't just
117+
be used as the pad's reference, because it wouldn't be unique. Dynamic pads are
118+
identified by [`Pad.ref/2`](`Membrane.Pad.ref/2`), that takes the pad's
119+
name and some unique reference as arguments. The result is a unique pad reference
120+
that is also associated with a given pad's specification through it's name.
121+
122+
#### MP4 Demuxer Example
123+
124+
An example of an element using dynamic pads is an
125+
[MP4 Demuxer](https://hexdocs.pm/membrane_mp4_plugin/Membrane.MP4.Demuxer.ISOM.html).
126+
This element has a input pad, from which it receives contents of a MP4
127+
container, and output pads, on which it'll send the different tracks
128+
that were in the container. The input pad can be static, however MP4 containers can
129+
have different numbers and kinds of tracks, so the output pad needs to be dynamic.
130+
131+
We'll consider the case when we don't have any prior information about the
132+
tracks in this MP4 container. Because of this, the parent pipeline or bin of this
133+
demuxer won't initially know how many pads should be connected. To solve this
134+
problem the demuxer will identify the tracks in the incoming stream and send a
135+
message to it's parent in the form of
136+
[`{:new_tracks, [{track_id :: integer(), content :: struct()}]}`](https://hexdocs.pm/membrane_mp4_plugin/Membrane.MP4.Demuxer.ISOM.html#t:new_tracks_t/0).
137+
The list contains a list of tuples corresponding to tracks, where the first
138+
element is a track id and will be used to identify corresponding pad,
139+
and the second a stream format contained in the track.
140+
141+
Initially the parent will only create the elements before and including the
142+
demuxer:
143+
144+
```elixir
145+
@impl true
146+
def handle_init(_context, state) do
147+
spec =
148+
child(:source, %Membrane.File.Source{location: "my_file.mp4"})
149+
|> child(:mp4_demuxer, Membrane.MP4.ISOM.Demuxer)
150+
151+
{[spec: spec], state}
152+
end
153+
```
154+
155+
The source will start providing the demuxer the MP4 container content,
156+
from which the demuxer will identify tracks and notify it's parent
157+
about them. The parent now has to connect an output pad of the demuxer for each
158+
track received, which can look like this:
159+
160+
```elixir
161+
@impl true
162+
def handle_child_notification({:new_tracks, tracks}, :mp4_demuxer, _context, state) do
163+
spec =
164+
Enum.map(tracks, fn {id, format} ->
165+
get_child(:mp4_demuxer)
166+
|> via_out(Pad.ref(:output, id))
167+
|> ...
168+
end)
169+
170+
{[spec: spec], state}
171+
end
172+
```
173+
174+
The elements that the output pads will be linked to should be based on what stream
175+
format is in `format` variable - different formats require different approaches.
176+
177+
After this spec is returned, the demuxer will now have the
178+
[`handle_pad_added/3`](`c:Membrane.Element.Base.handle_pad_added/3`) callback
179+
called for each new connected pad with pad reference of
180+
`Pad.ref(:output, track_id)`. It will now know that these pads are connected and
181+
ready to pass buffers forward.

0 commit comments

Comments
 (0)