Skip to content

Commit 916d3b0

Browse files
committed
Reformat to ml5 docs structure
Also address the highlighted example being too dark, by turning on normalizeDynamically
1 parent 20ac6e1 commit 916d3b0

File tree

2 files changed

+127
-51
lines changed

2 files changed

+127
-51
lines changed

docs/reference/depthestimation.md

Lines changed: 124 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -11,44 +11,36 @@ Get up and running with the [webcam example](https://editor.p5js.org/nasif-co/sk
1111

1212
[DEMO](iframes/depthestimation ":include :type=iframe width=100% height=550px")
1313

14-
## Initialization and options
15-
To get started, create an instance of `ml5.depthEstimation` in your preload function, to allow the model to load
14+
## Examples
15+
- [Webcam](https://editor.p5js.org/nasif-co/sketches/Pep6DjEtD): Show the live depth map of the video captured by the webcam.
16+
- [Video](https://editor.p5js.org/nasif-co/sketches/vifmzXg6o): Generate the depth map of a video file as it plays.
17+
- [Single Image](https://editor.p5js.org/nasif-co/sketches/_TcZofgrt): Depth map of an image using single-shot estimation.
18+
- [Mask Background](https://editor.p5js.org/nasif-co/sketches/Z_1xMhUPl): Showcases how to mask out the background from the depth result.
19+
- [Point Cloud](https://editor.p5js.org/nasif-co/sketches/VbT8hEoDz): Creates a live 3D point cloud visualization of the webcam video.
20+
- [Mesh](https://editor.p5js.org/nasif-co/sketches/X-e1DEZr4): Creates a live 3D mesh geometry of the webcam video.
21+
22+
## Step-by-Step Guide
23+
### Initialization and options
24+
Before starting, make sure you have included the ml5 library in your `index.html` file:
25+
26+
```html
27+
<script src="https://unpkg.com/ml5@1/dist/ml5.js"></script>
28+
```
29+
30+
?> For more information on importing the ml5 library, check out the [Getting Started](/?id=set-up-ml5js) page.
31+
32+
Create an instance of `ml5.depthEstimation` in your preload function, to allow the model to load
1633
```js
1734
function preload() {
1835
depthEstimator = ml5.depthEstimation({
19-
// Use options here to configure how the model behaves, like:
20-
flipHorizontal: true,
21-
// See a list of the available options below
36+
// Use options here to configure how the model behaves.
37+
// See a full list of options below, in the 'Methods' section of this reference page
2238
});
2339
}
2440
```
41+
For the full list of options, check out the [methods section](#ml5depthestimation) below!
2542

26-
The main available options are:
27-
28-
- `flipHorizontal`: Used to mirror the depth map horizontally
29-
- Default: `false`
30-
- Accepted values: `true`, `false` (boolean).
31-
- `dilationFactor`: Sets how many pixels around the detected edges of a person should be ignored. This is useful because depth values are inaccurate and noisy around the contours.
32-
- Default: `4`
33-
- Accepted values: `0` to `10` (integer).
34-
- `colormap`: Defines how the depth map is drawn; either Grayscale, mapping depth from black (far) to white (close), or Color, mapping depth using the whole range of color hues.
35-
- Default: `'GRAYSCALE'`
36-
- Accepted values: `'GRAYSCALE'` or `'COLOR'` (string).
37-
- `minDepth`: Sets the depth value that will map to the 'close' color.
38-
- Default: `0.2`
39-
- Accepted values: `0` to `1` (float). Must be less than `maxDepth`.
40-
- `maxDepth`: Sets the depth value that will map to the 'far' color.
41-
- Default: `0.75`
42-
- Accepted values: `0` to `1` (float). Must be greater than `minDepth`.
43-
- `normalizeDynamically`: Whether to do a manual mapping (using maxDepth and minDepth) or do it dynamically; recording the lowest and highest values detected in the depth map on every frame and using them as the mapping limits. This means that any particular color will not always represent the same absolute distance from the screen.
44-
- Default: `false`
45-
- Accepted values: `true`, `false` (boolean). Setting to `true` will ignore `minDepth` and `maxDepth` options
46-
- `normalizationSmoothingFactor`: Only used if normalizing dynamically. Sets how much to smooth the varying maximum and minimum depth values detected during normalization. Higher values result in faster reaction to changes. Lower values result in smoother changes.
47-
- Default: `0.5`
48-
- Accepted values: `0` to `1` (float).
49-
50-
51-
### p5.js 2.0
43+
#### p5.js 2.0
5244
You can also use this module with p5.js 2.0! Instead of creating `ml5.depthEstimation` in preload, do it using your async `setup()` and `await`:
5345
```js
5446
async function setup() {
@@ -60,10 +52,12 @@ async function setup() {
6052
}
6153
```
6254

63-
## Estimating depth
55+
### Estimating depth
6456
As with many other ml5 models, you have two options to run depth estimation on the image, video or webcam of your choice: _Continuous Estimation_ and _Single Shot Estimation_ .
6557

66-
### Continuous estimation
58+
For any of these, make sure you first load the image, video or start the webcam capture. This is the media we will pass to the model.
59+
60+
#### Continuous estimation
6761
This method is used to continuously estimate depth on every frame of a video or webcam feed.
6862
```js
6963
// Make sure to load the model in preload or async in p5 2.0!
@@ -81,7 +75,7 @@ function gotResults(result) {
8175
```
8276
Using this method, the depth estimator will take care of doing estimation of a frame and waiting for it to finish before working on the next frame. Any time a depth map is ready, it will fire the callback function to provide it.
8377

84-
### Single shot estimation
78+
#### Single shot estimation
8579
This method is used to estimate depth once, for a single image:
8680
```js
8781
// Make sure to load the image and the model in preload or asyn in p5 2.0!
@@ -96,7 +90,7 @@ function gotResults(result) {
9690
```
9791
Because the estimation takes time, we pass in a callback function that will fire when estimation is ready. The `estimate` method is called in setup because it **will only run once**. If calling it multiple times, it is prudent to wait for each operation to finish before starting the next one.
9892

99-
## Using the depth result
93+
### Using the depth result
10094
Whenever the callback function fires, we have acces to the depth result that contains all the depth information.
10195
```js
10296
let depthMap;
@@ -106,29 +100,109 @@ function gotResults(result) {
106100
depthMap = result;
107101
}
108102
```
109-
This result is an object, and it contains the following properties:
103+
The `result` is a `DepthEstimationResult` object that contains the depth map and other relevant data. Save it to variable so you can use it inside the p5 `draw()` loop!
104+
105+
For more information, on the structure and data contained in the result, check out [DepthEstimationResult Structure](#depthestimationresult) below.
106+
107+
## Methods
108+
109+
### ml5.depthEstimation()
110+
111+
This method is used to initialize the depth estimation object.
112+
113+
In p5.js 1.x.x, use it inside the `preload()` function:
114+
115+
```js
116+
const depthEstimator = ml5.depthEstimation(?options);
117+
```
118+
119+
In p5.js 2.0, use it in the `async setup()`:
120+
121+
```js
122+
const depthEstimator = await ml5.depthEstimation(?options);
123+
```
124+
125+
126+
**Options:**
127+
128+
- `flipHorizontal`: Used to mirror the depth map horizontally
129+
- Default: `false`
130+
- Accepted values: `true`, `false` (boolean).
131+
- `dilationFactor`: Sets how many pixels around the detected edges of a person should be ignored. This is useful because depth values are inaccurate and noisy around the contours.
132+
- Default: `4`
133+
- Accepted values: `0` to `10` (integer).
134+
- `colormap`: Defines how the depth map is drawn; either Grayscale, mapping depth from black (far) to white (close), or Color, mapping depth using the whole range of color hues.
135+
- Default: `'GRAYSCALE'`
136+
- Accepted values: `'GRAYSCALE'` or `'COLOR'` (string).
137+
- `minDepth`: Sets the depth value that will map to the 'close' color.
138+
- Default: `0.2`
139+
- Accepted values: `0` to `1` (float). Must be less than `maxDepth`.
140+
- `maxDepth`: Sets the depth value that will map to the 'far' color.
141+
- Default: `0.75`
142+
- Accepted values: `0` to `1` (float). Must be greater than `minDepth`.
143+
- `normalizeDynamically`: Whether to do a manual mapping (using maxDepth and minDepth) or do it dynamically; recording the lowest and highest values detected in the depth map on every frame and using them as the mapping limits. This means that any particular color will not always represent the same absolute distance from the screen.
144+
- Default: `false`
145+
- Accepted values: `true`, `false` (boolean). Setting to `true` will ignore `minDepth` and `maxDepth` options
146+
- `normalizationSmoothingFactor`: Only used if normalizing dynamically. Sets how much to smooth the varying maximum and minimum depth values detected during normalization. Higher values result in faster reaction to changes. Lower values result in smoother changes.
147+
- Default: `0.5`
148+
- Accepted values: `0` to `1` (float).
149+
150+
**Returns:**
151+
152+
- **Object**: `depthEstimation` object that contains the methods to run estimation.
153+
154+
### depthEstimator.estimateStart()
155+
This method is used for _Continuous Estimation_: estimating depth on a video/webcam continuously, for each frame. Calling it will initiate an estimation loop, running until `depthEstimator.estimateStop()` is called.
156+
157+
```js
158+
depthEstimator.estimateStart(media, callback)
159+
```
160+
161+
**Parameters:**
162+
163+
- **media**: An HTML or p5.js image, video, or canvas element to estimate a depth map for continuously.
164+
- **callback(result)**: A callback function that will be called *every time* an estimation result is available. The `result` is a `DepthEstimationResult` object. Check the section on it below for details on its structure.
165+
166+
### depthEstimator.estimateStop()
167+
This method is used to stop an estimation loop that was previously started by a call to `depthEstimator.estimateStart()`.
168+
169+
```js
170+
depthEstimator.estimateStop()
171+
```
172+
173+
### depthEstimator.estimate()
174+
This method is used for _Single Shot Estimation_: estimating depth one time on a single image or video/webcam frame.
175+
176+
```js
177+
depthEstimator.estimate(media, callback)
178+
```
179+
180+
**Parameters:**
181+
182+
- **media**: An HTML or p5.js image, video, or canvas element to estimate a depth map for.
183+
- **callback(result)**: A callback function that will be called when estimation is ready. The `result` is a `DepthEstimationResult` object. Check the section on it below for details on its structure.
184+
185+
186+
### DepthEstimationResult
187+
This is the object that is passed as an argument to the callback functions of `depthEstimator.estimateStart()` or `depthEstimator.estimate()`. It contains the result of the depth estimation process and other useful relevant data
188+
189+
These are its properties:
190+
110191
- `image`: A p5 image of the depth map in the chosen colormap.
111192
- Type: `p5.Image` object
112-
- `getDepthAt(x, y)`: Function that returns the depth value of the pixel in the location passed in to the x and y parameters.
113-
- Type: function.
193+
- `getDepthAt(x, y)`: Function that returns the depth value of the pixel at `x, y`.
194+
- Type: Function.
195+
- Returns: Floating point number in the 0 - 1 range.
114196
- `data`: The raw depth values for each pixel in a two dimensional array format.
115-
- Type: 2D array
197+
- Type: 2D array of floating point numbers in the 0 - 1 range.
116198
- `mask`: The mask of the people detected in the image and for whom depth values were estimated. It can be used directly with the `mask()` function in p5.
117199
- Type: `p5.Image` object
118-
- `sourceFrame`: The exact frame that was analyzed to create the depth map. Because depth estimation is not immediate, the result can fall out of sync from the source video. By using this value instead of the video, the depth data is guaranteed to be in sync. See it in action in the 'Keeping data in sync' section of [this article](https://ml5js.org/blog/bringing-depth-estimation/).
200+
- `sourceFrame`: The exact frame that was analyzed to create the depth map. Because depth estimation is not immediate, the result can fall out of sync from the source video. By using this value instead of the video, the depth data is guaranteed to be in sync. See a [demo sketch](https://editor.p5js.org/nasif-co/sketches/Z_1xMhUPl) showcasing the difference.
119201
- Type: `p5.Image`
120202
- `width`: Width of the depth map
121-
- Type: number
203+
- Type: number (integer)
122204
- `height`: Height of the depth map
123-
- Type: number
124-
125-
## Examples
126-
- [Webcam](https://editor.p5js.org/nasif-co/sketches/Pep6DjEtD): Show the live depth map of the video captured by the webcam.
127-
- [Video](https://editor.p5js.org/nasif-co/sketches/vifmzXg6o): Generate the depth map of a video file as it plays.
128-
- [Single Image](https://editor.p5js.org/nasif-co/sketches/_TcZofgrt): Depth map of an image using single-shot estimation.
129-
- [Mask Background](https://editor.p5js.org/nasif-co/sketches/Z_1xMhUPl): Showcases how to mask out the background from the depth result.
130-
- [Point Cloud](https://editor.p5js.org/nasif-co/sketches/VbT8hEoDz): Creates a live 3D point cloud visualization of the webcam video.
131-
- [Mesh](https://editor.p5js.org/nasif-co/sketches/X-e1DEZr4): Creates a live 3D mesh geometry of the webcam video.
205+
- Type: number (integer)
132206
133207
## Learn more
134208
Check out the community article [Finding the Z-axis](https://ml5js.org/blog/bringing-depth-estimation/) to learn more about the way depth estimation was implemented in ml5.

docs/reference/iframes/depthestimation/script.js

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,9 @@ let videoHeight = 480;
1616

1717
function preload() {
1818
// Load and start the depth estimation model
19-
depthEstimator = ml5.depthEstimation();
19+
depthEstimator = ml5.depthEstimation({
20+
normalizeDynamically: true,
21+
});
2022
}
2123

2224
function setup() {

0 commit comments

Comments
 (0)