Skip to content

Commit 923645d

Browse files
GabefireGabefire
authored
[SN-99] add model slice notebook (#1390)
1 parent c233986 commit 923645d

File tree

1 file changed

+338
-0
lines changed

1 file changed

+338
-0
lines changed

‎examples/model_experiments/model_slices.ipynb‎

Lines changed: 338 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,338 @@
1+
{
2+
"nbformat": 4,
3+
"nbformat_minor": 2,
4+
"metadata": {},
5+
"cells": [
6+
{
7+
"metadata": {},
8+
"source": [
9+
"<td>\n",
10+
" <a target=\"_blank\" href=\"https://labelbox.com\" ><img src=\"https://labelbox.com/blog/content/images/2021/02/logo-v4.svg\" width=256/></a>\n",
11+
"</td>"
12+
],
13+
"cell_type": "markdown"
14+
},
15+
{
16+
"metadata": {},
17+
"source": [
18+
"<td>\n",
19+
"<a href=\"https://colab.research.google.com/github/Labelbox/labelbox-python/blob/master/examples/model_experiments/model_slices.ipynb\" target=\"_blank\"><img\n",
20+
"src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"></a>\n",
21+
"</td>\n",
22+
"\n",
23+
"<td>\n",
24+
"<a href=\"https://github.com/Labelbox/labelbox-python/tree/master/examples/model_experiments/model_slices.ipynb\" target=\"_blank\"><img\n",
25+
"src=\"https://img.shields.io/badge/GitHub-100000?logo=github&logoColor=white\" alt=\"GitHub\"></a>\n",
26+
"</td>"
27+
],
28+
"cell_type": "markdown"
29+
},
30+
{
31+
"metadata": {},
32+
"source": [
33+
"# Model Slices\n",
34+
"\n",
35+
"Model Slice functions similarly to Catalog Slice, with both essentially being saved searches. However, there are key differences in their functionalities. While Catalog Slice searches within a specific data catalog, Model Slice extends its data row search across a model run in a model. You can construct a Model Slice by using one or more filters to curate a collection of data rows. Often users will combine filters to surface high-impact data and then save the results as a Model Slice.\n",
36+
"\n",
37+
"This notebook is used to go over some common Labelbox SDK methods to interact with Model Slices created through the Labelbox platform.\n",
38+
"\n",
39+
"See [Slices](https://docs.labelbox.com/docs/slices-1) for more information on modifying Model Slices."
40+
],
41+
"cell_type": "markdown"
42+
},
43+
{
44+
"metadata": {},
45+
"source": [
46+
"!pip install labelbox"
47+
],
48+
"cell_type": "code",
49+
"outputs": [],
50+
"execution_count": null
51+
},
52+
{
53+
"metadata": {},
54+
"source": [
55+
"import labelbox as lb\n",
56+
"import uuid"
57+
],
58+
"cell_type": "code",
59+
"outputs": [],
60+
"execution_count": null
61+
},
62+
{
63+
"metadata": {},
64+
"source": [
65+
"## API Key and Client\n",
66+
"See the developer guide for [creating an API key](https://docs.labelbox.com/reference/create-api-key)."
67+
],
68+
"cell_type": "markdown"
69+
},
70+
{
71+
"metadata": {},
72+
"source": [
73+
"# Add your API key\n",
74+
"API_KEY = \"\"\n",
75+
"# To get your API key go to: Workspace settings -> API -> Create API Key\n",
76+
"client = lb.Client(api_key=API_KEY)"
77+
],
78+
"cell_type": "code",
79+
"outputs": [],
80+
"execution_count": null
81+
},
82+
{
83+
"metadata": {},
84+
"source": [
85+
"## Create Model Slice\n",
86+
"\n",
87+
"In order to interact with model slices, you must create a Model Experiment with a Model Run and then create a Model Slice through the platform. The steps below go over this process. See [Model](https://docs.labelbox.com/reference/model) from our developer guides for more information."
88+
],
89+
"cell_type": "markdown"
90+
},
91+
{
92+
"metadata": {},
93+
"source": [
94+
"### Creating Model Experiment\n",
95+
"\n",
96+
"To create a Model Experiment you will need to create an ontology. See [Ontology](https://docs.labelbox.com/reference/ontology) for more information"
97+
],
98+
"cell_type": "markdown"
99+
},
100+
{
101+
"metadata": {},
102+
"source": [
103+
"#### Ontology"
104+
],
105+
"cell_type": "markdown"
106+
},
107+
{
108+
"metadata": {},
109+
"source": [
110+
"classification_features = [\n",
111+
" lb.Classification(\n",
112+
" class_type=lb.Classification.Type.CHECKLIST,\n",
113+
" name=\"Quality Issues\",\n",
114+
" options=[\n",
115+
" lb.Option(value=\"blurry\", label=\"Blurry\"),\n",
116+
" lb.Option(value=\"distorted\", label=\"Distorted\")\n",
117+
" ]\n",
118+
" )\n",
119+
"\n",
120+
"]\n",
121+
"\n",
122+
"ontology_builder = lb.OntologyBuilder(\n",
123+
" tools=[],\n",
124+
" classifications=classification_features\n",
125+
")\n",
126+
"\n",
127+
"ontology = client.create_ontology(\n",
128+
" \"Ontology from new features\",\n",
129+
" ontology_builder.asdict(),\n",
130+
" media_type=lb.MediaType.Image\n",
131+
")"
132+
],
133+
"cell_type": "code",
134+
"outputs": [],
135+
"execution_count": null
136+
},
137+
{
138+
"metadata": {},
139+
"source": [
140+
"#### Model Experiment"
141+
],
142+
"cell_type": "markdown"
143+
},
144+
{
145+
"metadata": {},
146+
"source": [
147+
"model = client.create_model(\n",
148+
" name=\"Model Slice Demo\",\n",
149+
" ontology_id=ontology.uid\n",
150+
")"
151+
],
152+
"cell_type": "code",
153+
"outputs": [],
154+
"execution_count": null
155+
},
156+
{
157+
"metadata": {},
158+
"source": [
159+
"### Creating a Model Run from Model Experiment\n",
160+
"\n",
161+
"On this step we will need to create a dataset to attach data rows to our model run. See [Dataset](https://docs.labelbox.com/reference/dataset) for more information."
162+
],
163+
"cell_type": "markdown"
164+
},
165+
{
166+
"metadata": {},
167+
"source": [
168+
"#### Dataset and Data Rows"
169+
],
170+
"cell_type": "markdown"
171+
},
172+
{
173+
"metadata": {},
174+
"source": [
175+
"# send a sample image as data row for a dataset\n",
176+
"global_key = str(uuid.uuid4())\n",
177+
"\n",
178+
"test_img_url = {\n",
179+
" \"row_data\":\n",
180+
" \"https://storage.googleapis.com/labelbox-datasets/image_sample_data/2560px-Kitano_Street_Kobe01s5s4110.jpeg\",\n",
181+
" \"global_key\":\n",
182+
" global_key\n",
183+
"}\n",
184+
"\n",
185+
"dataset = client.create_dataset(name=\"foundry-demo-dataset\")\n",
186+
"task = dataset.create_data_rows([test_img_url])\n",
187+
"task.wait_till_done()\n",
188+
"\n",
189+
"print(f\"Errors: {task.errors}\")\n",
190+
"print(f\"Failed data rows: {task.failed_data_rows}\")"
191+
],
192+
"cell_type": "code",
193+
"outputs": [],
194+
"execution_count": null
195+
},
196+
{
197+
"metadata": {},
198+
"source": [
199+
"#### Model Run and Attach Data Rows"
200+
],
201+
"cell_type": "markdown"
202+
},
203+
{
204+
"metadata": {},
205+
"source": [
206+
"model_run_name = \"Model Slice Demo\"\n",
207+
"example_config = {\n",
208+
" \"learning_rate\": 0.001, \n",
209+
" \"batch_size\": 32, \n",
210+
"}\n",
211+
"model_run = model.create_model_run(name=model_run_name, config=example_config)"
212+
],
213+
"cell_type": "code",
214+
"outputs": [],
215+
"execution_count": null
216+
},
217+
{
218+
"metadata": {},
219+
"source": [
220+
"model_run.upsert_data_rows(global_keys=[global_key])"
221+
],
222+
"cell_type": "code",
223+
"outputs": [],
224+
"execution_count": null
225+
},
226+
{
227+
"metadata": {},
228+
"source": [
229+
"### Make Model Slice Inside UI\n",
230+
"\n",
231+
"Creating a Model Slice is currently not supported through the SDK, however, to showcase how to interact with Model Slice, we are going to generate a Model Slice through the UI.\n",
232+
"\n",
233+
"#### Workflow\n",
234+
"\n",
235+
"1. Navigate to ***Model*** section of the Labelbox Platform, select the ***Experiment*** type, and select the Model Experiment that was created.\n",
236+
"2. You must have a filter created in order to save a slice. For this example, click ***Search your data*** dropdown and then ***Data row***.\n",
237+
"3. Change ***is one of*** dropdown to ***is not one of*** then type \"test\" into the ***Search for an id*** search box.\n",
238+
"4. Hit ***Enter*** and select ***Save slice***.\n",
239+
"5. Give the slice a name and select ***Save***.\n",
240+
"6. Above the ***Search your data*** dropdown you will see your slice's name. Select that dropdown and click ***Copy slice ID***.\n",
241+
"7. Paste the ***Slice ID*** below."
242+
],
243+
"cell_type": "markdown"
244+
},
245+
{
246+
"metadata": {},
247+
"source": [
248+
"SLICE_ID = \"\""
249+
],
250+
"cell_type": "code",
251+
"outputs": [],
252+
"execution_count": null
253+
},
254+
{
255+
"metadata": {},
256+
"source": [
257+
"### Get Model Slice"
258+
],
259+
"cell_type": "markdown"
260+
},
261+
{
262+
"metadata": {},
263+
"source": [
264+
"model_slice = client.get_model_slice(SLICE_ID)"
265+
],
266+
"cell_type": "code",
267+
"outputs": [],
268+
"execution_count": null
269+
},
270+
{
271+
"metadata": {},
272+
"source": [
273+
"### Obtain Data Row IDs from Model Slice"
274+
],
275+
"cell_type": "markdown"
276+
},
277+
{
278+
"metadata": {},
279+
"source": [
280+
"data_row_ids = model_slice.get_data_row_ids(model_run.uid)\n",
281+
"\n",
282+
"for data_row_id in data_row_ids:\n",
283+
" print(data_row_id)"
284+
],
285+
"cell_type": "code",
286+
"outputs": [],
287+
"execution_count": null
288+
},
289+
{
290+
"metadata": {},
291+
"source": [
292+
"### Obtain Data Row Identifiers Objects"
293+
],
294+
"cell_type": "markdown"
295+
},
296+
{
297+
"metadata": {},
298+
"source": [
299+
"data_rows = model_slice.get_data_row_identifiers(model_run.uid)\n",
300+
"\n",
301+
"for data_row in data_rows:\n",
302+
" print(data_row)"
303+
],
304+
"cell_type": "code",
305+
"outputs": [],
306+
"execution_count": null
307+
},
308+
{
309+
"metadata": {},
310+
"source": [
311+
"### Model Slice Attributes"
312+
],
313+
"cell_type": "markdown"
314+
},
315+
{
316+
"metadata": {},
317+
"source": [
318+
"# name (str)\n",
319+
"model_slice.name\n",
320+
"\n",
321+
"# description (str)\n",
322+
"model_slice.description\n",
323+
"\n",
324+
"# updated at (datetime)\n",
325+
"model_slice.updated_at\n",
326+
"\n",
327+
"# created at (datetime)\n",
328+
"model_slice.created_at\n",
329+
"\n",
330+
"# filter (list[dict])\n",
331+
"model_slice.filter"
332+
],
333+
"cell_type": "code",
334+
"outputs": [],
335+
"execution_count": null
336+
}
337+
]
338+
}

0 commit comments

Comments
 (0)