Skip to content

Commit 5c5d6ec

Browse files
jtsodapopjtsodapop
authored
Merge branch 'develop' into jtso/al-1278
2 parents c1bfe7b + 0cbc00e commit 5c5d6ec

File tree

115 files changed

+20375
-5428
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

115 files changed

+20375
-5428
lines changed

‎.github/workflows/python-package.yml‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,12 +17,15 @@ jobs:
1717
- python-version: 3.7
1818
prod-key: LABELBOX_API_KEY
1919
staging-key: STAGING_LABELBOX_API_KEY
20+
da-test-key: DA_GCP_LABELBOX_API_KEY
2021
- python-version: 3.8
2122
prod-key: PROD_LABELBOX_API_KEY_2
2223
staging-key: STAGING_LABELBOX_API_KEY_2
24+
da-test-key: DA_GCP_LABELBOX_API_KEY
2325
- python-version: 3.9
2426
prod-key: PROD_LABELBOX_API_KEY_3
2527
staging-key: STAGING_LABELBOX_API_KEY_3
28+
da-test-key: DA_GCP_LABELBOX_API_KEY
2629

2730
steps:
2831
- name: Cancel previous workflow
@@ -83,5 +86,7 @@ jobs:
8386
LABELBOX_TEST_API_KEY_PROD: ${{ secrets[matrix.prod-key] }}
8487

8588
LABELBOX_TEST_API_KEY_STAGING: ${{ secrets[matrix.staging-key] }}
89+
90+
DA_GCP_LABELBOX_API_KEY: ${{ secrets[matrix.da-test-key] }}
8691
run: |
87-
tox -e py -- -svvx
92+
tox -e py -- -svv --reruns 5 --reruns-delay 10

‎CHANGELOG.md‎

Lines changed: 118 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,129 @@
11
# Changelog
2-
# Version 3.14.0 (2022-02-10)
3-
## Added
4-
* Updated metrics for classifications to be per-answer
52

6-
# Version 3.13.0 (2022-02-07)
3+
# Version 3.22.1 (2022-05-23)
4+
## Updated
5+
* Renamed `custom_metadata` to `metadata_fields` in DataRow
6+
7+
# Version 3.22.0 (2022-05-20)
8+
## Added
9+
* `Dataset.create_data_row()` and `Dataset.create_data_rows()` now uploads metadata to data row
10+
* Added `media_attributes` and `metadata` to `BaseData`
11+
12+
## Updated
13+
* Removed `iou` from classification metrics
14+
15+
# Version 3.21.1 (2022-05-12)
16+
## Updated
17+
* `Project.create_batch()` timeout increased to 180 seconds
18+
19+
# Version 3.21.0 (2022-05-11)
20+
## Added
21+
* Projects can be created with a `media_type`
22+
* Added `media_type` attribute to `Project`
23+
* New `MediaType` enumeration
24+
25+
## Fix
26+
* Added back the mimetype to datarow bulk uploads for orgs that require delegated access
27+
28+
# Version 3.20.1 (2022-05-02)
29+
## Updated
30+
* Ontology Classification `scope` field is only set for top level classifications
31+
32+
# Version 3.20.0 (2022-04-27)
33+
## Added
34+
* Batches in a project can be retrieved with `project.batches()`
35+
* Added `Batch.remove_queued_data_rows()` to cancel remaining data rows in batch
36+
* Added `Batch.export_data_rows()` which returns `DataRow`s for a batch
37+
38+
## Updated
39+
* NDJsonConverter now supports Video bounding box annotations.
40+
* Note: Currently does not support nested classifications.
41+
* Note: Converting an export into Labelbox annotation types, and back to export will result in only keyframe annotations. This is to support correct import format.
42+
43+
44+
## Fix
45+
* `batch.project()` now works
46+
47+
# Version 3.19.1 (2022-04-14)
48+
## Fix
49+
* `create_data_rows` and `create_data_rows_sync` now uploads the file with a mimetype
50+
* Orgs that only allow DA uploads were getting errors when using these functions
51+
52+
# Version 3.19.0 (2022-04-12)
53+
## Added
54+
* Added Tool object type RASTER_SEGMENTATION for Video and DICOM ontologies
55+
56+
# Version 3.18.0 (2022-04-07)
57+
## Added
58+
* Added beta support for exporting labels from model_runs
59+
* LBV1Converter now supports data_split key
60+
* Classification objects now include `scope` key
61+
62+
## Fix
63+
* Updated notebooks
64+
65+
# Version 3.17.2 (2022-03-28)
66+
## Fix
67+
* Project.upsert_instructions now works properly for new projects.
68+
69+
# Version 3.17.1 (2022-03-25)
70+
## Updated
71+
* Remove unused rasterio dependency
72+
73+
# Version 3.17.0 (2022-03-22)
74+
## Added
75+
* Create batches from the SDK (Beta). Learn more about [batches](https://docs.labelbox.com/docs/batches)
76+
* Support for precision and recall metrics on Entity annotations
77+
78+
## Fix
79+
* `client.create_project` type hint added for its return type
80+
81+
## Updated
82+
* Removed batch MVP code
83+
84+
# Version 3.16.0 (2022-03-08)
85+
## Added
86+
* Ability to fetch a model run with `client.get_model_run()`
87+
* Ability to fetch labels from a model run with `model_run.export_labels()`
88+
- Note: this is only Experimental. To use, client param `enable_experimental` should
89+
be set to true
90+
* Ability to delete an attachment
91+
92+
## Fix
93+
* Logger level is no longer set to INFO
94+
95+
## Updated
96+
* Deprecation: Creating Dropdowns will no longer be supported after 2022-03-31
97+
- This includes creating/adding Dropdowns to an ontology
98+
- This includes creating/adding Dropdown Annotation Type
99+
- For the same functionality, use Radio
100+
- This will not affect existing Dropdowns
101+
102+
# Changelog
103+
# Version 3.15.0 (2022-02-28)
104+
## Added
105+
* Extras folder which contains useful applications using the sdk
106+
* Addition of ResourceTag at the Organization and Project level
107+
* Updates to the example notebooks
108+
109+
## Fix
110+
* EPSGTransformer now properly transforms Polygon to Polygon
111+
* VideoData string representation now properly shows VideoData
112+
113+
114+
# Version 3.14.0 (2022-02-10)
115+
## Added
116+
* Updated metrics for classifications to be per-answer
117+
118+
119+
# Version 3.13.0 (2022-02-07)
7120
## Added
8121
* Added `from_shapely` method to create annotation types from Shapely objects
9122
* Added `start` and `end` filter on the following methods
10123
- `Project.export_labels()`
11124
- `Project.label_generator()`
12125
- `Project.video_label_generator()`
13-
* Improved type hinting
126+
* Improved type hinting
14127

15128

16129
# Version 3.12.0 (2022-01-19)

‎Dockerfile‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
11
FROM python:3.7
22

3-
RUN pip install pytest pytest-cases
3+
RUN pip install pytest pytest-cases pytest-rerunfailures
44
RUN apt-get -y update
55
RUN apt install -y libsm6 \
66
libxext6 \
77
ffmpeg \
88
libfontconfig1 \
99
libxrender1 \
10-
libgl1-mesa-glx
10+
libgl1-mesa-glx \
11+
libgeos-dev
1112

1213
WORKDIR /usr/src/labelbox
1314
COPY requirements.txt /usr/src/labelbox

‎Makefile‎

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,17 +6,28 @@ build:
66
test-local: build
77
docker run -it -v ${PWD}:/usr/src -w /usr/src \
88
-e LABELBOX_TEST_ENVIRON="local" \
9+
-e DA_GCP_LABELBOX_API_KEY=${DA_GCP_LABELBOX_API_KEY} \
910
-e LABELBOX_TEST_API_KEY_LOCAL=${LABELBOX_TEST_API_KEY_LOCAL} \
10-
local/labelbox-python:test pytest $(PATH_TO_TEST) -svvx
11+
local/labelbox-python:test pytest $(PATH_TO_TEST)
1112

1213
test-staging: build
1314
docker run -it -v ${PWD}:/usr/src -w /usr/src \
1415
-e LABELBOX_TEST_ENVIRON="staging" \
16+
-e DA_GCP_LABELBOX_API_KEY=${DA_GCP_LABELBOX_API_KEY} \
1517
-e LABELBOX_TEST_API_KEY_STAGING=${LABELBOX_TEST_API_KEY_STAGING} \
16-
local/labelbox-python:test pytest $(PATH_TO_TEST) -svvx
18+
local/labelbox-python:test pytest $(PATH_TO_TEST)
1719

1820
test-prod: build
1921
docker run -it -v ${PWD}:/usr/src -w /usr/src \
2022
-e LABELBOX_TEST_ENVIRON="prod" \
23+
-e DA_GCP_LABELBOX_API_KEY=${DA_GCP_LABELBOX_API_KEY} \
2124
-e LABELBOX_TEST_API_KEY_PROD=${LABELBOX_TEST_API_KEY_PROD} \
22-
local/labelbox-python:test pytest $(PATH_TO_TEST) -svvx
25+
local/labelbox-python:test pytest $(PATH_TO_TEST)
26+
27+
test-onprem: build
28+
docker run -it -v ${PWD}:/usr/src -w /usr/src \
29+
-e LABELBOX_TEST_ENVIRON="onprem" \
30+
-e DA_GCP_LABELBOX_API_KEY=${DA_GCP_LABELBOX_API_KEY} \
31+
-e LABELBOX_TEST_API_KEY_ONPREM=${LABELBOX_TEST_API_KEY_ONPREM} \
32+
-e LABELBOX_TEST_ONPREM_HOSTNAME=${LABELBOX_TEST_ONPREM_HOSTNAME} \
33+
local/labelbox-python:test pytest $(PATH_TO_TEST)

‎README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,3 +121,6 @@ make test-prod # with an optional flag: PATH_TO_TEST=tests/integration/...etc LA
121121
```
122122
make -B {build|test-staging|test-prod}
123123
```
124+
125+
6. Testing against Delegated Access will be skipped unless the local env contains the key:
126+
DA_GCP_LABELBOX_API_KEY. These tests will be included when run against a PR. If you would like to test it manually, please reach out to the Devops team for information on the key.

‎docs/requirements.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
Sphinx==3.4.3
1+
Sphinx==4.5.0
22
sphinxcontrib-napoleon==0.7
33
sphinx-rtd-theme==0.5.1

‎e2e_tests_codefresh_pipeline.yaml‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
version: "1.0"
2+
stages:
3+
- "clone"
4+
- "build"
5+
- "e2e_test"
6+
steps:
7+
main_clone:
8+
title: "Cloning repository"
9+
type: "git-clone"
10+
repo: "Labelbox/labelbox-python"
11+
stage: "clone"
12+
revision: "${{CF_BRANCH}}"
13+
parallel_build:
14+
type: "parallel"
15+
stage: "build"
16+
steps:
17+
build_sdk_test:
18+
description: "Building Docker image for sdk tests"
19+
tag: "testing"
20+
type: "build"
21+
image_name: "sdk-tests"
22+
disable_push: true
23+
working_directory: "${{main_clone}}"
24+
dockerfile: "Dockerfile"
25+
e2e_test:
26+
title: "Python SDK e2e tests"
27+
description: "Running Python SDK e2e tests"
28+
fail_fast: false
29+
stage: "e2e_test"
30+
image: "${{build_sdk_test}}"
31+
working_directory: "IMAGE_WORK_DIR"
32+
commands:
33+
- pytest
34+
send_slack_success_to_pages_active_learning:
35+
title: "Alert on success to pages active learning"
36+
type: slack-message-sender
37+
arguments:
38+
WEBHOOK_URL: "${{slack_webhook_url}}"
39+
MESSAGE: "SDK Tests have succeeded-- ${{CF_BUILD_URL}}"
40+
when:
41+
condition:
42+
all:
43+
succeeded: e2e_test.result == 'success'
44+
send_slack_failure_to_e2e:
45+
title: "Alert on failure to e2e channel"
46+
type: slack-message-sender
47+
arguments:
48+
WEBHOOK_URL: "${{slack_webhook_e2e}}"
49+
MESSAGE: "SDK Tests have failed-- ${{CF_BUILD_URL}}"
50+
when:
51+
condition:
52+
all:
53+
failed: e2e_test.result == 'failure'
54+
send_slack_failure_to_deploy_prod:
55+
title: "Alert on failure to deployments prod channel"
56+
type: slack-message-sender
57+
arguments:
58+
WEBHOOK_URL: "${{slack_webhook_deployments_prod}}"
59+
MESSAGE: "SDK Tests have failed-- ${{CF_BUILD_URL}}"
60+
when:
61+
condition:
62+
all:
63+
failed: e2e_test.result == 'failure'
64+
check_for_failures:
65+
image: alpine
66+
commands:
67+
- exit 1
68+
when:
69+
condition:
70+
any:
71+
myCondition: e2e_test.result == 'failure'

‎examples/Makefile‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
2+
upload:
3+
python scripts/upload_documentation.py --config-path 'scripts/config.json'

‎examples/README.md‎

Lines changed: 13 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
| Notebook | Github | Google Colab |
1212
| --------------------------- | --------------------------------- | ------------ |
1313
| Fundamentals | [Github](basics/basics.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/basics/basics.ipynb) |
14+
| Batches | [Github](basics/batches.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/basics/batches.ipynb) |
1415
| Data Rows | [Github](basics/data_rows.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/basics/data_rows.ipynb) |
1516
| Data Row Metadata | [Github](basics/data_row_metadata.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/basics/data_row_metadata.ipynb) |
1617
| Datasets | [Github](basics/datasets.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/basics/datasets.ipynb) |
@@ -51,18 +52,16 @@ Learn more about annotation types in the [docs](https://docs.labelbox.com/docs/a
5152
| MAL Using Annotation Types | [Github](annotation_types/mal_using_annotation_types.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/annotation_types/mal_using_annotation_types.ipynb) |
5253

5354

54-
## [Model Assisted Labeling](model_assisted_labeling)
55+
## [Annotation Import (Ground Truth & MAL)](annotation_import)
5556

5657

5758
| Notebook | Github | Google Colab | Learn more |
5859
| --------------------------- | --------------------------------- | ------------ | ---------- |
59-
| MAL Basics | [Github](model_assisted_labeling/mal_basics.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/model_assisted_labeling/mal_basics.ipynb) | [Docs](https://docs.labelbox.com/docs/model-assisted-labeling) |
60-
| Image MAL | [Github](model_assisted_labeling/image_mal.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/model_assisted_labeling/image_mal.ipynb) |
61-
| Video MAL | [Github](model_assisted_labeling/video_mal.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/model_assisted_labeling/video_mal.ipynb) |
62-
| Named Entity Recognition MAL | [Github](model_assisted_labeling/ner_mal.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/model_assisted_labeling/ner_mal.ipynb) |
63-
| Debugging MAL | [Github](model_assisted_labeling/debugging_mal.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/model_assisted_labeling/debugging_mal.ipynb) |
64-
| MAL with Subclasses | [Github](model_assisted_labeling/mal_with_subclasses.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/model_assisted_labeling/mal_with_subclasses.ipynb) |
65-
| Tiled Imagery MAL | [Github](model_assisted_labeling/tiled_imagery_mal.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/model_assisted_labeling/tiled_imagery_mal.ipynb) |
60+
| Basics | [Github](annotation_import/basics.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/annotation_import/basics.ipynb) | [Docs](https://docs.labelbox.com/docs/model-assisted-labeling) |
61+
| Image Annotation Import | [Github](annotation_import/image.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/annotation_import/image.ipynb) |
62+
| Text Annotation Import | [Github](annotation_import/text.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/annotation_import/text.ipynb) |
63+
| Tiled Imagery Annotation Import | [Github](annotation_import/tiled.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/annotation_import/tiled.ipynb) |
64+
| Video Model-Assisted Labeling | [Github](annotation_import/video.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/annotation_import/video.ipynb) |
6665
------
6766

6867
## [Project Configuration](project_configuration)
@@ -82,3 +81,9 @@ Learn more about annotation types in the [docs](https://docs.labelbox.com/docs/a
8281
| Model Diagnostics Demo | [Github](model_diagnostics/model_diagnostics_demo.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/model_diagnostics/model_diagnostics_demo.ipynb) |
8382
| Custom Metrics Basics | [Github](model_diagnostics/custom_metrics_basics.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/model_diagnostics/custom_metrics_basics.ipynb) | [Docs](https://docs.labelbox.com/docs/model-diagnostics) |
8483
| Custom Metrics Demo | [Github](model_diagnostics/custom_metrics_demo.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/model_diagnostics/custom_metrics_demo.ipynb) |
84+
85+
86+
## [Extras](extras)
87+
| Notebook | Github | Google Colab | Learn more |
88+
|---------------------------------|------------------------------------------------| ------------ | ---------- |
89+
| Classification Confusion Matrix | [Github](extras/classification-confusion-matrix.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/extras/classification-confusion-matrix.ipynb) | |

0 commit comments

Comments
 (0)