Adding inference container URI for edit model handler #1030

kumar-shivam-ranjan · 2024-12-19T10:01:20Z

This PR intends to update EDIT model handler for editing registered model config in AQUA.
The edit feature is available for registered unverified model.
This will allow user to edit the following config for models in AQUA after the model is registered:

Deployment container
Deployment container URI for TEI (Text Embedding inference container)
Task information of the model
Finetuning config

curl --location --request PUT 'http://localhost:8888/aqua/model/ocid1.datasciencemodel.oc1.iad.xxxxx' \
--header 'Content-Type: application/json' \
--data '{
    "inference_container": "odsc-tei-serving",
    "inference_container_uri": "iad.ocir.io/ociodscdev/text-embeddings-inference:cpu-1.5",
    "task": "feature-extraction"
}'

github-actions · 2024-12-19T10:32:10Z

📌 Cov diff with main:

📌 Overall coverage:

github-actions · 2024-12-19T11:29:37Z

📌 Cov diff with main:

📌 Overall coverage:

VipulMascarenhas · 2024-12-19T22:13:35Z

ads/aqua/extension/model_handler.py

        if (
            inference_container is not None
            and inference_container not in inference_containers
+            and inference_container


To avoid hardcoding, would it be better to move AQUA_TEI_CONTAINER_FAMILY from InferenceContainerTypeFamily to a new enum CustomInferenceContainerTypeFamily(str, metaclass=ExtendedEnumMeta)? That way the final inference_containers list can be inference_containers.append(CustomInferenceContainerTypeFamily.values()).

VipulMascarenhas · 2024-12-19T22:15:59Z

ads/aqua/model/model.py

-                        replace=True,
-                    )
+                    if (
+                        inference_container


this can be replaced with inference_container in CustomInferenceContainerTypeFamily.values()

VipulMascarenhas · 2024-12-19T22:30:07Z

ads/aqua/model/model.py

+                        )
+                    else:
+                        raise AquaRuntimeError(
+                            f"Failed to edit model:{id}. Inference container URI can be edited only with TEI container."


f"Inference container URI can be edited only with container values: {CustomInferenceContainerTypeFamily.values()}".

github-actions · 2025-01-20T18:23:23Z

📌 Cov diff with main:

📌 Overall coverage:

mrDzurb · 2025-01-20T18:27:16Z

ads/aqua/extension/model_handler.py


        inference_container = input_data.get("inference_container")
+        inference_container_uri = input_data.get("inference_container_uri")
        inference_containers = AquaModelApp.list_valid_inference_containers()


Will it always return a list?

yes , either an empty list or list of valid containers from container_index.json stored in OSS bucket

mrDzurb · 2025-01-20T18:27:53Z

ads/aqua/model/model.py

                f"Failed to delete model:{model_id}. Only registered models or finetuned model can be deleted."
            )

    @telemetry(entry_point="plugin=model&action=delete", name="aqua")


Why in telemetry we say delete?

Updated. Thanks

mrDzurb · 2025-01-20T18:28:29Z

ads/aqua/model/model.py

        inference_container: str.
            The inference container family name
+        inference_container_uri: str
+            The inference container uri for embedding models


s it only for embedding models?

Yes as per now , we accept inference container URI for embedding models only

mrDzurb · 2025-01-20T18:30:54Z

ads/aqua/model/model.py

-                    )
+                    if (
+                        inference_container
+                        in CustomInferenceContainerTypeFamily.values()


ExtendedEnumMeta implement the __contains__ method, that means we can do something like this:

inference_container in CustomInferenceContainerTypeFamily

mrDzurb · 2025-01-20T18:32:04Z

ads/aqua/model/model.py

+                            "Inference container URI must be provided."
+                        )
+                    else:
+                        custom_metadata_list.add(


Do we really need to store this info in metadata? Can't it be extracted from MD?

This metadata contains deployment container. Via edit feature , user can edit/change deployment container as well. Hence , adding updated container to metadata.

However user can update deployment from console, and in this case the container stored on the model level will not match a real one?

yes but source of truth here should be model right , not the model deployment.
User can create multiple model deployments from a single model. What we are trying to do here is changing/updating the model config.
What if user registered a model for a particular deployment container. Now , they due to some reason want to change the deployment container config of the model.

github-actions · 2025-01-20T19:44:35Z

📌 Cov diff with main:

📌 Overall coverage:

github-actions · 2025-01-29T06:12:49Z

📌 Cov diff with main:

No lines with coverage information in this diff.

📌 Overall coverage:

Adding inference container uri for edit model

2295252

kumar-shivam-ranjan requested review from VipulMascarenhas, ahosler, darenr, mayoor, mrDzurb and qiuosier as code owners December 19, 2024 10:01

oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Dec 19, 2024

kumar-shivam-ranjan added 2 commits December 19, 2024 15:32

Formatting

3aedb48

Formatting

a95a678

kumar-shivam-ranjan added 3 commits December 19, 2024 16:26

inference container uri validation check

5b3a316

formatting

16b7ace

formatting

c6f87ec

VipulMascarenhas reviewed Dec 19, 2024

View reviewed changes

kumar-shivam-ranjan added 3 commits January 20, 2025 23:20

Addressing review comments

3a9fa5b

Formatting

f196331

Merge branch 'main' into ODSC-66457/update-edit-model-handler

8edce5c

mrDzurb reviewed Jan 20, 2025

View reviewed changes

kumar-shivam-ranjan added 2 commits January 21, 2025 00:50

Addressing review comments

e8d366f

Addressing review comments

557509e

kumar-shivam-ranjan requested a review from VipulMascarenhas January 20, 2025 19:31

mrDzurb approved these changes Jan 23, 2025

View reviewed changes

VipulMascarenhas approved these changes Jan 27, 2025

View reviewed changes

Merge branch 'main' into ODSC-66457/update-edit-model-handler

1ba166a

kumar-shivam-ranjan merged commit 410dbe0 into main Jan 29, 2025
19 of 20 checks passed

Adding inference container URI for edit model handler #1030

Adding inference container URI for edit model handler #1030

Uh oh!

Conversation

kumar-shivam-ranjan commented Dec 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 19, 2024

Uh oh!

github-actions bot commented Dec 19, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 20, 2025

Uh oh!

Uh oh!

github-actions bot commented Jan 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kumar-shivam-ranjan commented Dec 19, 2024 •

edited

Loading