Skip to content

Commit 937e79d

Browse files
qiuosierqiuosier
committed
Update runtime.rst
1 parent 0fd06ea commit 937e79d

File tree

1 file changed

+127
-32
lines changed

1 file changed

+127
-32
lines changed

‎docs/source/user_guide/jobs/runtime.rst‎

Lines changed: 127 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -7,76 +7,171 @@ and other configurations for the environment to run the workload.
77
Depending on the source code, ADS provides different types of *runtime* for defining a data science job,
88
including:
99

10-
* :py:class:`~ads.jobs.builders.runtimes.python_runtime.PythonRuntime`
10+
* :py:class:`~ads.jobs.PythonRuntime`
1111
for Python code stored locally, OCI object storage, or other remote location supported by
1212
`fsspec <https://filesystem-spec.readthedocs.io/en/latest/>`_
13-
* :py:class:`~ads.jobs.builders.runtimes.python_runtime.GitPythonRuntime`
13+
* :py:class:`~ads.jobs.GitPythonRuntime`
1414
for Python code from a Git repository.
15-
* :py:class:`~ads.jobs.builders.runtimes.python_runtime.NotebookRuntime`
15+
* :py:class:`~ads.jobs.NotebookRuntime`
1616
for a single Jupyter notebook stored locally, OCI object storage, or other remote location supported by
1717
`fsspec <https://filesystem-spec.readthedocs.io/en/latest/>`_
18-
* :py:class:`~ads.jobs.builders.runtimes.python_runtime.ScriptRuntime`
18+
* :py:class:`~ads.jobs.ScriptRuntime`
1919
for bash or shell scripts stored locally, OCI object storage, or other remote location supported by
2020
`fsspec <https://filesystem-spec.readthedocs.io/en/latest/>`_
21-
* :py:class:`~ads.jobs.builders.runtimes.python_runtime.ContainerRuntime` for container images.
21+
* :py:class:`~ads.jobs.ContainerRuntime` for container images.
2222

2323

24+
Environment Variables
25+
=====================
26+
You can set environment variables for a runtime by calling
27+
:py:meth:`~ads.jobs.PythonRuntime.with_environment_variable()`.
28+
Environment variables enclosed by ``${...}`` will be substituted. For example:
29+
30+
.. code-block:: python
31+
32+
from ads.jobs import PythonRuntime
33+
34+
runtime = (
35+
PythonRuntime()
36+
.with_environment_variable(
37+
HOST="10.0.0.1",
38+
PORT="443",
39+
URL="http://${HOST}:${PORT}/path/",
40+
ESCAPED_URL="http://$${HOST}:$${PORT}/path/",
41+
MISSING_VAR="This is ${UNDEFINED}",
42+
VAR_WITH_DOLLAR="$10",
43+
DOUBLE_DOLLAR="$$10"
44+
)
45+
)
46+
47+
for k, v in runtime.environment_variables.items():
48+
print(f"{k}: {v}")
49+
50+
will show the following environment variables for the runtime:
51+
52+
.. code-block:: text
53+
54+
HOST: 10.0.0.1
55+
PORT: 443
56+
URL: http://10.0.0.1:443/path/
57+
ESCAPED_URL: http://${HOST}:${PORT}/path/
58+
MISSING_VAR: This is This is ${UNDEFINED}
59+
VAR_WITH_DOLLAR: $10
60+
DOUBLE_DOLLAR: $10
61+
62+
Note that:
63+
64+
* You can use ``$$`` to escape the substitution.
65+
* Undefined variable enclosed by ``${...}`` will be ignored.
66+
* Double dollar signs ``$$`` will be substituted by a single one ``$``.
67+
68+
Command Line Arguments
69+
======================
70+
71+
The command line arguments for running your script or function can be configured by calling
72+
:py:meth:`~ads.jobs.PythonRuntime.with_argument()`. For example:
73+
74+
.. code-block:: python
75+
76+
from ads.jobs import PythonRuntime
77+
78+
runtime = (
79+
PythonRuntime()
80+
.with_source("oci://bucket_name@namespace/path/to/script.py")
81+
.with_arguments(
82+
"arg1", "arg2",
83+
key1="val1",
84+
key2="val2"
85+
)
86+
)
87+
88+
will configured the job to call your script by:
89+
90+
.. code-block:: bash
91+
92+
python script.py arg1 arg2 --key1 val1 --key2 val2
93+
94+
You can call :py:meth:`~ads.jobs.PythonRuntime.with_argument()` multiple times to set the arguments
95+
to your desired order. You can check ``runtime.args`` to see the added arguments.
96+
97+
Here are a few more examples:
98+
99+
.. code-block:: python
100+
101+
runtime = PythonRuntime().with_argument(key1="val1", key2="val2").with_argument("pos1")
102+
print(runtime.args)
103+
# ["--key1", "val1", "--key2", "val2", "pos1"]
104+
105+
runtime = PythonRuntime()
106+
runtime.with_argument("pos1")
107+
runtime.with_argument(key1="val1", key2="val2.1 val2.2")
108+
runtime.with_argument("pos2")
109+
print(runtime.args)
110+
# ['pos1', '--key1', 'val1', '--key2', 'val2.1 val2.2', 'pos2']
111+
112+
runtime = PythonRuntime()
113+
runtime.with_argument("pos1")
114+
runtime.with_argument(key1=None, key2="val2")
115+
runtime.with_argument("pos2")
116+
print(runtime.args)
117+
# ["pos1", "--key1", "--key2", "val2", "pos2"]
118+
24119
Conda Environment
25120
=================
26121

27-
Except for :py:class:`~ads.jobs.builders.runtimes.python_runtime.ContainerRuntime`,
122+
Except for :py:class:`~ads.jobs.ContainerRuntime`,
28123
all the other runtime options allow you to configure a
29124
`Conda Environment <https://docs.oracle.com/en-us/iaas/data-science/using/conda_understand_environments.htm>`_
30125
for your workload. You can use the slug name to specify a
31-
`conda environment provided by the data science service <https://docs.oracle.com/en-us/iaas/data-science/using/conda_viewing.htm#conda-dsenvironments>`_.
126+
`conda environment provided by the data science service
127+
<https://docs.oracle.com/en-us/iaas/data-science/using/conda_viewing.htm#conda-dsenvironments>`_.
32128
For example, to use the TensorFlow conda environment:
33129

34-
.. code-block:: python3
130+
.. code-block:: python
35131
36-
from ads.jobs import PythonRuntime
132+
from ads.jobs import PythonRuntime
37133
38-
runtime = (
39-
PythonRuntime()
40-
.with_source("oci://bucket_name@namespace/path/to/script.py")
41-
# Use slug name for conda environment provided by data science service
42-
.with_service_conda("tensorflow28_p38_cpu_v1")
43-
)
134+
runtime = (
135+
PythonRuntime()
136+
.with_source("oci://bucket_name@namespace/path/to/script.py")
137+
# Use slug name for conda environment provided by data science service
138+
.with_service_conda("tensorflow28_p38_cpu_v1")
139+
)
44140
45141
You can also use a custom conda environment published to OCI Object Storage
46-
by passing the ``uri`` to the ``with_custom_conda()`` method, for example:
142+
by passing the ``uri`` to :py:meth:`~ads.jobs.PythonRuntime.with_custom_conda`,
143+
for example:
47144

48-
.. code-block:: python3
145+
.. code-block:: python
49146
50147
runtime = (
51-
ScriptRuntime()
148+
PythonRuntime()
52149
.with_source("oci://bucket_name@namespace/path/to/script.py")
53150
.with_custom_conda("oci://bucket@namespace/conda_pack/pack_name")
54151
)
55152
56-
For more details on custom conda environment, see
57-
`Publishing a Conda Environment to an Object Storage Bucket in Your Tenancy <https://docs.oracle.com/en-us/iaas/data-science/using/conda_publishs_object.htm>`__.
58-
59-
Environment Variables
60-
=====================
153+
By default, ADS will try to determine the region based on the authenticated API key or resource principal.
154+
If your custom conda environment is stored in a different region,
155+
you can specify the ``region`` when calling :py:meth:`~ads.jobs.PythonRuntime.with_custom_conda`.
61156

62-
63-
CLI Arguments
64-
=============
157+
For more details on custom conda environment, see
158+
`Publishing a Conda Environment to an Object Storage Bucket in Your Tenancy
159+
<https://docs.oracle.com/en-us/iaas/data-science/using/conda_publishs_object.htm>`__.
65160

66161

67162
Override Configuration
68163
======================
69164

70-
When you call ``job.run()``, a new job run will be started with the configuration defined in the **job**.
165+
When you call :py:meth:`ads.jobs.Job.run`, a new job run will be started with the configuration defined in the **job**.
71166
You may want to override the configuration with custom variables. For example,
72167
you can customize job run display name, override command line argument, specify additional environment variables,
73168
and add free form tags:
74169

75170
.. code-block:: python3
76171
77172
job_run = job.run(
78-
name="<my_job_run_name>",
79-
args="new_arg --new_key new_val",
80-
env_var={"new_env": "new_val"},
81-
freeform_tags={"new_tag": "new_tag_val"}
82-
)
173+
name="<my_job_run_name>",
174+
args="new_arg --new_key new_val",
175+
env_var={"new_env": "new_val"},
176+
freeform_tags={"new_tag": "new_tag_val"}
177+
)

0 commit comments

Comments
 (0)