Update data_science_job.rst and run_python.rst

Author: qiuosier

docs/source/user_guide/jobs/data_science_job.rst

@@ -6,109 +6,20 @@ Quick Start
   Before creating a job, ensure that you have policies configured for Data Science resources.
   See also: :doc:`policies` and  `About Data Science Policies <https://docs.oracle.com/en-us/iaas/data-science/using/policies.htm>`_.
 
-In ADS, a job is defined by :doc:`infrastructure` and :doc:`runtime`.
-The Data Science Job infrastructure is configured through a :py:class:`~ads.jobs.DataScienceJob` instance.
-The runtime can be an instance of :py:class:`~ads.jobs.PythonRuntime`,
-:py:class:`~ads.jobs.GitPythonRuntime`,
-:py:class:`~ads.jobs.NotebookRuntime`,
-:py:class:`~ads.jobs.ScriptRuntime`, or
-:py:class:`~ads.jobs.ContainerRuntime`
-
+.. include:: ../jobs/components/toc_local.rst
 
 Create and Run a Job
 ====================
 
+In ADS, a job is defined by :doc:`infrastructure` and :doc:`runtime`.
+The Data Science Job infrastructure is configured through a :py:class:`~ads.jobs.DataScienceJob` instance.
+The runtime can be an instance of:
+
+.. include:: ../jobs/components/runtime_types.rst
+
 Here is an example to define and run a Python :py:class:`~ads.jobs.Job`:
 
-.. tabs::
-
-  .. code-tab:: python
-    :caption: Python
-
-    from ads.jobs import Job, DataScienceJob, PythonRuntime
-
-    job = (
-      Job(name="My Job")
-      .with_infrastructure(
-        DataScienceJob()
-        .with_log_group_id("<log_group_ocid>")
-        .with_log_id("<log_ocid>")
-        # The following infrastructure configurations are optional
-        # if you are in an OCI data science notebook session.
-        # The configurations of the notebook session will be used as defaults.
-        .with_compartment_id("<compartment_ocid>")
-        .with_project_id("<project_ocid>")
-        # For default networking, no need to specify subnet ID
-        .with_subnet_id("<subnet_ocid>")
-        .with_shape_name("VM.Standard.E3.Flex")
-        # Shape config details are applicable only for the flexible shapes.
-        .with_shape_config_details(memory_in_gbs=16, ocpus=1)
-        .with_block_storage_size(50)
-      )
-      .with_runtime(
-        PythonRuntime()
-        # Specify the service conda environment by slug name.
-        .with_service_conda("pytorch19_p37_cpu_v1")
-        # The job artifact can be a single Python script, a directory or a zip file.
-        .with_source("local/path/to/code_dir")
-        # Set the working directory
-        # When using a directory as source, the default working dir is the parent of code_dir.
-        # Working dir should be a relative path beginning from the source directory (code_dir)
-        .with_working_dir("code_dir")
-        # The entrypoint is applicable only to directory or zip file as source
-        # The entrypoint should be a path relative to the working dir.
-        # Here my_script.py is a file in the code_dir/my_package directory
-        .with_entrypoint("my_package/my_script.py")
-        # Add an additional Python path, relative to the working dir (code_dir/other_packages).
-        .with_python_path("other_packages")
-        # Copy files in "code_dir/output" to object storage after job finishes.
-        .with_output("output", "oci://bucket_name@namespace/path/to/dir")
-      )
-    )
-
-    # Create the job on OCI Data Science
-    job.create()
-    # Start a job run
-    run = job.run()
-    # Stream the job run outputs
-    run.watch()
-
-  .. code-tab:: yaml
-    :caption: YAML
-
-    kind: job
-    spec:
-      name: "My Job"
-      infrastructure:
-        kind: infrastructure
-        type: dataScienceJob
-        spec:
-          blockStorageSize: 50
-          compartmentId: <compartment_ocid>
-          jobInfrastructureType: STANDALONE
-          jobType: DEFAULT
-          logGroupId: <log_group_ocid>
-          logId: <log_ocid>
-          projectId: <project_ocid>
-          shapeConfigDetails:
-            memoryInGBs: 16
-            ocpus: 1
-          shapeName: VM.Standard.E3.Flex
-          subnetId: <subnet_ocid>
-      runtime:
-        kind: runtime
-        type: python
-        spec:
-          conda:
-            slug: pytorch19_p37_cpu_v1
-            type: service
-          entrypoint: my_package/my_script.py
-          outputDir: output
-          outputUri: oci://bucket_name@namespace/path/to/dir
-          pythonPath:
-          - other_packages
-          scriptPathURI: local/path/to/code_dir
-          workingDir: code_dir
+.. include:: ../jobs/tabs/python_runtime.rst
 
 For more details, see :doc:`infrastructure` configurations and see :doc:`runtime` configurations.
 
@@ -140,7 +51,7 @@ Here is an example of the logs:
 YAML
 ====
 
-A job can also be defined using YAML, as shown in the "YAML" tab.
+A job can be defined using YAML, as shown in the "YAML" tab.
 Here are some examples to load/save the YAML job configurations:
 
 .. code-block:: python
@@ -160,7 +71,7 @@ Here are some examples to load/save the YAML job configurations:
     infrastructure:
     kind: infrastructure
       ...
-  """")
+  """)
 
 The ``uri`` can be a local file path or a remote location supported by
 `fsspec <https://filesystem-spec.readthedocs.io/en/latest/>`_, including OCI object storage.
@@ -173,6 +84,8 @@ With the YAML file, you can create and run the job with ADS CLI:
 
 For more details on ``ads opctl``, see :doc:`../cli/opctl/_template/jobs`.
 
+The job infrastructure, runtime and job run also support YAML serialization/deserialization.
+
 
 Loading Existing Job or Job Run
 ===============================
@@ -226,3 +139,20 @@ You can also cancel a job run:
 .. code-block:: python
 
   run.cancel()
+
+
+Variable Substitution
+=====================
+
+When defining a job or starting a job run,
+you can use environment variable substitution for the names and ``output_uri`` argument of
+the :py:meth:`~ads.jobs.PythonRuntime.with_output` method.
+
+For example, the following job specifies the name based on the environment variable ``DATASET_NAME``,
+and ``output_uri`` based on the environment variables ``JOB_RUN_OCID``:
+
+.. include:: ../jobs/tabs/name_substitution.rst
+
+Note that ``JOB_RUN_OCID`` is an environment variable provided by the service after the job run is created.
+It is available for the ``output_uri`` but cannot be used in the job name.
+See also :ref:`Saving Outputs <_runtime_outputs>`