Updated instructions for running with Code Engine.

chcost · chcost · commit e46fd469baf8 · 2021-07-06T18:32:13.000-04:00
diff --git a/deploy/ibm_cloud_code_engine/README.md b/deploy/ibm_cloud_code_engine/README.md
@@ -18,32 +18,32 @@ limitations under the License.
 
 # CodeFlare on IBM Cloud Code Engine
 
-IBM Cloud Code Engine is a fully managed, serverless platform that runs your containerized workloads, including web apps, microservices, event-driven functions and batch jobs with run-to-completion characteristics. 
+IBM Cloud Code Engine is a fully managed, serverless platform that runs your containerized workloads, including web apps, microservices, event-driven functions, and batch jobs with run-to-completion characteristics. 
 
-CodeFlare Pipelines can use IBM Cloud Code Engine to seamlessly scale pipeline on a serverless backend.
+CodeFlare can use IBM Cloud Code Engine to seamlessly scale pipelines on a serverless backend.
 
-The following steps describe how to deploy the examples in [notebooks](./notebooks) on a serverless backend with Code Engine (they expand the steps to run Ray on Code Engine [here](https://www.ibm.com/cloud/blog/ray-on-ibm-cloud-code-engine))
+The following steps describe how to deploy the examples in [notebooks](https://github.com/project-codeflare/codeflare/tree/develop/notebooks) on a serverless backend with Code Engine (they expand the steps to run Ray on Code Engine [here](https://www.ibm.com/cloud/blog/ray-on-ibm-cloud-code-engine))
 
 ## Install pre-requisities
 
-Install the pre-requisites for Code Engine and Ray
+Install the pre-requisites for Code Engine and Ray:
 
 1. Get ready with IBM Code Engine:
 
 - [Set up your Code Engine CLI](https://cloud.ibm.com/docs/codeengine?topic=codeengine-install-cli)
-- [Create your first Code Engine Project using the CLI](https://cloud.ibm.com/docs/codeengine?topic=codeengine-manage-project)
+- [optional] [Create your first Code Engine Project using the CLI](https://cloud.ibm.com/docs/codeengine?topic=codeengine-manage-project)
 
 2. [Set up the Kubernetes CLI](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
 
-3. Install Kubernetes (assuming Ray and Python dependcies have been installed with the steps [here](../../README.md).
+3. Install Kubernetes:
 
 ```shell
 pip install kubernetes
 ```
 
 ## Step 1 - Define your Ray Cluster on Code Engine
 
-If not already done, create a project in Code Engine:
+If not already done in the previous step, create a Code Engine project:
 
 ```shell
 ibmcloud ce project create -n <your project name>
@@ -52,10 +52,10 @@ ibmcloud ce project create -n <your project name>
 Select the Code Engine project and switch the `kubectl` context to the project:
 
 ```shell
-ibmcloud ce project select -n <your project name>  -k
+ibmcloud ce project select -n <your project name> -k
 ```
 
-Extract the Kubernetes namespace assigned to your project. The namespace can be found in the NAME column in theoutput of the command:
+Extract the Kubernetes namespace assigned to your project. The namespace can be found in the `NAME` column in the output of the command:
 
 ```shell
 kubectl get namespace
@@ -67,7 +67,7 @@ Export the namespace:
 export NAMESPACE=<namespace from above>
 ```
 
-Update With the following command you can download a basic Ray cluster definition and customize it for your namespace:
+A reference Ray cluster definition can be customized for your namespace with the following commands:
 ```shell
 cd ./deploy/ibm_cloud_code_engine/
 sed "s/NAMESPACE/$NAMESPACE/" ./example-cluster.yaml.template > ./example-cluster.yaml
@@ -76,45 +76,42 @@ sed "s/NAMESPACE/$NAMESPACE/" ./example-cluster.yaml.template > ./example-cluste
 This reference deployment file will create a Ray cluster with following characteristics:
 
 - a cluster named example-cluster with up to 10 workers using the Kubernetes provider pointing to your Code Engine project
-- A head node type with 1 vCPU and 2GiB memory using the CodeFlare image codeflare:nightly
-- A worker node type with 1 vCPU and 2GiB memory using the ray image codeflare:nightly
+- A head node type with 1 vCPU and 2GB of memory using the CodeFlare image `codeflare:latest`
+- A worker node type with 1 vCPU and 2GB memory using the ray image `codeflare:latest`
 - The default startup commands to start Ray within the container and listen to the proper ports
 - The autoscaler upscale speed is set to 2 for quick upscaling in this short and simple demo
 
 ## Step 2 - Start Ray cluster
 
-You can start now the Ray cluster by running
-``` 
+You can now start the Ray cluster by running:
+
+```shell 
 ray up example-cluster.yaml
 ```
 
-This command will create the Ray head node as Kubernetes Pod in your Code Engine project. When you create the Ray cluster for the first time, it can take up to three minutes until the Ray image is downloaded from the Ray repository. 
+This command will create the Ray head node as Kubernetes Pod in your Code Engine project. When you create the Ray cluster for the first time, it can take few minutes until the Ray image is downloaded from the container registry. 
 
 ## Step 3 - Run sample Pipeline with Jupyter
 
-Edit the sample notebook to connect to a running Ray cluster with the command:
-```
-ray.init(address='auto', _redis_password='5241590000000000')
-```
+A Jupyter server will be automatically running on the head node. To access the server from your local machine, execute the command:
 
-```
-ray exec example-cluster.yaml 'jupyter notebook --port=8899 <add file here>'
+```shell
+kubectl -n $NAMESPACE port-forward <ray-cluster-name> 8888:8888
 ```
 
-To foward access to the Jupyter notebook, do:
-```
-kubectl -n $NAMESPACE port-forward <ray-cluster-name>  8899:8899
-```
+You can now access the Jupyter server by pointing your browser to the following url:
 
-Set up access to Jupyter notebook:
-< how to obtain the token >
+```shell
+http://127.0.0.1:8888/lab
+```
 
-<!-->
-To access Ray dashboard, do:
+Once in the the Jupyer envrionment, examples are found in the `codeflare/notebooks` directory in the container image. Documentation for reference use cases can be found in [Examples](https://codeflare.readthedocs.io/en/latest/).
 
-In your browser, go to:
--->
+To execute any of the notebooks with the Ray cluster running on Code Engine, edit the `ray.init()` line with the following parameters:
 
-Once in a Jupyer envrionment, refer to [notebooks](../../notebooks) for example pipeline. Documentation for reference use cases can be found in [Examples](https://codeflare.readthedocs.io/en/latest/).
+```python
+ray.init(address='auto', _redis_password='5241590000000000')
+```
 
+This change will allow pipelines to be auto-scaled on the underlying Ray cluster running on Code Engine (up to 10 workers with the reference deployment). The number of workers and scaling parameters can be adjusted in the `yaml` file.
 
diff --git a/docs/source/getting_started/starting.md b/docs/source/getting_started/starting.md
@@ -20,32 +20,32 @@ limitations under the License.
 
 ## CodeFlare on IBM Cloud Code Engine
 
-IBM Cloud Code Engine is a fully managed, serverless platform that runs your containerized workloads, including web apps, microservices, event-driven functions and batch jobs with run-to-completion characteristics. 
+IBM Cloud Code Engine is a fully managed, serverless platform that runs your containerized workloads, including web apps, microservices, event-driven functions, and batch jobs with run-to-completion characteristics. 
 
-CodeFlare Pipelines can use IBM Cloud Code Engine to seamlessly scale pipeline on a serverless backend.
+CodeFlare can use IBM Cloud Code Engine to seamlessly scale pipelines on a serverless backend.
 
-The following steps describe how to deploy the examples in [notebooks](./notebooks) on a serverless backend with Code Engine (they expand the steps to run Ray on Code Engine [here](https://www.ibm.com/cloud/blog/ray-on-ibm-cloud-code-engine))
+The following steps describe how to deploy the examples in [notebooks](https://github.com/project-codeflare/codeflare/tree/develop/notebooks) on a serverless backend with Code Engine (they expand the steps to run Ray on Code Engine [here](https://www.ibm.com/cloud/blog/ray-on-ibm-cloud-code-engine))
 
 ### Install pre-requisities
 
-Install the pre-requisites for Code Engine and Ray
+Install the pre-requisites for Code Engine and Ray:
 
 1. Get ready with IBM Code Engine:
 
 - [Set up your Code Engine CLI](https://cloud.ibm.com/docs/codeengine?topic=codeengine-install-cli)
-- [Create your first Code Engine Project using the CLI](https://cloud.ibm.com/docs/codeengine?topic=codeengine-manage-project)
+- [optional] [Create your first Code Engine Project using the CLI](https://cloud.ibm.com/docs/codeengine?topic=codeengine-manage-project)
 
 2. [Set up the Kubernetes CLI](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
 
-3. Install Kubernetes (assuming Ray and Python dependcies have been installed with the steps [here](../../README.md).
+3. Install Kubernetes:
 
 ```shell
 pip install kubernetes
 ```
 
 ### Step 1 - Define your Ray Cluster on Code Engine
 
-If not already done, create a project in Code Engine:
+If not already done in the previous step, create a Code Engine project:
 
 ```shell
 ibmcloud ce project create -n <your project name>
@@ -54,10 +54,10 @@ ibmcloud ce project create -n <your project name>
 Select the Code Engine project and switch the `kubectl` context to the project:
 
 ```shell
-ibmcloud ce project select -n <your project name>  -k
+ibmcloud ce project select -n <your project name> -k
 ```
 
-Extract the Kubernetes namespace assigned to your project. The namespace can be found in the NAME column in theoutput of the command:
+Extract the Kubernetes namespace assigned to your project. The namespace can be found in the `NAME` column in the output of the command:
 
 ```shell
 kubectl get namespace
@@ -69,7 +69,7 @@ Export the namespace:
 export NAMESPACE=<namespace from above>
 ```
 
-Update With the following command you can download a basic Ray cluster definition and customize it for your namespace:
+A reference Ray cluster definition can be customized for your namespace with the following commands:
 ```shell
 cd ./deploy/ibm_cloud_code_engine/
 sed "s/NAMESPACE/$NAMESPACE/" ./example-cluster.yaml.template > ./example-cluster.yaml
@@ -78,50 +78,48 @@ sed "s/NAMESPACE/$NAMESPACE/" ./example-cluster.yaml.template > ./example-cluste
 This reference deployment file will create a Ray cluster with following characteristics:
 
 - a cluster named example-cluster with up to 10 workers using the Kubernetes provider pointing to your Code Engine project
-- A head node type with 1 vCPU and 2GiB memory using the CodeFlare image codeflare:nightly
-- A worker node type with 1 vCPU and 2GiB memory using the ray image codeflare:nightly
+- A head node type with 1 vCPU and 2GB of memory using the CodeFlare image `codeflare:latest`
+- A worker node type with 1 vCPU and 2GB memory using the ray image `codeflare:latest`
 - The default startup commands to start Ray within the container and listen to the proper ports
 - The autoscaler upscale speed is set to 2 for quick upscaling in this short and simple demo
 
 ### Step 2 - Start Ray cluster
 
-You can start now the Ray cluster by running
-``` 
+You can now start the Ray cluster by running:
+
+```shell 
 ray up example-cluster.yaml
 ```
 
-This command will create the Ray head node as Kubernetes Pod in your Code Engine project. When you create the Ray cluster for the first time, it can take up to three minutes until the Ray image is downloaded from the Ray repository. 
+This command will create the Ray head node as Kubernetes Pod in your Code Engine project. When you create the Ray cluster for the first time, it can take few minutes until the Ray image is downloaded from the container registry. 
 
 ### Step 3 - Run sample Pipeline with Jupyter
 
-Edit the sample notebook to connect to a running Ray cluster with the command:
-```
-ray.init(address='auto', _redis_password='5241590000000000')
-```
+A Jupyter server will be automatically running on the head node. To access the server from your local machine, execute the command:
 
-```
-ray exec example-cluster.yaml 'jupyter notebook --port=8899 <add file here>'
+```shell
+kubectl -n $NAMESPACE port-forward <ray-cluster-name> 8888:8888
 ```
 
-To foward access to the Jupyter notebook, do:
-```
-kubectl -n $NAMESPACE port-forward <ray-cluster-name>  8899:8899
+You can now access the Jupyter server by pointing your browser to the following url:
+
+```shell
+http://127.0.0.1:8888/lab
 ```
 
-Set up access to Jupyter notebook:
-< how to obtain the token >
+Once in the the Jupyer envrionment, examples are found in the `codeflare/notebooks` directory in the container image. Documentation for reference use cases can be found in [Examples](https://codeflare.readthedocs.io/en/latest/).
 
-<!-->
-To access Ray dashboard, do:
+To execute any of the notebooks with the Ray cluster running on Code Engine, edit the `ray.init()` line with the following parameters:
 
-In your browser, go to:
--->
+```python
+ray.init(address='auto', _redis_password='5241590000000000')
+```
 
-Once in a Jupyer envrionment, refer to [notebooks](../../notebooks) for example pipeline. Documentation for reference use cases can be found in [Examples](https://codeflare.readthedocs.io/en/latest/).
+This change will allow pipelines to be auto-scaled on the underlying Ray cluster running on Code Engine (up to 10 workers with the reference deployment). The number of workers and scaling parameters can be adjusted in the `yaml` file.
 
 ## CodeFlare on OpenShift Container Platform (OCP)
 
-A few installation deployment targets are provided below.
+A few installation deployment targets are provided below:
 
 - [Ray Cluster Using Operator on Openshift](#Openshift-Ray-Cluster-Operator)
 - [Ray Cluster on Openshift](#Openshift-Cluster)
@@ -276,4 +274,4 @@ Jupyter setup demo [Reference repository](https://github.com/erikerlandson/ray-o
 
 ### Running examples
 
-Once in a Jupyer envrionment, refer to [notebooks](../../notebooks) for example pipeline. Documentation for reference use cases can be found in [Examples](https://codeflare.readthedocs.io/en/latest/).
+Once in a Jupyer envrionment, refer to [notebooks](https://github.com/project-codeflare/codeflare/tree/develop/notebooks) for example pipelines. Documentation for reference use cases can be found in [Examples](https://codeflare.readthedocs.io/en/latest/).
