Update s3 bucket endpoint url to remove https prefix (#643)

* Update s3 bucket endpoint url to remove https prefix
* Update e2e-readme wrt secure parameter used for minio client

Author: abhijeet-dhumal

docs/e2e.md

@@ -130,3 +130,4 @@ Currently the SDK doesn't support tolerations, so e2e tests can't be executed on
     AWS_STORAGE_BUCKET=<storage-bucket-name>
     AWS_STORAGE_BUCKET_MNIST_DIR=<storage-bucket-MNIST-datasets-directory>
     ```
+    Note : When using the Python Minio client to connect to a minio storage bucket, the `AWS_DEFAULT_ENDPOINT` environment variable by default expects secure endpoint where user can use endpoint url with https/http prefix for autodetection of secure/insecure endpoint.

tests/e2e/mnist.py

@@ -154,11 +154,20 @@ def prepare_data(self):
             secret_key = os.environ.get("AWS_SECRET_ACCESS_KEY")
             bucket_name = os.environ.get("AWS_STORAGE_BUCKET")
 
+            # remove prefix if specified in storage bucket endpoint url
+            secure = True
+            if endpoint.startswith("https://"):
+                endpoint = endpoint[len("https://") :]
+            elif endpoint.startswith("http://"):
+                endpoint = endpoint[len("http://") :]
+                secure = False
+
             client = Minio(
                 endpoint,
                 access_key=access_key,
                 secret_key=secret_key,
                 cert_check=False,
+                secure=secure,
             )
 
             if not os.path.exists(dataset_dir):