Merge pull request #3 from PatrickGhadban/main

Update README and provisioner name

Author: k8s-ci-robot

README.md

@@ -1,4 +1,6 @@
-[![Build Status](https://travis-ci.org/aws/aws-fsx-csi-driver.svg?branch=master)](https://travis-ci.org/aws/aws-fsx-csi-driver)
+[![Build Status](https://travis-ci.org/kubernetes-sigs/aws-file-cache-csi-driver.svg?branch=master)](https://travis-ci.org/kubernetes-sigs/aws-fsx-csi-driver)
+[![Coverage Status](https://coveralls.io/repos/github/kubernetes-sigs/aws-file-cache-csi-driver/badge.svg?branch=master)](https://coveralls.io/github/kubernetes-sigs/aws-file-cache-csi-driver?branch=master)
+[![Go Report Card](https://goreportcard.com/badge/github.com/kubernetes-sigs/aws-file-cache-csi-driver)](https://goreportcard.com/report/github.com/kubernetes-sigs/aws-file-cache-csi-driver)
 
 **WARNING**: This driver is in pre ALPHA currently. This means that there may potentially be backwards compatible breaking changes moving forward. Do NOT use this driver in a production environment in its current state.
 
@@ -16,91 +18,101 @@ This driver is in alpha stage. Basic volume operations that are functional inclu
 |---------------------------------------------------|-------|
 | master branch                                     | yes   |
 
-### Kubernetes Version Compability Matrix
-| AWS File Cache CSI Driver \ Kubernetes Version    | v1.17+ |
-|---------------------------------------------------|--------|
-| master branch                                     | yes    |
-
-## Features
+### Features
 Currently only static provisioning is supported. With static provisioning, a file cache should be created manually, then it could be mounted inside container as a persistence volume (PV) using File Cache CSI Driver.
 
 The following CSI interfaces are implemented:
 * Controller Service: 
 * Node Service: NodePublishVolume, NodeUnpublishVolume, NodeGetCapabilities, NodeGetInfo, NodeGetId
 * Identity Service: GetPluginInfo, GetPluginCapabilities, Probe
 
-## Examples
-This example shows how to make an Amazon File Cache availble inside container for the application to consume. Before this, get yourself familiar with how to setup kubernetes on AWS and [create an Amazon file cache](https://docs.aws.amazon.com/fsx/latest/FileCacheGuide/getting-started.html). And when creating an Amazon File Cache, make sure it is created inside the same VPC as kuberentes cluster or it is accessible through VPC peering.
+## Amazon File Cache CSI Driver on Kubernetes
 
-Once kubernetes cluster and an Amazon File Cache is created, create secret manifest file using [secret.yaml](../deploy/kubernetes/secret.yaml).
+---------
+The following sections are Kubernetes-specific. If you are a Kubernetes user, use the following for driver features, installation steps and examples.
 
-Then create the secret object:
-```sh
-kubectl apply -f deploy/kubernetes/secret.yaml 
-```
+### Kubernetes Version Compability Matrix
+| AWS File Cache CSI Driver \ Kubernetes Version    | v1.24+ |
+|---------------------------------------------------|--------|
+| master branch                                     | yes    |
+
+### Container Images
+| File Cache CSI Driver Version | Image                                                         |
+|-------------------------------|---------------------------------------------------------------|
+| master branch                 | public.ecr.aws/fsx-csi-driver/aws-filecache-csi-driver:latest |
+
+### Features
+* Static provisioning - Amazon File Cache needs to be created manually first, then it could be mounted inside container as a volume using the Driver.
+* Dynamic provisioning (currently not supported) - uses persistent volume claim (PVC) to let Kubernetes create the Amazon File Cache for you and consumes the volume from inside container.
+* Mount options - mount options can be specified in storageclass to define how the volume should be mounted.
 
-Deploy the Amazon file cache CSI driver:
+**Notes**:
+* For dynamically provisioned volumes, only one subnet is allowed inside a storageclass's `parameters.subnetId`. This is a [limitation](https://docs.aws.amazon.com/fsx/latest/APIReference/API_FileCacheCreating.html#FSx-Type-FileCacheCreating-SubnetIds) that is enforced by Amazon File Cache.
+
+### Installation
+#### Set up driver permission
+The driver requires IAM permission to talk to Amazon File Cache service to create/delete the filecache on user's behalf. There are several methods to grant driver IAM permission:
+* Using secret object - create an IAM user with proper permission, put that user's credentials in [secret manifest](../deploy/kubernetes/secret.yaml) then deploy the secret.
 
 ```sh
-kubectl apply -k deploy/kubernetes/base/
+curl https://raw.githubusercontent.com/kubernetes-sigs/aws-file-cache-csi-driver/master/deploy/kubernetes/secret.yaml > secret.yaml
+# Edit the secret with user credentials
+kubectl apply -f secret.yaml
 ```
 
-Edit the [persistence volume manifest file](../examples/kubernetes/static_provisioning/specs/pv.yaml):
+* Using worker node instance profile - grant all the worker nodes with proper permission by attach policy to the instance profile of the worker.
 ```sh
-apiVersion: v1
-kind: PersistentVolume
-metadata:
-  name: fc-pv
-spec:
-  capacity:
-    storage: 1200Gi
-  volumeMode: FileCache
-  accessModes:
-    - ReadWriteOnce
-  persistentVolumeReclaimPolicy: Recycle
-  storageClassName: fc-sc
-  csi:
-    driver: file.cache.csi.aws.com
-    volumeHandle: [FileCacheId]
-    volumeAttributes:
-      dnsname: [DNSName] 
+`kubectl annotate serviceaccount -n kube-system file-cache-csi-controller-sa \
+ eks.amazonaws.com/role-arn=arn:aws:iam::111111111111:role/AmazonEKSFileCacheCSIDriverFullAccess --overwrite=true
 ```
-Replace `volumeHandle` with `FileCacheId` and `dnsname` with `DNSName`. You can get both `FileCacheId` and `DNSName` using AWS CLI:
 
+
+#### Deploy driver
 ```sh
-aws fsx describe-file-caches
+kubectl apply -k deploy/kubernetes/base/
 ```
 
-Then create PV, persistence volume claim (PVC) and storage class:
+TODO: Add helm installation option
 ```sh
-kubectl apply -f examples/kubernetes/dynamic_provisioning/specs/storageclass.yaml
-kubectl apply -f examples/kubernetes/dynamic_provisioning/specs/pv.yaml
-kubectl apply -f examples/kubernetes/dynamic_provisioning/specs/claim.yaml
-kubectl apply -f examples/kubernetes/dynamic_provisioning/specs/pod.yaml
+
 ```
 
-After objects are created, verify that pod is running:
 
-```sh
-kubectl get pods
-```
 
-Make sure data is written onto Amazon File Cache:
 
-```sh
-kubectl exec -ti fc-app -- df -h
-kubectl exec -it fc-app -- ls /data
-```
+
+------------------
+
+
+### Examples
+Before the example, you need to:
+* Get yourself familiar with how to setup Kubernetes on AWS and [create Anmazon File Cache](https://docs.aws.amazon.com/fsx/latest/FileCacheGuide/getting-started.html) if you are using static provisioning.
+* When creating Amazon File Cache, make sure its VPC is accessible from Kuberenetes cluster's VPC and network traffic is allowed by security group.
+    * For FSx for Lustre VPC, you can either create an Amazon File Cache inside the same VPC as Kubernetes cluster or using VPC peering.
+    * For security group, make sure port 988 is allowed for the security groups that are attached the lustre filesystem ENI.
+* Install Amazon File Cache CSI driver following the [Installation](README.md#Installation) steps.
+
+#### Example Links
+* [Static provisioning](examples/kubernetes/static_provisioning/README.md)
+* [Dynamic provisioning](examples/kubernetes/dynamic_provisioning/README.md)
+* [Accessing the filesystem from multiple pods](examples/kubernetes/multiple_pods/README.md)
 
 ## Development
+
+----
 Please go through [CSI Spec](https://github.com/container-storage-interface/spec/blob/master/spec.md) and [General CSI driver development guideline](https://kubernetes-csi.github.io/docs/Development.html) to get some basic understanding of CSI driver before you start.
 
 ### Requirements
-* Golang 1.9+
+* Golang 1.19.0+
+
+### Dependency
+Dependencies are managed through go module. To build the project, first turn on go mod using `export GO111MODULE=on`, to build the project run: `make`
 
 ### Testing
 To execute all unit tests, run: `make test`
 
 ## License
+
+----
 This library is licensed under the Apache 2.0 License. 
 

deploy/kubernetes/base/csidriver.yaml

@@ -3,6 +3,6 @@
 apiVersion: storage.k8s.io/v1
 kind: CSIDriver
 metadata:
-  name: file.cache.csi.aws.com
+  name: filecache.csi.aws.com
 spec:
   attachRequired: false

deploy/kubernetes/base/node-daemonset.yaml

@@ -69,7 +69,7 @@ spec:
             - name: ADDRESS
               value: /csi/csi.sock
             - name: DRIVER_REG_SOCK_PATH
-              value: /var/lib/kubelet/plugins/file.cache.csi.aws.com/csi.sock
+              value: /var/lib/kubelet/plugins/filecache.csi.aws.com/csi.sock
             - name: KUBE_NODE_NAME
               valueFrom:
                 fieldRef:
@@ -99,5 +99,5 @@ spec:
             type: Directory
         - name: plugin-dir
           hostPath:
-            path: /var/lib/kubelet/plugins/file.cache.csi.aws.com/
+            path: /var/lib/kubelet/plugins/filecache.csi.aws.com/
             type: DirectoryOrCreate

examples/kubernetes/dynamic_provisioning/README.md

@@ -1,33 +1,45 @@
 ## Dynamic Provisioning Example
+
+---
+
 *~Not yet supported~*
 
-This example shows how to create an Amazon File Cache using persistence volume claim (PVC) and consumes it from a pod.
+This example shows how to create an Amazon File Cache using persistence volume claim (PVC) and consumes it from a pod. Please see the [CreateFileCache API Reference](https://docs.aws.amazon.com/fsx/latest/APIReference/API_CreateFileCache.html#FSx-CreateFileCache-request-DataRepositoryAssociations) for more information.
 
 
-### Edit [StorageClass](./specs/storageclass.yaml)
+### Edit [StorageClass](specs/storageclass.yaml)
 ```sh
 kind: StorageClass
 apiVersion: storage.k8s.io/v1
 metadata:
   name: fc-sc
-provisioner: file.cache.csi.aws.com
+provisioner: filecache.csi.aws.com
 parameters:
-  subnetId: subnet-0eabfaa81fb22bcaf
-  securityGroupIds: sg-068000ccf82dfba88
-  dataRepositoryAssociations: "fileCachePath=/ns1/,dataRepositoryPath=nfs://10.0.92.69/fsx/"
+  subnetId: [SubnetId]
+  securityGroupIds: [SecurityGroupId]
+  dataRepositoryAssociations: [DataRepositoryAssociations]
+  kmsKeyId: [KmsKeyId]
+  copyTagsToDataRepositoryAssociations: "false"
   fileCacheType: "LUSTRE"
   fileCacheTypeVersion: "2.12"
-  LustreConfiguration: "{DeploymentType=CACHE_1,MetadataConfiguration=2400,perUnitStorageThroughput=1000}"
-  weeklyMaintenanceStartTime: "6:00:00"
+  LustreConfiguration: "DeploymentType=CACHE_1,PerUnitStorageThroughput=1000,MetadataConfiguration={StorageCapacity=2400}"
+  weeklyMaintenanceStartTime: "d:HH:MM"
   extraTags: "Tag1=Value1,Tag2=Value2"
 ```
+*Update the parameters not marked as optional below.*
+
 * subnetId - The subnet ID that the Amazon File Cache should be created inside.
-* securityGroupIds - A comma separated list of security group IDs that should be attached to the filecache
-* dataRepositoryAssociations - A list of IDs of data repository associations that are associated with this cache.
+* securityGroupIds - A comma separated list of security group IDs that should be attached to the file cache.
+* dataRepositoryAssociations - A space separated ist of up to 8 configurations for data repository associations (DRAs) to be created during the cache creation. The DRAs link the cache to either an Amazon S3 data repository or a Network File System (NFS) data repository that supports the NFSv3 protocol. Please see [DataRepositoryAssociations](https://docs.aws.amazon.com/fsx/latest/APIReference/API_CreateFileCache.html#FSx-CreateFileCache-request-DataRepositoryAssociations) for the File Cache DRA configurations requirements.
+* copyTagsToDataRepositoryAssociations - A boolean flag indicating whether tags for the cache should be copied to data repository associations. This value defaults to false.
 * fileCacheType (Optional) - The type of cache, which must be LUSTRE.
 * fileCacheTypeVersion (Optional) - The Lustre version of the cache, which must be 2.12.
-* LustreConfiguration (Optional) - The configuration for the Amazon File Cache resource, please view [FileCacheLustreConfiguration](https://docs.aws.amazon.com/fsx/latest/APIReference/API_FileCacheLustreConfiguration.html) for more details on the contents.
 * weeklyMaintenanceStartTime (Optional) - The preferred start time to perform weekly maintenance, formatted d:HH:MM in the UTC time zone, where d is the weekday number, from 1 through 7, beginning with Monday and ending with Sunday. The default value is "7:09:00" (Sunday 09:00 UTC)
+* kmsKeyId (Optional) - Specifies the ID of the Key Management Service (KMS) key to use for encrypting data on an Amazon File Cache. If a KmsKeyId isn't specified, the Amazon FSx-managed KMS key for your account is used.
+* LustreConfiguration (Optional) - The configuration for the Amazon File Cache resource, please view [FileCacheLustreConfiguration](https://docs.aws.amazon.com/fsx/latest/APIReference/API_FileCacheLustreConfiguration.html) for more details on the contents.
+  * DeploymentType - Specifies the cache deployment type, which must be CACHE_1
+  * PerUnitStorageThroughput - Provisions the amount of read and write throughput for each 1 tebibyte (TiB) of cache storage capacity, in MB/s/TiB. The only supported value is 1000.
+  * MetadataConfiguration - The configuration for a Lustre MDT (Metadata Target) storage volume. The storage capacity of the Lustre MDT (Metadata Target) storage volume in gibibytes (GiB). The only supported value is 2400 GiB.
 * extraTags (Optional) - Tags that will be set on the FSx resource created in AWS, in the form of a comma separated list with each tag delimited by an equals sign (example - "Tag1=Value1,Tag2=Value2") . Default is a single tag with CSIVolumeName as the key and the generated volume name as it's value.
 
 ### Edit [Persistent Volume Claim Spec](./specs/claim.yaml)
@@ -44,7 +56,7 @@ spec:
     requests:
       storage: 1200Gi
 ```
-Update `spec.resource.requests.storage` with the storage capacity to request. The storage capacity value will be rounded up to 1200 GiB, 2400 GiB, or a multiple of 3600 GiB for SSD. If the storageType is specified as HDD, the storage capacity will be rounded up to 6000 GiB or a multiple of 6000 GiB if the perUnitStorageThroughput is 12, or rounded up to 1800 or a multiple of 1800 if the perUnitStorageThroughput is 40.
+*Update `spec.resource.requests.storage` with the storage capacity to request. The storage capacity value will be rounded up to 1200 GiB, 2400 GiB, or a multiple of 2400 GiB.*
 
 ### Deploy the Application
 Create PVC, storageclass and the pod that consumes the PV:
@@ -64,5 +76,5 @@ After the objects are created, verify that pod is running:
 Also verify that data is written onto Amazon File Cache:
 
 ```sh
->> kubectl exec -ti fsx-app -- tail -f /data/out.txt
+>> kubectl exec -ti fc-app -- tail -f /data/out.txt
 ```

examples/kubernetes/dynamic_provisioning/specs/pv.yaml

@@ -2,15 +2,14 @@ kind: StorageClass
 apiVersion: storage.k8s.io/v1
 metadata:
   name: fc-sc
-provisioner: file.cache.csi.aws.com
+provisioner: filecache.csi.aws.com
 parameters:
-  subnetId: subnet-0eabfaa81fb22bcaf
-  securityGroupIds: sg-068000ccf82dfba88
-  dataRepositoryAssociations: "fileCachePath=/ns1/,dataRepositoryPath=nfs://10.0.92.69/fsx/"
-  fileCacheType: "Lustre"
+  subnetId: "subnet-0d7b5e117ad7b4961"
+  securityGroupIds: "sg-05a37bfe01467059"
+  dataRepositoryAssociations: "FileCachePath=/ns1/,DataRepositoryPath=nfs://10.0.92.69/fsx/,NFS={Version=NFS3} FileCachePath=/ns2/,DataRepositoryPath=nfs://19.12.12.12/fsx/,NFS={Version=NFS3}"
+  fileCacheType(Optional): "Lustre"
   fileCacheTypeVersion(Optional): "2.12"
-  weeklyMaintenanceStartTime: "6:00:00"
-  LustreConfiguration: "{DeploymentType=CACHE_1,MetadataConfiguration=2400,perUnitStorageThroughput=1000}"
-  extraTags: "Tag1=Value1,Tag2=Value2"
+  weeklyMaintenanceStartTime(Optional): "7:09:00"
+  LustreConfiguration(Optional): "DeploymentType=CACHE_1,PerUnitStorageThroughput=1000,MetadataConfiguration={StorageCapacity=2400}"
 mountOptions:
   - flock