Forked for SRE Handbook defaults

Author: jeffmccune

README.md

@@ -1,15 +1,31 @@
-# fio-kubernetes
+# fio k8s
 
-Example approaches for running parallel FIO tests in Kubernetes to test either RWO or RWX volumes.
+Forked to have better defaults sourced from the [SRE
+Handbook](https://s905060.gitbooks.io/site-reliability-engineer-handbook/content/fio.html).
 
-For more detailed explanations please see the [accompanying blog post](https://medium.com/@joshua_robinson/storage-benchmarking-with-fio-in-kubernetes-14cf29dc5375).
+# Proxmox 6.4 VE
 
-Two different approaches are shown: 1) a simple deployment and PVC for RWX volumes and 2) a statefulset for RWO/RWX. Tradeoffs between the two approaches discussed below.
+My use case is to test a base install of the following stack:
 
-FIO job configs are stored as a configMap object in configs.yaml. There is currently one config, but additional configs could be added.
+ 1. 3x Dell R620 servers
+   * Each: CPU(s) 32 x Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (2 Sockets)
+   * 256 MB Ram
+   * 4x 556GB SAAS drives
+ 2. Proxmox 6.4 VE
+ 3. ZFS Raid 10
+ 4. Debian 10 VM
+ 5. Kubernetes 1.21.0
+ 6. 3x controller nodes
+ 7. 4x worker nodes
 
-The docker image is a simple Alpine image based on FIO and is only ~10MB in size.
+With a relatively un-tuned system the system is brought to a halt.  The kube
+API starts failing with errors related to etcd storage.  Luckily, deleting the
+deployment quickly resolves the situation.
 
-Deployment + PVC : The deployment makes scaling simple, with all pods connecting to the same dynamically provisioned persistentVolume and using a different output path to avoid collisions. This approach requires a single RWX volume, testing performance on a single shared filesystem.
+```
+Error from server: etcdserver: request timed out
+```
 
-Statefulset: each replica creates it's own volume, making this approach suitable for RWO volumes as well as RWX. Note that some shared filesystems will perform better if each node is using it's own filesystem, i.e., no true sharing.
+# Reference
+
+Forked from [this post](https://medium.com/@joshua_robinson/storage-benchmarking-with-fio-in-kubernetes-14cf29dc5375)

configs.yaml

@@ -4,21 +4,19 @@ apiVersion: v1
 metadata:
   name: fio-job-config
 data:
+  # https://s905060.gitbooks.io/site-reliability-engineer-handbook/content/fio.html
+  # 8K block, 16 jobs
   fio.job: |-
     [global]
-    ioengine=psync
+    ioengine=libaio
+    iodepth=128
+    rw=randrw
+    bs=8K
     direct=1
-    buffered=0
-    size=10G
-    iodepth=1000
-    numjobs=100
+    size=1G
+    numjobs=12
+    ramp_time=5
+    runtime=60
+    time_based
     group_reporting
-    refill_buffers
-    rwmixread=80
-    norandommap
-    randrepeat=0
-    percentage_random=0
-    bs=512K
-    buffer_compress_percentage=50
-    rw=read
     [testjob]

fio_deployment_pvc.yaml renamed to fio.yaml

@@ -15,6 +15,16 @@ spec:
       labels:
         app: fio
     spec:
+      affinity:
+        podAntiAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+          - labelSelector:
+              matchExpressions:
+              - key: app
+                operator: In
+                values:
+                - fio
+            topologyKey: "kubernetes.io/hostname"
       containers:
       - name: fio
         image: joshuarobinson/fio:3.19
@@ -32,17 +42,4 @@ spec:
         configMap:
           name: fio-job-config
       - name: fio-data
-        persistentVolumeClaim:
-          claimName: fio-claim
----
-kind: PersistentVolumeClaim
-apiVersion: v1
-metadata:
-  name: fio-claim
-spec:
-  storageClassName: pure-file
-  accessModes:
-  - ReadWriteMany
-  resources:
-    requests:
-      storage: 1Ti
+        emptyDir: {}