Merge branch 'master' into chaos

ewoutp · ewoutp · commit bed3cb373fb6 · 2018-04-03T14:55:37.000+02:00
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -0,0 +1,103 @@
+# Change Log
+
+## [Unreleased](https://github.com/arangodb/kube-arangodb/tree/HEAD)
+
+[Full Changelog](https://github.com/arangodb/kube-arangodb/compare/0.0.1...HEAD)
+
+**Closed issues:**
+
+- make sure scripts terminate to avoid hanging CI [\#63](https://github.com/arangodb/kube-arangodb/issues/63)
+- prefix environment variables [\#62](https://github.com/arangodb/kube-arangodb/issues/62)
+- warning when passing string literal "None" as spec.tls.caSecretName [\#60](https://github.com/arangodb/kube-arangodb/issues/60)
+
+**Merged pull requests:**
+
+- Added helper to prepull arangodb \(enterprise\) image. This allows the normal tests to have decent timeouts while prevent a timeout caused by a long during image pull. [\#94](https://github.com/arangodb/kube-arangodb/pull/94)
+- Tracking recent pod terminations [\#91](https://github.com/arangodb/kube-arangodb/pull/91)
+- Enable LONG on kube-arangodb-long test [\#90](https://github.com/arangodb/kube-arangodb/pull/90)
+- Tests/modes [\#88](https://github.com/arangodb/kube-arangodb/pull/88)
+- increase timeout for long running tests [\#87](https://github.com/arangodb/kube-arangodb/pull/87)
+- fix rocksdb\_encryption\_test [\#86](https://github.com/arangodb/kube-arangodb/pull/86)
+- fix - /api/version will answer on all servers \(not leader only\) [\#85](https://github.com/arangodb/kube-arangodb/pull/85)
+- fixes required after merge [\#84](https://github.com/arangodb/kube-arangodb/pull/84)
+- Deployment state -\> phase [\#83](https://github.com/arangodb/kube-arangodb/pull/83)
+- Added detection on unschedulable pods [\#82](https://github.com/arangodb/kube-arangodb/pull/82)
+- AsOwner no longer things the owner refers to a controller. It refers to the ArangoDeployment [\#81](https://github.com/arangodb/kube-arangodb/pull/81)
+- Store & compare hash of secrets. [\#80](https://github.com/arangodb/kube-arangodb/pull/80)
+- Control jenkins from git commit log. [\#79](https://github.com/arangodb/kube-arangodb/pull/79)
+- Fix scale-up [\#78](https://github.com/arangodb/kube-arangodb/pull/78)
+- Added terminated-pod cleanup to speed up re-creation of pods. [\#77](https://github.com/arangodb/kube-arangodb/pull/77)
+- add upgrade tests [\#76](https://github.com/arangodb/kube-arangodb/pull/76)
+- check result of api version call [\#75](https://github.com/arangodb/kube-arangodb/pull/75)
+- Also watch changes in PVCs and Services [\#74](https://github.com/arangodb/kube-arangodb/pull/74)
+- Feature/test individual pod deletion [\#72](https://github.com/arangodb/kube-arangodb/pull/72)
+- Moved low level resource \(pod,pvc,secret,service\) creation & inspection to resources sub-package. [\#71](https://github.com/arangodb/kube-arangodb/pull/71)
+- Moved reconciliation code to separate package [\#70](https://github.com/arangodb/kube-arangodb/pull/70)
+- Test/different deployments resilient [\#69](https://github.com/arangodb/kube-arangodb/pull/69)
+- Store accepted spec [\#68](https://github.com/arangodb/kube-arangodb/pull/68)
+- Fixed behavior for scaling UI integration wrt startup of the cluster [\#67](https://github.com/arangodb/kube-arangodb/pull/67)
+- Fixed immitable `mode` field. [\#66](https://github.com/arangodb/kube-arangodb/pull/66)
+- Integrate with scaling web-UI [\#65](https://github.com/arangodb/kube-arangodb/pull/65)
+- add test for different deployments [\#64](https://github.com/arangodb/kube-arangodb/pull/64)
+- Fixed validation of tls.caSecretName=None [\#61](https://github.com/arangodb/kube-arangodb/pull/61)
+- Feature/add tests for immutable cluster parameters [\#59](https://github.com/arangodb/kube-arangodb/pull/59)
+- rename test function [\#58](https://github.com/arangodb/kube-arangodb/pull/58)
+- Detecting ImageID & ArangoDB version. [\#57](https://github.com/arangodb/kube-arangodb/pull/57)
+- Adds ssl support for scaling test [\#53](https://github.com/arangodb/kube-arangodb/pull/53)
+- Rotation support for members. [\#49](https://github.com/arangodb/kube-arangodb/pull/49)
+- begin to add tests for `apis/storage/v1alpha` [\#36](https://github.com/arangodb/kube-arangodb/pull/36)
+
+## [0.0.1](https://github.com/arangodb/kube-arangodb/tree/0.0.1) (2018-03-20)
+**Merged pull requests:**
+
+- Changed scope of ArangoLocalStorage to Cluster. [\#56](https://github.com/arangodb/kube-arangodb/pull/56)
+- External crd creation [\#55](https://github.com/arangodb/kube-arangodb/pull/55)
+- Rename default docker image to kube-arangodb [\#54](https://github.com/arangodb/kube-arangodb/pull/54)
+- Splitting operator in two parts [\#52](https://github.com/arangodb/kube-arangodb/pull/52)
+- Turn on TLS by default [\#51](https://github.com/arangodb/kube-arangodb/pull/51)
+- Rename repository to `kube-arangodb` [\#48](https://github.com/arangodb/kube-arangodb/pull/48)
+- Use single image tag to prevent polluting the docker hub [\#47](https://github.com/arangodb/kube-arangodb/pull/47)
+- Renamed pkg/apis/arangodb to pkg/apis/deployment [\#46](https://github.com/arangodb/kube-arangodb/pull/46)
+- Added release code [\#45](https://github.com/arangodb/kube-arangodb/pull/45)
+- Cleaning up deployment, avoiding docker overrides [\#44](https://github.com/arangodb/kube-arangodb/pull/44)
+- TLS support [\#43](https://github.com/arangodb/kube-arangodb/pull/43)
+- Adds "Storage Resource" to user README [\#42](https://github.com/arangodb/kube-arangodb/pull/42)
+- Reworked TLS spec [\#41](https://github.com/arangodb/kube-arangodb/pull/41)
+- Set sesion affinity for coordinator [\#40](https://github.com/arangodb/kube-arangodb/pull/40)
+- Set PublishNotReadyAddresses on coordinator&syncmasters service [\#39](https://github.com/arangodb/kube-arangodb/pull/39)
+- Prepare test cluster [\#38](https://github.com/arangodb/kube-arangodb/pull/38)
+- Run tests on multiple clusters in parallel [\#37](https://github.com/arangodb/kube-arangodb/pull/37)
+- Implemented isDefault behavior of storage class [\#35](https://github.com/arangodb/kube-arangodb/pull/35)
+- add some tests for util/k8sutil/erros.go [\#32](https://github.com/arangodb/kube-arangodb/pull/32)
+- Adding `ArangoLocalStorage` resource \(wip\) [\#31](https://github.com/arangodb/kube-arangodb/pull/31)
+- Added custom resource spec for ArangoDB Storage operator. [\#30](https://github.com/arangodb/kube-arangodb/pull/30)
+- Added unit tests for k8s secrets & utility methods [\#28](https://github.com/arangodb/kube-arangodb/pull/28)
+- Added unit test for creating affinity [\#27](https://github.com/arangodb/kube-arangodb/pull/27)
+- More simple tests [\#26](https://github.com/arangodb/kube-arangodb/pull/26)
+- Changed default storage engine to RocksDB [\#24](https://github.com/arangodb/kube-arangodb/pull/24)
+- Adding command line tests for arangod commandlines. [\#23](https://github.com/arangodb/kube-arangodb/pull/23)
+- UnitTests for plan\_builder [\#22](https://github.com/arangodb/kube-arangodb/pull/22)
+- Unit tests for apis/arangodb/v1alpha package [\#21](https://github.com/arangodb/kube-arangodb/pull/21)
+- Fix bash error [\#20](https://github.com/arangodb/kube-arangodb/pull/20)
+- Renamed Controller to Operator [\#19](https://github.com/arangodb/kube-arangodb/pull/19)
+- Cleanup kubernetes after tests [\#18](https://github.com/arangodb/kube-arangodb/pull/18)
+- Adding rocksdb encryption key support [\#17](https://github.com/arangodb/kube-arangodb/pull/17)
+- Adding test design [\#16](https://github.com/arangodb/kube-arangodb/pull/16)
+- avoid sub-shell creation [\#15](https://github.com/arangodb/kube-arangodb/pull/15)
+- Adding authentication support [\#14](https://github.com/arangodb/kube-arangodb/pull/14)
+- Scaling deployments [\#13](https://github.com/arangodb/kube-arangodb/pull/13)
+- Test framework [\#11](https://github.com/arangodb/kube-arangodb/pull/11)
+- Change docs to "authentication default on" [\#10](https://github.com/arangodb/kube-arangodb/pull/10)
+- Pod monitoring [\#9](https://github.com/arangodb/kube-arangodb/pull/9)
+- Pod affinity [\#8](https://github.com/arangodb/kube-arangodb/pull/8)
+- Extended storage docs wrt local storage [\#7](https://github.com/arangodb/kube-arangodb/pull/7)
+- Adding event support [\#6](https://github.com/arangodb/kube-arangodb/pull/6)
+- Added pod probes [\#5](https://github.com/arangodb/kube-arangodb/pull/5)
+- Creating pods [\#4](https://github.com/arangodb/kube-arangodb/pull/4)
+- Extending spec & status object. Implementing service & pvc creation [\#3](https://github.com/arangodb/kube-arangodb/pull/3)
+- Initial API objects & vendoring [\#2](https://github.com/arangodb/kube-arangodb/pull/2)
+- Added specification of custom resource [\#1](https://github.com/arangodb/kube-arangodb/pull/1)
+
+
+
+\* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)*
diff --git a/Makefile b/Makefile
@@ -273,6 +273,20 @@ endif
 
 # Release building
 
+.PHONY: patch-readme
+patch-readme:
+	$(ROOTDIR)/scripts/patch_readme.sh $(VERSION_MAJOR_MINOR_PATCH)
+
+.PHONY: update-changelog
+changelog:
+	docker run -it --rm \
+		-e CHANGELOG_GITHUB_TOKEN=$(shell cat ~/.arangodb/github-token) \
+		-v "$(ROOTDIR)":/usr/local/src/your-app \
+		ferrarimarco/github-changelog-generator \
+		--user arangodb \
+		--project kube-arangodb \
+		--no-author
+
 .PHONY: docker-push
 docker-push: docker
 ifneq ($(DOCKERNAMESPACE), arangodb)
diff --git a/main.go b/main.go
@@ -48,6 +48,7 @@ import (
 	"github.com/arangodb/kube-arangodb/pkg/operator"
 	"github.com/arangodb/kube-arangodb/pkg/util/constants"
 	"github.com/arangodb/kube-arangodb/pkg/util/k8sutil"
+	"github.com/arangodb/kube-arangodb/pkg/util/probe"
 	"github.com/arangodb/kube-arangodb/pkg/util/retry"
 )
 
@@ -82,6 +83,8 @@ var (
 	chaosOptions struct {
 		allowed bool
 	}
+	deploymentProbe probe.Probe
+	storageProbe    probe.Probe
 )
 
 func init() {
@@ -137,7 +140,9 @@ func cmdMainRun(cmd *cobra.Command, args []string) {
 		cliLog.Fatal().Err(err).Msg("Failed to get hostname")
 	}
 
-	//http.HandleFunc(probe.HTTPReadyzEndpoint, probe.ReadyzHandler)
+	http.HandleFunc("/health", probe.LivenessHandler)
+	http.HandleFunc("/ready/deployment", deploymentProbe.ReadyHandler)
+	http.HandleFunc("/ready/storage", storageProbe.ReadyHandler)
 	http.Handle("/metrics", prometheus.Handler())
 	listenAddr := net.JoinHostPort(server.host, strconv.Itoa(server.port))
 	go http.ListenAndServe(listenAddr, nil)
@@ -188,11 +193,13 @@ func newOperatorConfigAndDeps(id, namespace, name string) (operator.Config, oper
 		AllowChaos:       chaosOptions.allowed,
 	}
 	deps := operator.Dependencies{
-		LogService:    logService,
-		KubeCli:       kubecli,
-		KubeExtCli:    kubeExtCli,
-		CRCli:         crCli,
-		EventRecorder: eventRecorder,
+		LogService:      logService,
+		KubeCli:         kubecli,
+		KubeExtCli:      kubeExtCli,
+		CRCli:           crCli,
+		EventRecorder:   eventRecorder,
+		DeploymentProbe: &deploymentProbe,
+		StorageProbe:    &storageProbe,
 	}
 
 	return cfg, deps, nil
diff --git a/manifests/templates/deployment/deployment.yaml b/manifests/templates/deployment/deployment.yaml
@@ -6,6 +6,8 @@ metadata:
   namespace: {{ .Deployment.Operator.Namespace }}
 spec:
   replicas: 1
+  strategy:
+    type: Recreate
   template:
     metadata:
       labels:
@@ -27,3 +29,18 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.name
+        ports:
+        - name: metrics
+          containerPort: 8528
+        livenessProbe:
+          httpGet:
+            path: /health
+            port: 8528
+          initialDelaySeconds: 5
+          periodSeconds: 10
+        readinessProbe:
+          httpGet:
+            path: /ready/deployment
+            port: 8528
+          initialDelaySeconds: 5
+          periodSeconds: 10
diff --git a/manifests/templates/storage/deployment.yaml b/manifests/templates/storage/deployment.yaml
@@ -14,6 +14,8 @@ metadata:
   namespace: {{ .Storage.Operator.Namespace }}
 spec:
   replicas: 1
+  strategy:
+    type: Recreate
   template:
     metadata:
       labels:
@@ -35,3 +37,19 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.name
+        ports:
+        - name: metrics
+          containerPort: 8528
+        livenessProbe:
+          httpGet:
+            path: /health
+            port: 8528
+          initialDelaySeconds: 5
+          periodSeconds: 10
+        readinessProbe:
+          httpGet:
+            path: /ready/storage
+            port: 8528
+          initialDelaySeconds: 5
+          periodSeconds: 10
+      
diff --git a/pkg/operator/operator.go b/pkg/operator/operator.go
@@ -39,6 +39,7 @@ import (
 	"github.com/arangodb/kube-arangodb/pkg/generated/clientset/versioned"
 	"github.com/arangodb/kube-arangodb/pkg/logging"
 	"github.com/arangodb/kube-arangodb/pkg/storage"
+	"github.com/arangodb/kube-arangodb/pkg/util/probe"
 )
 
 const (
@@ -71,11 +72,13 @@ type Config struct {
 }
 
 type Dependencies struct {
-	LogService    logging.Service
-	KubeCli       kubernetes.Interface
-	KubeExtCli    apiextensionsclient.Interface
-	CRCli         versioned.Interface
-	EventRecorder record.EventRecorder
+	LogService      logging.Service
+	KubeCli         kubernetes.Interface
+	KubeExtCli      apiextensionsclient.Interface
+	CRCli           versioned.Interface
+	EventRecorder   record.EventRecorder
+	DeploymentProbe *probe.Probe
+	StorageProbe    *probe.Probe
 }
 
 // NewOperator instantiates a new operator from given config & dependencies.
diff --git a/pkg/operator/operator_deployment.go b/pkg/operator/operator_deployment.go
@@ -58,6 +58,7 @@ func (o *Operator) runDeployments(stop <-chan struct{}) {
 		DeleteFunc: o.onDeleteArangoDeployment,
 	}, cache.Indexers{})
 
+	o.Dependencies.DeploymentProbe.SetReady()
 	informer.Run(stop)
 }
 
diff --git a/pkg/operator/operator_local_storage.go b/pkg/operator/operator_local_storage.go
@@ -58,6 +58,7 @@ func (o *Operator) runLocalStorages(stop <-chan struct{}) {
 		DeleteFunc: o.onDeleteArangoLocalStorage,
 	}, cache.Indexers{})
 
+	o.Dependencies.StorageProbe.SetReady()
 	informer.Run(stop)
 }
 
diff --git a/pkg/util/probe/health.go b/pkg/util/probe/health.go
@@ -0,0 +1,32 @@
+//
+// DISCLAIMER
+//
+// Copyright 2018 ArangoDB GmbH, Cologne, Germany
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Copyright holder is ArangoDB GmbH, Cologne, Germany
+//
+// Author Ewout Prangsma
+//
+
+package probe
+
+import (
+	"net/http"
+)
+
+// LivenessHandler writes back the HTTP status code 200 to indicate a healthy operator.
+func LivenessHandler(w http.ResponseWriter, r *http.Request) {
+	w.WriteHeader(http.StatusOK)
+}
diff --git a/pkg/util/probe/ready.go b/pkg/util/probe/ready.go
@@ -0,0 +1,48 @@
+//
+// DISCLAIMER
+//
+// Copyright 2018 ArangoDB GmbH, Cologne, Germany
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Copyright holder is ArangoDB GmbH, Cologne, Germany
+//
+// Author Ewout Prangsma
+//
+
+package probe
+
+import (
+	"net/http"
+	"sync/atomic"
+)
+
+// Probe wraps a readiness probe handler.
+type Probe struct {
+	ready int32
+}
+
+// SetReady marks the probe as ready.
+func (p *Probe) SetReady() {
+	atomic.StoreInt32(&p.ready, 1)
+}
+
+// ReadyHandler writes back the HTTP status code 200 if the operator is ready, and 500 otherwise.
+func (p *Probe) ReadyHandler(w http.ResponseWriter, r *http.Request) {
+	isReady := atomic.LoadInt32(&p.ready) != 0
+	if isReady {
+		w.WriteHeader(http.StatusOK)
+	} else {
+		w.WriteHeader(http.StatusInternalServerError)
+	}
+}
diff --git a/scripts/patch_readme.sh b/scripts/patch_readme.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+# Updates the installation instructions in README.md to reflect the current 
+# version.
+
+VERSION=$1
+
+if [ -z $VERSION ]; then
+    echo "Specify a version argument"
+    exit 1
+fi
+
+f=README.md
+sed -e "s@^kubectl apply -f https://raw.githubusercontent.com/arangodb/kube-arangodb/.*/manifests/crd.yaml\$@kubectl apply -f https://raw.githubusercontent.com/arangodb/kube-arangodb/${VERSION}/manifests/crd.yaml@g" -i "" $f
+sed -e "s@^kubectl apply -f https://raw.githubusercontent.com/arangodb/kube-arangodb/.*/manifests/arango-deployment.yaml\$@kubectl apply -f https://raw.githubusercontent.com/arangodb/kube-arangodb/${VERSION}/manifests/arango-deployment.yaml@g" -i "" $f
+sed -e "s@^kubectl apply -f https://raw.githubusercontent.com/arangodb/kube-arangodb/.*/manifests/arango-storage.yaml\$@kubectl apply -f https://raw.githubusercontent.com/arangodb/kube-arangodb/${VERSION}/manifests/arango-storage.yaml@g" -i "" $f
diff --git a/tools/release/release.go b/tools/release/release.go
@@ -63,9 +63,12 @@ func main() {
 		"IMAGETAG":        version,
 		"MANIFESTSUFFIX":  "-",
 	})
+	make("patch-readme", nil)
 	make("build-ghrelease", nil)
 	gitCommitAll(fmt.Sprintf("Updated manifest to %s", version)) // Commit manifest
 	gitTag(version)
+	make("changelog", nil)
+	gitCommitAll(fmt.Sprintf("Updated changelog for %s", version)) // Commit CHANGELOG.md
 	githubCreateRelease(version)
 	bumpVersion("devel")
 }

Original file line number	Diff line number	Diff line change
`@@ -58,6 +58,7 @@ func (o *Operator) runDeployments(stop <-chan struct{}) {`
`58`	`58`	`DeleteFunc: o.onDeleteArangoDeployment,`
`59`	`59`	`}, cache.Indexers{})`
`60`	`60`
	`61`	`+ o.Dependencies.DeploymentProbe.SetReady()`
`61`	`62`	`informer.Run(stop)`
`62`	`63`	`}`
`63`	`64`