Disco is a Service Discovery and Internal Custom DNS for Kubernetes.
It allows to customize Pod and Service discovery by annotating them with an annotation disco.min.io, this annotation supports jsonpath expressions, and either adding Disco as a DNS service either at cluster level or pod level via dnsPolicy
For example, after setting up Disco as a service we can configure a new Statefulset to use it to resolve the name of the peer replicas. So if we setup a MinIO instance that looks for it's peers at at hostname zone-1-{0...3}.zone-1 we can indicate this to the Disco via the annotation:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: zone-1
namespace: default
spec:
podManagementPolicy: Parallel
replicas: 4
selector:
matchLabels:
app: minio
controller: zone-1
serviceName: zone-1
template:
metadata:
labels:
app: minio
controller: zone-1
annotations:
disco.min.io: '{.metadata.name}.{.metadata.labels.controller}'
spec:
dnsPolicy: "None"
dnsConfig:
nameservers:
- 10.110.109.99
containers:
- name: minio
args:
- server
- http://zone-1-{0...3}.zone-1/data{1...4}You could have Disco announce a service by annotating it as well.
apiVersion: v1
kind: Service
metadata:
name: myservice
annotations:
disco.min.io: mycustom.domain.xyz
namespace: default
spec:
clusterIP: 10.0.11.208
ports:
- name: http-minio
port: 9000
protocol: TCP
targetPort: 9000
selector:
v1.min.io/instance: bigdata
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}You can deploy with kustomize
kustomize build deployment/base | kubectl apply -f -After deploying disco you should find out the IP for the minio-disco service since that is going to be used to configure the top level minio.local domain.
$ kubectl get svc minio-disco -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
minio-disco ClusterIP 10.109.234.52 <none> 53/UDP,53/TCP 12m app=minio-disco
Here we can see the IP is 10.109.234.52 so we are going to add that to the Corefile stored in the coredns configmap inside the kube-system namespace.
$ kubectl -n kube-system edit configmap corendsand add at the end of Corefile
minio.local:53 {
errors
cache 30
forward . 10.109.234.52
}The file should look like this
apiVersion: v1
kind: ConfigMap
metadata:
name: coredns
namespace: kube-system
data:
Corefile: |
.:53 {
errors
health {
lameduck 5s
}
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
}
prometheus :9153
forward . /etc/resolv.conf
cache 30
loop
reload
loadbalance
}
minio.local:53 {
errors
cache 30
forward . 10.109.234.52
}Afterwards, restart the coredns pods on the kube-system namespace
$ kubectl -n kube-system delete pod $(kubectl -n kube-system get pods | grep coredns | awk '{print $1}')Use of disco is governed by the AGPLv3 license that can be found in the LICENSE file.