[POC] Introduce production-ready Kubernetes Cluster

[mrnicegyu11]

TLDR

• this issue presents a general overview of fundamental tasks / progress done to switch to Kubernets. It contains all critical milestones but not all issues (use kubernetes label to find all issues)

Tasks: Running services

• Kubernetes: run adminer on a cloud machine via k8s. #669
• Kubernetes: move adminer to k8s in in-house master #695
• Kubernetes: aws master simcore resource usage tracker #750

Tasks: Cluster and System Requirements

• provision kubernetes cluster

  • document cluster creation https://git.speag.com/oSparc/osparc-infra/-/merge_requests/226/
  • automate with makefile https://git.speag.com/oSparc/osparc-infra/-/merge_requests/251
  • fine tune for on-premise deployments https://git.speag.com/oSparc/osparc-infra/-/merge_requests/242

• Kubernetes: multi-node cluster and node-labelled pods #805

• Kubernetes: add 2nd node for in-house Master #948

• Kubernets: introduce production ready volume management #855

• Kubernetes: Add monitoring / tracing / logging #1036

• Add test monitoring cluster certificates expiration (ones used by api server and others [see related PR])

  • update admin.conf for all users (k8s certs renew from kubespray only updates admin.conf for root)

• Kubernetes: properly reuse TLS certificates  #1228

• Authenticate docker pulls to avoid rate limits

• Kubernetes: set up local deployment & tooling #696

• k8s - Add Service Overview (Portainer) #819

Wishes:

• Have a read-only user functionality (e.g. to safely execute e2e cluster state tests)
• Managing charts on demand in different CI Pipelines (e.g. delete all charts for automatic shutdown)

Tasks: CI/CD pipelines

• Deploy / Update a particular service only --> use helmfile selectors

Security

• Kubernetes: security #1122

Open questions / improvements

• Use node labels and attach run charts based on labels
• How to separate all simcore charts ?
• Kubernetes: simcore images are not updated on the master stage #946
• k8s-csi-s3 has no label and no resource constraints #947
• Kubernetes enhancements #688
• Design / Define version updates flow (kubernetes versions, cluster component [e.g. calico], chart versions,etc)

[mrnicegyu11]

Task: Run adminer on a cloud machine via k8s.

YH Update: it is done ✔️