Portainer on MicroK8S + OpenEBS, MetalLB and Ingress

First some pre-reqs:

• Make sure that you run the commands as root
• Latest version of the Ubuntu server any other linux distribution/OS that can run MicroK8s (installation details can be found here: https://microk8s.io/docs)
• 3 nodes running MicroK8s v1.21
• Before deploying Portainer we have to make sure that MicroK8s has been deployed with the following addons enabled:
  • ha-cluster (installed by default on MicroK8S)
  • dns
  • rbac
  • storage

1. First step is to install MicroK8S on all three nodes by running snap install microk8s --classic --channel=1.21/stable:
   
   
   
2. Now you need to create the Kubernetes cluster by adding the nodes to a cluster by running microk8s add-node on the machine you will use and your main cluster server. Make sure you run the previous command on the main cluster node for every additional node you intend to include on the cluster. You cannot run the same output for all the nodes.
   
   
3. Let's create an alias for kubectl to make things easier bu running echo "alias kubectl='microk8s kubectl'" >> .bash_aliases && source .bash_aliases

4. Now check if all nodes are ready by running kubectl get nodes:
   
   
   
5. Once all nodes are with a Ready status you need to enable the iscsid driver required by OpenEBS by running on every node the following command: systemctl enable --now iscsid

6. We will now only work on the main node you can now close terminal access to the 2 other nodes.
   
   
7. Let's deploy the required plugins on your MicroK8S cluster by running microk8s enable rbac dns openebs storage
   
   This might take up to 5 minutes depending on your cluster setup.
   
   
8. OpenEBS might take some time to install. You can check that the plugin has finished deploying all it's pods by running watch microk8s kubectl get pod -n openebs:
   
9. Once the installation of the plugins above finish we need to make sure the OpenEBS plugin becomes the default StorageClass for your cluster by running the following two commands:
   
   kubectl patch storageclass openebs-jiva-default -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

kubectl patch storageclass microk8s-hostpath -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
   
   
10. This is required so you can install Portainer. You cannot have two default StorageClasses defined on your cluster otherwise Portainer will complain during the installation.
    
    
11. Now we can finally install Portainer by running microk8s enable portainer. Portainer on MicroK8S with OpenEBS takes slightly longer than with the default storage plugin because of the storage replicas that OpenEBS needs to create. You can check the progress of the deployment with watch microk8s kubectl get pod -n portainer.
    
    
12. Deploying Metallb and Ingress is quite simple by running microk8s enable ingress metallb

13. During the Metallb installation, you were asked to provide an IP range to be managed by the Load Balancer. Make sure you have an advanced route rule on your route/gateway that points to the main IP address of your main node. For example, in my case I have a routing rule on my router that directs any traffic of the 192.168.10.0 subnet to the IP address of my main node:
    
    
    
14. Once Portainer, Metallb, and Ingress are deployed you can open your browser and finish the installation by using the IP address of the main node and port 30777.
    
    
15. Final steps are:
    1. Create the first admin user:
       
    2. Connect Portainer to the Kubernetes cluster:
       
    3. Finish the Cluster Setup according to your Kubernetes environment:
       1. Enable MetalLB and the Ingress Controller. The name of the ingress class for the MicroK8S ingress plugin is public and the type ingress:
          
       2. Enable the openebs-jiva-default StorageClass: