CNI/Cannot list pods forbidden user.

I'm currently on the 
Bootstrapping the Kubernetes Worker Nodes section and
I get the following error when I run the sudo systemctl status containerd kubelet kube-proxy command on my worker node:

containerd[4150]: time="2018-12-20T23:37:17Z" level=error msg="Failed to load cni during init, please check CRI plugin status before setting up network for pods" error="cni config load failed: no network config found in /etc/cni/net.d: cni plugin not initialized: failed to load cni config"

kubelet[4158]: E1220 23:57:43.963669    4158 reflector.go:205] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list pods

When I run the kubectl get nodes from the controller node. I only see one of my two worker nodes.

I've redownloaded the binaries from this section and recreated the relevant config files from the bootstrapping the kubernetes worker nodes section and the same issue appears. Any idea where to debug from here?

  • post-author-pic
    Michael M
    12-21-2018

    Hello, 


    It is possible that the issues are related to one another, for the issue with containerd it is possible that you have a port conflict. 
    Either that or the error is actually what is indicated. 
    The second error can be related to RBAC but I am not sure that this applies in your case. 
    I would check that there is nothing that is running on the port that containerd wants I think it is in the 10050 10010 range. Validate that the ports are not conflicting with something else.  Once you get containerd to start then work the second issue if it is still there.


  • post-author-pic
    Victor C
    12-22-2018

    Have you installed the weave net already ?

    Run,  kubectl get pods -n kube-system  ,  to check the status of the  weave pods.  Then run,   kubectl describe pod weave-pod-name  -n kube-system   ,  on the failing pod to get more details.  When the weave pod is initialized, the config (10-weave.conflist) is copied to /etc/cni/net.d on the worker node.  

    For the second error, it looks like the worker node is not authenticated properly. Check your client certificate if the hostname and IP address are valid. 

    Run,  openssl x509 -in worker.pem -text -noout | grep "IP\|DNS"   , to display the hostname and IP. Substitute your worker's hostname.
     

Looking For Team Training?

Learn More