Multiple lines for the same host in SWARM cluster

hi 
Can any one explain the behavior..
Whenever, i remove the worker node, manager node does not remove the worker from its list 'docker node ls' and show it as DOWN. And, whenever, i again add worker on the same host, manager host shows new line item as READY. Old entry remains as it..
Q. When the worker has left the swarm then why Manager is not deleting the same from its cluster group. is there any configured time when these down entries would be deleted. If yes, where it is configured.

ID                            HOSTNAME                       STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
3oppmfpkt35gakwmfbtb2tcdc * tulip2.mylabserver.com Ready Active Leader 18.09.1
befmoz32tpq2d8m5kscuwl55b tulip6.mylabserver.com Down Active 18.09.1
uoeyq091r0wowms2cuw2t9wdg tulip6.mylabserver.com Ready Active 18.09.1
xr35jnhfr3se4jzt3yoh113ns tulip6.mylabserver.com Down Active 18.09.1




  • post-author-pic
    Terrence C
    01-20-2019

    The removed node would be completely removed from the list at the next daemon restart. To test, remove a node and view it shows as down. Restart the docker daemon on the management node, relist the nodes that are detected and the removed/down node should no longer be in the list. Subsequently adding it back will show it as ready. Each remove/add, will assign it a new ID in the cluster.

  • post-author-pic
    Sunil A
    01-20-2019

    Hi Terrence, 

    Thanks for your revert. I stopped and started docker service during swarm lock and unlock exercise, however, it did not helped. Tried again, but no relief, is there any other way to remove these DOWN entries..

    [user@tulip2 ~]$ docker node ls
    ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
    3oppmfpkt35gakwmfbtb2tcdc * tulip2.mylabserver.com Ready Active Leader 18.09.1
    befmoz32tpq2d8m5kscuwl55b tulip6.mylabserver.com Down Active 18.09.1
    uoeyq091r0wowms2cuw2t9wdg tulip6.mylabserver.com Ready Active 18.09.1
    xr35jnhfr3se4jzt3yoh113ns tulip6.mylabserver.com Down Active 18.09.1

    [user@tulip2 ~]$ sudo systemctl stop docker
    [sudo] password for user:

    [user@tulip2 ~]$ docker node ls
    Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

    [user@tulip2 ~]$ sudo systemctl start docker

    [user@tulip2 ~]$ docker node ls
    ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
    3oppmfpkt35gakwmfbtb2tcdc * tulip2.mylabserver.com Ready Active Leader 18.09.1
    befmoz32tpq2d8m5kscuwl55b tulip6.mylabserver.com Down Active 18.09.1
    uoeyq091r0wowms2cuw2t9wdg tulip6.mylabserver.com Ready Active 18.09.1
    xr35jnhfr3se4jzt3yoh113ns tulip6.mylabserver.com Down Active 18.09.1


  • post-author-pic
    Sunil A
    01-20-2019

    One more question, docker node ls shows the worker nodes as DOWN if their hosting machine is down (not booted). Will Daemon restart on the management node will remove those worker nodes also, although these nodes have not left the swarm.

    Can you please share link to any reference document to understand the underlying messages flow during cluster formation and message flow when worker machine comes in service after reboot etc.

Looking For Team Training?

Learn More