docker swarm部署rabbitmq集群

swarm集群搭建看这里，本篇不做概述

在现在的docker swarm 环境上搭建RabbitMQ

1.使用docker-compose编排构建集群

参考https://github.com/oprearocks/RabbitMQ-Docker-cluster
进行forked后修改为适合自己的，看源码的文件列表

1 2	参考https://github.com/oprearocks/RabbitMQ-Docker-cluster 进行forked后修改为适合自己的，看源码的文件列表

按作者的源码，需要将 definitions.json 和 rabbitmq.config 放到所有需要mq的机器上，看源码的docker-compose.yml 其中需要将两个配置文件映射到容器中

1	按作者的源码，需要将 definitions.json 和 rabbitmq.config 放到所有需要mq的机器上，看源码的docker-compose.yml 其中需要将两个配置文件映射到容器中

volumes:
      - "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"
      - "./definitions.json:/etc/rabbitmq/definitions.json"

volumes:

- "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"

- "./definitions.json:/etc/rabbitmq/definitions.json"

并且还需要事先创建network

docker network create -d overlay rabbitmq-cluster

1	docker network create -d overlay rabbitmq-cluster

按照docker-compose.yml中的绝对路径和需要加载的文件配置，将如下两个文件放到每个服务器的相应位置

mkdir /etc/rabbitmq  #这里应该会自动创建,可以自己试试,我是事先创建好了

按照docker-compose.yml中的绝对路径和需要加载的文件配置，将如下两个文件放到每个服务器的相应位置

mkdir /etc/rabbitmq #这里应该会自动创建,可以自己试试,我是事先创建好了

docker-composer.yaml

version: "3"
services:
  rabbit1:
    image: rabbitmq:3-management
    hostname: rabbit1
    environment:
      RABBITMQ_ERLANG_COOKIE: "secret string"
      RABBITMQ_NODENAME: rabbit1
    volumes:
      - "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"
      - "./definitions.json:/etc/rabbitmq/definitions.json"
    ports:
      - "4369:4369"
      - "5671:5671"
      - "5672:5672"
      - "15671:15671"
      - "15672:15672"
      - "25672:25672"

  rabbit2:
    image: rabbitmq:3-management
    hostname: rabbit2
    depends_on:
      - rabbit1
    environment:
      RABBITMQ_ERLANG_COOKIE: "secret string"
      RABBITMQ_NODENAME: rabbit2
    volumes:
      - "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"
      - "./definitions.json:/etc/rabbitmq/definitions.json"
    
  rabbit3:
    image: rabbitmq:3-management
    hostname: rabbit3
    depends_on:
      - rabbit1
    environment:
      RABBITMQ_ERLANG_COOKIE: "secret string"
      RABBITMQ_NODENAME: rabbit3
    volumes:
      - "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"
      - "./definitions.json:/etc/rabbitmq/definitions.json"

networks:
  default:
    external:
      name: rabbitmq-cluster

version: "3"

services:

rabbit1:

image: rabbitmq:3-management

hostname: rabbit1

environment:

RABBITMQ_ERLANG_COOKIE: "secret string"

RABBITMQ_NODENAME: rabbit1

volumes:

- "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"

- "./definitions.json:/etc/rabbitmq/definitions.json"

ports:

- "4369:4369"

- "5671:5671"

- "5672:5672"

- "15671:15671"

- "15672:15672"

- "25672:25672"

rabbit2:

image: rabbitmq:3-management

hostname: rabbit2

depends_on:

- rabbit1

environment:

RABBITMQ_ERLANG_COOKIE: "secret string"

RABBITMQ_NODENAME: rabbit2

volumes:

- "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"

- "./definitions.json:/etc/rabbitmq/definitions.json"

rabbit3:

image: rabbitmq:3-management

hostname: rabbit3

depends_on:

- rabbit1

environment:

RABBITMQ_ERLANG_COOKIE: "secret string"

RABBITMQ_NODENAME: rabbit3

volumes:

- "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"

- "./definitions.json:/etc/rabbitmq/definitions.json"

networks:

default:

external:

name: rabbitmq-cluster

进入目录执行部署

docker stack deploy rabbit -c docker-compose.yml

1	docker stack deploy rabbit -c docker-compose.yml

在Marster服务器上查看service rabbit_rabbit1的具体部署

docker service ps rabbit_rabbit1

docker service ps -f "desired-state=running" rabbit_rabbit{1..3}

docker service ps rabbit_rabbit1

docker service ps -f "desired-state=running" rabbit_rabbit{1..3}

三个rabbitmq 都装在了master机器（这个问题最后边解释处理）

访问 rabbitmq管理后台：http://localhost:15672/ 查看

糟糕的是rabbit1和rabbit2建立了集群，但是rabbit3并没有。下面继续在xubuntu3号节点上用CLI查看rabbitMQ集群部署情况。

docker container ls 
docker container exec -it rabbit_rabbit3.1.ccu7cqc2gqkme5g44s31lhllw bash
#查看集群状况
#rabbitmqctl cluster_status

docker container ls

docker container exec -it rabbit_rabbit3.1.ccu7cqc2gqkme5g44s31lhllw bash

#查看集群状况

#rabbitmqctl cluster_status

发现确实只有rabbit1和rabbit2两个节点集群，下面让rabbit3也加入集群。我承认这个解决方法并不完美，但是可用… 应该能直接让docker启动的rabbit容器自动集群，有待优化

进入rabbitm3容器

docker container exec -it rabbit_rabbit3.1.ccu7cqc2gqkme5g44s31lhllw bash

[root@swarm-master rabbmq]# docker container exec -it rabbit_rabbit3.1.ccu7cqc2gqkme5g44s31lhllw bash
root@rabbit3:/# rabbitmqctl stop_app
Stopping rabbit application on node rabbit3@rabbit3 ...
root@rabbit3:/# rabbitmqctl join_cluster rabbit1@rabbit1
Clustering node rabbit3@rabbit3 with rabbit1@rabbit1
root@rabbit3:/# rabbitmqctl start_app
Starting node rabbit3@rabbit3 ...
 completed with 3 plugins.
root@rabbit3:/#

docker container exec -it rabbit_rabbit3.1.ccu7cqc2gqkme5g44s31lhllw bash

[root@swarm-master rabbmq]# docker container exec -it rabbit_rabbit3.1.ccu7cqc2gqkme5g44s31lhllw bash

root@rabbit3:/# rabbitmqctl stop_app

Stopping rabbit application on node rabbit3@rabbit3 ...

root@rabbit3:/# rabbitmqctl join_cluster rabbit1@rabbit1

Clustering node rabbit3@rabbit3 with rabbit1@rabbit1

root@rabbit3:/# rabbitmqctl start_app

Starting node rabbit3@rabbit3 ...

completed with 3 plugins.

root@rabbit3:/#

登录rabbitMQ管理后台，发现已经成功加入

三个服务都装在了Master1节点

猜想:可能因为我把其他工作节点之前都升级为了Master节点(高可用), 而我执行yaml脚本后，由于主节点找不到工作节点，所以将所有的服务装在了自身节点:

验证：

1.取消其他2个节点为master节点

docker node demote swarm-node1
docker node demote swarm-node2

1 2	docker node demote swarm-node1 docker node demote swarm-node2

查看

2.再次执行yaml脚本

docker stack deploy rabbit -c rabbit-proxy.yaml

1	docker stack deploy rabbit -c rabbit-proxy.yaml

现在主节点只有swarm-master

最终找到了原因，是因为其他节点下，使用上面那个脚本，需要将那2个配置文件放置到每一个要启动的节点，因为我没有放，所以起不来，所以系统最终将所有服务安装在了master节点

按作者的源码，需要将 definitions.json 和 rabbitmq.config 放到所有需要mq的机器上，看源码的docker-compose.yml 其中需要将两个配置文件映射到容器中

volumes:
      - "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"
      - "./definitions.json:/etc/rabbitmq/definitions.json"

并且还需要事先创建network

以下是我修改后的版本 修改后的源码
修改后的docker-compose.yml，不需要在每台mq的机器上放置两个配置文件，而是使用了docker-compose编排3.3版本的docker config

按作者的源码，需要将 definitions.json 和 rabbitmq.config 放到所有需要mq的机器上，看源码的docker-compose.yml 其中需要将两个配置文件映射到容器中

volumes:

- "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"

- "./definitions.json:/etc/rabbitmq/definitions.json"

并且还需要事先创建network

以下是我修改后的版本修改后的源码

修改后的docker-compose.yml，不需要在每台mq的机器上放置两个配置文件，而是使用了docker-compose编排3.3版本的docker config

version: "3"
services:
  rabbit1:
    image: rabbitmq:3-management
    hostname: rabbit1
    environment:
      RABBITMQ_ERLANG_COOKIE: "secret string"
      RABBITMQ_NODENAME: rabbit1
    volumes:
      - "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"
      - "./definitions.json:/etc/rabbitmq/definitions.json"
    ports:
      - "4369:4369"
      - "5671:5671"
      - "5672:5672"
      - "15671:15671"
      - "15672:15672"
      - "25672:25672"

  rabbit2:
    image: rabbitmq:3-management
    hostname: rabbit2
    depends_on:
      - rabbit1
    environment:
      RABBITMQ_ERLANG_COOKIE: "secret string"
      RABBITMQ_NODENAME: rabbit2
    volumes:
      - "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"
      - "./definitions.json:/etc/rabbitmq/definitions.json"
    
  rabbit3:
    image: rabbitmq:3-management
    hostname: rabbit3
    depends_on:
      - rabbit1
    environment:
      RABBITMQ_ERLANG_COOKIE: "secret string"
      RABBITMQ_NODENAME: rabbit3
    volumes:
      - "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"
      - "./definitions.json:/etc/rabbitmq/definitions.json"

networks:
  default:
    external:
      name: rabbitmq-cluster

version: "3"

services:

rabbit1:

image: rabbitmq:3-management

hostname: rabbit1

environment:

RABBITMQ_ERLANG_COOKIE: "secret string"

RABBITMQ_NODENAME: rabbit1

volumes:

- "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"

- "./definitions.json:/etc/rabbitmq/definitions.json"

ports:

- "4369:4369"

- "5671:5671"

- "5672:5672"

- "15671:15671"

- "15672:15672"

- "25672:25672"

rabbit2:

image: rabbitmq:3-management

hostname: rabbit2

depends_on:

- rabbit1

environment:

RABBITMQ_ERLANG_COOKIE: "secret string"

RABBITMQ_NODENAME: rabbit2

volumes:

- "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"

- "./definitions.json:/etc/rabbitmq/definitions.json"

rabbit3:

image: rabbitmq:3-management

hostname: rabbit3

depends_on:

- rabbit1

environment:

RABBITMQ_ERLANG_COOKIE: "secret string"

RABBITMQ_NODENAME: rabbit3

volumes:

- "./rabbitmq.config:/etc/rabbitmq/rabbitmq.config"

- "./definitions.json:/etc/rabbitmq/definitions.json"

networks:

default:

external:

name: rabbitmq-cluster

docker stack deploy rabbit -c rabbit.yaml

1	docker stack deploy rabbit -c rabbit.yaml

工作节点排除/允许manager，manager只作为管理节点

1.排除(排除后manager只作为管理节点)
docker node update --availability drain swarm-master
2.允许
docker node update –availability active swarm-master

1.排除(排除后manager只作为管理节点)

docker node update --availability drain swarm-master

2.允许

docker node update –availability active swarm-master

本文固定链接: https://www.yoyoask.com/?p=3333
转载请注明: shooter 2019年09月22日于 SHOOTER 发表

最后编辑：2020-05-26

作者：shooter

这个作者貌似有点懒，什么都没有留下。

站内专栏

您可能还会对这些文章感兴趣！