|
1 2 3 4 5 6 |
单Master模式,配置简单，方便部署.但是存在缺点 缺点：这种方式风险较大，一旦Broker重启或者宕机时，会导致整个服务不可用，不建议线上环境使用 1、因为只买了一台阿里云服务器，所以RocketMQ集群都部署在单台服务器上只是端口不同，如果实际开发，可以分别部署在多台服务器上。 2、这里有关 Broker 和 NameServer 分别都做了了集群部署(各部署两个），且BroKer是按两主进行部署。 |
1. 在docker部署单机版的基础上,部署集群,都是基于内网ip部署
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
创建几个目录 mkdir -p /opt/rocketmq/logs/nameserver-a mkdir -p /opt/rocketmq/logs/nameserver-b mkdir -p /opt/rocketmq/store/nameserver-a mkdir -p /opt/rocketmq/store/nameserver-b mkdir -p /opt/rocketmq/logs/broker-a mkdir -p /opt/rocketmq/logs/broker-b mkdir -p /opt/rocketmq/store/broker-a mkdir -p /opt/rocketmq/store/broker-b mkdir -p /opt/rocketmq/broker-a/ mkdir -p /opt/rocketmq/broker-b/ #这里持久化日志文件必须给写入权限才可以,否则无法启动 chmod -R 777 /opt/rocketmq/store |
配置文件详解
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
#broker集群名称，用于划分broker brokerClusterName=MQCluster001 #broker名称，用于主从配对，相同名称的broker才能做主从设置 brokerName=mq_broker_1 #用于标识主从关系，0为主，其他大于0的为从（不能小于0）master设置0，slave设置1。Master角色的Broker支持读和写，Slave角色的Broker仅支持读，也就是Producer只能和Master角色的Broker连接写人消息：Consumer可以连接Master角色的Broker，也可以连接Slave角色的Broker来读取消息。 #Master节点设置 brokerId=0 #Slave节点设置 #brokerId=1 #name server服务器地址及端口，可以是多个，分号隔开 namesrvAddr=192.168.1.100:9876 #创建topic时，若未指定topic下的队列数，则取该默认值作为默认队列数 defaultTopicQueueNums=8 #是否自动创建默认topic，生产需保持关闭 autoCreateTopicEnable=true #是否自动创建topic的订阅组，默认开启 autoCreateSubscriptionGroup=true #broker服务监听端口 listenPort=10911 #未消费的持久化消息清理时间点 deleteWhen=04 #持久化消息保存周期（单位：小时），超过该周期将被清理 fileReservedTime=24 #单个commitLog文件的大小限制（单位：字节） mapedFileSizeCommitLog=1073741824 #单个consumeQueue大小限制（存储的消息条数 * 每条消息的索引大小20） mapedFileSizeConsumeQueue=8000000 #存储使用率阀值，当使用率超过阀值时，将拒绝发送消息请求 diskMaxUsedSpaceRatio=88 #持久化消息存储根路径(如果是容器启动,这里写容器内部目录即可,因为有外部挂载) storePathRootDir=/data/store(如果是容器启动,这里写容器内部目录即可,因为有外部挂载) #commitLog文件存储路径 storePathCommitLog=/data/store/commitlog #最大消息大小限制（单位：字节） maxMessageSize=65536 #commitLog最少刷盘page数 flushCommitLogLeastPages=4 #consumeQueue最少刷盘page数 flushConsumeQueueLeastPages=2 #commitLog刷盘间隔时间 flushCommitLogThoroughInterval=10000 #consumeQueue刷盘间隔时间 flushConsumeQueueThoroughInterval=60000 #处理消息发送线程池大小 sendMessageThreadPoolNums=128 #处理消息拉取线程池大小 pullMessageThreadPoolNums=128 #broker角色（SYNC_MASTER:同步双写Master、ASYNC_MASTER:异步复制Master、SLAVE:Slave） brokerRole=ASYNC_MASTER #Slave节点设置 #brokerRole=SLAVE #刷盘方式（ASYNC_FLUSH:异步刷盘、SYNC_FLUSH:同步刷盘） flushDiskType=ASYNC_FLUSH |
2 编写broker.conf
|
1 |
vim /opt/rocketmq/broker-a/broker-a.conf |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
brokerClusterName = rocketmq-cluster brokerName = broker-a brokerId = 0 # 这个ip配置为内网访问，让mq只能内网访问，不配置默认为内网 #brokerIP1 = xxxxx deleteWhen = 04 fileReservedTime = 48 brokerRole = ASYNC_MASTER flushDiskType = ASYNC_FLUSH #nameServer的容器地址,broken与nameServer通信地址,可以是docker网桥地址,也可以是内网地址，只要能够正常通信即可 namesrvAddr=172.21.0.10:9876;172.21.0.11:9877 autoCreateTopicEnable=true #Broker 对外服务的监听端口, listenPort = 11911 #Broker角色 #- ASYNC_MASTER 异步复制Master #- SYNC_MASTER 同步双写Master #- SLAVE brokerRole=ASYNC_MASTER #刷盘方式 #- ASYNC_FLUSH 异步刷盘 #- SYNC_FLUSH 同步刷盘 flushDiskType=ASYNC_FLUSH |
|
1 |
vim /opt/rocketmq/broker-b/broker-b.conf |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
brokerClusterName = rocketmq-cluster brokerName = broker-b brokerId = 0 # 这个ip配置为内网访问，让mq只能内网访问,不配置默认为内网 #brokerIP1 = xxxxx deleteWhen = 04 fileReservedTime = 48 brokerRole = ASYNC_MASTER flushDiskType = ASYNC_FLUSH #nameServer的容器地址,broken与nameServer通信地址 namesrvAddr=172.21.0.10:9876;172.21.0.11:9877 autoCreateTopicEnable=true #Broker 对外服务的监听端口, listenPort = 11911 #Broker角色 #- ASYNC_MASTER 异步复制Master #- SYNC_MASTER 同步双写Master #- SLAVE brokerRole=ASYNC_MASTER #刷盘方式 #- ASYNC_FLUSH 异步刷盘 #- SYNC_FLUSH 同步刷盘 flushDiskType=ASYNC_FLUSH |
编写docker-compose.yml文件
|
1 2 3 4 5 6 7 8 9 |
从下面配置文件可以大致看出几点： 1、通过Docker总共拉取了5条镜像记录。rmqnamesrv-a、rmqnamesrv-b、rmqbroker-a、rmqbroker-b、rmqconsole 2、rmqconsole是一个可视化的工具，可以通过页面来查看RocketMQ相关信息。 3、可以看出对于broker的配置文件broker.conf已经被broker-a/b.conf替换。 |
|
1 |
docker-compose.yml |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 |
version: '3.5' services: rmqnamesrv-a: image: rocketmqinc/rocketmq:4.3.0 container_name: rmqnamesrv-a restart: always deploy: restart_policy: condition: on-failure ports: - 9876:9876 volumes: - /opt/rocketmq/logs/nameserver-a:/opt/logs - /opt/rocketmq/store/nameserver-a:/opt/store command: sh mqnamesrv networks: rmq: aliases: - rmqnamesrv-a rmqnamesrv-b: image: rocketmqinc/rocketmq:4.3.0 container_name: rmqnamesrv-b restart: always deploy: restart_policy: condition: on-failure ports: - 9877:9876 volumes: - /opt/rocketmq/logs/nameserver-b:/opt/logs - /opt/rocketmq/store/nameserver-b:/opt/store command: sh mqnamesrv networks: rmq: aliases: - rmqnamesrv-b rmqbroker-a: image: rocketmqinc/rocketmq:4.3.0 container_name: rmqbroker-a restart: always deploy: restart_policy: condition: on-failure ports: - 11909:10909 - 11911:10911 volumes: - /opt/rocketmq/logs/broker-a:/opt/logs - /opt/rocketmq/store/broker-a:/opt/store - /home/rocketmq/broker-a/broker-a.conf:/opt/rocketmq-4.3.0/conf/broker.conf #- /home/rocketmq/broker-a/broker.conf:/opt/rocketmq-4.3.0/conf/broker.conf environment: TZ: Asia/Shanghai NAMESRV_ADDR: "rmqnamesrv-a:9876" JAVA_OPTS: " -Duser.home=/opt" JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m" command: sh mqbroker -c /opt/rocketmq-4.3.0/conf/broker.conf autoCreateTopicEnable=true & links: - rmqnamesrv-a:rmqnamesrv-a - rmqnamesrv-b:rmqnamesrv-b networks: rmq: aliases: - rmqbroker-a rmqbroker-b: image: rocketmqinc/rocketmq:4.3.0 container_name: rmqbroker-b restart: always deploy: restart_policy: condition: on-failure ports: - 10909:10909 - 10911:10911 volumes: - /opt/rocketmq/logs/broker-b:/opt/logs - /opt/rocketmq/store/broker-b:/opt/store - /home/rocketmq/broker-b/broker-b.conf:/opt/rocketmq-4.3.0/conf/broker.conf #- /home/rocketmq/broker-b:/opt/rocketmq-4.3.0/conf environment: TZ: Asia/Shanghai NAMESRV_ADDR: "rmqnamesrv-b:9876" JAVA_OPTS: " -Duser.home=/opt" JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m" command: sh mqbroker -c /opt/rocketmq-4.3.0/conf/broker.conf autoCreateTopicEnable=true & links: - rmqnamesrv-a:rmqnamesrv-a - rmqnamesrv-b:rmqnamesrv-b networks: rmq: aliases: - rmqbroker-b rmqconsole: image: styletang/rocketmq-console-ng container_name: rmqconsole restart: always deploy: restart_policy: condition: on-failure ports: - 9001:8080 environment: JAVA_OPTS: -Drocketmq.namesrv.addr=rmqnamesrv-a:9876;rmqnamesrv-b:9877 -Dcom.rocketmq.sendMessageWithVIPChannel=false -Duser.timezone='Asia/Shanghai' networks: rmq: aliases: - rmqconsole networks: rmq: name: rmq driver: bridge |
因为每台broker节点与所有的nameService保持长连接及心跳，并定时将Topic信息注册到nameService中，所以这里我固定了namesrv的容器ip，这样上面的broker配置文件里也就提前知道了namesrvAddr要配置的ip，并且这个ip也不会随着容器重启而丢失。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 |
version: '3.5' services: rmqnamesrv-a: image: rocketmqinc/rocketmq:4.3.0 container_name: rmqnamesrv-a ports: - 9876:9876 volumes: - /opt/rocketmq/logs/nameserver-a:/opt/logs - /opt/rocketmq/store/nameserver-a:/opt/store command: sh mqnamesrv networks: rmq: ipv4_address: 172.21.0.10 aliases: - rmqnamesrv-a rmqnamesrv-b: image: rocketmqinc/rocketmq:4.3.0 container_name: rmqnamesrv-b ports: - 9877:9876 volumes: - /opt/rocketmq/logs/nameserver-b:/opt/logs - /opt/rocketmq/store/nameserver-b:/opt/store command: sh mqnamesrv networks: rmq: ipv4_address: 172.21.0.11 aliases: - rmqnamesrv-b rmqbroker-a: image: rocketmqinc/rocketmq:4.3.0 container_name: rmqbroker-a ports: - 11909:10909 - 11911:10911 volumes: - /opt/rocketmq/logs/broker-a:/opt/logs - /opt/rocketmq/store/broker-a:/opt/store - /opt/rocketmq/broker-a/broker-a.conf:/opt/rocketmq-4.3.0/conf/broker.conf environment: TZ: Asia/Shanghai NAMESRV_ADDR: "rmqnamesrv-a:9876" JAVA_OPTS: " -Duser.home=/opt" JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m" command: sh mqbroker -c /opt/rocketmq-4.3.0/conf/broker.conf autoCreateTopicEnable=true & links: - rmqnamesrv-a:rmqnamesrv-a - rmqnamesrv-b:rmqnamesrv-b depends_on: - rmqnamesrv-a - rmqnamesrv-b networks: rmq: aliases: - rmqbroker-a rmqbroker-b: image: rocketmqinc/rocketmq:4.3.0 container_name: rmqbroker-b ports: - 10909:10909 - 10911:10911 volumes: - /opt/rocketmq/logs/broker-b:/opt/logs - /opt/rocketmq/store/broker-b:/opt/store - /opt/rocketmq/broker-b/broker-b.conf:/opt/rocketmq-4.3.0/conf/broker.conf environment: TZ: Asia/Shanghai NAMESRV_ADDR: "rmqnamesrv-b:9876" JAVA_OPTS: " -Duser.home=/opt" JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m" command: sh mqbroker -c /opt/rocketmq-4.3.0/conf/broker.conf autoCreateTopicEnable=true & links: - rmqnamesrv-a:rmqnamesrv-a - rmqnamesrv-b:rmqnamesrv-b depends_on: - rmqnamesrv-a - rmqnamesrv-b networks: rmq: aliases: - rmqbroker-b rmqconsole: image: styletang/rocketmq-console-ng container_name: rmqconsole ports: - 9001:8080 environment: JAVA_OPTS: -Drocketmq.namesrv.addr=rmqnamesrv-a:9876;rmqnamesrv-b:9877 -Dcom.rocketmq.sendMessageWithVIPChannel=false networks: rmq: aliases: - rmqconsole networks: rmq: name: rmq driver: bridge ipam: config: - subnet: 172.21.0.0/16 gateway: 172.21.0.1 |
启动:
|
1 |
docker-compose -f docker-compose.yml up -d |
访问：
|
1 |
http://192.168.66.180:9001/#/ |
注意：
|
1 2 3 4 |
防火墙开放RocketMQ相关端口 NameServer的9876端口、Broker的10911、11011、10909、11009端口 可以选择关闭服务器防火墙或者开放相应的端口 |
如果容器需要资源限制请添加如下设置
|
1 |
limit用于限制最大的资源使用数量，reservation为最低的资源占用量。 |
|
1 2 3 4 5 6 7 8 |
deploy: resources: limits: cpus: '0.50' memory: 50M reservations: cpus: '0.25' memory: 20M |
other 持久化配置
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
brokerClusterName = rocketmq-cluster brokerName = broker-c brokerId = 0 brokerIP1 = 192.168.0.101 brokerRole = SYNC_FLUSH flushDiskType = SYNC_FLUSH namesrvAddr=192.168.0.101:9876 deleteWhen=48 fileReservedTime=168 mapedFileSizeCommitLog=1073741824 mapedFileSizeConsumeQueue=8000000 diskMaxUsedSpaceRatio=88 storePathRootDir=/opt/store storePathCommitLog=/opt/store/commitlog maxMessageSize=65536 flushCommitLogLeastPages=4 flushConsumeQueueLeastPages=2 flushCommitLogThoroughInterval=10000 flushConsumeQueueThoroughInterval=60000 sendMessageThreadPoolNums=128 pullMessageThreadPoolNums=128 flushDiskType=SYNC_FLUSH autoCreateTopicEnable=true traceTopicEnable=true |
- 本文固定链接: https://www.yoyoask.com/?p=5907
- 转载请注明: shooter 于 SHOOTER 发表
这个作者貌似有点懒，什么都没有留下。