CentOS配置RabbitMQ集群

三台RabbitMQ服务器配置集群


主要参考:https://blog.csdn.net/jxdl6655/article/details/78194191

三台机器分别是192.168.220.128、192.168.220.129、192.168.220.130,已经安装好Erlang和RabbitMQ。

1、分别在3台机器上配置/etc/hosts,如下
mnd1 192.168.220.128
mnd2 192.168.220.129
mnd3 192.168.220.130

2、复制erlang cookie,保持三台机器内容一致
使用rpm方式安装时,在/var/lib/rabbitmq/目录下。
把第一台服务器的.erlang.cookie上传到第二、三台服务器。
注意:
如果通过编辑的方式改成一样,chmod 777 .erlang.cookie以后需要改回去,chmod 400 .erlang.cookie,否则启动会报错。
如果通过先下载后上传的方式,需要把所属用户和所属组改成rabbitmq, chown rabbitmq:rabbitmq .erlang.cookie

3、新增一个用户并赋予权限
添加用户并设置密码:
rabbitmqctl add_user admin 123456

添加权限(使admin用户对虚拟主机“/” 具有所有权限):
rabbitmqctl set_permissions -p “/” admin “.*” “.*” “.*”

修改用户角色(加入administrator用户组)
rabbitmqctl set_user_tags admin administrator

4、
在第二台机器上,加入节点1的集群,节点1的名称不要写错:
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl join_cluster rabbit@mnd1
rabbitmqctl start_app

在第三台机器上,加入节点1的集群,节点1的名称不要写错:
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl join_cluster rabbit@mnd1
rabbitmqctl start_app

如果提示连接节点1失败,关闭节点1的防火墙:
systemctl stop firewalld.service
systemctl disable firewalld.service

最后效果:

—————————————————————————————————————-
遇到的两个坑:
复制.erlang.cookie以后启动报错:

1
2
# systemctl start rabbitmq-server.service
Job for rabbitmq-server.service failed. See 'systemctl status rabbitmq-server.service' and 'journalctl -xn' for details.

用journalctl -xn命令查看日志

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
7月 02 21:10:37 localhost.localdomain su[3677]: pam_unix(su:session): session opened for user rabbitmq by (uid=0)
7月 02 21:10:38 localhost.localdomain su[3677]: pam_unix(su:session): session closed for user rabbitmq
7月 02 21:10:39 localhost.localdomain su[3684]: pam_unix(su:session): session closed for user rabbitmq
7月 02 21:10:39 localhost.localdomain rabbitmq-server[3436]: Starting rabbitmq-server: FAILED - check /var/log/rabbitmq/startup_{log, _err}
7月 02 21:10:39 localhost.localdomain rabbitmq-server[3436]: rabbitmq-server.
7月 02 21:10:39 localhost.localdomain systemd[1]: rabbitmq-server.service: control process exited, code=exited status=1
7月 02 21:10:39 localhost.localdomain systemd[1]: Failed to start LSB: Enable AMQP service provided by RabbitMQ broker.
-- Subject: Unit rabbitmq-server.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit rabbitmq-server.service has failed.
--
-- The result is failed.
7月 02 21:10:39 localhost.localdomain systemd[1]: Unit rabbitmq-server.service entered failed state.
7月 02 21:10:39 localhost.localdomain systemd[1]: rabbitmq-server.service failed.
7月 02 21:10:39 localhost.localdomain polkitd[670]: Unregistered Authentication Agent for unix-process:3430:16972 (system bus name :1.60, object path /org/fr

其中标红的:Failed to start LSB: Enable AMQP service provided by RabbitMQ broker.

根据这个搜索到的一些解决方案:
https://stackoverflow.com/questions/25816918/not-able-to-start-rabbitmq-server-in-centos-7-using-systemctL
https://blog.csdn.net/testcs_dn/article/details/52514199

试了很多方法,包括:
–确认/etc/hosts/,缩短node名称
–配置ERLANG_HOME
–关闭防火墙
–disable SELinux
–vi /etc/rabbitmq/rabbitmq-env.conf ,添加DENAME=rabbit@localhost

差点要重新安装Erlang。

最后是用前台方式启动rabbitmq
/usr/sbin/rabbitmq-server

看到日志:
{error_logger,{{2018,7,2},{21,29,16}},”Error when reading /var/lib/rabbitmq/.erlang.cookie: eacces”,[]}

说明没有/var/lib/rabbitmq/.erlang.cookie读权限(因为后两台服务器的cookie文件是从第一台下载后上传的)。
改成初始权限:
chmod 400 /var/lib/rabbitmq/.erlang.cookie

还是报错:
{error_logger,{{2018,7,2},{21,25,59}},”Cookie file /var/lib/rabbitmq/.erlang.cookie must be accessible by owner only”,[]}

因为是直接用root用户上传的,要改成rabbitmq用户:
chown rabbitmq:rabbitmq .erlang.cookie

终于启动成功了。

—————————————————————————————————————-
第二个坑:
加入集群失败的问题

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[root@mnd2 ~]# rabbitmqctl join_cluster rabbitmq-cli-60@mnd1
Clustering node rabbit@mnd2 with 'rabbitmq-cli-60@mnd1'
Error: unable to connect to nodes ['rabbitmq-cli-60@mnd1']: nodedown

DIAGNOSTICS
===========

attempted to contact: ['rabbitmq-cli-60@mnd1']

rabbitmq-cli-60@mnd1:
  * unable to connect to epmd (port 4369) on mnd1: address (cannot connect to host/port)

current node details:
- node name: 'rabbitmq-cli-99@mnd2'
- home dir: /var/lib/rabbitmq
- cookie hash: L2zzYyP5riESYzMX0Dwxjw==

一直以为是/etc/hosts的问题

1
2
3
192.168.220.128 mnd1
192.168.220.129 mnd2
192.168.220.130 mnd3

加入集群的命令

1
2
3
4
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl join_cluster rabbit@mnd1
rabbitmqctl start_app

IP放在后面或者在节点名前面加rabbit@都没用。重启虚拟机也没用。
其中犯了一个错误,集群前面是rabbit@ 而不是 rabbitmq@。改正确以后出现了防火墙的提示:

1
2
3
4
5
rabbit@mnd1:
  * connected to epmd (port 4369) on mnd1
  * epmd reports node 'rabbit' running on port 25672
  * can't establish TCP connection, reason: ehostunreach (host is unreachable)
  * suggestion: blocked by firewall?

关掉防火墙并禁止开机启动:

1
2
systemctl stop firewalld.service
systemctl disable firewalld.service

加入集群成功:

1
2
3
4
5
6
7
8
[root@mnd2 ~]# rabbitmqctl stop_app
Stopping rabbit application on node rabbit@mnd2
[root@mnd2 ~]# rabbitmqctl reset
Resetting node rabbit@mnd2
[root@mnd2 ~]# rabbitmqctl join_cluster rabbit@mnd1
Clustering node rabbit@mnd2 with rabbit@mnd1
[root@mnd2 ~]# rabbitmqctl start_app
Starting node rabbit@mnd2