MongoDB副本集

MongoDB副本集

Administrator 935 2018-09-05
  1. MongoDB副本集介绍
  2. MongoDB副本集搭建
  3. MongoDB副本集测试

MongoDB副本集介绍

  • 早期MongoDB高可用使用的是master-slave架构，一主一从与MySQL类似，但在此架构中slave为只读模式，当主库宕机后，从库不能自动切换为主；

  • 目前版本的MongoDB则采用的是副本集架构的高可用，这种模式下有一个主（primary），以及多个从（secondary），其中从是只读的，并且支持设置权重，当主宕机后，权重最高的从切换为主；

  • 另外在副本集架构中还可以建立一个仲裁（arbiter）的角色，它只负责裁决主库是否宕机，而不存储数据；

  • 副本集架构中，读写数据都是在主上，要想实现负载均衡的目的，则需要手动指定读库的目标server；

  • 副本集架构图

    iSMgAA.md.png

MongoDB副本集搭建

  • 首先在准备三台主机，IP分别为192.168.49.128，192.168.49.129,192.168.49.130，三台主机均安装MongoDB，并关闭selinux和防火墙，其中128为主机，另外两台为从机；

  • 修改MongoDB的配置文件/etc/mongod.conf，将bindIp监听地址增加本地内网地址，并且取消replcation选项的注释，三台机器的实际配置相同，如下：

    net:
  port: 27017
  bindIp: 127.0.0.1,192.168.49.130

replication:
  oplogSizeMB: 20
  replSetName: evobot
    • oplogSizeMB是定义oplog日志的大小，可以不配置；
    • replSetName则是定义副本集的名字，三台机器的配置相同。

  • 然后连接主的MongoDB命令行，执行下面的命令：

    > use admin
switched to db admin
> config={_id:"evobot",members:[{_id:0,host:"192.168.49.128:27017"},{_id:1,host:"192.168.49.129"},{_id:2,host:"192.168.49.130"}]}
    • id:0是主，剩下两个为从

  • 然后执行rs.initiate(config)初始化命令：

    > rs.initiate(config)
{
	"ok" : 1,
	"operationTime" : Timestamp(1536155549, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1536155549, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
    • 显示"ok": 1则表示副本集创建成功。

  • rs.status()命令可以查看副本集的状态：

    evobot:SECONDARY> rs.status()
{
	"set" : "evobot",
	"date" : ISODate("2018-09-05T13:53:06.737Z"),
	"myState" : 1,
	"term" : NumberLong(1),
	"syncingTo" : "",
	"syncSourceHost" : "",
	"syncSourceId" : -1,
	"heartbeatIntervalMillis" : NumberLong(2000),
	"optimes" : {
		"lastCommittedOpTime" : {
			"ts" : Timestamp(1536155581, 1),
			"t" : NumberLong(1)
		},
		"readConcernMajorityOpTime" : {
			"ts" : Timestamp(1536155581, 1),
			"t" : NumberLong(1)
		},
		"appliedOpTime" : {
			"ts" : Timestamp(1536155581, 1),
			"t" : NumberLong(1)
		},
		"durableOpTime" : {
			"ts" : Timestamp(1536155581, 1),
			"t" : NumberLong(1)
		}
	},
	"members" : [
		{
			"_id" : 0,
			"name" : "192.168.49.128:27017",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
			"uptime" : 2446,
			"optime" : {
				"ts" : Timestamp(1536155581, 1),
				"t" : NumberLong(1)
			},
			"optimeDate" : ISODate("2018-09-05T13:53:01Z"),
			"syncingTo" : "",
			"syncSourceHost" : "",
			"syncSourceId" : -1,
			"infoMessage" : "could not find member to sync from",
			"electionTime" : Timestamp(1536155559, 1),
			"electionDate" : ISODate("2018-09-05T13:52:39Z"),
			"configVersion" : 1,
			"self" : true,
			"lastHeartbeatMessage" : ""
		},
		{
			"_id" : 1,
			"name" : "192.168.49.129:27017",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 37,
			"optime" : {
				"ts" : Timestamp(1536155581, 1),
				"t" : NumberLong(1)
			},
			"optimeDurable" : {
				"ts" : Timestamp(1536155581, 1),
				"t" : NumberLong(1)
			},
			"optimeDate" : ISODate("2018-09-05T13:53:01Z"),
			"optimeDurableDate" : ISODate("2018-09-05T13:53:01Z"),
			"lastHeartbeat" : ISODate("2018-09-05T13:53:05.606Z"),
			"lastHeartbeatRecv" : ISODate("2018-09-05T13:53:06.026Z"),
			"pingMs" : NumberLong(0),
			"lastHeartbeatMessage" : "",
			"syncingTo" : "192.168.49.128:27017",
			"syncSourceHost" : "192.168.49.128:27017",
			"syncSourceId" : 0,
			"infoMessage" : "",
			"configVersion" : 1
		},
		{
			"_id" : 2,
			"name" : "192.168.49.130:27017",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 37,
			"optime" : {
				"ts" : Timestamp(1536155581, 1),
				"t" : NumberLong(1)
			},
			"optimeDurable" : {
				"ts" : Timestamp(1536155581, 1),
				"t" : NumberLong(1)
			},
			"optimeDate" : ISODate("2018-09-05T13:53:01Z"),
			"optimeDurableDate" : ISODate("2018-09-05T13:53:01Z"),
			"lastHeartbeat" : ISODate("2018-09-05T13:53:05.601Z"),
			"lastHeartbeatRecv" : ISODate("2018-09-05T13:53:06.032Z"),
			"pingMs" : NumberLong(0),
			"lastHeartbeatMessage" : "",
			"syncingTo" : "192.168.49.128:27017",
			"syncSourceHost" : "192.168.49.128:27017",
			"syncSourceId" : 0,
			"infoMessage" : "",
			"configVersion" : 1
		}
	],
	"ok" : 1,
	"operationTime" : Timestamp(1536155581, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1536155581, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
    • 查看输出信息中的members，是否是一个PRIMARY，两个SECONDARY。

  • 如果rs.status()中的members中，两个从的状态为"stateStr":"STARTUP"，则需要执行如下命令重新配置：

    //重新配置config
> var config={_id:"evobot",members:[{_id:0,host:"192.168.49.128:27017"},{_id:1,host:"192.168.49.129"},{_id:2,host:"192.168.49.130"}]}
//重新初始化
> rs.reconfig(config)

  • 在副本集的创建过程中，想要哪一台机器作为主，就要在哪一台主机上执行配置副本集的命令。

MongoDB副本集测试

副本集可用性测试

  • 在主上创建库，并插入文档：

    evobot:PRIMARY> use mydb
switched to db mydb
evobot:PRIMARY> db.acc.insert({AccountID:1,UserName:"123",password:"123456"})
WriteResult({ "nInserted" : 1 })

  • 查看创建的文档：

    evobot:PRIMARY> show dbs
admin   0.000GB
config  0.000GB
local   0.000GB
mydb    0.000GB

evobot:PRIMARY> use mydb
switched to db mydb
evobot:PRIMARY> show tables
acc
evobot:PRIMARY> db.acc.find()
{ "_id" : ObjectId("5b8fe273ef0e7acc8f910353"), "AccountID" : 1, "UserName" : "123", "password" : "123456" }

  • 然后登录从机器的命令行，执行show dbs：

    evobot:SECONDARY> show dbs
2018-09-05T22:07:54.141+0800 E QUERY    [thread1] Error: listDatabases failed:{
	"operationTime" : Timestamp(1536156471, 1),
	"ok" : 0,
	"errmsg" : "not master and slaveOk=false",
	"code" : 13435,
	"codeName" : "NotMasterNoSlaveOk",
	"$clusterTime" : {
		"clusterTime" : Timestamp(1536156471, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:65:1
shellHelper.show@src/mongo/shell/utils.js:849:19
shellHelper@src/mongo/shell/utils.js:739:15
@(shellhelp2):1:1
    • 出现上面的报错slaveOk=false，说明从的状态不正确，此时需要执行rs.slaveOk()设置从：
    evobot:SECONDARY> rs.slaveOk()
evobot:SECONDARY> show dbs
admin   0.000GB
config  0.000GB
local   0.000GB
mydb    0.000GB
evobot:SECONDARY> use mydb
switched to db mydb
evobot:SECONDARY> show tables
acc
evobot:SECONDARY> db.acc.find()
{ "_id" : ObjectId("5b8fe273ef0e7acc8f910353"), "AccountID" : 1, "UserName" : "123", "password" : "123456" }

  • 同样的，在另一台从上也执行rs.slaveOk()。

更改权重模拟宕机

  • 默认情况下，三台机器的权重都是1，使用rs.config()查看三台机器的权重priority：

    evobot:PRIMARY> rs.config()
{
	"_id" : "evobot",
	"version" : 1,
	"protocolVersion" : NumberLong(1),
	"members" : [
		{
			"_id" : 0,
			"host" : "192.168.49.128:27017",
			"arbiterOnly" : false,
			"buildIndexes" : true,
			"hidden" : false,
			"priority" : 1,
			"tags" : {

			},
			"slaveDelay" : NumberLong(0),
			"votes" : 1
		},
		{
			"_id" : 1,
			"host" : "192.168.49.129:27017",
			"arbiterOnly" : false,
			"buildIndexes" : true,
			"hidden" : false,
			"priority" : 1,
			"tags" : {

			},
			"slaveDelay" : NumberLong(0),
			"votes" : 1
		},
		{
			"_id" : 2,
			"host" : "192.168.49.130:27017",
			"arbiterOnly" : false,
			"buildIndexes" : true,
			"hidden" : false,
			"priority" : 1,
			"tags" : {

			},
			"slaveDelay" : NumberLong(0),
			"votes" : 1
		}
	],
	"settings" : {
		"chainingAllowed" : true,
		"heartbeatIntervalMillis" : 2000,
		"heartbeatTimeoutSecs" : 10,
		"electionTimeoutMillis" : 10000,
		"catchUpTimeoutMillis" : -1,
		"catchUpTakeoverDelayMillis" : 30000,
		"getLastErrorModes" : {

		},
		"getLastErrorDefaults" : {
			"w" : 1,
			"wtimeout" : 0
		},
		"replicaSetId" : ObjectId("5b8fdf9cc66be9d311189b52")
	}
}

  • 模拟宕机，在主上添加一条防火墙规则，然后在从上使用rs.status()查看副本集状态：

    iptables -I INPUT -p tcp --dport 27017 -j DROP

    evobot:SECONDARY> rs.status()
{
	"set" : "evobot",
	"date" : ISODate("2018-09-05T14:23:03.844Z"),
	"myState" : 1,
	"term" : NumberLong(2),
	"syncingTo" : "",
	"syncSourceHost" : "",
	"syncSourceId" : -1,
	"heartbeatIntervalMillis" : NumberLong(2000),
	"optimes" : {
		"lastCommittedOpTime" : {
			"ts" : Timestamp(1536157361, 1),
			"t" : NumberLong(1)
		},
		"readConcernMajorityOpTime" : {
			"ts" : Timestamp(1536157361, 1),
			"t" : NumberLong(1)
		},
		"appliedOpTime" : {
			"ts" : Timestamp(1536157361, 1),
			"t" : NumberLong(1)
		},
		"durableOpTime" : {
			"ts" : Timestamp(1536157361, 1),
			"t" : NumberLong(1)
		}
	},
	"members" : [
		{
			"_id" : 0,
			"name" : "192.168.49.128:27017",
			"health" : 0,
			"state" : 8,
			"stateStr" : "(not reachable/healthy)", //可以看到主的状态已经异常
			"uptime" : 0,
			"optime" : {
				"ts" : Timestamp(0, 0),
				"t" : NumberLong(-1)
			},
			"optimeDurable" : {
				"ts" : Timestamp(0, 0),
				"t" : NumberLong(-1)
			},
			"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
			"optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
			"lastHeartbeat" : ISODate("2018-09-05T14:22:52.216Z"),
			"lastHeartbeatRecv" : ISODate("2018-09-05T14:23:03.591Z"),
			"pingMs" : NumberLong(0),
			"lastHeartbeatMessage" : "Operation timed out",
			"syncingTo" : "",
			"syncSourceHost" : "",
			"syncSourceId" : -1,
			"infoMessage" : "",
			"configVersion" : -1
		},
		{
			"_id" : 1,
			"name" : "192.168.49.129:27017",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY", //从自动变成了主
			"uptime" : 4160,
			"optime" : {
				"ts" : Timestamp(1536157361, 1),
				"t" : NumberLong(1)
			},
			"optimeDate" : ISODate("2018-09-05T14:22:41Z"),
			"syncingTo" : "",
			"syncSourceHost" : "",
			"syncSourceId" : -1,
			"infoMessage" : "could not find member to sync from",
			"electionTime" : Timestamp(1536157376, 1),
			"electionDate" : ISODate("2018-09-05T14:22:56Z"),
			"configVersion" : 1,
			"self" : true,
			"lastHeartbeatMessage" : ""
		},
		{
			"_id" : 2,
			"name" : "192.168.49.130:27017",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 1833,
			"optime" : {
				"ts" : Timestamp(1536157361, 1),
				"t" : NumberLong(1)
			},
			"optimeDurable" : {
				"ts" : Timestamp(1536157361, 1),
				"t" : NumberLong(1)
			},
			"optimeDate" : ISODate("2018-09-05T14:22:41Z"),
			"optimeDurableDate" : ISODate("2018-09-05T14:22:41Z"),
			"lastHeartbeat" : ISODate("2018-09-05T14:23:02.474Z"),
			"lastHeartbeatRecv" : ISODate("2018-09-05T14:23:03.834Z"),
			"pingMs" : NumberLong(0),
			"lastHeartbeatMessage" : "",
			"syncingTo" : "192.168.49.128:27017",
			"syncSourceHost" : "192.168.49.128:27017",
			"syncSourceId" : 0,
			"infoMessage" : "",
			"configVersion" : 1
		}
	],
	"ok" : 1,
	"operationTime" : Timestamp(1536157361, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1536157376, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}

  • 此时删除防火墙规则，模拟主宕机恢复，由于权重都为1，PRIMARY并不会回到原来的机器上；

  • 为副本集配置权重，由于现在PRIMARY已经变到了192.168.49.129上，所以需要在新的PRIMARY的MongoDB命令行中进行如下操作，其中数字越大，权重越大：

    > cfg=rs.conf()
> cfg.members[0].priority=3
> cfg.members[1].priority=2
> cfg.members[2].priority=1
> rs.reconfig(cfg)

  • 在执行了rs.reconfig(cfg)之后，由于主上的防火墙规则已经删除，会发现原本是PRIMARY的从，再次变为SECONDARY。

MongoDB