ceph monitor---总结1

参考

ceph代码和名词解释 http://accelazh.github.io/ceph/Ceph-Code-Deep-Dive

ceph 网页书籍  http://blog.sina.com.cn/s/blog_153c9453d0102xvwi.html

ceph-deploy源码剖析  有其他博客  http://www.hl10502.com/2017/06/15/ceph-deploy-cli/

ceph在终端敲命令后怎么调用 https://blog.csdn.net/qq_36118718/article/details/79195621

Ceph monitor and paxos ：http://catkang.github.io/2016/07/17/ceph-monitor-and-paxos.html

Ceph monitor 实现 https://www.jianshu.com/p/60b34ba5cdf2

ceph 源码剖析 https://blog.csdn.net/qq_36118718/article/details/79234737

src/ceph_mon.cc    monitor启动流程 main（）函数： https://blog.csdn.net/kakaxi8891/article/details/10921297

ceph-monmap命令处理流程 https://blog.csdn.net/carny/article/details/52583560

src/mon/

monitor leader选举源码分析 https://blog.csdn.net/sy_yu/article/details/79102202

                                                         https://segmentfault.com/a/1190000010413557

博客    https://blog.csdn.net/sy_yu?t=1

monitor leader选举 https://blog.csdn.net/sy_yu/article/details/79102202

Monitor 选举机制 https://blog.csdn.net/scaleqiao/article/details/52315468

https://www.cnblogs.com/shanno/p/3967116.html

MonitorDB源码分析 monmap同步  https://blog.csdn.net/qq_36118718/article/details/79234737

paxos 源码注释  https://blog.csdn.net/fishermandong/article/details/72805660

paxos 源码 phase1 https://blog.csdn.net/fishermandong/article/details/72805660

                  phase2  https://blog.csdn.net/fishermandong/article/details/76360237

paxos 算法 https://blog.csdn.net/qq_36118718/article/details/79134887

                       https://blog.csdn.net/skdkjzz/article/details/41979521

Maps   https://ceph-doc.readthedocs.io/en/latest/Monitor/

PGmap，OSDmap  http://ju.outofmemory.cn/entry/76367

A Ceph Monitor maintains a master copy of the cluster map. A robust ceph cluster usually contains a cluster of monitors which provide the cluster map to the clients. 

the basic framework of a monitor

A ceph monitor consists of  K/V store, paxos and the paxosService. The K/V store is for persistent store of monitor data. The paxos provides consistent data access logic for the paxosService layer. Each paxosService represents a kind of state information of the cluster. They change their data to the form of Key-value and then write to the paxos layer.

the initialization and leader election of a monitor

The monitor will connect other monitors according to the monmap once it starts or restarts.  If it starts at the first time, it needs to build the monmap using the ceph configuration file and store it to the MonitorDBStore. If not the first time, it gets the monmap from MonitorDBStore. So once a monitor starts, it initializes the MonitorDBStore. Messenger is the network thread module. The monitor initializes it and registers the callback function which will be executed after the reception of requests. The paxos and paxosService will be described in detail later. The bootstrap process will be called time and again,which plays an important role in the lifecycle of a monitor.

After bootstrap, the monitor is in STATE_PROBING, it communicates and synchronizes with other monitors. After synchronization the cluster starts election, and decides the roles of the monitors. The detailed process is as follows.

Probing and synchronizing process:

 Leader election process:

Paxos : recovery and propose

The following data structures are important in Paxos. They need to be kept in the DBStore.

last_pn	Last Proposal Number
accepted_pn	The last Proposal Number we have accepted.On the Leader, it will be the Proposal Number picked by the Leader itself. On the Peon, however, it will be the proposal sent by the Leader and it will only be updated if its value is higher than the one already known by the Peon.
uncommitted_pn	Uncommitted value's Proposal Number.We use this variable to assess if the Leader should take into consideration an uncommitted value sent by a Peon. Given that the Peon will send back to the Leader the last Proposal Number it accepted, the Leader will be able to infer if this value is more recent than the one the Leader has, thus more relevant.
first_committed	First committed value's version
last_committed	Last committed value's version. On both the Leader and the Peons, this is the last value's version that was accepted by a given quorum and thus committed, that this instance knows about.
uncommitted_v	Uncommitted value's version.If we have, or end up knowing about, an uncommitted value, then its version will be kept in this variable.
uncommitted_value	If the system fails in-between the accept replies from the Peons and the instruction to commit from the Leader, then we may end up with accepted but yet-uncommitted values. During the Leader's recovery, it will attempt to bring the whole system to the latest state, and that means committing past accepted but uncommitted values. This variable will hold an uncommitted value, which may originate either on the Leader, or learnt by the Leader from a Peon during the collect phase.

After the leader election process, the roles of leader and peon are clear. Before the consistent read and write, the mon cluster should do phase1:RECOVERY to make PN( proposal number) consistent firstly. The flow is as follows.

After phase1, we go to phase2 which is the working flow of proposing, accepting and committing when the monitors are under normal working.

 

The detailed processes are as follows.

how the client's requests are dealt with

When the client sends a request to the monitor, the monitor firstly dispatches the request to the corresponding PaxosService. Then PaxosService calls the functions according to whether it's a reading operation or writing. And it decides whether the propose process should be triggered.