两阶段提交实际项目V1

最新推荐文章于 2023-02-19 18:52:14 发布

小屋子大侠

最新推荐文章于 2023-02-19 18:52:14 发布

阅读量262

点赞数

分类专栏：分布式相关文章标签： 2pc 两阶段提交演示

本文链接：https://blog.csdn.net/qq_33339479/article/details/102865538

版权

分布式相关专栏收录该内容

18 篇文章 1 订阅

订阅专栏

项目介绍

两阶段提交项目主要是实际用代码演示复现一下，两阶段提交的执行过程，仅供学习参考。本次主要分析的版本为V1版本，主要实现的流程包括服务端的基础架构编写，客户端的基础架构编写，完成事务提交的过程。

两阶段项目的用例模型

客户端注册

用例名称	客户端启动注册
主要参与者	客户端
涉及关注点	客户端：希望将自己注册到服务器端，从而能够向服务端提交事务数据或者接受其他客户端的提交事务的数据
前置条件	服务器正常运行（后续可考虑设置固定的客户端数量）
后置条件	注册完成后，返回注册成功信息
基本流程	1.客户端启动，客户端启动一个线程去连接服务器；2.服务端处理请求并返回正常

客户端发起事务提交数据

用例名称	客户端发起事务提交数据
主要参与者	客户端，服务端
涉及关注点	客户端：客户端先将待提交数据发送给服务端，服务端将数据提交给其他客户端。
前置条件	服务器正常运行，并且没有其他新加入的客户端
后置条件	客户端接受服务端返回的数据提交成功
基本流程	1.客户端提交事务数据到服务端；2.服务端此时将数据发送给其他客户端；3.此时客户端进入等待状态；4.服务端等待其他客户端的反馈；5.如果其他所有客户端都反馈成功，则向该客户端发送数据提交成功消息，并让通知其他所有客户端提交该数据

客户端发送终止事务

用例名称	客户端发送终止事务
主要参与者	客户端，服务端
涉及关注点	客户端：客户端已经将事务发送给服务端，服务端等待其他客户端的事务提交成功反馈。
前置条件	服务器正常运行，并且没有其他新加入的客户端
后置条件	接收到事务终止消息
基本流程	1.客户端提交事务数据到服务端；2.服务端此时将数据发送给其他客户端；3.此时客户端进入等待状态；4.服务端等待其他客户端的反馈；5.其中有任意一个客户端发送abort消息，则向该客户端发送停止消息到其他客户端，并让其他客户端终止该事务。

本文V1版本主要就是围绕这三种基本用例模型来进行实现。包括客户端注册，事务的提交与客户端发出的终止信号。

两阶段项目的流程图

服务端的流程图

服务端的流程图，主要的实现思路参考IO复用事件驱动来进行数据的处理，并且实现思路也是通过单线程来实现。

客户端的流程图

客户端的功能，主要就是获取用户输入的指令或者数据，将该数据发送给服务端，然后客户端需要监听来自服务端的数据，是否有服务端的消息发送回来，如果有消息则处理该消息。

两阶段项目V1版本的时序图

两阶段项目V1代码

服务端代码

import selectors
import socket
from uuid import uuid4

import logging


logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(thread)d - %(lineno)d - %(levelname)s - %(message)s'
)

logger = logging.getLogger(__name__)


HEAD_DELIMITER = "\r\n"
BODY_DELIMITER = "\n"

"""
code: msg
404 : 未找到相关的处理
500 : 处理异常出错
401 : 
200 : 处理成功
"""


class BaseHandler(object):
    type = "base"
    head_delimiter = HEAD_DELIMITER
    body_delimiter = BODY_DELIMITER
    body = "error"

    def __init__(self, request_data=None, c=None):
        self.request_data = request_data
        self.protocl = c

    def serilizer(self):
        raise NotImplementedError

    def serilizer_data(self, body):
        if not body:
            body = self.body
        length = len(BODY_DELIMITER) + len(self.type) + len(body)
        return "{0}{1}{2}{3}{4}".format(length, HEAD_DELIMITER, self.type, BODY_DELIMITER, body)

    def finish(self, body):
        content = self.serilizer_data(body)
        self.protocl.write_result(content)

    def deserialize(self, data):
        return self.serilizer_data(data)


class ErrorMsg(BaseHandler):
    type = "error"

    def serilizer(self):
        return self.finish("404")


class RegisterHandler(BaseHandler):
    type = "register"

    def serilizer(self):
        name = self.request_data
        dispatch.register_client(name, self.protocl.stream)
        logger.info(dispatch.clients)
        return self.finish("200")

    def deserialize(self, data):
        return self.serilizer_data(data)


class MultiHandler(BaseHandler):
    body_parse = None

    def parse_body(self):
        raise NotImplementedError


class PreSubmit(MultiHandler):
    type = "presubmit"
    body_parse = "\t"

    def write_body(self, xid, key, value):
        content = "{0}{1}{2}{3}{4}".format(xid, self.body_parse, key, self.body_parse, value)
        return self.serilizer_data(content)

    def parse_body(self):
        loc = self.request_data.find(self.body_parse)
        if loc != -1:
            client = self.request_data[:loc]
            data = self.request_data[(loc + len(self.body_parse)):]
            loc_v = data.find(self.body_parse)
            if loc_v != -1:
                key = data[:loc_v]
                value = data[(loc_v + len(self.body_parse)):]
                return client, key, value
        return None, None, None

    def serilizer(self):
        # 检查有什么问题 没什么问题则 返回ok
        xid, key, value = self.parse_body()
        logger.info("client recv :  xid :  {0}  key : {1}  value : {2}".format(xid, key, value))
        return xid

    def deserialize(self, data):
        return self.serilizer_data(data)


class ClientPostData(MultiHandler):
    type = "client_start"
    body_parse = "\t"

    def write_body(self, name, key, value):
        content = "{0}{1}{2}{3}{4}".format(name, self.body_parse, key, self.body_parse, value)
        return self.serilizer_data(content)

    def parser_body(self):
        loc = self.request_data.find(self.body_parse)
        if loc != -1:
            client = self.request_data[:loc]
            data = self.request_data[(loc+len(self.body_parse)):]
            loc_v = data.find(self.body_parse)
            if loc_v != -1:
                key = data[:loc_v]
                value = data[(loc_v+len(self.body_parse)):]
                return client, key, value
        return None, None, None

    def serilizer(self):
        # 开始将数据提交到各个参与者
        # 生成一个事务ID
        xid = str(uuid4())

        client_id, key, value = self.parser_body()

        logger.info("client_id   :   {0}    key :   {1}   value :  {2}".format(client_id, key, value))
        transaction = Transaction(dispatch, client_id, xid)
        transaction.presubmit(client_id, key, value, self)

    def deserialize(self, data):
        return self.serilizer_data(data)


class PreperPost(BaseHandler):
    type = "preperpost"


class PreSubmitAck(MultiHandler):
    """
    客户端调用返回ack
    """
    type = "presubmitack"
    body_parse = "\t"

    def write_body(self, xid, client_id):
        content = "{0}{1}{2}".format(xid, self.body_parse, client_id)
        return self.serilizer_data(content)

    def parser_body(self):
        loc = self.request_data.find(self.body_parse)
        if loc != -1:
            xid = self.request_data[:loc]
            client_id = self.request_data[(loc + len(self.body_parse)):]
            return xid, client_id
        return None, None

    def serilizer(self):
        logger.info("request_data {0}   dispath prepares {1}".format(self.request_data, dispatch.prepares))
        xid, client_id = self.parser_body()
        logger.info(" parser body   xid : {0}   client_id  : {1}".format(xid, client_id))
        if xid in dispatch.prepares:
            tran = dispatch.prepares[xid]

            tran.presubmit_ack()

            logger.info("vote {0}   dispath prepares {1}".format(tran, dispatch.prepares))


class SubmitAbort(BaseHandler):
    """
    客户端调用 终止提交
    """
    type = "submitabort"
    body_parse = "\t"

    def write_body(self, xid, client_id):
        content = "{0}{1}{2}".format(xid, self.body_parse, client_id)
        return self.serilizer_data(content)

    def parser_body(self):
        loc = self.request_data.find(self.body_parse)
        if loc != -1:
            xid = self.request_data[:loc]
            client_id = self.request_data[(loc + len(self.body_parse)):]
            return xid, client_id
        return None, None

    def serilizer(self):
        logger.info("request_data {0}   dispath prepares {1}".format(self.request_data, dispatch.prepares))
        xid, client_id = self.parser_body()
        logger.info(" parser body   xid : {0}   client_id  : {1}".format(xid, client_id))
        if xid in dispatch.prepares:
            tran = dispatch.prepares[xid]

            logger.info("vote {0}   dispath prepares {1}".format(tran, dispatch.prepares))
            # 如果返回的事务id不存在，则证明本次的事务失败需要向client发送终止
            tran.tran_abort(client_id)

        return self.finish("abort ok")


class Transaction(object):
    """
    事务处理类
    """
    def __init__(self, dispatch, client_id, xid):
        self.dispatch = dispatch
        self.originator = client_id
        # 当前设计为一次只能一个客户端发起一个事务
        self.participants_nums = len(dispatch.clients) - 1
        self.participants_vote = 0
        self.xid = xid

    def finish(self):
        if self.xid in self.dispatch.prepares:
            self.dispatch.prepares.pop(self.xid)

    def presubmit(self, client_id, key, value, handler):
        p = PreSubmit()

        if client_id is None:
            handler.finish("client_id none")
            return
        for client in dispatch.clients:
            stream = dispatch.clients[client]
            logger.info(
                "client :  {0}  clients : {1}".format(client, dispatch.clients))
            if client != client_id:
                result = p.write_body(self.xid, key, value)
                logger.info(" stream  result : {0}".format(result))
                stream.write(result)
                logger.info(" stream {0}  ".format(stream))

        self.dispatch.register_tran(self.xid, self)

    def presubmit_ack(self):
        self.participants_vote += 1

        if self.participants_vote == self.participants_nums:
            for client in dispatch.clients:
                stream = dispatch.clients[client]
                if self.originator == client:
                    # 此时证明可以向所有的参与者 发送提交指令
                    c = ClientPostData()
                    logger.info("stream  {0}".format(stream))
                    stream.write(c.serilizer_data("finish"))
                else:
                    p = PreperPost()
                    logger.info("stream  {0}".format(stream))
                    stream.write(p.serilizer_data(self.xid))
            self.finish()
            logger.info("after  {0}  ".format(self.dispatch.prepares))

    def tran_abort(self, client_id):
        for client in dispatch.clients:
            stream = dispatch.clients[client]
            if not stream.check_client_id(client_id):
                abort = SubmitAbort()
                stream.write(abort.write_body(self.xid, client_id))
        self.finish()
        logger.info("after  {0}  ".format(self.dispatch.prepares))


class RegisterClient(object):
    """
    客户端注册的客户端
    """

    def __init__(self, stream, client_id):
        self.stream = stream
        self.client_id = client_id

    def write(self, data):
        self.stream.write(data)
        self.stream.update_io_state(selectors.EVENT_WRITE)

    def update_io_state(self, mask):
        self.stream.update_io_state(mask)

    def check_stream(self, stream):
        return self.stream == stream

    def check_client_id(self, client_id):
        return self.client_id == client_id


class Dispatch(object):

    def __init__(self, mess):
        self.mess = mess
        self.messages = {}
        self.clients = {}
        self.prepares = {}
        self.init()

    def init(self):
        for mes in self.mess:
            self.register_message(mes)

    def register_tran(self, xid, tran):
        self.prepares[xid] = tran

    def register_client(self, client_id, stream):
        self.clients[client_id] = RegisterClient(stream, client_id)

    def register_message(self, message):
        if not issubclass(message, BaseHandler):
            return
        if hasattr(message, "type"):
            mes_type = getattr(message, "type")
            if mes_type:
                self.messages[mes_type] = message

    def execute(self, type, request_data, protocl):
        if type in self.messages.keys():
            self.messages[type](request_data, protocl).serilizer()
        else:
            ErrorMsg(request_data, protocl).serilizer()

    def close(self, stream):
        remove_client = None
        for key in self.clients:
            if self.clients[key].check_stream(stream):
                logger.info("found  error  key:  {0}  straem: {1} ".format(key, stream))
                remove_client = key

        if remove_client:
            self.clients.pop(remove_client)


messages = [RegisterHandler, ClientPostData, PreSubmitAck, SubmitAbort]
dispatch = Dispatch(messages)
logger.info(dispatch.messages)


class CooProtocl(object):

    def __init__(self, stream):
        self.stream = stream
        self._read_delimiter = HEAD_DELIMITER
        self._body_delimiter = BODY_DELIMITER
        self.stream.read_util(self._read_delimiter, self.parse_length)

    def parse_length(self, data):
        body_length = data[:len(self._read_delimiter)]
        if body_length.isdigit():
            self.stream.read_nums_util(int(body_length), self.parse_body)

    def write_result(self, data):
        self.stream.write(data)

    def parse_body(self, data):
        loc = data.find(self._body_delimiter)
        msg_type = data[:loc]
        msg_content = data[(loc+len(self._body_delimiter)):]
        logger.info("{0}   {1}".format(msg_type, msg_content))
        dispatch.execute(msg_type, msg_content, self)
        self.stream.read_util(self._read_delimiter, self.parse_length)


class Stream(object):

    def __init__(self, conn, addr, server):
        self.conn = conn
        self.addr = addr
        self.conn.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 1024)
        self.conn.setblocking(False)
        self.recv_buffer = ""
        self.write_buffer = b""
        self._read_delimiter = None
        self._read_callback = None
        self._read_length = None
        self._read_nums_callback = None
        self.server = server
        self.state = selectors.EVENT_READ
        self.server.selector.register(self.conn, self.state, self.handle_event)

    def _consume(self, loc):
        data = self.recv_buffer[:loc]
        self.recv_buffer = self.recv_buffer[loc:]
        return data

    def update_io_state(self, mask):
        self.state = mask
        self.server.modify(self.conn, mask, self.handle_event)

    def read_util(self, delimiter, callback):
        loc = self.recv_buffer.find(delimiter)
        if loc != -1:
            callback(self._consume(loc+len(delimiter)))
            return
        # 如果没有解析对的格式则表示还需要等待数据输入
        self._read_delimiter = delimiter
        self._read_callback = callback
        self.update_io_state(selectors.EVENT_READ)

    def read_nums_util(self, read_length, callback):
        if len(self.recv_buffer) >= read_length:
            callback(self._consume(read_length))
            return
        self._read_length = read_length
        self._read_nums_callback = callback
        self.update_io_state(selectors.EVENT_READ)

    def close(self):
        if self.state != -1:
            self.state = -1
            dispatch.close(self)
            self.server.modify(self.conn, self.state)

    def handle_read(self):
        try:
            data = self.conn.recv(1)
        except socket.error as e:
            if e.errno in (socket.EAGAIN, socket.EWOULDBLOCK):
                logger.info("connectreset error : {0}".format(e))
                return
            else:
                logger.info("handler read error : {0}".format(e))
                self.close()
                return

        if not data:
            self.close()
            return

        self.recv_buffer += data.decode("utf-8")

        if self._read_delimiter:
            loc = self.recv_buffer.find(self._read_delimiter)
            if loc != -1:
                callback = self._read_callback
                data = self._consume(loc+len(self._read_delimiter))
                self._read_delimiter = None
                self._read_callback = None
                callback(data)
        elif self._read_length:
            if len(self.recv_buffer) >= self._read_length:
                callback = self._read_nums_callback
                data = self._consume(self._read_length)
                self._read_length = None
                self._read_nums_callback = None
                callback(data)

    def write(self, data):
        self.write_buffer += data.encode("utf-8")

    def handle_write(self):
        while self.write_buffer:
            try:
                size = self.conn.send(self.write_buffer)
            except socket.error as e:
                if e.errno in (socket.EWOULDBLOCK, socket.EAGAIN):
                    logger.error("send error : {0}".format(e))
                    break
                else:
                    self.close()
                    logger.error("handle write close error : {0}".format(e))
                    return
            else:
                self.write_buffer = self.write_buffer[size:]
            logger.info("left data :    {0}".format(self.write_buffer))

    def handle_event(self, conn, mask):
        if mask == selectors.EVENT_READ:
            self.handle_read()
        elif mask == selectors.EVENT_WRITE:
            self.handle_write()

        if self.state == -1:
            return

        state = 0
        if self._read_length or self._read_delimiter:
            state = selectors.EVENT_READ | state
        if self.write_buffer:
            state = selectors.EVENT_WRITE | state
        if state != self.state:
            logger.info("290   {0}".format(state, self.state))
            self.state = state
            self.update_io_state(state)


class Server(object):

    def __init__(self, ip=None, port=None):
        self.ip = ip or "127.0.0.1"
        self.port = port or 4545
        self.sock = None
        self.running = False
        self.selector = selectors.DefaultSelector()
        self.init()

    def init(self):
        self.sock = socket.socket()
        self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.sock.bind((self.ip, self.port))
        self.sock.listen(100)
        self.sock.setblocking(False)
        self.selector.register(self.sock, selectors.EVENT_READ, self.accept)

    def modify(self, conn, mask, callback=None):
        logger.info("302  {0}   {1}   {2}".format(conn, mask, callback))
        if mask != -1:
            self.selector.modify(conn, mask, callback)
        else:
            self.selector.unregister(conn)

    def accept(self, conn, mask):
        conn, addr = conn.accept()
        logger.info("recv conn    {0}".format(conn))
        s = Stream(conn, addr, self)
        CooProtocl(s)

    def start(self):
        if not self.running:
            self.running = True
        while self.running:
            events = self.selector.select(timeout=1)
            for key, mask in events:
                callback = key.data
                callback(key.fileobj, mask)

    def stop(self):
        if self.running:
            self.running = False


if __name__ == '__main__':
    s = Server()
    s.start()

客户端代码

import socket
import selectors
import threading
import click
import random
import time
from prompt_toolkit import prompt

from test_select import *

logger = logging.getLogger()

presubmit_flag = True
abort_flag = False


class Client(object):

    def __init__(self, name, ip=None, port=None):
        self.ip = ip or "127.0.0.1"
        self.port = port or 4545
        self.client_name = name
        self.sock = None
        self.running = False
        self.recv_buffer = ""
        self.write_buffer = b""
        self.selector = selectors.DefaultSelector()
        self.init()

    def init(self):
        self.sock = socket.socket()
        self.sock.connect((self.ip, self.port))
        self.sock.setblocking(False)
        self.selector.register(self.sock, selectors.EVENT_READ, self.read)

    def handle_execute(self, msg_type, msg_content):
        if presubmit_flag and msg_type == "presubmit":
            xid = PreSubmit(request_data=msg_content).serilizer()
            if abort_flag:
                abort = SubmitAbort()
                self.write_result(abort.write_body(xid, self.client_name).encode("utf-8"))
            else:
                p_ack = PreSubmitAck()
                self.write_result(p_ack.write_body(xid, self.client_name).encode("utf-8"))

    def read(self, conn, mask):
        data = conn.recv(1000)
        if data:
            logger.info("client recv data :  {0}".format(repr(data)))
            self.recv_buffer += data.decode("utf-8")
            while True:
                loc = self.recv_buffer.find(HEAD_DELIMITER)
                logger.info("loc index : {0}   recv buffer  {1}".format(loc, self.recv_buffer))
                if loc != -1:
                    length = int(self.recv_buffer[:loc])
                    self.recv_buffer = self.recv_buffer[(loc+len(HEAD_DELIMITER)):]
                    if length <= len(self.recv_buffer):
                        data = self.recv_buffer[:length]
                        body_loc = data.find(BODY_DELIMITER)
                        if body_loc != -1:
                            msg_type = data[:body_loc]
                            msg_content = data[(body_loc+len(BODY_DELIMITER)):]
                            logger.info("parse from server  {0}    {1}  ".format(msg_type, msg_content))
                            self.handle_execute(msg_type, msg_content)
                        self.recv_buffer = self.recv_buffer[length:]
                else:
                    break
        else:
            self.close()

    def close(self):
        logger.info("client close")
        self.selector.unregister(self.sock)
        self.sock.close()

        if self.running:
            time.sleep(1)
            self.init()

    def handle_send(self):
        while self.write_buffer:
            try:
                size = self.sock.send(self.write_buffer)
            except socket.error as e:
                if e.errno in (socket.EWOULDBLOCK, socket.EAGAIN):
                    logger.error("send error : {0}".format(e))
                    break
                else:
                    logger.error("handle write close error : {0}".format(e))
                    return
            else:
                self.write_buffer = self.write_buffer[size:]

    def write_result(self, data):
        if self.sock:
            logger.info("send   {0}".format(repr(data)))
            while data:
                try:
                    size = self.sock.send(data)
                except socket.error as e:
                    if e.errno in (socket.EWOULDBLOCK, socket.EAGAIN):
                        logger.error("send error : {0}".format(e))
                        break
                    else:
                        logger.error("handle write close error : {0}".format(e))
                        return
                else:
                    logger.info("send  size  {0}".format(size))
                    data = data[size:]

    def stop(self):
        if self.running:
            self.running = False
            print("close")

    def start(self):
        t = threading.Thread(target=self.worker)
        t.setDaemon(False)
        t.start()

    def worker(self):
        if not self.running:
            self.running = True

        print(self.running)
        while self.running:
            events = self.selector.select(timeout=1)
            for key, mask in events:
                callback = key.data
                callback(key.fileobj, mask)
            self.handle_send()
        print("work over")


class Command(object):

    def __init__(self, name):
        self.name = name

    def parse_command(self, args):
        raise NotImplementedError


class SendCommand(Command):

    def __init__(self, name, callback):
        super().__init__(name)
        self.callback = callback

    def parse_command(self, args):

        if len(args) != 3:
            print(" {0} args must be 3  {1}".format(self.name, args))
            return

        key, value = args[1], args[2]
        self.callback(key, value)


class AbortCommand(Command):

    def parse_command(self, args):
        if len(args) != 2:
            print(" {0} args must be 2 ".format(self.name))
            return
        data = {
            "true": True,
            "false": False,
        }
        if args[1] not in data:
            print(" {0} value not correct ".format(self.name))
            return
        global abort_flag
        abort_flag = data[args[1]]
        print("abort_flag : {0}".format(abort_flag))


class PreSubmitCommand(Command):

    def parse_command(self, args):
        if len(args) != 2:
            print(" {0} args must be 2 ".format(self.name))
            return
        data = {
            "true": True,
            "false": False,
        }
        if args[1] not in data:
            print(" {0} value not correct ".format(self.name))
            return
        global presubmit_flag
        presubmit_flag = data[args[1]]
        print(" presubmit_flag : {0}".format(presubmit_flag))


class CommandDispatch(object):

    def __init__(self):
        self.commands = {}

    def register(self, command):
        if isinstance(command, Command):
            self.commands[command.name] = command

    def execute(self, command_name, args):
        if command_name not in self.commands:
            print(" commands not found ")
            return
        self.commands[command_name].parse_command(args)


@click.command()
@click.option('--name', default=str(random.randint(1, 1000)), help="must unique client id")
def main(name):
    c = Client(name=name)
    c.start()

    c.write_result(RegisterHandler().serilizer_data(name).encode("utf-8"))

    def write(key, value):
        c.write_result(ClientPostData().write_body(name, key, value).encode("utf-8"))

    dis = CommandDispatch()

    dis.register(AbortCommand("abort"))
    dis.register(PreSubmitCommand("presubmit"))
    dis.register(SendCommand("send", write))

    while True:
        answer = prompt('input> ').strip()
        args = answer.split(" ")
        counts = args.count("")
        for i in range(counts):
            args.remove("")
        if len(args) >= 2:
            command = args[0]
            print("args : {0}".format(args))
            dis.execute(command, args)
        else:
            print("args not correct")


if __name__ == '__main__':
    main()

代码联调

首先启动服务端，让服务端等待请求

(venv) wuzideMacBook-Pro:2pc wuzi$ python 2pc_server.py 
2019-11-01 21:35:56,379 - 4603864512 - 346 - INFO - {'register': <class '__main__.RegisterHandler'>, 'client_start': <class '__main__.ClientPostData'>, 'presubmitack': <class '__main__.PreSubmitAck'>, 'submitabort': <class '__main__.SubmitAbort'>}

接着我们就启动三个客户端来处理分别为client1,client2,client3。

(venv) wuzideMacBook-Pro:2pc wuzi$ python 2pc_client.py --name=client1 
2019-11-01 21:37:01,343 - 4439086528 - 437 - INFO - {'type1': <class 'test_select.MsgOne'>, 'type2': <class 'test_select.MsgTwo'>, 'register': <class 'test_select.RegisterHandler'>, 'client_start': <class 'test_select.ClientPostData'>, 'presubmitack': <class 'test_select.PreSubmitAck'>, 'submitabort': <class 'test_select.SubmitAbort'>}
True
2019-11-01 21:37:01,346 - 4439086528 - 96 - INFO - send   b'16\r\nregister\nclient1'
2019-11-01 21:37:01,346 - 4439086528 - 108 - INFO - send  size  20
2019-11-01 21:37:01,348 - 123145470545920 - 49 - INFO - client recv data :  b'12\r\nregister\n200'
2019-11-01 21:37:01,348 - 123145470545920 - 53 - INFO - loc index : 2   recv buffer  12
register
200
2019-11-01 21:37:01,348 - 123145470545920 - 63 - INFO - parse from server  register    200  
2019-11-01 21:37:01,349 - 123145470545920 - 53 - INFO - loc index : -1   recv buffer  
input>

(venv) wuzideMacBook-Pro:2pc wuzi$ python 2pc_client.py --name client2
2019-11-01 21:37:29,948 - 4399826368 - 437 - INFO - {'type1': <class 'test_select.MsgOne'>, 'type2': <class 'test_select.MsgTwo'>, 'register': <class 'test_select.RegisterHandler'>, 'client_start': <class 'test_select.ClientPostData'>, 'presubmitack': <class 'test_select.PreSubmitAck'>, 'submitabort': <class 'test_select.SubmitAbort'>}
True
2019-11-01 21:37:29,950 - 4399826368 - 96 - INFO - send   b'16\r\nregister\nclient2'
2019-11-01 21:37:29,951 - 4399826368 - 108 - INFO - send  size  20
2019-11-01 21:37:29,952 - 123145410183168 - 49 - INFO - client recv data :  b'12\r\nregister\n200'
2019-11-01 21:37:29,952 - 123145410183168 - 53 - INFO - loc index : 2   recv buffer  12
register
200
2019-11-01 21:37:29,952 - 123145410183168 - 63 - INFO - parse from server  register    200  
2019-11-01 21:37:29,953 - 123145410183168 - 53 - INFO - loc index : -1   recv buffer  
input>

(venv) wuzideMacBook-Pro:2pc wuzi$ python 2pc_client.py --name=client3
2019-11-01 21:38:17,543 - 4597429696 - 437 - INFO - {'type1': <class 'test_select.MsgOne'>, 'type2': <class 'test_select.MsgTwo'>, 'register': <class 'test_select.RegisterHandler'>, 'client_start': <class 'test_select.ClientPostData'>, 'presubmitack': <class 'test_select.PreSubmitAck'>, 'submitabort': <class 'test_select.SubmitAbort'>}
True
2019-11-01 21:38:17,546 - 4597429696 - 96 - INFO - send   b'16\r\nregister\nclient3'
2019-11-01 21:38:17,547 - 4597429696 - 108 - INFO - send  size  20
2019-11-01 21:38:17,549 - 123145484255232 - 49 - INFO - client recv data :  b'12\r\nregister\n200'
2019-11-01 21:38:17,549 - 123145484255232 - 53 - INFO - loc index : 2   recv buffer  12
register
200
2019-11-01 21:38:17,549 - 123145484255232 - 63 - INFO - parse from server  register    200  
2019-11-01 21:38:17,549 - 123145484255232 - 53 - INFO - loc index : -1   recv buffer  
input>

事务提交测试

让client1在终端中输入需要提交的数据；

input> send key data                                                                                                                                                                                    
args : ['send', 'key', 'data']
2019-11-01 21:39:14,687 - 4439086528 - 96 - INFO - send   b'29\r\nclient_start\nclient1\tkey\tdata'
2019-11-01 21:39:14,687 - 4439086528 - 108 - INFO - send  size  33
2019-11-01 21:39:14,717 - 123145470545920 - 49 - INFO - client recv data :  b'19\r\nclient_start\nfinish'
input> 2019-11-01 21:39:14,719 - 123145470545920 - 53 - INFO - loc index : 2   recv buffer  19                                                                                                          
client_start
finish
2019-11-01 21:39:14,719 - 123145470545920 - 63 - INFO - parse from server  client_start    finish  
2019-11-01 21:39:14,719 - 123145470545920 - 53 - INFO - loc index : -1   recv buffer

此时client2与client3收到的数据如下；

 2019-11-01 21:39:14,690 - 123145410183168 - 49 - INFO - client recv data :  b'55\r\npresubmit\n5d677fb2-01dd-4962-b66b-24d3b572ca1d\tkey\tdata'                                                  
2019-11-01 21:39:14,691 - 123145410183168 - 53 - INFO - loc index : 2   recv buffer  55
presubmit
5d677fb2-01dd-4962-b66b-24d3b572ca1d    key     data
2019-11-01 21:39:14,691 - 123145410183168 - 63 - INFO - parse from server  presubmit    5d677fb2-01dd-4962-b66b-24d3b572ca1d    key     data  
2019-11-01 21:39:14,691 - 123145410183168 - 188 - INFO - client recv :  xid :  5d677fb2-01dd-4962-b66b-24d3b572ca1d  key : key  value : data
2019-11-01 21:39:14,691 - 123145410183168 - 96 - INFO - send   b'57\r\npresubmitack\n5d677fb2-01dd-4962-b66b-24d3b572ca1d\tclient2'
2019-11-01 21:39:14,691 - 123145410183168 - 108 - INFO - send  size  61
2019-11-01 21:39:14,691 - 123145410183168 - 53 - INFO - loc index : -1   recv buffer  
2019-11-01 21:39:14,709 - 123145410183168 - 49 - INFO - client recv data :  b'47\r\npreperpost\n5d677fb2-01dd-4962-b66b-24d3b572ca1d'
2019-11-01 21:39:14,710 - 123145410183168 - 53 - INFO - loc index : 2   recv buffer  47
preperpost
5d677fb2-01dd-4962-b66b-24d3b572ca1d
2019-11-01 21:39:14,710 - 123145410183168 - 63 - INFO - parse from server  preperpost    5d677fb2-01dd-4962-b66b-24d3b572ca1d  
2019-11-01 21:39:14,710 - 123145410183168 - 53 - INFO - loc index : -1   recv buffer

2019-11-01 21:39:14,695 - 123145484255232 - 49 - INFO - client recv data :  b'55\r\npresubmit\n5d677fb2-01dd-4962-b66b-24d3b572ca1d\tkey\tdata'                                                  
2019-11-01 21:39:14,695 - 123145484255232 - 53 - INFO - loc index : 2   recv buffer  55
presubmit
5d677fb2-01dd-4962-b66b-24d3b572ca1d    key     data
2019-11-01 21:39:14,695 - 123145484255232 - 63 - INFO - parse from server  presubmit    5d677fb2-01dd-4962-b66b-24d3b572ca1d    key     data  
2019-11-01 21:39:14,695 - 123145484255232 - 188 - INFO - client recv :  xid :  5d677fb2-01dd-4962-b66b-24d3b572ca1d  key : key  value : data
2019-11-01 21:39:14,695 - 123145484255232 - 96 - INFO - send   b'57\r\npresubmitack\n5d677fb2-01dd-4962-b66b-24d3b572ca1d\tclient3'
2019-11-01 21:39:14,696 - 123145484255232 - 108 - INFO - send  size  61
2019-11-01 21:39:14,696 - 123145484255232 - 53 - INFO - loc index : -1   recv buffer  
2019-11-01 21:39:14,711 - 123145484255232 - 49 - INFO - client recv data :  b'47\r\npreperpost\n5d677fb2-01dd-4962-b66b-24d3b572ca1d'
2019-11-01 21:39:14,711 - 123145484255232 - 53 - INFO - loc index : 2   recv buffer  47
preperpost
5d677fb2-01dd-4962-b66b-24d3b572ca1d
2019-11-01 21:39:14,711 - 123145484255232 - 63 - INFO - parse from server  preperpost    5d677fb2-01dd-4962-b66b-24d3b572ca1d  
2019-11-01 21:39:14,711 - 123145484255232 - 53 - INFO - loc index : -1   recv buffer

从日志中可以看出，数据以及提交到了本地，并且通过了提交。此时可以看出client1发出的事务提交，是得到了client2与client3的答复之后，就确认提交成功了。基本的事务提交功能完成。

事务中止测试

假如还是由client1发送事务数据，此时让client2在接受到是否可以进行事务提交的时候发挥abort信息，首先在client2中输入如下数据更改本例中的测试标志位；

input> abort true                                                                                                                                                                                       
args : ['abort', 'true']
abort_flag : True
input>

此时我们继续在client1中输入数据；

send key2 value2                                                                                                                                                                                 
args : ['send', 'key2', 'value2']
2019-11-01 21:47:36,660 - 4439086528 - 96 - INFO - send   b'32\r\nclient_start\nclient1\tkey2\tvalue2'
2019-11-01 21:47:36,661 - 4439086528 - 108 - INFO - send  size  36
2019-11-01 21:47:36,674 - 123145470545920 - 49 - INFO - client recv data :  b'56\r\nsubmitabort\n85ccd0b5-d883-4550-9726-16da44abb084\tclient2'
2019-11-01 21:47:36,674 - 123145470545920 - 53 - INFO - loc index : 2   recv buffer  56
submitabort
85ccd0b5-d883-4550-9726-16da44abb084    client2
2019-11-01 21:47:36,674 - 123145470545920 - 63 - INFO - parse from server  submitabort    85ccd0b5-d883-4550-9726-16da44abb084  client2  
2019-11-01 21:47:36,674 - 123145470545920 - 53 - INFO - loc index : -1   recv buffer

此时client2的日志如下；

 2019-11-01 21:47:36,662 - 123145410183168 - 49 - INFO - client recv data :  b'58\r\npresubmit\n85ccd0b5-d883-4550-9726-16da44abb084\tkey2\tvalue2'                                               
2019-11-01 21:47:36,662 - 123145410183168 - 53 - INFO - loc index : 2   recv buffer  58
presubmit
85ccd0b5-d883-4550-9726-16da44abb084    key2    value2
2019-11-01 21:47:36,663 - 123145410183168 - 63 - INFO - parse from server  presubmit    85ccd0b5-d883-4550-9726-16da44abb084    key2    value2  
2019-11-01 21:47:36,663 - 123145410183168 - 188 - INFO - client recv :  xid :  85ccd0b5-d883-4550-9726-16da44abb084  key : key2  value : value2
2019-11-01 21:47:36,663 - 123145410183168 - 96 - INFO - send   b'56\r\nsubmitabort\n85ccd0b5-d883-4550-9726-16da44abb084\tclient2'
2019-11-01 21:47:36,663 - 123145410183168 - 108 - INFO - send  size  60
2019-11-01 21:47:36,663 - 123145410183168 - 53 - INFO - loc index : -1   recv buffer  
2019-11-01 21:47:36,666 - 123145410183168 - 49 - INFO - client recv data :  b'20\r\nsubmitabort\nabort ok'
2019-11-01 21:47:36,666 - 123145410183168 - 53 - INFO - loc index : 2   recv buffer  20
submitabort
abort ok
2019-11-01 21:47:36,666 - 123145410183168 - 63 - INFO - parse from server  submitabort    abort ok  
2019-11-01 21:47:36,666 - 123145410183168 - 53 - INFO - loc index : -1   recv buffer

从日志中可以看出，在接受到presubmit消息的时候，此时client2返回的是abort的消息，并且服务器返回了一条abort成功的消息。继续分析client3的消息。

2019-11-01 21:47:36,663 - 123145484255232 - 49 - INFO - client recv data :  b'58\r\npresubmit\n85ccd0b5-d883-4550-9726-16da44abb084\tkey2\tvalue2'
2019-11-01 21:47:36,663 - 123145484255232 - 53 - INFO - loc index : 2   recv buffer  58
presubmit
85ccd0b5-d883-4550-9726-16da44abb084    key2    value2
2019-11-01 21:47:36,663 - 123145484255232 - 63 - INFO - parse from server  presubmit    85ccd0b5-d883-4550-9726-16da44abb084    key2    value2  
2019-11-01 21:47:36,663 - 123145484255232 - 188 - INFO - client recv :  xid :  85ccd0b5-d883-4550-9726-16da44abb084  key : key2  value : value2
2019-11-01 21:47:36,663 - 123145484255232 - 96 - INFO - send   b'57\r\npresubmitack\n85ccd0b5-d883-4550-9726-16da44abb084\tclient3'
2019-11-01 21:47:36,663 - 123145484255232 - 108 - INFO - send  size  61
2019-11-01 21:47:36,663 - 123145484255232 - 53 - INFO - loc index : -1   recv buffer  
2019-11-01 21:47:36,667 - 123145484255232 - 49 - INFO - client recv data :  b'56\r\nsubmitabort\n85ccd0b5-d883-4550-9726-16da44abb084\tclient2'
2019-11-01 21:47:36,667 - 123145484255232 - 53 - INFO - loc index : 2   recv buffer  56
submitabort
85ccd0b5-d883-4550-9726-16da44abb084    client2
2019-11-01 21:47:36,667 - 123145484255232 - 63 - INFO - parse from server  submitabort    85ccd0b5-d883-4550-9726-16da44abb084  client2  
2019-11-01 21:47:36,667 - 123145484255232 - 53 - INFO - loc index : -1   recv buffer

从client3的日志中可以看出，在接收到presubmit的消息的时候，已经给服务端返回了预提交成功的消息，但是在之后就立马收到了服务端发送的submitabort的消息，从而得到事务中止的消息，至此事务中止的演示完成。

事务异常

当其中一个客户端在消息预提交的时候，不回复服务端的时候，此时相对服务端而言就得不到所有的客户端的成功回复，该事务就会一直处于等待状态，在版本V1中并没有对该情况进行其他处理，会导致该事务一直保存在服务端。

在client2中输入;

input> presubmit false                                                                                                                                                                                  
args : ['presubmit', 'false']
 presubmit_flag : False

此时继续在client1中发送数据；

send key3  value3                                                                                                                                                                                
args : ['send', 'key3', 'value3']
2019-11-01 22:02:10,690 - 4439086528 - 96 - INFO - send   b'32\r\nclient_start\nclient1\tkey3\tvalue3'
2019-11-01 22:02:10,690 - 4439086528 - 108 - INFO - send  size  36

此时client2的信息如下；

2019-11-01 22:02:10,698 - 123145410183168 - 49 - INFO - client recv data :  b'58\r\npresubmit\n048df5a7-00e7-4344-91fa-1700c7b738ba\tkey3\tvalue3'                                               
2019-11-01 22:02:10,699 - 123145410183168 - 53 - INFO - loc index : 2   recv buffer  58
presubmit
048df5a7-00e7-4344-91fa-1700c7b738ba    key3    value3
2019-11-01 22:02:10,699 - 123145410183168 - 63 - INFO - parse from server  presubmit    048df5a7-00e7-4344-91fa-1700c7b738ba    key3    value3  
2019-11-01 22:02:10,699 - 123145410183168 - 53 - INFO - loc index : -1   recv buffer

此时从信息中可以看见，client2收到了预提交数据，但是没有回复服务端ack，此时client3的消息如下；

2019-11-01 22:02:10,700 - 123145484255232 - 49 - INFO - client recv data :  b'58\r\npresubmit\n048df5a7-00e7-4344-91fa-1700c7b738ba\tkey3\tvalue3'
2019-11-01 22:02:10,701 - 123145484255232 - 53 - INFO - loc index : 2   recv buffer  58
presubmit
048df5a7-00e7-4344-91fa-1700c7b738ba    key3    value3
2019-11-01 22:02:10,702 - 123145484255232 - 63 - INFO - parse from server  presubmit    048df5a7-00e7-4344-91fa-1700c7b738ba    key3    value3  
2019-11-01 22:02:10,702 - 123145484255232 - 188 - INFO - client recv :  xid :  048df5a7-00e7-4344-91fa-1700c7b738ba  key : key3  value : value3
2019-11-01 22:02:10,702 - 123145484255232 - 96 - INFO - send   b'57\r\npresubmitack\n048df5a7-00e7-4344-91fa-1700c7b738ba\tclient3'
2019-11-01 22:02:10,702 - 123145484255232 - 108 - INFO - send  size  61
2019-11-01 22:02:10,703 - 123145484255232 - 53 - INFO - loc index : -1   recv buffer

从消息中，可以看出client3收到了预提交数据并回复了可以提交，但是就是一直没有收到可以提交数据的确认消息。此时就会一直在服务端保存该事务，该版本的实现中，因为客户端可以再动态的加入到服务端，此时如果当前正有事务在进行的话，如果加入了一个新的客户端，此时就会导致该事务得不到新加入的回复，导致该事务不会被提交，这个问题不在V1版本的考虑范围。

总结

本文主要就是实现了两阶段提交的基本流程，并没有考虑到其他异常的情况，主要完成了最基本的事务正常提交的流程，事务中止的流程等，并且在实现过程中可以通过加入相对的日志来提高性能，但是在V1版本中并没有实现，而且本文中的实例更像是类似与Zookeeper的主从复制的原理，而且在本版本中并没有加入超时回调，在后续的版本中可能会考虑更多的细节情况，后续会努力去完善类图的关系与设计。由于本人才疏学浅，如有错误请批评指正。

小屋子大侠

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
两阶段提交实际项目V1

项目介绍两阶段提交项目主要是实际用代码演示复现一下，两阶段提交的执行过程，仅供学习参考。本次主要分析的版本为V1版本，主要实现的流程包括服务端的基础架构编写，客户端的基础架构编写，完成事务提交的过程。两阶段项目的用例模型客户端注册用例名称客户端启动注册主要参与者客户端涉及关注点客户端：希望将自己注册到服务器端，从而能够向服务端提交事务数据或者接受其他客户端的提...
复制链接

扫一扫

专栏目录