python 网络框架twisted基础学习及详细讲解

twisted网络框架的三个基础模块：Protocol, ProtocolFactory, Transport.这三个模块是构成twisted服务器端与客户端程序的基本。

Protocol：Protocol对象实现协议内容，即通信的内容协议
ProtocolFactory: 是工厂模式的体现，在这里面生成协议
Transport: 是用来收发数据，服务器端与客户端的数据收发与处理都是基于这个模块

在windows中安装twisted需要先安装pywin32,自己去下载下就行。随后pip install twisted就会帮我们安装twisted以及zope。

我们结合一张图，以及一段程序来理解下twisted的基础实现：

然后我们首先看看服务器端程序：

# coding=utf-8
from twisted.internet.protocol import Protocol
from twisted.internet.protocol import Factory
from twisted.internet.endpoints import TCP4ServerEndpoint
from twisted.internet import reactor


clients = []


class Spreader(Protocol):
    def __init__(self, factory):
        self.factory = factory

    def connectionMade(self):
        self.factory.numProtocols = self.factory.numProtocols + 1
        self.transport.write(
            "欢迎来到Spread Site, 你是第%s个客户端用户!
" % (self.factory.numProtocols)
        )
        print "new connect: %d" % (self.factory.numProtocols)
        clients.append(self)

    def connectionLost(self, reason):
        self.factory.numProtocols = self.factory.numProtocols - 1
        clients.remove(self)
        print "lost connect: %d" % (self.factory.numProtocols)

    def dataReceived(self, data):
        if data == "close":
            self.transport.loseConnection()
            for client in clients:
                if client != self:
                    client.transport.write(data)
        else:
            print data


class SpreadFactory(Factory):
    def __init__(self):
        self.numProtocols = 0

    def buildProtocol(self, addr):
        return Spreader(self)


endpoint = TCP4ServerEndpoint(reactor, 8007)
endpoint.listen(SpreadFactory())
reactor.run()

创建一个TCP的IPv4版本的终结点，随后就开始监听listen, 在这里我们传入协议工厂对象作为参数, 先看看我们自定义的工厂类SpreadFactory, 它派生自Factory, 我们看看这个类的源码(此时你需要有道词典了:) )：

@implementer(interfaces.IProtocolFactory, interfaces.ILoggingContext)
@_oldStyle
class Factory:
    """
    This is a factory which produces protocols.

    By default, buildProtocol will create a protocol of the class given in
    self.protocol.
    """

    # put a subclass of Protocol here:
    protocol = None

    numPorts = 0
    noisy = True

    @classmethod
    def forProtocol(cls, protocol, *args, **kwargs):
        """
        Create a factory for the given protocol.

        It sets the C{protocol} attribute and returns the constructed factory
        instance.

        @param protocol: A L{Protocol} subclass

        @param args: Positional arguments for the factory.

        @param kwargs: Keyword arguments for the factory.

        @return: A L{Factory} instance wired up to C{protocol}.
        """
        factory = cls(*args, **kwargs)
        factory.protocol = protocol
        return factory


    def logPrefix(self):
        """
        Describe this factory for log messages.
        """
        return self.__class__.__name__


    def doStart(self):
        """Make sure startFactory is called.

        Users should not call this function themselves!
        """
        if not self.numPorts:
            if self.noisy:
                _loggerFor(self).info("Starting factory {factory!r}",
                                      factory=self)
            self.startFactory()
        self.numPorts = self.numPorts + 1

    def doStop(self):
        """Make sure stopFactory is called.

        Users should not call this function themselves!
        """
        if self.numPorts == 0:
            # this shouldn't happen, but does sometimes and this is better
            # than blowing up in assert as we did previously.
            return
        self.numPorts = self.numPorts - 1
        if not self.numPorts:
            if self.noisy:
                _loggerFor(self).info("Stopping factory {factory!r}",
                                      factory=self)
            self.stopFactory()

    def startFactory(self):
        """This will be called before I begin listening on a Port or Connector.

        It will only be called once, even if the factory is connected
        to multiple ports.

        This can be used to perform 'unserialization' tasks that
        are best put off until things are actually running, such
        as connecting to a database, opening files, etcetera.
        """

    def stopFactory(self):
        """This will be called before I stop listening on all Ports/Connectors.

        This can be overridden to perform 'shutdown' tasks such as disconnecting
        database connections, closing files, etc.

        It will be called, for example, before an application shuts down,
        if it was connected to a port. User code should not call this function
        directly.
        """


    def buildProtocol(self, addr):
        """
        Create an instance of a subclass of Protocol.

        The returned instance will handle input on an incoming server
        connection, and an attribute "factory" pointing to the creating
        factory.

        Alternatively, L{None} may be returned to immediately close the
        new connection.

        Override this method to alter how Protocol instances get created.

        @param addr: an object implementing L{twisted.internet.interfaces.IAddress}
        """
        p = self.protocol()
        p.factory = self
        return p

在这里很重要的一个函数就是buildProtocol, 此函数就是在工厂模式中创建协议的.我们是基于基类Factory来实现这个函数的, 下面我们看看派生自Protocol的协议类Spread,Spread的__Init__参数中，我们给它传入的是自定义的SpreadFactory, 然后我们看下基类Protocol的源代码

@implementer(interfaces.IProtocol, interfaces.ILoggingContext)
class Protocol(BaseProtocol):
    """
    This is the base class for streaming connection-oriented protocols.

    If you are going to write a new connection-oriented protocol for Twisted,
    start here.  Any protocol implementation, either client or server, should
    be a subclass of this class.

    The API is quite simple.  Implement L{dataReceived} to handle both
    event-based and synchronous input; output can be sent through the
    'transport' attribute, which is to be an instance that implements
    L{twisted.internet.interfaces.ITransport}.  Override C{connectionLost} to be
    notified when the connection ends.

    Some subclasses exist already to help you write common types of protocols:
    see the L{twisted.protocols.basic} module for a few of them.
    """

    def logPrefix(self):
        """
        Return a prefix matching the class name, to identify log messages
        related to this protocol instance.
        """
        return self.__class__.__name__


    def dataReceived(self, data):
        """Called whenever data is received.

        Use this method to translate to a higher-level message.  Usually, some
        callback will be made upon the receipt of each complete protocol
        message.

        @param data: a string of indeterminate length.  Please keep in mind
            that you will probably need to buffer some data, as partial
            (or multiple) protocol messages may be received!  I recommend
            that unit tests for protocols call through to this method with
            differing chunk sizes, down to one byte at a time.
        """

    def connectionLost(self, reason=connectionDone):
        """Called when the connection is shut down.

        Clear any circular references here, and any external references
        to this Protocol.  The connection has been closed.

        @type reason: L{twisted.python.failure.Failure}
        """

而Protocol又是派生自BaseProtocol的，继续看这个类的源代码：

@_oldStyle
class BaseProtocol:
    """
    This is the abstract superclass of all protocols.

    Some methods have helpful default implementations here so that they can
    easily be shared, but otherwise the direct subclasses of this class are more
    interesting, L{Protocol} and L{ProcessProtocol}.
    """
    connected = 0
    transport = None

    def makeConnection(self, transport):
        """Make a connection to a transport and a server.

        This sets the 'transport' attribute of this Protocol, and calls the
        connectionMade() callback.
        """
        self.connected = 1
        self.transport = transport
        self.connectionMade()

    def connectionMade(self):
        """Called when a connection is made.

        This may be considered the initializer of the protocol, because
        it is called when the connection is completed.  For clients,
        this is called once the connection to the server has been
        established; for servers, this is called after an accept() call
        stops blocking and a socket has been received.  If you need to
        send any greeting or initial message, do it here.
        """

connectionDone=failure.Failure(error.ConnectionDone())
connectionDone.cleanFailure()

可以看到，我们自定义的Spread不过是实现了基类的函数。接下来我们滚一边实现逻辑:
首先，我们定义一个列表clients，以便存储多个客户端的连接。当服务器端接收到了客户端的连接后，调用connectionMade函数，同时，我们给使用Transport客户端发送消息, 通知客户端我们已收到连接。当客户端连接失去的时候，我们调用ConnectionLost, 同时移除列表中的客户端连接, dataReceived函数来接受数据，当客户端传来"close"命令时，我们就主动关闭连接, 否则，我们就把data输出来。

看看客户端的代码:

# coding=utf-8
from twisted.internet.protocol import Protocol, ClientFactory
from twisted.internet import reactor
import threading
import time
import sys
import datetime


class Echo(Protocol):
    def __init__(self):
        self.connected = False

    def connectionMade(self):
        self.connected = True

    def connectionLost(self, reason):
        self.connected = False

    def dataReceived(self, data):
        print data.decode("utf-8")


class EchoClientFactory(ClientFactory):
    def __init__(self):
        self.protocol = None

    def startedConnecting(self, connector):
        print "Start to Connect..."

    def buildProtocol(self, addr):
        print "Connected..."
        self.protocol = Echo()
        return self.protocol

    def clientConnectionLost(self, connector, reason):
        print "Lost connection. Reason: ", reason

    def clientConnectionFailed(self, connector, reason):
        print "Connection is failed, Reason: ", reason


bStop = False


def routine(factory):
    while not bStop:
        if factory.protocol and factory.protocol.connected:
            factory.protocol.transport.write("hello, I'm %s %s" % (
                sys.argv[0], datetime.datetime.now()
            ))
            print sys.argv[0], datetime.datetime.now()
        time.sleep(5)


host = '127.0.0.1'
port = 8007
factory = EchoClientFactory()
reactor.connectTCP(host, port, factory)
threading.Thread(target=routine, args=(factory,)).start()
reactor.run()
bStop = True

一开始我们建立TCP连接, 传入主机地址, 端口, 协议工厂对象作为参数，随后reactor.run挂起运行。
下面我们看看ClientFactory基类，因为我们自定义的协议工厂EchoClientFactory派生自它。源码:

class ClientFactory(Factory):
    """A Protocol factory for clients.

    This can be used together with the various connectXXX methods in
    reactors.
    """

    def startedConnecting(self, connector):
        """Called when a connection has been started.

        You can call connector.stopConnecting() to stop the connection attempt.

        @param connector: a Connector object.
        """

    def clientConnectionFailed(self, connector, reason):
        """Called when a connection has failed to connect.

        It may be useful to call connector.connect() - this will reconnect.

        @type reason: L{twisted.python.failure.Failure}
        """

    def clientConnectionLost(self, connector, reason):
        """Called when an established connection is lost.

        It may be useful to call connector.connect() - this will reconnect.

        @type reason: L{twisted.python.failure.Failure}
        """

同样的，我们自定义的EchoClientFactory不过就是实现了基类中没有实现的函数，其中最重要的还是buildProtocol, 它为我们生成一个协议，下面看下我们自定义的协议类Echo, 基类源代码与上面的是一样的.
客户端的协议函数与服务器端的协议函数是一样的，在这里就不多说。

客户端的twisted模块讲完了，随后我们创建了一个线程去和服务器端通信, 并且定时发送, 当然，在这里我们为了以防万一，需要判断是否已经与服务器取得了连接，随后才发送消息.

大概讲了下基础部分，所有的代码都是来自《python高效开发实战》里的代码，在这里也向大家推荐这本书，学习twisted还有两个不错的教程，在最后我会发百度网盘共享。之所以写这篇基础的，就是为了能够理解高效开发实战里的最后一个项目: 用Twisted开发跨平台物联网消息网关。因为第一次实习就接触到了物联网通信，在工作时，滚了一遍项目的源代码(java写的，但我毕竟也是学了C#, .net两年的人, 看懂项目源码没压力, mvc orm都是与.net中的EF, MVC差不多, 不过就是语法有点不同)，正好和书上的这个项目差不多，书上将服务器与客户端的通信协议指令都讲得很清楚。因此这是一本不容错过的好书, 也是学习, 精通twisted的最好途径

最后就是运行测试:

服务器端:

客户端：

twisted教程: http://pan.baidu.com/s/1dEBPGhN