zoukankan      html  css  js  c++  java
  • Hadoop RPC protocol description--转

    原文地址:https://spotify.github.io/snakebite/hadoop_rpc.html

    Snakebite currently implements the following protocol in snakebite.channel.SocketRpcChannel to communicate with the NameNode.

    Connection

    The Hadoop RPC protocol works as described below. On connection, headers are sent to setup a session. After that, multiple requests can be sent within the session.

    FunctionTypeDefault
    Header bytes “hrpc”
    Version uint8 7
    Auth method uint8 80 (Auth method SIMPLE)
    Serialization type uint8 0 (protobuf)
    IpcConnectionContextProto length uint32  
    IpcConnectionContextProto bytes  

    Sending messages

    When sending a message, the following is sent to the sever:

    FunctionType
    Length of the next two parts uint32
    RpcPayloadHeaderProto length varint
    RpcPayloadHeaderProto protobuf serialized message
    HadoopRpcRequestProto length varint
    HadoopRpcRequestProto protobuf serialized message

    varint is a Protocol Buffer variable int.

    Note

    The Java protobuf implementation uses writeToDelimited to prepend the message with their lenght, but the python implementation doesn’t implement such a method (yet).

    Next to an rpcKind (snakebites default is RPC_PROTOCOL_BUFFER), an rpcOp (snakebites default isRPC_FINAL_PAYLOAD), the RpcPayloadHeaderProto message defines a callId that is added in the RPC response (described below).

    The HadoopRpcRequestProto contains a methodName field that defines what server method is called and a has a property request that contains the serialized actual request message.

    Receiving messages

    After a message is sent, the response can be read in the following way:

    FunctionType
    Length of the RpcResponseHeaderProto varint
    RpcResponseHeaderProto bytes
    Length of the RPC response uint32
    Serialized RPC response bytes

    The RpcResponseHeaderProto contains the callId of the request and a status field. The status can beSUCCESSERROR or FAILURE. In case SUCCESS the rest of response is a complete protobuf response.

    In case of ERROR, the response looks like follows:

    FunctionType
    Length of the RpcResponseHeaderProto varint
    RpcResponseHeaderProto bytes
    Length of the RPC response uint32
    Length of the Exeption class name uint32
    Exception class name utf-8 string
    Length of the stack trace uint32
    Stack trace utf-8 string
  • 相关阅读:
    在Android中,使用Kotlin的 API请求简易方法
    Android开发者的Kotlin:书
    用Kotlin开发Android应用(IV):定制视图和Android扩展
    用Kotlin开发Android应用(III):扩展函数和默认值
    zookeeper应用
    BigDecimal的setScale()方法无效(坑)
    Linux命令详解之—less命令
    jdk10 var定义变量的由来
    Mysql DataPacketTooBigException异常处理
    JDK自带的监控工具方法
  • 原文地址:https://www.cnblogs.com/davidwang456/p/4773884.html
Copyright © 2011-2022 走看看