zoukankan      html  css  js  c++  java
  • 新的契机,因为本人en文不好,所以只能从片言只语来理解,不过有利于思考what is key value )答案在最后!

    Despite our many discussions regarding keys and values, we have yet to mention their
    types. The MapReduce framework won’t allow them to be any arbitrary class. For example,
    although we can and often do talk about certain keys and values as integers,
    strings, and so on, they aren’t exactly standard Java classes, such as Integer, String,
    and so forth. This is because the MapReduce framework has a certain defined way of
    serializing the key/value pairs to move them across the cluster’s network, and only
    classes that support this kind of serialization can function as keys or values in the
    framework.

    尽管我们讨论许多关于keys and values,我们还没有提到它们的类型.这个mapreduce框架不允许它们随意的定义成任何类型.例如,尽管我们常常讨论关于无疑是keys and values 是整型,或者是字符串类型,等等,他们不是明确确标准的java 类,例如integer,string,等等.这是因为mapreduce框架有一种定义序列化key/value pairs 在集群网络中传输,并且只这些类支持这种类型的序列能在做为keys or values 在这个框架起作用.


    More specifically, classes that implement the Writable interface can be values, and
    classes that implement the WritableComparable<T> interface can be either keys
    or values. Note that the WritableComparable<T> interface is a combination of the
    Writable and java.lang.Comparable<T> interfaces . We need the comparability
    requirement for keys because they will be sorted at the reduce stage, whereas values
    are simply passed through.


    Hadoop comes with a number of predefined classes that implement
    WritableComparable, including wrapper classes for all the basic data types, as seen
    in table 3.1.
    Table 3.1 List of frequently used types for the key/value pairs . These classes all implement the
    WritableComparable interface.

    列举常用的key/value pairs类型.

    Class                    Description


    BooleanWritable Wrapper for a standard Boolean variable
    ByteWritable Wrapper for a single byte
    DoubleWritable Wrapper for a Double
    FloatWritable Wrapper for a Float
    IntWritable Wrapper for a Integer
    LongWritable Wrapper for a Long
    Text Wrapper to store text using the UTF8 format
    NullWritable Placeholder when the key or value is not needed


    Keys and values can take on types beyond the basic ones which Hadoop natively supports.
    You can create your own custom type as long as it implements the Writable
    (or WritableComparable<T>) interface. For example, listing 3.2 shows a class
    that can represent edges in a network. This may represent a flight route between
    two cities.

    所以最后的答案就是,key/value type是一种用于有mapreduce framework cluster进行传输的格式.

    包括你在map 在reduce 进口的通讯交互格式,这机话,在不同slave里面执行时最后产生的一个格式返回汇总.

    有意思,细节有待于我进一步的阅读这不好懂的英文.

    这种格式

  • 相关阅读:
    几种编辑器的markdown-toc生成目录在github上的表现
    Docker 镜像文件的导入和导出
    Python 的 os 与 sys 模块
    IPython 自动重载魔术
    一种新的python局部调试手法
    Python 动态从文件中导入类或函数的方法
    python2中新式类和经典类的多重继承调用顺序
    Python运行目录或压缩文件
    子类化内置类型
    Django 数据库访问性能优化
  • 原文地址:https://www.cnblogs.com/chenli0513/p/2290862.html
Copyright © 2011-2022 走看看