zoukankan      html  css  js  c++  java
  • libhdfs的配置和使用

    测试环境:centos6.10,hadoop2.7.3,jdk1.8

    测试代码:HDFSCSample.c

    #include "hdfs.h"
    #include <string.h>
    #include <stdio.h>
    #include <stdlib.h>
    
    int main(int argc, char **argv) {
    
        hdfsFS fs = hdfsConnect("default", 0);
        const char* writePath = "/tmp/testfile.txt";
        hdfsFile writeFile = hdfsOpenFile(fs, writePath, O_WRONLY |O_CREAT, 0, 0, 0);
        if(!writeFile) {
              fprintf(stderr, "Failed to open %s for writing!
    ", writePath);
              exit(-1);
        }
        char* buffer = "Hello, World!";
        tSize num_written_bytes = hdfsWrite(fs, writeFile, (void*)buffer, strlen(buffer)+1);
        if (hdfsFlush(fs, writeFile)) {
               fprintf(stderr, "Failed to 'flush' %s
    ", writePath);
              exit(-1);
        }
        hdfsCloseFile(fs, writeFile);
    }

    编译脚本:

      compile.sh

    #!/bin/bash
    export JAVA_HOME=/root/softs/jdk1.8.0_172
    export HADOOP_HOME=/root/softs/hadoop-2.7.3/
    
    gcc -I$HADOOP_HOME/include -L$HADOOP_HOME/lib/native -lhdfs -L$JAVA_HOME/jre/lib/amd64/server -ljvm HDFSCSample.c

    执行命令: 

      # chmod +x compile.sh

      # ./compile.sh

    执行脚本:

      execute.sh

    #!/bin/bash
    export JAVA_HOME=/root/softs/jdk1.8.0_172
    export HADOOP_HOME=/root/softs/hadoop-2.7.3/
    export LD_LIBRARY_PATH=$HADOOP_HOME/lib/native:$JAVA_HOME/jre/lib/amd64/server/
    
    CLASSPATH=./
    for f in $HADOOP_HOME/share/hadoop/common/*.jar; do
      CLASSPATH=${CLASSPATH}:$f;
    done
    for f in $HADOOP_HOME/share/hadoop/common/lib/*.jar; do
      CLASSPATH=${CLASSPATH}:$f;
    done
    for f in $HADOOP_HOME/share/hadoop/hdfs/*.jar; do
      CLASSPATH=${CLASSPATH}:$f;
    done
    for f in $HADOOP_HOME/share/hadoop/hdfs/lib/*.jar; do
      CLASSPATH=${CLASSPATH}:$f;
    done
    export CLASSPATH=$CLASSPATH
    
    ./a.out

    执行命令:

      # chmod +x execute.sh

      # ./execute.sh

    打开/tmp/testFile.txt,可以看到里面写入了一条 Hello, World!

      

  • 相关阅读:
    Oracle- 表的自增长创建
    C#- 写Windows服务
    基于redis分布式缓存实现(新浪微博案例)
    分布式集群系统下的高可用session解决方案
    Hibernate 缓存介绍
    MongoDB 安装(Window/Linux)
    MongoDB 优点
    MongoDB 介绍
    浅析数据一致性
    mysql常用函数汇总
  • 原文地址:https://www.cnblogs.com/cqdxwjd/p/9835420.html
Copyright © 2011-2022 走看看