zoukankan      html  css  js  c++  java
  • HDFS 断点续传,写文件功能

    实际上这是个 HDFS 的工具类部分代码。 首先

    public static Configuration configuration = null;
    public static FileSystem fileSystem = null;

    static {
    try {
    if (null == configuration) {
    configuration = new Configuration();
    }
    if (null == fileSystem) {
    fileSystem = FileSystem.get(URI.create(RockyConstants.HDFS_PATH), configuration);
    }
    }
    catch (IOException e) {
    e.printStackTrace()
    ;
    }
    }

    /**
    * 整文件存入 HDFS
    *
    * @throws Exception
    */
    public static boolean putHDFS(String filePath, byte[] info) {
    try {
    FSDataOutputStream writer = fileSystem.create(
    new Path(filePath), true);
    writer.write(info);
    writer.flush();
    writer.close();
    } catch (IOException e) {
    e.printStackTrace();
    return false;
    }
    return true;
    }

    /**
    * 断点续传存入
    * @throws IOException
    */
    public static void continueUpload(String targetPath, byte[] info) throws IOException{
    Path fsPath = new Path(targetPath);
    // 第一次
    if (!fileSystem.exists(fsPath)) {
    putHDFS(targetPath,info);
    } else {
    // 续传
    FSDataOutputStream writer = fileSystem.append(fsPath);
    writer.write(info);
    writer.flush();
    writer.close();
    }
    }






    God has given me a gift. Only one. I am the most complete fighter in the world. My whole life, I have trained. I must prove I am worthy of someting. rocky_24
  • 相关阅读:
    寒假日报day10
    寒假日报day9
    周计划06(20201026-20201101)
    周计划05(20201019-20201025)
    周总结2
    编程语言的实现模式读后感1
    软工总结
    哈夫曼编码算法
    hive表查询——排序
    假期总结4
  • 原文地址:https://www.cnblogs.com/rocky24/p/5032385.html
Copyright © 2011-2022 走看看