Java NIO

Java NIO 是Java新的IO类库(相对于旧IO来说),它的目的是提高速度.虽然旧IO已经使用NIO重新实现过,但是显示使用NIO对于文件IO和网络IO的速度还是有很大提升.

NIO的体系结构比较简单,主要围绕的是FileChannel和ByteBuffer来使用

• FileChannel相当于IO读取的内容数据源,可以通过InputStream,OutputStream和RandomAccessFile获得
• ByteBuffer则是存储当前读取出来的数据或者等待写入数据源FileChannel的数据的媒介,FileChannel数据的读取和写入必须通过ByteBuffer来实现

1. 简单示例

public class SimpeTest {
    private static final String FILE_PATH = "e:\in.test";
    private static final int BSIZE = 1024;

    public static void main(String[] args) throws IOException {
        // OutputStream
        FileChannel fc = new FileOutputStream(FILE_PATH).getChannel();
        fc.write(ByteBuffer.wrap("Some data ".getBytes()));
        fc.close();

        // RandomAccessFile
        fc = new RandomAccessFile(FILE_PATH, "rw").getChannel();
        fc.position(fc.size()); // 移动到文件尾
        fc.write(ByteBuffer.wrap("some more data".getBytes()));
        fc.close();

        // InputStream
        ByteBuffer bb = ByteBuffer.allocate(BSIZE);
        System.out.println(bb);
        fc = new FileInputStream(FILE_PATH).getChannel();
        fc.read(bb);
        bb.flip(); // 将limit设为position并将position设为0
        while (bb.hasRemaining()) {
            System.out.print((char) bb.get());
        }
        fc.close();
    }
}

要理解ByteBuffer的用法,理解清楚ByteBuffer的内部结构很重要,下面是示意图

其中

mark: 标记值,使用mark()函数可以标记当前position,在使用reset()后会将position重置为mark值,默认为-1,即没有mark值

position: 当前位置

limit: 限制值

capacity: buffer总容量

常用方法:

mark(): 将mark设为position

get(): 读取position位置数据,使用后position后移

put(): 在position位置写入数据,使用后position后移

remaining(): 获取position和limit之间的元素个数

flip(): 将limit设为position,并将position设为0,mark置为-1,一般来讲在准备读取buffer数据前会调用此方法

rewind(): 将position设为0,并丢弃标志位

reset(): 将position设为mark

clear(): 将position设为 0,并将limit设为capacity,不要误会这个方法的作用,它并没有清空buffer内的数据,一般来讲在写入buffer数据前会调用此方法

 2. 对FileChannel做数据转移

public class TransferTest {
    private static final int BSIZE = 1024;
    private static final String IN_FILE_PATH = "e:\in.test";
    private static final String OUT_FILE_PATH = "e:\out.test";

    public static void main(String[] args) throws IOException {
        FileChannel in = new FileInputStream(IN_FILE_PATH).getChannel();
        FileChannel out = new FileOutputStream(OUT_FILE_PATH).getChannel();
        in.transferTo(0, in.size(), out);
        // or
        // out.transferFrom(0, in.size(), in);
        in.close();
        out.close();
    }
}

对文件的数据转移的最佳方式是使用一个通道直接与另一个通道直接相连,调用FileChannel.transferTo()或FileChannel.transferFrom()方法直接解决问题

3. 对文件编码

public class EncodeTest {

    private static final int BSIZE = 1024;
    private static final String OUT_FILE_PATH = "e:\out.test";

    public static void main(String[] args) throws IOException {
        FileChannel fc = new FileOutputStream(OUT_FILE_PATH).getChannel();
        fc.write(ByteBuffer.wrap("test data".getBytes()));
        fc.close();

        ByteBuffer bb = ByteBuffer.allocate(BSIZE);
        fc = new FileInputStream(OUT_FILE_PATH).getChannel();
        fc.read(bb);
        fc.close();

        // 编码有问题情况:写入的时候使用的是UTF-8,而ByteBuffer.asCharBuffer()的解码编码是UTF16-BE
        bb.flip();
        System.out.println("Bad encoding : " + bb.asCharBuffer());

        // 使用指定编码UTF-8 decode
        String encoding = System.getProperty("file.encoding");
        System.out.println("Using charset '" + encoding + "' : " + Charset.forName(encoding).decode(bb));

        // 或者使用指定编码UTF-16写入文件
        fc = new FileOutputStream(OUT_FILE_PATH).getChannel();
        fc.write(ByteBuffer.wrap("test data".getBytes("UTF-16BE")));
        fc.close();

        bb.clear();
        fc = new FileInputStream(OUT_FILE_PATH).getChannel();
        fc.read(bb);
        fc.close();

        bb.flip();
        System.out.println("Using charset 'UTF-16BE' to write file : " + bb.asCharBuffer());

        // 或者选择直接使用CharBuffer写入
        bb.clear();
        bb.asCharBuffer().put("test data 2");
        fc = new FileOutputStream(OUT_FILE_PATH).getChannel();
        fc.write(bb);
        fc.close();

        fc = new FileInputStream(OUT_FILE_PATH).getChannel();
        bb.clear();
        fc.read(bb);
        bb.flip();
        System.out.println("Use CharBuffer to write file : " + bb.asCharBuffer());
    }
}

输出:

Bad encoding : 瑥獴⁤慴
Using charset 'UTF-8' : test data
Using charset 'UTF-16BE' to write file : test data
Use CharBuffer to write file : test data 2

ByteBuffer可以在写入或读取的时候指定编码,默认的编码式Buffer编码是"UTF-16BE"(Big Endian) 

3. View Buffer

使用ViewBuffer可以实现多种不同的基本数据类型Buffer的读取与写入,例子

public class ViewBufferTest {

    public static void main(String[] args) {
        ByteBuffer bb = ByteBuffer.wrap(new byte[] { 0, 0, 0, 0, 0, 0, 0, 'a' });

        // ByteBuffer
        bb.rewind();
        System.out.print("ByteBuffer: ");
        while (bb.hasRemaining()) {
            System.out.print(bb.position() + " -> " + bb.get() + ", ");
        }
        System.out.println();

        // CharBuffer
        bb.rewind();
        CharBuffer cb = bb.asCharBuffer();
        System.out.print("CharBuffer: ");
        while (cb.hasRemaining()) {
            System.out.print(cb.position() + " -> " + cb.get() + ", ");
        }
        System.out.println();

        // ShortBuffer
        bb.rewind();
        ShortBuffer sb = bb.asShortBuffer();
        System.out.print("ShortBuffer: ");
        while (sb.hasRemaining()) {
            System.out.print(sb.position() + " -> " + sb.get() + ", ");
        }
        System.out.println();

        // IntBuffer
        bb.rewind();
        IntBuffer ib = bb.asIntBuffer();
        System.out.print("IntBuffer: ");
        while (ib.hasRemaining()) {
            System.out.print(ib.position() + " -> " + ib.get() + ", ");
        }
        System.out.println();

        // LongBuffer
        bb.rewind();
        LongBuffer lb = bb.asLongBuffer();
        System.out.print("LongBuffer: ");
        while (lb.hasRemaining()) {
            System.out.print(lb.position() + " -> " + lb.get() + ", ");
        }
        System.out.println();

        // FloatBuffer
        bb.rewind();
        FloatBuffer fb = bb.asFloatBuffer();
        System.out.print("FloatBuffer: ");
        while (fb.hasRemaining()) {
            System.out.print(fb.position() + " -> " + fb.get() + ", ");
        }
        System.out.println();

        // DoubleBuffer
        bb.rewind();
        DoubleBuffer db = bb.asDoubleBuffer();
        System.out.print("DoubleBuffer: ");
        while (db.hasRemaining()) {
            System.out.println(db.position() + " -> " + db.get() + ", ");
        }
        System.out.println();
    }
}

输出

可以看出不同的ViewBuffer每次读取数据的长度不一样,其读取原理如下

4. Big Endian 与 Little Endian

ByteBuffer可以使用Big Endian的存储方式,也可以使用Little Endian的存储方式,默认是Big Endian

public class EndiansTest {

    public static void main(String[] args) {
        ByteBuffer bb = ByteBuffer.wrap(new byte[12]);
        bb.asCharBuffer().put("abcdef");
        System.out.println(Arrays.toString(bb.array()));

        // Big Endian
        bb.rewind();
        bb.order(ByteOrder.BIG_ENDIAN);
        bb.asCharBuffer().put("abcdef");
        System.out.println(Arrays.toString(bb.array()));

        // Little Endian
        bb.rewind();
        bb.order(ByteOrder.LITTLE_ENDIAN);
        bb.asCharBuffer().put("abcdef");
        System.out.println(Arrays.toString(bb.array()));
    }
}

输出:

[0, 97, 0, 98, 0, 99, 0, 100, 0, 101, 0, 102]
[0, 97, 0, 98, 0, 99, 0, 100, 0, 101, 0, 102]
[97, 0, 98, 0, 99, 0, 100, 0, 101, 0, 102, 0]

由于Char在Java中占用两个字节,所以使用Big Endian和Little Endian的区别就是两个字节对调.

5. 更为精细的IO操作

下面例子为使用ByteBuffer的操作方法实现字符的两两调换的操作

public class UsingBuffers {
    private static void symmetricsScamble(CharBuffer buffer) {
        while (buffer.hasRemaining()) {
            buffer.mark();
            char c1 = buffer.get();
            char c2 = buffer.get();
            buffer.reset();
            buffer.put(c2).put(c1);
        }
    }

    public static void main(String[] args) throws UnsupportedEncodingException {
        // 此处也可以使用CharBuffer来写入数据,从而避免乱码问题
        byte[] data = "UsingBuffers".getBytes("UTF-16BE");
        ByteBuffer bb = ByteBuffer.allocate(data.length * 2);
        bb.put(data);

        bb.rewind();
        System.out.println(bb.asCharBuffer());

        bb.rewind();
        symmetricsScamble(bb.asCharBuffer());
        System.out.println(bb.asCharBuffer());

        bb.rewind();
        symmetricsScamble(bb.asCharBuffer());
        System.out.println(bb.asCharBuffer());
    }
}

输出

UsingBuffers

sUniBgfuefsr

UsingBuffers

6. 内存映射文件

内存映射文件允许你读入整个太大而不能读入内存的的大文件,并假定整个文件都已经在内存中,将其当做一个大数组来操作,这种方式简化了文件修改的代码.

他可以将整个或部分文件映射到虚拟内存, 用这种方式我们读取到整个文件的内容, 从而减少磁盘 IO, 提升读取性能

下面是一个读取文件的性能对比

public class MappedIOTest {
    private static final String TEST_FILE = "e:\test.file";
    private static int numOfInts = 4000000;
    private static int numOfUbuffInts = 200000;

    private abstract static class Tester {
        private String name;

        public Tester(String name) {
            this.name = name;
        }

        // 效率测试模板方法
        public void runTest() {
            System.out.println(name + " : ");
            try {
                long start = System.nanoTime();
                test();
                double duration = System.nanoTime() - start;
                System.out.format("%.2f
", duration / 1.0e9);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        }

        public abstract void test() throws IOException;
    }

    private static Tester[] testers = { new Tester("Stream Write") {

        @Override
        public void test() throws IOException {
            DataOutputStream out = new DataOutputStream(new BufferedOutputStream(new FileOutputStream(TEST_FILE)));
            int i = 0;
            while (i++ < numOfInts) {
                out.writeInt(i);
            }
            out.close();
        }
    }, new Tester("Mapped Write") {

        @Override
        public void test() throws IOException {
            FileChannel fc = new RandomAccessFile(TEST_FILE, "rw").getChannel();
            IntBuffer ib = fc.map(FileChannel.MapMode.READ_WRITE, 0, fc.size()).asIntBuffer();
            int i = 0;
            while (i++ < numOfInts) {
                ib.put(i);
            }
            fc.close();
        }
    }, new Tester("Stream Read") {

        @Override
        public void test() throws IOException {
            DataInputStream in = new DataInputStream(new BufferedInputStream(new FileInputStream(TEST_FILE)));
            int i = 0;
            while (i++ < numOfInts) {
                in.readInt();
            }
            in.close();
        }
    }, new Tester("Mapped Read") {

        @Override
        public void test() throws IOException {
            FileChannel fc = new FileInputStream(TEST_FILE).getChannel();
            IntBuffer ib = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size()).asIntBuffer();
            ib.rewind();
            while (ib.hasRemaining()) {
                ib.get();
            }
            fc.close();
        }
    }, new Tester("Stream Read/Write") {

        @Override
        public void test() throws IOException {
            RandomAccessFile raf = new RandomAccessFile(TEST_FILE, "rw");
            raf.writeInt(1);
            int i = 0;
            while (i++ < numOfUbuffInts) {
                raf.seek(raf.length() - 4);
                raf.writeInt(raf.readInt());
            }
            raf.close();
        }
    }, new Tester("Mapped Read/Write") {

        @Override
        public void test() throws IOException {
            FileChannel fc = new RandomAccessFile(TEST_FILE, "rw").getChannel();
            IntBuffer ib = fc.map(FileChannel.MapMode.READ_WRITE, 0, fc.size()).asIntBuffer();
            ib.put(0);
            int i = 0;
            while (i++ < numOfUbuffInts) {
                ib.put(ib.get(i - 1));
            }
            fc.close();
        }
    } };

    public static void main(String[] args) throws IOException {
        for (Tester tester : testers) {
            tester.runTest();
        }
    }
}

输出:

Stream Write :
0.45
Mapped Write :
0.03
Stream Read :
0.43
Mapped Read :
0.03
Stream Read/Write :
8.08
Mapped Read/Write :
0.00

7. 文件锁

NIO引入了文件锁,文件锁跟线程锁有个很大的区别,对文件的操作有可能是在不同的JVM的两个线程,甚至有可能是一个是Java线程,另一个是操作系统的其他本地线程.而文件锁对操作系统进程是可见的,因为java的文件锁直接映射到本地操作系统的加锁工具,所以它的作用于是整个系统的,而非在JVM内.

FileChannel有tryLock()和lock()方法,他们分别对应非阻塞锁和阻塞锁

tryLock()在无法获得锁时不会被阻塞,而是直接往下执行代码

lock()则会在无法获得锁时进入等待阶段

此外,Java的FileChannel文件锁还支持对文件的部分加锁,就像Mapped文件映射一样,下面是一个例子

public class FileLockTest {
    private static final String LOCK_FILE = "e:\lock.file";
    private static final int LENGTH = 0x8FFFFFF; // 128M
    private static FileChannel fc;

    public static void main(String[] args) throws IOException {
        fc = new RandomAccessFile(LOCK_FILE, "rw").getChannel();
        MappedByteBuffer mbb = fc.map(FileChannel.MapMode.READ_WRITE, 0, LENGTH);
        for (int i = 0; i < LENGTH; i++) {
            mbb.put((byte) 'x');
        }
        new LockAndModify(mbb, 0, LENGTH / 3);
        new LockAndModify(mbb, LENGTH / 2, LENGTH / 2 + LENGTH / 4);
    }

    private static class LockAndModify extends Thread {
        private ByteBuffer buffer;
        private int start, end;

        public LockAndModify(ByteBuffer bb, int start, int end) {
            this.start = start;
            this.end = end;
            bb.limit(end);
            bb.position(start); // position必须要比limit小,所以position()必须在limit()之后
            buffer = bb.slice();
            start();
        }

        @Override
        public void run() {
            try {
                // 非共享锁
                FileLock fl = fc.lock(start, end, false);
                System.out.println("Locked: " + start + " to " + end);
                // 修改锁内文件数据
                while (buffer.position() < buffer.limit() - 1) {
                    buffer.put((byte) (buffer.get() + 1));
                }
                System.out.println("Released: " + start + " to " + end);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        }
    }
}

总的来说,Java NIO的优势有:

1. 效率高

2. 文件写入读取的编码控制

3. 文件映射

4. 文件锁

5. ByteBuffer的灵活性

6. 基础数据类型Buffer的支持