zoukankan      html  css  js  c++  java
  • 跟着Rocketmq学习堆外内存

    刚好看rocketmq关于存储的源码,

      org.apache.rocketmq.store.MappedFile

      MappedFile#init

    private void init(final String fileName, final int fileSize) throws IOException {
            this.fileName = fileName;
            this.fileSize = fileSize;
            this.file = new File(fileName);
            this.fileFromOffset = Long.parseLong(this.file.getName());
            boolean ok = false;
    
            ensureDirOK(this.file.getParent());
    
            try {
                this.fileChannel = new RandomAccessFile(this.file, "rw").getChannel();
                this.mappedByteBuffer = this.fileChannel.map(MapMode.READ_WRITE, 0, fileSize);
                TOTAL_MAPPED_VIRTUAL_MEMORY.addAndGet(fileSize);
                TOTAL_MAPPED_FILES.incrementAndGet();

      大名鼎鼎的rocketmq获取堆外内存也是用这么朴素的方式

      appendMessagesInner

    public AppendMessageResult appendMessagesInner(final MessageExt messageExt, final AppendMessageCallback cb) {
            assert messageExt != null;
            assert cb != null;
    
            int currentPos = this.wrotePosition.get();
    
            if (currentPos < this.fileSize) {
                ByteBuffer byteBuffer = writeBuffer != null ? writeBuffer.slice() : this.mappedByteBuffer.slice();
                byteBuffer.position(currentPos);

      重点就是看看这个slice,先看jdk中的注释

    Creates a new byte buffer whose content is a shared subsequence of this buffer's content.
    The content of the new buffer will start at this buffer's current position. Changes to this buffer's content will be visible in the new buffer, and vice versa; the two buffers' position, limit, and mark values will be independent.
    The new buffer's position will be zero, its capacity and its limit will be the number of bytes remaining in this buffer, and its mark will be undefined. The new buffer will be direct if, and only if, this buffer is direct, and it will be read-only if, and only if, this buffer is read-only.
    Returns:
    The new byte buffer

      翻译过来就是两块内存,内容一样,一个变化另一个也能看到,同时新的buffer的起始地址是旧的buffer的position。新buffer的容量是旧buffer的剩余容量。

           下面就通过代码试一下

      

    private static String path = "d://output.txt";
    
        public static void main(String[] args) {
            int capacity = 5;
    
            try {
    
                FileChannel fileChannel = new RandomAccessFile(path, "rw").getChannel();
    
                ByteBuffer bb1 = fileChannel.map(FileChannel.MapMode.READ_WRITE, 0, 5);
    
                bb1.put("a".getBytes());
                bb1.put("b".getBytes());
    
                ByteBuffer bb2 = bb1.slice();
                bb2.put("c".getBytes());
    
            } catch (IllegalArgumentException | FileNotFoundException e) {
    
                System.out.println("IllegalArgumentException catched");
            }
            catch (ReadOnlyBufferException | IOException e) {
    
                System.out.println("ReadOnlyBufferException catched");
            }
        }

      

     结果正好说明三件事

    1 通过slice出来的buffer确实能像旧buffer一样,映射到相同的物理文件

    2 新buffer的起始位置正好是2,所以c在b后面,而不是覆盖a

    3 直接内存,不需要手动刷盘

    总结

      本篇就是说明下slice的作用

  • 相关阅读:
    CSP2020 T1儒略日 暴力模拟90pts代码
    CSP-S 2019 D1T2括号树
    P3593 [POI2015]TAB
    P5145 漂浮的鸭子
    CH0503 奇数码问题
    [NOIP2012]国王游戏 -高精度-贪心-
    费解的开关
    P1040 加分二叉树
    初步学习线段树
    P2758 编辑距离 简单DP
  • 原文地址:https://www.cnblogs.com/juniorMa/p/14964405.html
Copyright © 2011-2022 走看看