zoukankan      html  css  js  c++  java
  • flink1.10 taskmanager内存分配

    taskmanager.memory.process.size: 4096m

    -Xmx1748185876(1667M,flink web界面显示的java heap size) -Xms1748185876 -XX:MaxDirectMemorySize=510698460(487M) -XX:MaxMetaspaceSize=100663296(96M)

    4096-JVM specific memory = total flink memory (M)(当配置taskmanager.memory.flink.size时,这里不需要减,配置的内存除JVM Metaspace和JVM Overhead之外全部作为flink  memory)

    4096m * 0.88 = 3590M(flink memory,可以拿taskmanager.memory.network.min/0.1算出来)
    taskmanager.memory.framework.off-heap.size=134217728b 128M
    taskmanager.memory.framework.heap.size=134217728b 128M
    taskmanager.memory.network.max=376480732b 359M
    taskmanager.memory.network.min=376480732b 359M 3590*0.1
    taskmanager.memory.managed.size=1505922928b 1436M 3590*0.4
    taskmanager.memory.task.heap.size=1613968148b 1539M (3590-taskmanager.memory.framework.off-heap.size-taskmanager.memory.framework.heap.size
    -taskmanager.memory.network.max-taskmanager.memory.managed.size)=1539
    taskmanager.memory.task.off-heap.size=0b

    Flink taskmanager JVM heap(T)=taskmanager.memory.task.heap.size+taskmanager.memory.framework.heap.size=1539M +128M=1667M

    通过以上可快速得出 total flink memory (M):M=(T+128)*2   #####(1667+128)*2=3590

    T-taskmanager.memory.framework.heap.size=taskmanager.memory.task.heap.size=M-taskmanager.memory.framework.off-heap.size-taskmanager.memory.framework.heap.size
    -taskmanager.memory.network.max-taskmanager.memory.managed.size

    T-128=M-128-128-M*0.1-M*0.4

    M=(T+128)*2

    另外,当配成taskmanager.memory.flink.size时,内存除了JVM Metaspace和JVM Overhead全作为 total flink memory (M),因此通过配置时的taskmanager.memory.flink.size值可以迅速计算出T=M/2-128    #4096/2-128=1920M

  • 相关阅读:
    Media change : please insert the disk labeled
    ubuntu 关闭和开启防火墙
    CentOS6.3上部署Ceph
    Keepalived_vrrp: ip address associated with VRID not present in received packet
    Python 错误和异常小结
    nova network-vif-plugged 事件分析1
    ansible 之条件语句 when
    ansible 判断和循环
    openvswitch dpdk
    ES6之Promise
  • 原文地址:https://www.cnblogs.com/lyy-blog/p/12470330.html
Copyright © 2011-2022 走看看