使用juicefs 让s3 更好兼容posix协议

juicefs 是一个很不错的云原生高性能共享文件系统，以下是s3的测试

环境准备

• docker-compose

version: "3"

services:

  s3:

    image: minio/minio

    environment:

      - "MINIO_ACCESS_KEY=minio"

      - "MINIO_SECRET_KEY=minio123"

    command: server /data --console-address ":9001"

    ports:

      - "9000:9000"

      - "9001:9001"

• juicefs mac 客户端

可以参考官方资料
https://www.cnblogs.com/rongfengliang/p/15734513.html

使用

测试多客户端挂载文件通知

• format

 

juicefs format \

    --storage minio \

    --bucket http://localhost:9000/appdemo  \

    --bucket http://localhost:9000/appdemo2  \

    --bucket http://localhost:9000/appdemo4  \

    --access-key minio \

    --secret-key minio123 \

    sqlite3://myjfs.db \

    miniofs

• mount

juicefs mount sqlite3://myjfs.db miniofs

juicefs mount sqlite3://myjfs.db miniofs2

• nodejs watch 文件
  基于chokidar

 

const chokidar = require('chokidar');

const watcher = chokidar.watch(["/Users/dalong/mylearning/juicefs-rongfl/miniofs","/Users/dalong/mylearning/juicefs-rongfl/miniofs2"], {

    ignored: /(^|[\/\\\\])\../, // ignore dotfiles

    persistent: true

  });

 

  // Something to use when events are received.

  const log = console.log.bind(console);

 

  // More possible events.

  watcher

    .on('addDir', path => log(`Directory ${path} has been added`))

    .on('unlinkDir', path => log(`Directory ${path} has been removed`))

    .on('error', error => log(`Watcher error: ${error}`))

    .on('ready', () => log('Initial scan complete. Ready for changes'))

    .on('raw', (event, path, details) => { // internal

      log('Raw event info:', event, path, details);

    });

• 效果

说明

以上是基于juicefs 可以很明显的提升s3 挂载的性能，而且可以更好的兼容posix（尤其是进行多端挂载进行文件处理的时候），大家如果在测试
s3fs-fuse 的时候是不能达到多端挂载实时数据变动通知的（可以参考Limitations章节）

参考资料

https://www.npmjs.com/package/chokidar
https://juicefs.com/docs/zh/community/getting-started/for_distributed
https://github.com/s3fs-fuse/s3fs-fuse