zoukankan      html  css  js  c++  java
  • 大文件读写独占磁盘引发的故障

     https://en.wikipedia.org/wiki/Mount_(computing)

    Mounting is a process by which the operating system makes files and directories on a storage device (such as hard driveCD-ROM, or network share) available for users to access via the computer's file system.[1]

    In general, the process of mounting comprises operating system acquiring access to the storage medium; recognizing, reading, processing file system structure and metadata on it; before registering them to the virtual file system (VFS) component.

    The location in VFS that the newly-mounted medium was registered is called mount point; when the mounting process is completed, the user can access files and directories on the medium from there.

    An opposite process of mounting is called unmounting, in which the operating system cuts off all user access to files and directories on the mount point, writes the remaining queue of user data to the storage device, refreshes file system metadata, then relinquishes access to the device; making the storage device safe for removal.

    Normally, when the computer is shutting down, every mounted storage will undergo an unmounting process to ensure that all queued data got written, and to preserve integrity of file system structure on the media.

    The 

    Anatomy of the Linux file system
    https://www.ibm.com/developerworks/linux/library/l-linux-filesystem/?S_TACT=105AGX52&S_CMP=cn-a-l

    Linux 文件系统剖析
    https://www.ibm.com/developerworks/cn/linux/l-linux-filesystem/

    A layered structure-based review

    When it comes to file systems, Linux® is the Swiss Army knife of operating systems. Linux supports a large number of file systems, from journaling to clustering to cryptographic. Linux is a wonderful platform for using standard and more exotic file systems and also for developing file systems. This article explores the virtual file system (VFS)—sometimes called the virtual filesystem switch—in the Linux kernel and then reviews some of the major structures that tie file systems together.

    按照分层结构讨论 Linux 文件系统

    在文件系统方面,Linux® 可以算得上操作系统中的 “瑞士军刀”。Linux 支持许多种文件系统,从日志型文件系统到集群文件系统和加密文件系统。对于使用标准的和比较奇特的文件系统以及开发文件系统来说,Linux 是极好的平台。本文讨论 Linux 内核中的虚拟文件系统(VFS,有时候称为虚拟文件系统交换器),然后介绍将文件系统连接在一起的主要结构。

    Basic file system architecture

    The Linux file system architecture is an interesting example of abstracting complexity. Using a common set of API functions, a large variety of file systems can be supported on a large variety of storage devices. Take, for example, the read function call, which allows some number of bytes to be read from a given file descriptor. The read function is unaware of file system types, such as ext3 or NFS. It is also unaware of the particular storage medium upon which the file system is mounted, such as AT Attachment Packet Interface (ATAPI) disk, Serial-Attached SCSI (SAS) disk, or Serial Advanced Technology Attachment (SATA) disk. Yet, when the read function is called for an open file, the data is returned as expected. This article explores how this is done and investigates the major structures of the Linux file system layer.

    基本的文件系统体系结构

    Linux 文件系统体系结构是一个对复杂系统进行抽象化的有趣例子。通过使用一组通用的 API 函数,Linux 可以在许多种存储设备上支持许多种文件系统。例如,read 函数调用可以从指定的文件描述符读取一定数量的字节。read 函数不了解文件系统的类型,比如 ext3 或 NFS。它也不了解文件系统所在的存储媒体,比如 AT Attachment Packet Interface(ATAPI)磁盘、Serial-Attached SCSI(SAS)磁盘或 Serial Advanced Technology Attachment(SATA)磁盘。但是,当通过调用 read 函数读取一个文件时,数据会正常返回。本文讲解这个机制的实现方法并介绍 Linux 文件系统层的主要结构。

    Basic file system architecture

    The Linux file system architecture is an interesting example of abstracting complexity. Using a common set of API functions, a large variety of file systems can be supported on a large variety of storage devices. Take, for example, the read function call, which allows some number of bytes to be read from a given file descriptor. The read function is unaware of file system types, such as ext3 or NFS. It is also unaware of the particular storage medium upon which the file system is mounted, such as AT Attachment Packet Interface (ATAPI) disk, Serial-Attached SCSI (SAS) disk, or Serial Advanced Technology Attachment (SATA) disk. Yet, when the read function is called for an open file, the data is returned as expected. This article explores how this is done and investigates the major structures of the Linux file system layer.

    What is a file system?

    I'll start with an answer to the most basic question, the definition of a file system. A file system is an organization of data and metadata on a storage device. With a vague definition like that, you know that the code required to support this will be interesting. As I mentioned, there are many types of file systems and media. With all of this variation, you can expect that the Linux file system interface is implemented as a layered architecture, separating the user interface layer from the file system implementation from the drivers that manipulate the storage devices.

    File systems as protocols

    Mounting

    Associating a file system to a storage device in Linux is a process called mounting. The mountcommand is used to attach a file system to the current file system hierarchy (root). During a mount, you provide a file system type, a file system, and a mount point.

    To illustrate the capabilities of the Linux file system layer (and the use of mount), create a file system in a file within the current file system. This is accomplished first by creating a file of a given size using dd (copy a file using /dev/zero as the source) -- in other words, a file initialized with zeros, as shown in Listing 1.

    Listing 1. Creating an initialized file

    什么是文件系统?

    首先回答最常见的问题,“什么是文件系统”。文件系统是对一个存储设备上的数据和元数据进行组织的机制。由于定义如此宽泛,支持它的代码会很有意思。正如前面提到的,有许多种文件系统和媒体。由于存在这么多类型,可以预料到 Linux 文件系统接口实现为分层的体系结构,从而将用户接口层、文件系统实现和操作存储设备的驱动程序分隔开。

    挂装

    在 Linux 中将一个文件系统与一个存储设备关联起来的过程称为挂装(mount)。使用 mount 命令将一个文件系统附着到当前文件系统层次结构中(根)。在执行挂装时,要提供文件系统类型、文件系统和一个挂装点。

    为了说明 Linux 文件系统层的功能(以及挂装的方法),我们在当前文件系统的一个文件中创建一个文件系统。实现的方法是,首先用 dd 命令创建一个指定大小的文件(使用 /dev/zero 作为源进行文件复制)—— 换句话说,一个用零进行初始化的文件,见清单 1。

    dd if=/dev/zero of=file.img bs=1k count=10000
    well@well:~$ cd /
    well@well:/$ ls
    bin   dev  home  lib64  mnt  proc  run   srv  tmp  var
    boot  etc  lib   media  opt  root  sbin  sys  usr
    well@well:/$ cd home
    well@well:/home$ ll
    total 8
    drwxr-xr-x. 1 well well   73 Apr 18 04:10 ./
    drwxr-xr-x. 1 root root   92 Apr 18 04:09 ../
    drwxr-xr-x. 1 well well   18 Apr 11 21:51 bin/
    drwxr-xr-x. 1 well well   21 Apr 18 04:09 etc/
    drwxr-xr-x. 1 well well   95 Apr 19 20:38 project/
    -rwxrwxr-x. 1 well well  360 Mar 14 11:59 readme.php*
    drwxr-xr-x. 4 well well 4096 Apr 19 20:50 well/
    well@well:/home$ dd if=/dev/zero of=file.img bs=1k count=10000
    10000+0 records in
    10000+0 records out
    10240000 bytes (10 MB) copied, 0.0123252 s, 831 MB/s
    well@well:/home$ ll
    total 10008
    drwxr-xr-x. 1 well well       89 Apr 20 02:55 ./
    drwxr-xr-x. 1 root root       92 Apr 18 04:09 ../
    drwxr-xr-x. 1 well well       18 Apr 11 21:51 bin/
    drwxr-xr-x. 1 well well       21 Apr 18 04:09 etc/
    -rw-rw-r--. 1 well well 10240000 Apr 20 02:55 file.img
    drwxr-xr-x. 1 well well       95 Apr 19 20:38 project/
    -rwxrwxr-x. 1 well well      360 Mar 14 11:59 readme.php*
    drwxr-xr-x. 4 well well     4096 Apr 19 20:50 well/
    losetup /dev/loop0 file.img
    well@well:/dev$ sudo ls
    console  fd    mqueue  ptmx  random  stderr  stdout  urandom
    core     full  null    pts   shm     stdin   tty     zero
    well@well:/dev$ cd /
    well@well:/$ losetup /dev/loop0 file.img
    file.img: No such file or directory
    well@well:/$ cd -
    /dev
    well@well:/dev$ cd -
    /
    well@well:/$ cd home
    well@well:/home$ ls
    bin  etc  file.img  project  readme.php  well
    well@well:/home$ losetup /dev/loop0 file.img
    /dev/loop0: No such file or directory
    well@well:/home$ cd /  
    well@well:/$ cd dev
    well@well:/dev$ mkdir 777 loop0
    mkdir: cannot create directory '777': Permission denied
    mkdir: cannot create directory 'loop0': Permission denied
    well@well:/dev$ sudo mkdir 777 loop0
    well@well:/dev$ ll
    total 0
    drwxr-xr-x. 7 root root    400 Apr 20 03:27 ./
    drwxr-xr-x. 1 root root     92 Apr 18 04:09 ../
    drwxr-xr-x. 2 root root     40 Apr 20 03:27 777/
    crw-------. 1 root root 136, 1 Apr 19 16:50 console
    lrwxrwxrwx. 1 root root     11 Apr 19 16:50 core -> /proc/kcore
    lrwxrwxrwx. 1 root root     13 Apr 19 16:50 fd -> /proc/self/fd/
    crw-rw-rw-. 1 root root   1, 7 Apr 19 16:50 full
    drwxr-xr-x. 2 root root     40 Apr 20 03:27 loop0/
    drwxrwxrwt. 2 root root     40 Apr 19 16:50 mqueue/
    crw-rw-rw-. 1 root root   1, 3 Apr 19 16:50 null
    lrwxrwxrwx. 1 root root      8 Apr 19 16:50 ptmx -> pts/ptmx
    drwxr-xr-x. 2 root root      0 Apr 19 16:50 pts/
    crw-rw-rw-. 1 root root   1, 8 Apr 19 16:50 random
    drwxrwxrwt. 2 root root     40 Apr 19 16:50 shm/
    lrwxrwxrwx. 1 root root     15 Apr 19 16:50 stderr -> /proc/self/fd/2
    lrwxrwxrwx. 1 root root     15 Apr 19 16:50 stdin -> /proc/self/fd/0
    lrwxrwxrwx. 1 root root     15 Apr 19 16:50 stdout -> /proc/self/fd/1
    crw-rw-rw-. 1 root root   5, 0 Apr 20 03:19 tty
    crw-rw-rw-. 1 root root   1, 9 Apr 19 16:50 urandom
    crw-rw-rw-. 1 root root   1, 5 Apr 19 16:50 zero
    well@well:/dev$  cd /
    well@well:/$ cd home
    well@well:/home$ ll
    total 10008
    drwxr-xr-x. 1 well well       89 Apr 20 02:55 ./
    drwxr-xr-x. 1 root root       92 Apr 18 04:09 ../
    drwxr-xr-x. 1 well well       18 Apr 11 21:51 bin/
    drwxr-xr-x. 1 well well       21 Apr 18 04:09 etc/
    -rw-rw-r--. 1 well well 10240000 Apr 20 02:55 file.img
    drwxr-xr-x. 1 well well       95 Apr 19 20:38 project/
    -rwxrwxr-x. 1 well well      360 Mar 14 11:59 readme.php*
    drwxr-xr-x. 4 well well     4096 Apr 19 20:50 well/
    well@well:/home$ losetup /dev/loop0 file.img
    /dev/loop0: Is a directory
    well@well:/home$ mke2fs -c /dev/loop0 10000
    mke2fs 1.42.9 (4-Feb-2014)
    /dev/loop0 is not a block special device.
    Proceed anyway? (y,n) 
    mke2fs: Filesystem larger than apparent device size.
    Proceed anyway? (y,n) 
    warning: Unable to get device geometry for /dev/loop0
    /dev/loop0: Is a directory while setting up superblock
    well@well:/home$ 

    Major structures

    Linux views all file systems from the perspective of a common set of objects. These objects are the superblock, inode, dentry, and file. At the root of each file system is the superblock, which describes and maintains state for the file system. Every object that is managed within a file system (file or directory) is represented in Linux as an inode. The inode contains all the metadata to manage objects in the file system (including the operations that are possible on it). Another set of structures, called dentries, is used to translate between names and inodes, for which a directory cache exists to keep the most-recently used around. The dentry also maintains relationships between directories and files for traversing file systems. Finally, a VFS file represents an open file (keeps state for the open file such as the write offset, and so on).

    主要结构

    Linux 以一组通用对象的角度看待所有文件系统。这些对象是超级块(superblock)、inode、dentry 和文件。超级块在每个文件系统的根上,超级块描述和维护文件系统的状态。文件系统中管理的每个对象(文件或目录)在 Linux 中表示为一个 inode。inode 包含管理文件系统中的对象所需的所有元数据(包括可以在对象上执行的操作)。另一组结构称为 dentry,它们用来实现名称和 inode 之间的映射,有一个目录缓存用来保存最近使用的 dentry。dentry 还维护目录和文件之间的关系,从而支持在文件系统中移动。最后,VFS 文件表示一个打开的文件(保存打开的文件的状态,比如写偏移量等等)。

    w

    李智慧

  • 相关阅读:
    如何避免自己上传的视频或者音频资源被下载
    定期备份服务器上的项目到本地服务器
    查看项目中的laravel的版本
    PHP高并发和大流量的解决方案
    wordpress的安装及使用
    openstack及组件简要介绍
    Java中 如何把Object类型强转成Map<String, String>类型
    JSch基本使用
    Ganymed SSH-2 for Java
    全面解析NIO
  • 原文地址:https://www.cnblogs.com/rsapaper/p/6734939.html
Copyright © 2011-2022 走看看