Why does Docker need a Union File System

It is used to:

avoid duplicating a complete set of files each time you run an image as a new container
isolate changes to a container filesystem in its own layer, allowing for that same container to be restarted from a known content (since the layer with the changes will have been dismissed when the container is removed)

implements a union mount for other file systems. It allows files and directories of separate file systems, known as branches, to be transparently overlaid, forming a single coherent file system.
Contents of directories which have the same path within the merged branches will be seen together in a single merged directory, within the new, virtual filesystem.

This allows a file system to appear as writable, but without actually allowing writes to change the file system, also known as copy-on-write

If you didn't have UnionFS, an 200MB image run 5 times as 5 separates containers would mean 1GB of disk space.

See more at "How does a Docker image work?".
For more technical details, see:

How Docker Images Work: Union File Systems for Dummies

So How Does it Work? (From a technical perspective)

Layers Involved

So basically with the overlay(fs), and more specifically the overlay2 storage driver, 4 directories must exist beforehand:

Base Layer

This is where the base files for your file system are stored, this layer ( from the overlay view) is read only. If you want to think about this in terms of Docker images you can think of this layer as your base image.

Overlay Layer

The Overlay layer is where the user operates, it initially offers a view of the base layer and gives the user the ability to interact with files and even “write” to them! When you write to this layer changes are stored in our next layer.

When changes are made this layer will offer a union view of the Base and Diff layer with the Diff layer’s files superseding取代 the Base layer’s.

Again if you want to think about this in terms of Docker images you can think of this layer as the layer you see whenever you run a container.

Disclaimer: I know some people are groaning抱怨 right now because I’m over-simplifying the last statement but please bear with me.

Diff Layer

So right now you’re probably thinking, but what if you made changes to something that’s already found in the base layer? Well worry not some smart person a while ago thought about this as well!

Whenever you write to something that’s already found in the base layer the overlay(fs) will copy the file over to the Diff Layer and then make the modifcations you just tried to write. This type of operation is known as a copy-on-write operation and is probably the most important part of making a Union File System function correctly.

Maybe an image would help?

In the above image the right pane is the view of the overlayed file layers with the top right being the base layer and the bottom right being the diff layer.

As you can see foundation/file1 exists in both the base and diff layer but the diff layer’s file is used instead as it is the higher layer.

I also created another file named fruit/pear and we can see that it only exists in the diff layer.