Append-only sequence of records on disk. Many distributed systems can be built around append-only logs.
Write-ahead-logs are used with databases where each database first writes to a log before changing a table for durability.
We already know how logs are used for Replication.
In many such cases, logs are the database, they decouple systems, they power several database data structures, enable recovery, and provide ordering.
Note
An interesting observation is that a log is a type of circular buffer, where the buffer size is the size of the disk the log is writing to. There will come a point, where older data needs to be cleared to make room for new incoming data.
Log based derived-data is very promising for integrating different data systems.
Using Logs for Durability and Correctness
We can often avoid distributed transactions (Atomic Commits) by turning the workflow into durable, retryable, idempotent steps backed by logs. in many cases, where we first log intent and then perform that intended action (these are however typically done in the same atomic commit). When performing the request, we insert the operation ID into a requests table and log our action to the WAL. If there is a system crash, we can check our current state by reading the log to determine what we need to do next (retry / move on)