Git Internals Under the Hood: A Step-by-Step Walkthrough
A comprehensive developer's systems guide to Git internals — covering blobs, trees, commits, SHA-1 key storage, plumbing commands, and Directed Acyclic Graphs.
Almost every modern software project relies on Git for version control. Yet to many developers, Git is a black box of commands like add, commit, and push. When merge conflicts arise or commits go missing, this lack of understanding leads to frustration and lost productivity.
Git is not a traditional delta-based version control system (like CVS or SVN) that records differences between file versions. Under the hood, Git is a simple, elegant **Content-Addressable Database** wrapped in a virtual filesystem. It captures snapshots of your project structure and stores them as compressed object files. This guide details the internal structure of the .git directory, explains how Git objects represent files and folders, walks you through the math of SHA-1 addressability, and traces commit structures using our interactive graph simulator.
1. The Mental Model of Git: A Content-Addressable Database
1.1 Snapshots vs Delta Lists
Traditional version control systems store revision histories as lists of file deltas (differences). To reconstruct a file at version 5, the engine starts with version 1 and sequentially applies the differences. Git discards this model. Every commit in Git is a complete snapshot of the entire repository at a single point in time. If a file has not changed in a commit, Git does not duplicate the file bytes; instead, it stores a reference link pointing to the existing file object.
This design enables Git to perform branches and merges almost instantly. A branch is not a copy of directories; it is simply a 40-character text file containing the hash of a commit node. When you switch branches, Git swaps the files in your working directory to match the commit snapshot.
1.snap vs delta
A delta database accumulates diff files. A content-addressable database stores complete snapshots via pointers, saving storage when files remain unchanged.
Common Misconception — Git duplicates files to save snapshots: A common developer misconception is that storing complete snapshots instead of diffs makes Git repositories bloated. In reality, Git applies Zlib compression to all objects, and periodically packs loose object files into a single packfile using delta compression. This results in repositories that are often smaller than their SVN equivalents.
2. The Git Directory Structure: Inside the .git Folder
2.1 The Control Files
When you run git init, Git creates a hidden directory named .git. This folder contains all the database tables, history, configuration, and index headers for your project. Key elements include: (1) **objects/**: the database directory storing all compressed blobs, trees, and commits. (2) **refs/**: references directory storing pointers to branches, tags, and remotes. (3) **HEAD**: a text file pointing to the currently active branch or commit. (4) **index**: the binary staging area file mapping working directory paths to object hashes.
Mermaid Diagram: The structural breakdown of the internal .git metadata directory, routing active references to database files.
3. The Three Git Objects: Blobs, Trees, and Commits
3.1 Git Object Definitions
Git handles version control using three primary object types:
- **Blob**: stores the raw file contents (excluding file metadata like name, path, or permissions).
- **Tree**: represents a directory. It lists filenames, file permissions, and maps them to their respective blob or child tree hashes.
- **Commit**: represents a snapshot checkpoint. It contains metadata (author, committer, date, message), a pointer to a root tree hash, and pointers to one or more parent commit hashes.
4. Hashing and Content Addressability: The Role of SHA-1
4.1 Address Calculation
Git is content-addressable. This means that objects are retrieved from the database using a key calculated directly from their content. Git calculates this key using the Secure Hash Algorithm 1 (SHA-1), yielding a 160-bit hash represented as a 40-character hexadecimal string (e.g. 246b9a8...).
Before hashing, Git prepends a header containing the object type, content length in bytes, and a null character. This ensures that a blob containing "hello" generates a different hash than a tree containing "hello". The content-derived key guarantees data integrity: if even a single byte of a file changes, its hash changes, propagating modifications up the directory tree.
4.2 Cryptographic Header Formatting and Object Database Partitioning
To generate the unique SHA-1 key, the Git engine formats the input content into a standardized byte sequence before applying the hashing algorithm. Let $o$ be a Git object containing a raw payload $P$, let $\text{len}(P)$ be the length of the payload in bytes, and let $\text{type}(o)$ be a string literal indicating the object type (e.g. "blob", "tree", or "commit"). The hash value $H(o)$ is calculated as:
For example, if you write the string `"Git Internals"` (13 bytes) as a blob, Git formats the header as "blob 13\0". The concatenated byte array is passed through the SHA-1 hashing pipeline, producing a 20-byte binary digest. This digest is formatted as a 40-character lowercase hexadecimal string.
To store this object efficiently on the host filesystem without creating flat directories containing millions of files (which degrades OS directory scan performance), Git splits the 40-character hex key into two parts:
- **Directory Prefix**: The first two characters of the hex string (e.g.
"24") are used to name a subdirectory inside.git/objects/. - **Filename**: The remaining 38 characters (e.g.
"6b9a8...") are used as the filename for the compressed object file.
Thus, the file path mapping can be represented as:
Below is a comparison table of object database address resolutions:
| Hexadecimal Hash (SHA-1) | Partitioned Folder Path | Partitioned Filename | Object Type |
|---|---|---|---|
| 246b9a8f4c28f4c28f4c28f4c28f4c28f4c28f4c | .git/objects/24/ |
6b9a8f4c28f4c28f4c28f4c28f4c28f4c28f4c |
Blob |
| f3e2b10a48a48a48a48a48a48a48a48a48a48a48 | .git/objects/f3/ |
e2b10a48a48a48a48a48a48a48a48a48a48a48 |
Tree |
Pitfall — Modifying object folder contents manually: Renaming or editing files inside .git/objects/ directly breaks content-addressability. Since the folder and filename are derived from the SHA-1 hash, any modification to the compressed content will cause a hash mismatch when Git attempts to read or verify the object, corrupting the repository.
5. The Directed Acyclic Graph (DAG) Structure
5.1 Commit Node Links
A Git repository history forms a Directed Acyclic Graph (DAG). In this graph, each commit node is a vertex, and parent links are edges. Commits point backward to their parents. A standard commit has one parent. A merge commit combines two histories and has two (or more) parents. A root commit has zero parents.
Because parent references are cryptographic hashes of the parent commit content, the history cannot be altered without changing all subsequent commit hashes. This structural integrity guarantees that your project history cannot be secretly modified.
6. Referencing Commits: Refs and HEAD
6.1 Active Pointers
Branch and tag names are simply text files inside the .git/refs/ directory containing a single 40-character commit hash. A branch is a dynamic pointer: when you make a new commit, Git automatically updates the branch text file with the new hash. A tag is static: it remains anchored to its target commit hash.
The HEAD file indicates where you are currently checked out. If HEAD contains ref: refs/heads/main, you are on the main branch. If HEAD contains a raw commit hash (e.g. after checking out a specific commit), you are in a **detached HEAD** state. Any commits made here will not update any branch files and can be lost if you switch branches.
6.2 References Resolution and Packed Refs Internals
To resolve a reference to its target commit hash, the Git engine performs a series of lookups. For example, if you run a command targeting `main`, Git checks several paths in order: (1) .git/refs/heads/main, (2) .git/refs/tags/main, and (3) .git/packed-refs. The packed-refs file is an optimization: instead of storing thousands of tiny files for every branch and tag (which wastes disk block allocations), Git writes them into a single sorted text file:
In this file, the prefix character ^ indicates a "peeled" tag. This means the tag is an annotated tag object, and the line below it contains the direct commit hash that the tag points to. The lookup path mapping can be represented as a function $R(\text{name})$ returning the SHA-1 hash:
When a branch pointer is updated, the change is atomic: Git writes the new hash to a temporary file (e.g. .git/refs/heads/main.lock) and then renames it to overwrite the existing file, preventing race conditions during parallel updates.
Pitfall — Overwriting refs manually: Editing branch files inside .git/refs/heads/ directly with a text editor is dangerous. If you introduce a newline character or save with incorrect encoding, Git will fail to parse the ref, leading to a "corrupted reference" error. Always use the plumbing command git update-ref to modify branch pointers.
7. Advanced: The Index (Staging Area) and Git plumbing commands
7.1 Staging Internals
The staging area, or index, is the workspace buffer between your working directory files and the object database. Git commands are split into **porcelain** (user-facing high-level commands like add, commit) and **plumbing** (systems-level low-level commands like hash-object, write-tree, commit-tree). Understanding plumbing is the key to mastering Git's internal architecture.
Using plumbing, you can bypass the standard workflow and construct objects manually in the database, verifying how Git formats commit objects dynamically.
7.2 Binary Structure of the Index and the Directory Tree Hash Formula
The Git index is stored as a binary file at .git/index. It contains a 12-byte header, followed by sorted index entries, and ends with a 20-byte SHA-1 signature over the file contents. Each entry contains file metadata (ctime, mtime, dev, ino, mode, uid, gid, file size), followed by the 20-byte SHA-1 blob hash and the file path. When you run git write-tree, Git reads the sorted index entries and recursively parses them into tree objects. Let $d$ be a directory containing child files and subdirectories. The hash of the tree object representing $d$, denoted as $T(d)$, is calculated by hashing the concatenation of its sorted entries:
Where each entry $E_i$ is formatted as a binary string containing the file mode (e.g. 100644 for normal files, 040000 for subdirectories), a space character, the filename, a null byte, and the raw 20-byte SHA-1 hash of the child blob or tree:
This recursive hashing means that the root tree hash of a commit acts as a cryptographic fingerprint of the entire project state. If a single file in a sub-sub-directory is altered, its blob hash changes, forcing all parent trees up to the root directory to generate new hashes:
Additionally, all objects are compressed using Zlib before writing to disk. The compression ratio $C_r$ is defined as:
This keeps the object files compact. Below is a comparison table of Git's file mode formats:
| File Mode (Octal) | Git Object Representation | Permissions Meaning | Common Extension |
|---|---|---|---|
| 100644 | Blob | Regular non-executable user file | .txt, .json, .kt |
| 100755 | Blob | Executable shell script or binary file | .sh, .exe |
| 120000 | Blob | Symbolic link pointing to another file | Symlink files |
| 040000 | Tree | Directory structure container | Folder nodes |
Pitfall — Staging large media files: Because Git compresses blobs individually, committing large binary media files (like 100MB videos) causes repository bloat. The compression ratio for pre-compressed formats (like mp4 or zip) is near 1.0, and since Git keeps a complete copy of the blob for every change, your repository size will balloon. Use Git LFS (Large File Storage) to handle big media files.
8. Advanced: Comparison of Git Object Types
8.1 Database Objects Comparison
The table below compares the format, properties, and internal storage formats of the primary Git database objects:
| Object Type | Prepend Header Format | Primary Responsibility | Content Dependency |
|---|---|---|---|
| Blob | "blob [length]\0" |
Store raw file contents in compressed format | File contents only (ignores path and name) |
| Tree | "tree [length]\0" |
Map filenames and directory structures to hashes | Permissions, names, and hashes of children |
| Commit | "commit [length]\0" |
Link root tree snapshot and assign commit metadata | Root tree, parent hashes, message, author, time |
| Tag | "tag [length]\0" |
Create static reference point pointing to a commit | Commit hash, tagger info, tag name, message |
Pitfall — Re-writing committed tags: Modifying a git tag that has already been pushed to a remote repository causes confusion for other developers. Because tags are static pointers, local caches do not update tags automatically. If you must change a tag, delete it first and push the deletion before recreating it.
9. Step-by-Step Manual Object Assembly Walkthrough
9.1 Plumbing Commands Walkthrough
The following terminal script demonstrates how to create a commit object manually using plumbing commands in an empty repository:
10. Interactive: Git DAG Graph Visualizer
Click "Add Commit" to add commits. Watch how HEAD and branch pointers update, and see nodes point back to their parents:
Packfile Compression Latency comparison
The chart below compares the storage size (in kilobytes) of loose object files vs a single compressed packfile for repositories of different sizes:
12. Frequently Asked Questions
Q1: If Git uses SHA-1, can hash collisions corrupt my repository?
Historically possible but extremely rare. Git repositories hold millions of objects, but the probability of a collision is lower than a hardware failure. Modern Git versions are incorporating SHA-256 support to address this.
Q2: How does Git know a file has been renamed?
Git does not track renames explicitly. During a commit, Git hashes the file content. If the content hash is identical to a deleted file in the same commit, Git detects the match dynamically during diff rendering.
Q3: What is a detached HEAD state?
A detached HEAD state occurs when HEAD points directly to a raw commit hash instead of a branch reference. Commits made in this state are orphans and can be deleted by garbage collection if you switch branches.
Q4: Why does Git store file content in blobs instead of lines of changes?
Storing complete file blobs simplifies data structures and speeds up branch operations. Git only calculates diff lines on the fly when you run commands like git diff or git log -p.
Q5: What does the git gc command do?
The git gc (garbage collection) command cleans up loose files, deletes orphaned commits, and packs objects into a single compressed packfile to optimize disk space.
Q6: What is a merge commit?
A merge commit combines two distinct development histories. It is unique because it points to two parent commits, merging branches in the DAG.
Q7: Can I read objects stored inside the .git/objects/ folder directly?
No, because Git compresses object files using Zlib. To view the contents of an object file, you must decompress it first or use the plumbing command git cat-file -p [hash].
Q8: How do I recover a deleted commit?
Verify: (1) use git reflog to find the hash of the lost commit, (2) check the commit details with git show, (3) check out a branch pointing to that hash, and (4) verify that garbage collection has not run yet.