Lesson 16 advanced

Git Submodules

Include one repository inside another with submodules: how they pin a commit, adding them, cloning with --recurse-submodules, updating to new versions, working inside a submodule, removing one cleanly, using them in CI, and when git subtree or a package manager is the better choice.

Sometimes a project needs another repository’s files inside it: shared Terraform modules, a design system, a vendored library. A submodule is a Git repository nested inside another, where the outer repository records which exact commit of the inner one to use. Submodules are powerful and precise, and also the Git feature people complain about most, mainly because they do not update themselves. This lesson shows how to use them without the surprises.

How a submodule works

Orbit’s infrastructure repository wants to use the platform team’s shared Terraform modules, which live in their own repository. With a submodule:

  • The outer repository stores a .gitmodules file with the submodule’s path and URL.
  • Instead of the submodule’s files, the outer repository’s commit stores one commit ID: the exact version of the submodule to check out.
  • The submodule folder is a full Git repository of its own, usually in detached HEAD at that commit.

Because a commit is pinned, everyone who checks out a given commit of Orbit gets exactly the same version of the shared modules. Upgrading is an explicit change that goes through review like any other.

Adding a submodule

$ git submodule add https://github.com/orbit-dev/terraform-modules.git modules/shared
Cloning into '/home/ana/orbit-infra/modules/shared'...
$ cat .gitmodules
[submodule "modules/shared"]
	path = modules/shared
	url = https://github.com/orbit-dev/terraform-modules.git
$ git status -s
A  .gitmodules
A  modules/shared
$ git commit -m "Add shared Terraform modules as a submodule"

git diff --staged shows the submodule as a single line, Subproject commit 4f2a9c1.... That line is all the outer repository knows about it.

Cloning a repository with submodules

A plain git clone leaves submodule folders empty. This is the most common submodule surprise. Either clone with submodules:

$ git clone --recurse-submodules git@github.com:orbit-dev/orbit-infra.git

or initialise them after the fact:

$ git submodule update --init --recursive

--recursive also handles submodules inside submodules.

Staying in sync after pull and switch

When a teammate upgrades the submodule and you pull, your outer repository now points at a new commit, but the submodule folder still has the old one checked out:

$ git pull
$ git status -s
 M modules/shared           # the folder does not match the recorded commit
$ git submodule update --recursive

To make pull, switch and checkout update submodules automatically, set:

$ git config set --global submodule.recurse true

This one setting removes most day-to-day submodule pain.

Upgrading a submodule

To move to a newer version of the shared modules, update the submodule and commit the new pointer in the outer repository:

$ git submodule update --remote modules/shared       # fetch and check out the latest of its branch
$ git diff --submodule=log
Submodule modules/shared 4f2a9c1..b81d3e0:
  > Add WAF rules to the alb module
  > Pin the AWS provider to 6.x
$ git commit -am "Update shared modules to b81d3e0"

--remote follows the submodule’s default branch, or the one set with git config -f .gitmodules submodule.modules/shared.branch release. You can also go into the folder and check out a specific tag, then commit in the outer repository:

$ git -C modules/shared fetch --tags
$ git -C modules/shared switch --detach v2.3.0
$ git add modules/shared && git commit -m "Pin shared modules to v2.3.0"

Set diff.submodule log and status.submoduleSummary true so diffs and status always show the commit messages behind a submodule change.

Working inside a submodule

To change the shared modules themselves while working on Orbit, remember the submodule is in detached HEAD. Create or switch to a branch first, or your commits will not be on any branch:

$ cd modules/shared
$ git switch -c feature/waf-rules
$ git commit -am "Add WAF rules to the alb module"
$ git push -u origin feature/waf-rules
$ cd ../..
$ git add modules/shared && git commit -m "Use shared modules with WAF rules"

Push the submodule before the outer repository. If the outer repository points at a submodule commit that exists only on your laptop, everyone else’s submodule update fails. git push --recurse-submodules=check refuses to push in that situation, and on-demand pushes the submodules for you.

To run a command in every submodule:

$ git submodule foreach 'git status -s'
$ git submodule status
 b81d3e0 modules/shared (v2.3.0-4-gb81d3e0)

Removing a submodule

$ git rm modules/shared                    # removes the folder and its .gitmodules entry
$ git commit -m "Remove shared modules submodule"
$ rm -rf .git/modules/modules/shared       # optional: delete the stored copy of its repository

Submodules in CI

CI checkouts also start with empty submodule folders unless configured:

# GitLab CI
variables:
  GIT_SUBMODULE_STRATEGY: recursive
# GitHub Actions
- uses: actions/checkout@v5
  with:
    submodules: recursive

If the submodule is private, the CI job’s token must be allowed to read it. Relative URLs in .gitmodules, such as ../terraform-modules.git, make the submodule use the same host and credentials as the main repository.

Alternatives

OptionHow it worksChoose it when
SubmodulePins a commit of another repositoryYou need an exact version and independent history
git subtreeCopies the other repository’s files into a folder, history merged inConsumers should not need to know about it; changes are rare
Package managernpm, pip, Go modules, Helm, the Terraform registryThe code is published as versioned packages
MonorepoEverything in one repositoryOne team or org that changes things together

For Terraform modules specifically, referencing a tagged Git source in the module block, such as source = "git::https://github.com/orbit-dev/terraform-modules.git//alb?ref=v2.3.0", often replaces a submodule entirely (see the Terraform Tutorial).

Quick reference

TaskCommand
Addgit submodule add <url> <path>
Clone with submodulesgit clone --recurse-submodules <url>
Fill empty submodule foldersgit submodule update --init --recursive
Keep in sync automaticallygit config set --global submodule.recurse true
Upgrade to latest of its branchgit submodule update --remote <path>
See what changedgit diff --submodule=log
Run in every submodulegit submodule foreach '<command>'
Statusgit submodule status
Removegit rm <path>

Exercise

A new teammate clones orbit-infra and the modules/shared folder is empty. What fixes it?

  1. git pull –force
  2. git clone the shared modules repository into the folder by hand
  3. git restore modules/shared
  4. git submodule update –init –recursive
Show answer

Answer: D. A plain clone does not fetch submodules. update –init registers and checks out each submodule at the commit the outer repository records. Next time, clone with –recurse-submodules.

Exercise

What does the outer repository actually store about a submodule in each commit?

  1. A full copy of the submodule’s files
  2. The name of the submodule’s branch to follow
  3. The exact commit ID of the submodule to check out, plus its path and URL in .gitmodules
  4. Nothing; submodules always use their latest commit
Show answer

Answer: C. Each commit of the outer repository pins one submodule commit. That is why upgrading is an explicit commit, and why everyone gets the same version for the same outer commit.

Exercise

You committed a change inside a submodule and updated the pointer in the outer repository. Teammates’ submodule updates now fail. What went wrong?

  1. The .gitmodules file must be deleted after each change
  2. You pushed the outer repository but not the submodule commit it points to
  3. Submodules cannot be changed once added
  4. Teammates need submodule.recurse set to false
Show answer

Answer: B. The outer repository points at a commit that exists only on your machine. Push the submodule first, or use git push –recurse-submodules=on-demand to push both in the right order.

Key takeaways

  • A submodule is a nested repository pinned to one exact commit, recorded in the outer repository with .gitmodules.
  • Clone with --recurse-submodules or run git submodule update --init --recursive to fill the folders.
  • submodule.recurse true keeps submodules in sync on pull and switch.
  • Upgrade with git submodule update --remote and commit the new pointer; push submodule commits first.
  • Configure CI checkouts for submodules, and consider subtree, packages or a monorepo when pinning is not needed.

FAQ

Why is my submodule always in detached HEAD?

Because the outer repository pins a commit, not a branch. Git checks out that commit directly. To make changes inside the submodule, switch to a branch first.

Why does git status show my submodule as modified when I did not change it?

The submodule’s checked-out commit differs from what the outer repository records, usually after a pull or switch. Run git submodule update --recursive, or set submodule.recurse true so it happens automatically.

Should I use submodules or git subtree?

Submodules keep the other project separate and pinned, which is precise but needs extra commands. Subtree copies the files in, so anyone can clone and build with no extra steps, but contributing changes back is more work. If consumers should not need to know, prefer subtree or a package.

Can a submodule follow a branch automatically?

Not on its own: the outer repository always records a commit. git submodule update --remote moves it to the latest commit of a configured branch, and you commit that change. Bots like Dependabot or Renovate can open these update pull requests for you.

How do I change a submodule’s URL?

Edit the URL in .gitmodules, run git submodule sync --recursive to copy it into your local configuration, and commit the .gitmodules change so others get it too.

↑ ↓ navigate Enter open Esc close