Make Git Forget
Git is your faithful repository of all code and comments, faithfully saving everything you give it. What if you need it to forget?
Git is used by developers, engineers and technical writers to store code and comments on the changes to that code. We use it so that we never have to worry about losing old code, that was removed: because Git never forgets.
If you delete a line from a file - say a secret (like a password or a key) that might be gone from the current version of the file, but anyone who can see the repo can trivially get the key out again, just by looking at the history.
git log -S passwordOops. Oh dear:
commit cef63cd76777c66c9ce4aa81bd3bef6a33544e25 (HEAD -> main)
Author: Sarah Smith <sarah@example.com>
Date: Tue Feb 4 09:32:38 2025 +1000
store the password - will delete it laterYou thought you could just delete the file. Even if I delete my entire local repository, SURELY it’s gone then?
# DO *** NOT *** DO THIS.
# You will lose stuff. Branches, stashes.
cd ..
rm -Rf my-git-repo/If I pushed it, then it’s in the cloud too. Git remembers:
git show cef63cd76777c66c9ce4aa81bd3bef6a33544e25
commit cef63cd76777c66c9ce4aa81bd3bef6a33544e25 (HEAD -> main)
Author: Sarah Smith <sarah@example.com>
Date: Tue Feb 4 09:32:38 2025 +1000
store the password - will delete it later
diff --git a/temp.txt b/temp.txt
new file mode 100644
index 0000000..a7e5b23
--- /dev/null
+++ b/temp.txt
@@ -0,0 +1 @@
+password: s3kr1t
So it’s worse - it’s in the main repo where bad actors - or your colleagues, boss, the intern - could find it.
And the same thing applies to anything not a secret - the hot take you shared about a colleague in a code file - you might have put in Git that you want gone. It’s there for perpetuity.
Unless you know how to fix it. For the rest of this article, I’m going to assume its a secret - in the technical sense. That is a token of some sort that gives access to assets.
Cleaning Secrets from a Repo
So someone messed up and now you have to fix things. Slow down, and carefully follow these steps — the order & detail is important. I promise you we’ll get through this together.
FIRST: Revoke the secrets from whatever assets or platform they provide access to, eg:
Assume that the keys have already fallen in to the hands of bad actors, and immediately change the locks so that the keys are no use.
Cleaning House
HAVE YOU REALLY REVOKED ALL THE KEYS?
Go back and revoke them (see previous section) if you have not.
Alright, the secrets cannot hurt you now. But. You may still want to clean up the repo where the keys are exposed to remove traces of the keys — even tho’ you have now rendered them useless.
Maybe because:
Other authors in the repo might find the keys and
…mistakenly issue & expose new ones
To protect the guilty — cover your tracks — as an act of contrition
Note that cleaning house like this will not remove the keys from any fork or offline copy someone may (read as “already has”) taken of the repo. That’s why you have to revoke the secrets first.
Install BFG
The tool we are going to use is much faster than git filter branch, and more flexible. Go to the website (or jump to below where I just curl it down):
Grab the latest jar from the download button on the right-hand-side. Below I’ll assume it was version 1.14.0 (which is what I'm using currently).
I also assume:
you know how to use a command line,
you have a Unix-like command line - eg WSL/mingw, Mac or Linux, and
are using bash, or can translate my bash-isms below to your shell.
I assume a Unix command line & bash - translate to the shell you have
Or if you really want to be a bit more secure:
click through to Github,
verify the Github site via SSL and
get the latest code from the releases tab on the right.
verify the code in whatever way you like
The author of BFG does not provide SHA checksums to verify. But I am too lazy to build it, and I just use the pre-built binaries anyway. 🤷♀️
I install my copy in a handy-dandy local ~/bin folder where I can run all my useful scripts and binaries from:
# Download it - if you didn't already
curl -O https://repo1.maven.org/maven2/com/madgag/bfg/1.15.0/bfg-1.15.0.jar
# Install it
mkdir -p ~/bin
mv ~/Downloads/bfg-1.14.0.jar ~/bin/.
# I use a personal bin directory for scripts - add to bash_profile
export PATH="$HOME/bin:$PATH"
# Test it worked
java -jar ~/bin/bfg-1.14.0.jar If the last java -jar command didn’t work, check you have a Java JDK in your path.
Hint: if you have an Android SDK installed on Mac then editing your bash profile like this might work:
# Java from Android SDK
export JAVA_HOME="/Applications/Android\
Studio.app/Contents/jbr/Contents/Home"
# Add to path
export PATH="${JAVA_HOME}/bin:${PATH}"Otherwise install a Java JDK and try again. Load up a new Terminal session so you get the new Java tools in your path.
Get Ready for the BFG
Close all PR’s on your Github or whatever server and get rid of any stale branches
Stop any CI/CD activity or anything that could update your repo
Clean out the secrets
Don’t get messed up by whatever state your current working copy is in
Even create a fresh area to do this work — below its src
…but use a directory name you’re not already using
Make a fresh clone of your repo
You could do this with your existing repo, but I suggest using a fresh clone
If things are in a panic, its the public repo you want to focus on
Git is of course a distribute source control system so theoretically all repositories are equal - but in practice its the Github one everyone can see we worry about
# Get the SSH clone link from your repo
# Check it out with a distinctive name
mkdir -p ~/src && cd ~/src
git clone git@github.com:my-git-repo/RepoWithExposedSecrets.git\
freshRepoWithExposedSecretsIf you host Git on some other platform, eg GitLab or BitBucket, sub in the appropriate details in the above.
Remove the file containing the secrets
cd freshRepoWithExposedSecrets
git rm Credentials.json
git commit -m "Remove credentials file"Add another commit so the secret removal is not on HEAD
This is important — the BFG cannot clean HEAD commit history
If you just removed a sensitive file then the commit will show the secret
echo "Please never add secrets to this repo" >> README.md
git add README.md
git commit -m "Document secrets policy"Push the results
git pushRun the BFG
cd ~/src
git clone --mirror git@github.com:my-git-repo/RepoWithExposedSecrets.git
java -jar ~/bin/bfg-1.14.0.jar --delete-files Credentials.json \
RepoWithExposedSecrets.gitAfter the above you should get output like:
Using repo : /Users/my-git-name/src/RepoWithExposedSecrets.gitFound 23 objects to protect
Found 4 commit-pointing refs : HEAD, refs/heads/ex0, refs/heads/ex1, refs/heads/masterProtected commits
-----------------These are your protected commits, and so their contents will NOT be altered: * commit 593dbab8 (protected by 'HEAD')Cleaning
--------Found 29 commits
Cleaning commits: 100% (29/29)
Cleaning commits completed in 74 ms.Updating 1 Ref
-------------- Ref Before After
---------------------------------------
refs/heads/master | 594dbcb8 | 033618fc
Updating references: 100% (1/1)
...Ref update completed in 11 ms.Commit Tree-Dirt History
------------------------ Earliest Latest
| |
.....................DmDmDDmm D = dirty commits (file tree fixed)
m = modified commits (commit message or parents changed)
. = clean commits (no changes to file tree) Before After
-------------------------------------------
First modified commit | 56be179e | 127e3e8b
Last dirty commit | edad0d34 | 1cfe71d0Deleted files
------------- Filename Git id
-------------------------------------------
Credentials.json | 453b2544 (117 B )
In total, 9 object ids were changed. Full details are logged here: /Users/my-git-name/src/RepoWithExposedSecrets.git.bfg-report/2021-07-18/15-22-55
BFG run is complete!
When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressiveThe format of this report is a bit hard to read, and I have not bothered to try to un-garble the above.
If you get that line telling you to run git reflog then you are all good. 👍
If the output says something about protected commits and zero changes then you have a protected commit that is blocking the BFG from cleaning your repo. 🥲
Most likely you did not add an additional commit. Check my step above where it says:
Add another commit so the secret removal is not on HEAD.
More details can be read on SO, but seriously its as simple as making sure that
the commit that removes the file containing the secrets
is not the most recent.
If it didn’t work
If you get the above-mentioned protected commits message, you will need to remove the mirror repo & fresh clone and start again from the top of this document.
# It didn't work? Oh no!
# Let's slow down and start again
cd ~/src
rm -Rf RepoWithExposedSecrets.git
rm -Rf freshRepoWithExposedSecretsGo to the top and start again
Really?
Yes.
Push the BFG results
OK — it worked? The BFG should have printed out a line like:
BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressive…copy the run part and execute it in the mirror repo:
You should see something like this:
Enumerating objects: 139, done.
Counting objects: 100% (139/139), done.
Delta compression using up to 16 threads
Compressing objects: 100% (138/138), done.
Writing objects: 100% (139/139), done.
Building bitmaps: 100% (29/29), done.
Total 139 (delta 89), reused 47 (delta 0), pack-reused 0Now push the cleaned repo up to your mainline:
git pushYou may need to do a force push, or git may figure that out. What you should see on push is:
Enumerating objects: 38, done.
Counting objects: 100% (38/38), done.
Writing objects: 100% (108/108), 16.04 KiB | 16.04 MiB/s, done.
Total 108 (delta 38), reused 38 (delta 38), pack-reused 70
remote: Resolving deltas: 100% (77/77), completed with 13 local objects.
To github.com:sarah-j-smith/my-git-repo.git
+ 594dbab...033628f master -> master (forced update)Go to the Repo on Github and check
Check the relevant file is gone
Check that commit history with the diff showing the secrets is not visible
Make Sure you don’t commit more secrets
Make sure your readme files don’t have the literal secrets. Instructions and scripts probably refer to .env files or credentials.json or similar — add those to your .gitignore so that future git commits won’t pick them up.
Consider using Talisman - a free tool made by ThoughtWorks, the company I used to consult for - to check for secrets using a commit hook.
This article is a bit more technical. I also post non-technical stuff about team dynamics - just the thing if you’ve suddenly found yourself with leadership duties. I know what I’m talking about.
Two articles a week, landing in your inbox 6am Wednesday and Saturday. Subscribe to get my startup stories with real deal info inside, team dynamics and the occasional tech deep dive. Don’t miss it.