What a software audit actually looks at

People ask for a software audit when something feels wrong and nobody can say what. Releases take longer than they used to, one developer is the only person who dares to touch a module, a security question from a customer waited a week for an answer. The audit is meant to replace that feeling with a list. This is what the list looks like, what we read to write it, and what we will not do along the way.

What the report looks like

The report is shorter than most people expect. It has four parts.

• Findings ranked by consequence. Each finding says what happens if you do nothing, how likely that is and roughly what it takes to close.
• What to leave alone. A section of the report names the parts that are fine as they are, so nobody spends a quarter modernising something that was never the problem.
• A first week of work. The cheapest items that remove the most risk, grouped so a team can start on Monday without us.
• Where our part ends. The report says what we would do next if asked and what your own team can handle. A good audit does not quietly become a promise to manage implementation forever.

It is written in plain language, it is yours to keep, and it is written so any team can act on it, including a team that is not ours.

What we read

Software problems rarely live in one place, so we read the code and everything around it. The documentation tells us what people intended. The repository, the infrastructure and the real flow of work tell us what happened.

• Structure: how the system is divided, where the boundaries are and which of them are still respected in the code.
• Dependencies: which libraries and platforms it stands on, which of those still receive security patches and which have quietly stopped.
• Shortcuts: the places where someone took the fast route under pressure. Every system has them. The question is which ones will hold and which are now carrying weight they were never meant to.
• Security that fits the real risk: the measures that match your data, your users and the consequences of a breach. No security theatre, and no list of controls copied from a bank when you run a planning tool.
• Why releases hurt: how a change travels from an idea to production, where it waits, what breaks on the way and what would make shipping predictable again.

We also talk to the people who do the work before we measure anything. Their worries are usually right about the symptom and sometimes wrong about the cause, and the gap between the two is the most useful thing an audit can find.

Two patterns we keep finding

The specifics differ every time. The shapes repeat, and two of them say most about what an audit is for.

The first is the module everyone wants to rewrite. It is the oldest part of the system, written in a style nobody would choose today, and it looks like the obvious target. Then we read the change history and find that it has barely been touched in years, that it does one job and that it does that job without incident. The recommendation is to leave it alone: wrap it, document the one thing people need to know about it and spend the attention where change actually happens. A rewrite would cost months and throw away what the code already learned. Technical debt is measured by its consequence, not by its age.

The second is the system that runs on luck. Backups run, but nobody has ever restored one. The administrators share a single account, and its password has survived two departures. One person holds the knowledge of how a release works, and that person takes holidays. None of this shows up as a bug. All of it is a week of work to fix: a tested restore, personal accounts with two-factor authenticationA second proof of identity beside the password, such as a code on a phone, so a leaked password alone does not open an admin account., one page that describes a release. The audit ranks that week above every architectural ambition in the report, because it removes the worst risk for the least money.

What we refuse to do

• Score people. We judge the work. Most shortcuts were reasonable decisions under the constraints of the day, and a report that assigns blame is a report nobody acts on.
• Recommend ourselves by default. If follow-up work makes sense we say so, and the decision is yours. The advice stands alone and may well conclude that you need nothing from us.
• Promise results. Our general terms say that advice is an obligation of meansA commitment to work skilfully and carefully with the resources the assignment calls for, without guaranteeing a particular outcome. Our general terms use it for consultancy, advice and development.: we work carefully and commit what the assignment calls for, and we do not guarantee an outcome. An audit that promises painless releases by spring is selling something other than an audit.

Before you ask for one

If you want a first read yourself, our software health check asks twenty questions about one system in production and ends with a count you can act on. If the count surprises you, or if the questions you cannot answer are the ones about who holds the knowledge, that is the moment an audit earns its place. Keeping the findings from coming back is a separate habit, and we have written about what that routine looks like.