half-built · grails · ember · 2024
A system nobody could run, repaired instead of replaced
A South American environment ministry
A government reporting system declared inoperative four years after another team built it, audited and put back into service without replacing what it ran on.
The operation
A government environment ministry in South America. It runs the country's climate projects on money from international funds, and every one of those projects has to be accounted for: what it will cost, what it is meant to achieve, how far along it is, and the documents standing behind each answer.
The problem
The system for doing that already existed. A different government team had built it in 2020, handed it over, and moved on. By 2024 the funder's own paperwork described it as inoperative and listed it among the obstacles standing between the country and further climate financing.
Nobody who wrote it was available. That is the ordinary version of this situation rather than the extreme one: the code, the server and four years of accumulated data were the only witnesses left.
What the scope stage found
The first job was to read it and run it, not to plan its replacement.
What came back was a great deal smaller than the word inoperative suggests. The reports tab returned an error. Any link pointing straight at a page inside the application returned a not found, because the web server in front of it had never been told to hand those addresses over. Parts of the interface had never been translated.
That is not a broken system. It is a working system with three things wrong with it, and the distance between those two readings is the entire budget.
What we decided to build, and why
Keep it, repair it, and add the one thing it genuinely lacked. The reporting the ministry owed its funders needed four kinds of record the system had never held, covering money received, technology, training and the reporting itself, along with the exports that turn them into the spreadsheets and documents a reporting deadline actually consumes.
Three departments were asked what they needed and they did not agree. One wanted precisely that new reporting. One wanted nothing, because its figures already lived in the finance system. The third asked for continuous air quality readings to be stored here as well, and that request was declined in writing with the reasoning attached: everything in this system hangs off a project with a start and an end date, and a sensor reading has neither.
Turning a request down on the record, rather than quietly building it, is what kept the system coherent. It is also the part of a rescue that a client is paying for and rarely gets.
What changed
The reports error turned out not to be a mistake in the code at all. The code was correct against the data it was written for, and wrong against the data the ministry had since accumulated. Four years of real use had produced projects with no agency recorded and budgets with no currency, and the original code read straight past those gaps without ever checking whether anything was there. The repair went exactly where a report reaches through something that might be missing.
The same shape appeared again once staff began pasting images into the text fields and quietly outgrew limits set in 2020, so thirty two columns were converted to hold text with no ceiling.
Underneath all of it, the system had no safe way to change its own database: no migration files, no record of what had been applied, and no way back. Four new tables and every one of those column changes therefore had to be right the first time, against a live database holding the ministry's real records.
Where it stands
In production and in use. Eleven defects and change requests were closed and written up one by one, the people who have to run it were trained on it, and the system, the decisions and the reasoning behind them were handed over in writing.
The stack was left where it was found. It runs on Grails and Java 8 with an Ember front end, both long out of support, and both deliberately not rewritten. Replacing them was not what the ministry needed, and not what the money was for.
Different business, same shape of problem?
Every system written up here started with somebody describing an operation that had outgrown the way it was being run. If that sounded familiar, tell me how yours works today.
Talk through your version