See a condensed version of this article as it ran in The Hartford Courant. I also was interviewed on WFSB Channel 3’s Face the State about this topic in September 2020.
When a city or company is hacked, its leaders usually don’t face the press. They hide behind a statement and news of the attack gradually emerges over days or weeks. They don’t want to field questions about what they would have, could have, should have done.
Earlier in September 2020 when Hartford Public Schools canceled the first day of classes while the city recovered from a cyber attack, the mayor, school superintendent, police chief and head of IT for the city held a joint press conference. They confidently explained the situation and the city’s response.
To those of us in the cybersecurity field, it was clear that the city had invested time and financial resources and was ready for this attack and that its leaders were following a response plan. It was very different from the scrambling we’re used to seeing, especially on the municipal level.
Canceling the first day of school—especially in this fraught and fragile school year—was an unfortunate outcome. But it could have been so much worse. Look no further than other Connecticut towns and cities that, in recent years, have paid hackers’ ransoms (or debated at length about doing so), or spent weeks or months attempting to recover lost data.
The National Institute of Standards and Technology (NIST) provides a four-step incident response cycle, which provides insight into why Hartford fared so well.
Preparation
At that first-day-of-school press conference, Mayor Luke Bronin described a recent investment of just under half a million dollars to shore up the city’s cyber defenses. The upgrade was well-timed, and without it, the story of this cyber attack would likely be very different.
Part of the investment, it would seem, included robust backup systems. Without the ability to restore data from backups quickly—and professionals available who know how to do so in a crisis—a ransomware attack like this one (in which hackers lock data and demand payment to restore it) can be devastating and long-lasting.
In the press conference, Mayor Bronin said Hartford is the subject of cyber attacks on a regular basis. He wisely recognized that it was only a matter of time before one of those attacks was successful in some capacity.
Detection & Analysis
Reportedly, hackers gained access to the City of Hartford’s IT systems on September 3rd and their presence was detected when information began to be encrypted on September 5th. It may be surprising for people outside of the IT industry, but two days is a very short period of time to detect a cyber attack.
Hackers love undetected access to systems in order to gather more information or expand their access over time. According to a recent IBM study, hackers remain undetected for about 197 days on average after first gaining access.
Containment, Eradication & Recovery
It typically takes about 2-3 months for an organization to contain and recover from a cyber attack. While we don’t know exactly where the City of Hartford’s recovery effort stands, we do know that 200 of the city’s 300 servers were affected, and yet school opened successfully on day two. It came close to opening on day one, were it not for a system that operates school transportation that had not yet been restored.
Restoring large amounts of data and complicated systems from backups takes time and frequently users find that their backup or backup process are unworkable or broken exactly when they need it the most! There are solutions that keep a physical copy of data on site in addition to a cloud backup, which can make data recovery almost instant.
However, for a famously cash-strapped city, I am extremely impressed with Hartford’s ability to get back online so quickly. They way the city seemingly immediately involved the FBI and the Connecticut National Guard Defense Cyber Operations Element team to lend their assistance and expertise was impressive as well.
Post-Incident Activity
While the City of Hartford’s response was quite strong, it is, of course, better to stop ransomware before it enters the system at all. Typically, an employee unwittingly enables a ransomware attack.
The sudden transition to remote work this year has caused cybersecurity best practices to fray at many organizations, and the number of cyber attacks to increase.
Though the city hasn’t detailed exactly how the Hartford hack occurred, it’s important for municipalities and companies alike to make sure that their cybersecurity measures and training have adjusted to the current reality of how work is done.
Simply not physically being together in the same office makes it so much easier for a bad actor to create a situation where it seems urgent and important to give up some data that should not be handed out.
At this time, it appears that none of the data involved was actually stolen or exfiltrated. It’s hard to say if this was due to preparation in security measures, or the rapid nature of the response, but this is very fortunate.
Sometimes, the way this type of attack plays out is that the victim’s data is encrypted and cannot be accessed until a ransom is paid and there is a an explicit threat to post confidential data in a public forum.
Organizations throughout Connecticut can look at Hartford’s ransomware incident as a new type of example to aid in their cybersecurity planning and decision making. There seems to be an endless stream of cases demonstrating what can go wrong. In Hartford, we have a case study in what it takes to weather a ransomware attack with minimal harm.
