To understand why it is so difficult to defend computers from even moderately capable hackers, consider the case of the security flaw officially known as CVE-2017-0199.
The bug was unusually dangerous but of a common genre: it was in Microsoft software, could allow a hacker to seize control of a personal computer with little trace, and was fixed April 11 in Microsoft’s regular monthly security update.
But it had traveled a rocky, nine-month journey from discovery to resolution, which cyber security experts say is an unusually long time.
Google’s security researchers, for example, give vendors just 90 days’ warning before publishing flaws they find.
Microsoft Corp. declined to say how long it usually takes to patch a flaw.
While Microsoft investigated, hackers found the flaw and manipulated the software to spy on unknown Russian speakers, possibly in Ukraine.
And a group of thieves used it to bolster their efforts to steal from millions of online bank accounts in Australia and other countries.
Those conclusions and other details emerged from interviews with researchers at cyber security firms who studied the events and analyzed versions of the attack code.
Microsoft confirmed the sequence of events.
The tale began last July, when Ryan Hanson, a 2010 Idaho State University graduate and consultant at boutique security firm Optiv Inc. in Boise, found a weakness in the way that Microsoft Word processes documents from another format. That allowed him to insert a link to a malicious program that would take control of a computer.
Combining flaws
Hanson spent some months combining his find with other flaws to make it more deadly, he said on Twitter. Then in October he told Microsoft. The company often pays a modest bounty of a few thousands dollars for the identification of security risks.
Soon after that point six months ago, Microsoft could have fixed the problem, the company acknowledged. But it was not that simple. A quick change in the settings on Word by customers would do the trick, but if Microsoft notified customers about the bug and the recommended changes, it would also be telling hackers about how to break in.
Alternatively, Microsoft could have created a patch that would be distributed as part of its monthly software updates.
But the company did not patch immediately and instead dug deeper. It was not aware that anyone was using Hanson’s method, and it wanted to be sure it had a comprehensive solution.
“We performed an investigation to identify other potentially similar methods and ensure that our fix addresses more than just the issue reported,” Microsoft said through a spokesman, who answered emailed questions on the condition of anonymity. “This was a complex investigation.”
Hanson declined interview requests.
The saga shows that Microsoft’s progress on security issues, as well as that of the software industry as a whole, remains uneven in an era when the stakes are growing dramatically.
The United States has accused Russia of hacking political party emails to interfere in the 2016 presidential election, a charge Russia denies, while shadowy hacker groups opposed to the U.S. government have been publishing hacking tools used by the Central Intelligence Agency and National Security Agency.
Attack begin
It is unclear how the unknown hackers initially found Hanson’s bug. It could have been through simultaneous discovery, a leak in the patching process, or even hacking against Optiv or Microsoft.
In January, as Microsoft worked on a solution, the attacks began.
The first known victims were sent emails enticing them to click on a link to documents in Russian about military issues in Russia and areas held by Russian-backed rebels in eastern Ukraine, researchers said. Their computers were then infected with eavesdropping software made by Gamma Group, a private company that sells to agencies of many governments.
The best guess of cyber security experts is that one of Gamma’s customers was trying to get inside the computers of soldiers or political figures in Ukraine or Russia; either of those countries, or any of their neighbors or allies, could have been responsible. Such government espionage is routine.
The initial attacks were carefully aimed at a small number of targets and so stayed below the radar. But in March, security researchers at FireEye Inc noticed that a notorious piece of financial hacking software known as Latenbot was being distributed using the same Microsoft bug.
FireEye probed further, found the earlier Russian-language attacks, and warned Microsoft. The company, which confirmed it was first warned of active attacks in March, got on track for an April 11 patch.
Then, what counts as disaster in the world of bug-fixers struck. Another security firm, McAfee, saw some attacks using the Microsoft Word flaw on April 6.
After what it described as “quick but in-depth research,” it established that the flaw had not been patched, contacted Microsoft, and then blogged about its discovery on April 7.
The blog post contained enough detail that other hackers could mimic the attacks.
Other software security professionals were aghast that McAfee did not wait, as Optiv and FireEye were doing, until the patch came out.
McAfee Vice President Vincent Weafer blamed “a glitch in our communications with our partner Microsoft” for the timing. He did not elaborate.
By April 9, a program to exploit the flaw was on sale on underground markets for criminal hackers, said FireEye researcher John Hultquist.
The next day, attacks were mainstream. Someone used it to send documents booby-trapped with Dridex banking-fraud software to millions of computers in Australia.
Finally, on the Tuesday, about six months after hearing from Hanson, Microsoft made the patch available. As always, some computer owners are lagging behind and have not installed it.
Ben-Gurion University employees in Israel were hacked, after the patch, by attackers linked to Iran who took over their email accounts and sent infected documents to their contacts at technology companies and medical professionals, said Michael Gorelik, vice president of cyber security firm Morphisec.
When Microsoft patched, it thanked Hanson, a FireEye researcher and its own staff.
A six-month delay is bad but not unheard of, said Marten Mickos, chief executive of HackerOne, which coordinates patching efforts between researchers and vendors.
“Normal fixing times are a matter of weeks,” Mickos said.
Privately-held Optiv said through a spokeswoman that it usually gives vendors 45 days to make fixes before publishing research when appropriate, and that it “materially followed” that practice in this case.
Optiv is now comparing the details of what Hanson told Microsoft with what the spies and criminals used in the wild, trying to find out if the researcher’s work was partly responsible for the worldwide hacking spree, the spokeswoman said.
The spree included one or more people who created a hacking tool for what FireEye’s Hultquist said is probably a national government – and then appearing to double-dip by also selling it to a criminal group.
If the patching took time, others who learned of the flaw moved quickly.
On the final weekend before the patch, the criminals could have sold it along to the Dridex hackers, or the original makers could have cashed in a third time, Hultquist said, effectively staging a last clearance sale before it lost peak effectiveness.
It is unclear how many people were ultimately infected or how much money was stolen.