Hackers last year conducted a 'dry run' of SolarWinds breach

Kim Zetter
·Contributor
·7 min read
Westend61/Getty Images
Westend61/Getty Images

Hackers who breached federal agency networks through software made by a company called SolarWinds appear to have conducted a test run of their broad espionage campaign last year, according to sources with knowledge of the operation.

The hackers distributed malicious files from the SolarWinds network in October 2019, five months before previously reported files were sent to victims through the company’s software update servers. The October files, distributed to customers on Oct. 10, did not have a backdoor embedded in them, however, in the way that subsequent malicious files that victims downloaded in the spring of 2020 did, and these files went undetected until this month.

“We’re thinking they wanted to test whether or not it was going to work and whether it would be detected. So it was more or less a dry run,” a source familiar with the investigation told Yahoo News. “They took their time. They decided to not go out with an actual backdoor right away. That signifies that they’re a little bit more disciplined and deliberate.”

The October files were discovered in the systems of several victims, but investigators have so far found no signs that the hackers engaged in any additional malicious activity on those systems after the files landed on them.

Five months later, the hackers added new malicious files to the SolarWinds software update servers that got distributed and installed on the networks of federal government agencies and other customers. These new files installed a backdoor on victim networks that allowed the hackers to directly access them. Once inside an infected network, the attackers could have used the SolarWinds software to learn about the structure of the network or alter the configuration of network systems. But they would also have been able to breach other systems on the network or download new malicious files directly to those systems.

The specific number of infected victims remains unknown at this time, but some of the victims breached with the spring 2020 files reportedly include: divisions within the U.S. Treasury and Commerce departments, the Department of Homeland Security, national labs working for the Department of Energy, and the National Nuclear Security Administration, which oversees the national nuclear weapons stockpile. In the commercial sector, the security firm FireEye was also breached by the hackers through SolarWinds software, and late Tuesday Microsoft acknowledged that it had found malicious SolarWinds files on its network as well. Not all SolarWinds customers downloaded the malicious updates.

FireEye was the first to expose the espionage campaign, in a blog post on Dec. 8 after discovering the hackers in its network, though it did not mention that SolarWinds was the origin of the breach into its network. The company did not become aware of the SolarWinds connection until after publishing its post, according to a source.

The U.S. Treasury Department building in Washington. (Patrick Semansky/AP)
The Treasury Building in Washington, D.C. (Patrick Semansky/AP)

The new information about the 2019 files expands the previously reported timeline around the intrusions and indicates that the hackers had already compromised SolarWinds’ software update system at least five months earlier than reported.

“This tells us the actor had access to SolarWinds’ environment much earlier than this year. We know at minimum they had access Oct. 10, 2019. But they would certainly have had to have access longer than that,” says the source. “So that intrusion [into SolarWinds] has to originate probably at least a couple of months before that — probably at least mid-2019 [if not earlier].”

The files distributed to victims in October 2019 were signed with a legitimate SolarWinds certificate to make them appear to be authentic code for the company’s Orion Platform software, a tool used by system administrators to monitor and configure servers and other computer hardware on their network.

SolarWinds would not answer questions about how long the attackers were in their network, but a spokesperson directed Yahoo News to a list of frequently asked questions published Friday morning that addresses the 2019 files. It indicates that in October 2019, SolarWinds distributed versions of its software that “contained test modifications to the code base ... it is the first version in which we have seen activity from the attacker at this time.” The company noted that subsequent software releases it made in 2019 “did not include either test modifications contained” in that October 2019 version or the backdoor added to the spring 2020 versions.

The company did not say, however, that these files had been found on victims’ machines.

The files that infected customers on Oct. 10 were compiled the same day customers got infected with them, as were files released in the spring of 2020, infecting customers within hours — and in some cases minutes — after they were compiled.

Programmers first write code in a programming language before compiling it into a binary file that computers can read.

The SolarWinds logo is seen outside its headquarters in Austin, Texas, U.S., December 18, 2020. (Sergio Flores/Reuters)
SolarWinds headquarters in Austin, Texas. (Sergio Flores/Reuters)

It’s not clear when each individual customer was infected by the backdoor when the files first became available for customers to download in the spring of 2020. Charles Carmakal, senior vice president and chief technology officer at Mandiant, FireEye’s incident response arm, wouldn’t say when his company was breached but did say the attackers were not in his company for the full eight months between when the malicious software updates first were made available for customers to download from the SolarWinds server and the time that FireEye discovered the breach. He told Yahoo News that their investigation shows that other SolarWinds customers infected by the malware did not download and install the malicious update until months after it became available on the update server.

There has been some confusion about how FireEye discovered the hackers in its network. A story published on Wednesday quoted sources on Capitol Hill who said the hackers had duped a FireEye employee into revealing their credentials for accessing the company’s network.

But Carmakal told Yahoo News this is incorrect. The breach was discovered after the hackers enrolled a device into FireEye’s multifactor authentication system, which FireEye employees use to remotely sign into the company’s VPN. The multifactor authentication system works similarly to how Gmail users access their accounts in a secure manner. It generates a unique code on the user’s phone that they enter, along with their username and password, each time they access their account so that even if someone has their username and password, they cannot access the account without the unique code. That unique code gets generated only on the account holder’s mobile phone that they have tied to their account.

Attendees walk by the FireEye booth during the 2016 Black Hat cyber-security conference in Las Vegas, Nevada, U.S. August 3, 2016. (David Becker/Reuters)
Attendees walk by the FireEye booth during the 2016 Black Hat cybersecurity conference in Las Vegas. (David Becker/Reuters)

After the hackers registered their device with the FireEye network to obtain the unique codes that would normally go only to that employee’s device, FireEye’s security system issued an automatic alert to the employee and to the company’s security team that an unknown device was registered to the company’s multifactor authentication system as if the device belonged to the employee.

“They had to provide credentials to authenticate [their device] to the [multifactor authentication system] in order to authenticate to the FireEye VPN,” Carmakal said. “It was the process the attacker followed to enroll in the MFA solution which is what generated the alert. But at this point the attacker already had the employee’s username and password.”

The employee whose credentials were compromised told FireEye’s security team that the device didn’t belong to him, and in the course of investigating how the hackers may have obtained the employee’s credentials, they discovered the hackers had gained access to the network through malicious SolarWinds software.

Carmakal didn’t say how the hackers obtained the credentials after that or how many employee credentials they stole. But once inside a network, it’s common for skilled hackers to seek access to critical system files where employee account credentials are stored in order to use those credentials to gain deeper access into additional parts of the network.

_____

Read more from Yahoo News: