Once again, cloud services very predictably show why they can be less secure than running your own.
We’ve warned for many years of cloud insider abuse like this, using examples from Uber, Google and Facebook.
In many of these cases it’s male engineers in American technology companies using their power and privilege to stalk and abuse women.
The US Department of Justice has posted details of a 34 year old man who is said to have worked at Yahoo.
In pleading guilty, Ruiz, a former Yahoo software engineer, admitted to using his access through his work at the company to hack into about 6,000 Yahoo accounts. Ruiz cracked user passwords, and accessed internal Yahoo systems to compromise the Yahoo accounts. Ruiz admitted to targeting accounts belonging to younger women, including his personal friends and work colleagues. He made copies of images and videos that he found in the personal accounts without permission, and stored the data at his home. Once he had access to the Yahoo accounts, Ruiz admitted to compromising the iCloud, Facebook, Gmail, DropBox, and other online accounts of the Yahoo users in search of more private images and videos. After his employer observed the suspicious account activity, Ruiz admitted to destroying the computer and hard drive on which he stored the images.
That last sentence is concerning for anyone who has done digital forensics. How was Ruiz tipped off that he was being observed by an internal investigations team?
He wasn’t just a software engineer, he was a Site Reliability Engineer (SRE). And he wasn’t just a Yahoo engineer…
That career path reveals a far worse story than what is being reported right now.
A SRE is a person with deep access inside the cloud provider. They are trusted with the most sensitive data because, in theory, without giving them access a system could become unreliable or go offline.
For example, here’s a single line command in virtual (VMware) environments that exports a copy of an entire server. In a disaster (planning) scenario it could be essential to keeping services running:
Copy-DatastoreItem vmstore:\Datacenter01\StorageArray01\DBNodes\* C:\SREisGod\StolenUserSecrets
Imagine instead, as you can see from the destination path name at the end of that line, any evil SRE just wants to steal ALL the data. SRE staff literally have keys to any kingdom trusting their employer. Even if the data was stored encrypted, using this command it’s decrypted by design.
I’ve repeatedly designed systems to protect against exactly this kind of insider threat and customers need to explicitly ask for proof that one exists. This is a disaster for both Okta and Yahoo if they cannot account for SRE access on their systems, particularly during the hours Ruiz was working.
His eight months at Okta, a widely used identity management company, could be an even bigger problem than Yahoo. Although to be fair the timing is interesting for both cases. Yahoo in 2007 when he joined was the biggest identity provider in the world. In 2018 Okta was claiming to be the leader in this space.
It looks like Okta apparently fired him as soon as the indictments were unsealed that detailed his long-term abuse of being a privileged SRE to game identity management. What Okta hasn’t said is whether they’ve concluded an internal investigation of all his access to identity as an SRE.
This is huge. I can’t overstate enough that an identity management cloud provider, holding the secrets of millions of people, hired an identity thief. It’s like saying a bank hired a bank robber to guard their safe.
Given inside knowledge and access at the service provider he allegedly cracked passwords of thousands of young women, including those he knew and worked with, in order to steal their images. Then he used their identity information to pivot through their cloud accounts that shared the same password to steal more images.
Two lessons here:
One. Okta is a core identity management company that hired a predator who clearly joined companies to commit crimes. Anyone using Okta or a similar service needs to be prepared for this level of insider threat being reported. Although we can pressure Okta on reasons screening didn’t block this hire, we can’t assume screens will be perfect and instead should demand they prove his actions were limited and detected.
Two. Re-use of passwords is what made one evil cloud staff member able to access so many other cloud accounts. Impersonation was possible by Ruiz because users didn’t setup different passwords on each cloud service. Password managers are free and a baseline requirement for users today. Also multi-factor authentication (MFA) would have made SRE theft of user secrets less effective and should be considered another baseline requirement (caveat: nothing is perfect. see new FBI warning on MFA bypass).
There’s a third point about avoiding tipping off suspects during investigations, and preserving evidence, but we don’t have enough details yet on why or how badly that security team at Yahoo was compromised.