Twitter’s power users in the media and commentariat have been pathologically obsessed with Elon Musk’s acquisition of the platform since April. For his part, Musk has been happy to add fuel to the fire by providing several independent journalists with the Twitter Files, which exposed the activist nature of the company’s content moderation during the 2020 presidential election cycle. But back in July, well before the Musk/Twitter discourse went nuclear, Twitter’s former head of security released a shocking whistleblower report alleging, among other things: 

• that agents of foreign governments such as those of China and India had infiltrated the company and were on its payroll

• that Twitter’s security systems were egregiously porous and could have allowed external organizations and foreign governments to monitor employee laptops and steal user data such as physical location of last login, phone number, etc.

• that its database architecture was so flimsy that Twitter nearly permanently collapsed in the spring of 2021

• that Twitter’s laughably poor management of user data prevented the company from even detecting, identifying, and resolving security breaches when they happened

• and that company executives, particularly Parag Agrawal, preferred to lie about all these problems, rather than fix them

But compared to the discourse around Musk’s acquisition and The Twitter Files, the whistleblower report fell completely flat on media Twitter when it was published, and has received little attention in the narrative since. This couldn’t be because of the whistleblower’s lack of credibility: Peiter “Mudge” Zatko was a cybersecurity pioneer in the internet’s early days, and his pre-Twitter resume includes stints at Google, Stripe, and DARPA. Maybe the report’s allegations simply added up to a nothingburger? Not the case either—Mudge’s claims prompted a Congressional testimony, where he painted a picture of a company endlessly plagued by serious security issues and data breaches, exacerbated by an effete executive team that repeatedly denied, misdirected, and downplayed the extent of the problems. Let’s take a look.

Mudge joined Twitter’s executive team as the company’s Security Lead in November 2020 and, according to his report, quickly discovered that the company’s privacy practices were terrible. The company could only account for the origin and content of about 20% of its data, which for example enabled teams inside Twitter to misuse private information originally designated for security purposes in marketing campaigns. When users deleted their accounts, “not only had [their] data not been properly deleted, it couldn’t even be accounted for.” In his Congressional testimony, Mudge put it this way:

Senator Mazie Hirono (D-HI): [A]re you sure that you discovered Twitter compromises its user data long after the user has closed their accounts? In fact, you stated that the accounts are simply deactivated while the data is not fully deleted[.]

Peiter Zatko: Yes… I was told straight out by the chief privacy officer that the FTC had come and asked… does Twitter delete user information when they leave the platform? And the reason this person told me this is, he said, I need you to know this because [other] regulators are asking us and this ruse is not going to hold up. So instead of answering whether we delete user data, we intentionally have replied, we deactivate users and try to sidestep the program because we know we do not delete user data…

According to Mudge, Twitter’s information security was essentially nil. In his report, he alleges that many employees installed spyware on work computers at the behest of external organizations. And because Twitter didn’t actively monitor employee devices, it mostly discovered such spyware by accident. This spyware—essentially a malicious program that logs user activity and steals data—could have been used by rival social media firms or foreign governments to access sensitive information on users, including their addresses, phone numbers, physical location of their last login, and financial information. And the spyware’s access to Twitter’s systems, Mudge says, could have been exacerbated by the fact that many employees had disabled security updates, firewalls, and settings that would have prevented unauthorized users from remotely controlling their computers. But ironically, if unauthorized users had accessed Twitter’s systems, Twitter wouldn’t be able to know, because it didn’t log who was accessing which systems, or what they did with that access, making it impossible for Mudge (or anyone at Twitter) to identify malicious activity.

One particularly insane detail in the whistleblower report is that Twitter didn’t have separate development, test, staging, and production environments. In his testimony before Congress, Mudge described it as actively tinkering with a plane’s engines while flying with passengers. In general, to only have a production environment is a practice rarely seen in early-stage startups, and is practically unheard of at major tech companies. In Twitter’s case, it meant that about 5,000 employees had access to production. In other words, most employees, and any of the organizations that had access to Twitter’s systems via an employee’s computer, could have sought out and accessed information on a personal rival, romantic interest, or political dissident—or sabotaged Twitter itself. And again: Twitter was apparently utterly incapable of discovering if, when, or by whom any of this may have happened.

As one report to Twitter’s Board of Directors put it: “Every new employee has access to data they do not need to have access to.” Presumably that included employees known by Twitter to be agents of the Indian and Chinese governments, of which there were several. From Mudge’s Congressional testimony:

Senator Chuck Grassley, (R-IA): In your disclosure, you mentioned that the FBI notified Twitter that one of their employees was suspected of being a Chinese foreign asset. Were you and others at Twitter at all surprised by that?

Peiter Zatko: This was made aware to me… I had been told because the [corporate security team] had been contacted and told that there was at least one agent of the MSS, which is one of China’s intelligence services on the payroll inside Twitter[.]

Elsewhere in his testimony, Mudge recalled one Twitter executive’s reaction to the discovery of a foreign agent: “Well since we have one, what does it matter if we have more?”

According to the whistleblower report, one consequence of these and many other poor practices was a nearly constant stream of security breaches, with serious incidents occurring almost weekly throughout 2020. One such breach made headlines when a group of teenagers hijacked several major accounts, including those of former President Barack Obama, future Twitter CEO Elon Musk, Apple, and Uber, and used them to solicit transfers of bitcoin. The teen hackers’ level of access was “enough to achieve 'God Mode,' where the teenagers could imposter-tweet from any account they wanted,” the report explains. “Twitter's solution was to impose a system-wide shutdown of system access to all of its employees, lasting days. For about a month, hiring was paused and the company essentially shut down many basic operations to diagnose the symptoms, not the causes, of the hack.”

But in the aftermath of this attack, Twitter claimed that access to the internal tools used to take over these accounts was 

strictly limited and is only granted for valid business reasons. We have zero tolerance for misuse of credentials or tools, actively monitor for misuse, regularly audit permissions, and take immediate action if anyone accesses account information without a valid business reason.

In fact, according to the whistleblower report, access to the tools was not limited. Mudge claims that Twitter did not and could not actively monitor for misuse of credentials and tools, and if Twitter ever audited permissions, it was not with any regularity. If what Mudge says is true, Twitter’s falsehoods at the very least misled users about the safety of the Twitter experience. Mudge alleges they represent securities fraud as well.

The high-profile 2020 breach, which again was perpetrated by teenagers, was not technically demanding: the teens simply called Twitter employees, claimed to be Twitter IT, and convinced the employees to give them the necessary information to access Twitter’s internal systems. But because of Twitter’s atrocious systems architecture, it might remain woefully vulnerable to more “traditional” hacks to this day. In late 2021, a major vulnerability in the popular software development tool Log4j was discovered. Called the “most serious vulnerability I have seen in my decades-long career” by Jen Easterly, the director of the U.S. Cybersecurity and Infrastructure Security Agency, the Log4j bug represented one of the biggest software security threats ever identified, and companies across the world scrambled to protect their systems. Investigating Twitter’s exposure, Mudge found that the company had “over 300 corporate systems and more than 10,000 services” potentially at risk from the Log4j bug. Specifically, it could allow attackers to inject code into Twitter’s system, exfiltrate users’ private personal information, hijack accounts, or sabotage the platform itself. But Twitter engineers had no way of knowing whether any particular instance of the vulnerability had been fixed, in part because it could not reliably pinpoint where in their systems those thousands of vulnerabilities were. Given everything Mudge’s report has alleged about Twitter’s ability and appetite for fixing major security flaws, how likely is it that the company remains exposed to the Log4j issue today?

Mudge alleges that when he brought these and other glaring security defects to the attention of then-CTO Parag Agrawal, on whose watch many of these problems had festered for years, Agrawal stonewalled or actively undermined him. When Mudge informed Agrawal that there were as many as 3,000 failed logins to Twitter’s engineering system every day—meaning 3,000 daily attempts to access Twitter’s sensitive user data, user account access, and code—Agrawal told Mudge he had been unaware of the problem, and proceeded to do nothing about it. When Mudge shared his belief that even a brief datacenter outage could lead to the permanent collapse of Twitter with the rest of the executive team, Agrawal gave strong pushback. And when Mudge prepared to inform the Board of this potentially existential risk to the company, Agrawal told him to only present the information verbally, not in writing.

Ultimately, Mudge’s worries proved prescient: months later, a series of cascading datacenter problems did put Twitter at risk of “permanent irreparable failure,” and was only prevented by the herculean efforts of a team of Twitter engineers. Every account, every bit of code, every tweet, like, retweet, quote-tweet, DM—everything that constitutes the company, platform, and community known as Twitter—was nearly lost forever during this incident. A key piece of the global information system, poof, gone, and with no way to bring it back. A multibillion-dollar company obliterated in an instant, the biggest 404 error in history, caused not by hackers, but by incredible negligence.

After Agrawal took over as CEO in November 2021, Mudge alleges that prior to his first Board meeting as company chief, Agrawal planned to mislead the Board on a number of security and compliance issues, and required convincing not to do so. And in advance of a meeting with the Board’s Risk Committee, Agrawal announced his plans to present misleading data yet again. This time neither Mudge nor other concerned employees were able to stop him, but after Mudge noted that the events of the meeting could constitute fraud, Twitter’s Audit Committee investigated and ultimately agreed. Mudge began working on a report to correct the record with the Board, but Agrawal fired him the next day.

The above is merely a small sampling from the whistleblower report, almost every sentence of which details fairly mind-blowing allegations of Twitter’s negligence to protect its systems and user data. But it is worth asking why the grossly negligent and potentially criminal manner in which Twitter has been run is less interesting to journalists and Twitter’s power users than the machinations of the site’s moderation team, as unprincipled and capricious as it may have been. Yes, it is bad that Yoel Roth and co. seemed to look for any reason to purge undesirables from the site. But Twitter’s potential misuse of private data, infiltration by foreign spies, pathologically negligent security practices, and the threat of the total, permanent collapse of Twitter, as this report claims nearly happened, are much, much worse. Maybe the story is simply too straightforward: executives more concerned with their own advancement than the good of the company and its users—and a board uninterested in providing real oversight—nearly destroyed one of the most important communications platforms in the world. There’s no scissor, no debatable interpretive point on which turns the question of whether the libs or conservatives have been owned, and so no discourse. Whatever the reason, those of us who enjoy Twitter can only hope that the new management will be as vigorous in tackling its predecessor’s technical debt as it has its many moral failings.