This is especially problematic for a site like Twitter, which can have an unforeseen spike in traffic and user interest, says Krueger. Krueger contrasts Twitter with online retailers, where companies can prepare for major traffic events like Black Friday with some predictability. “When it comes to Twitter, they’re likely to have Black Friday on any given day at any time of the day,” he said. “On any given day, a number of news events can occur that can have a significant impact on the conversation.” Responding to that is harder to do when you get rid of up to 80% of your SRE — a number that Krueger says has been introduced in the industry but cannot be confirmed by the MIT Technology Review. The Twitter engineer agrees that the percentage sounds “reasonable”.
That engineer doesn’t see a way out of the problem — other than reversing the layoff (the company has supposedly tried to go back somewhat). “If we are pushing at a breakneck pace, everything is going to fall apart,” he said. “There is no way around that. We are accumulating technical debt much faster than before – almost as quickly as we are accumulating financial debt.”
The list is long
He offers a backward future where problems pile up as the backlog of maintenance and repair tasks grows longer and longer. “Everything will be broken. Things will break more often. Everything will be broken in the longer term. Things are going to break down in more serious ways,” he said. “Things get complicated until, eventually, it’s unusable.”
The collapse of Twitter into an unusable wreck has been some time, the engineer said, but signs of rotting are already there. It starts with the little things: “Fault on any part of whatever client they’re using; whatever service on the back end they’re trying to use. They’ll be minor annoyances at the start, but when the fixes behind the scenes are being delayed, things will build up until everyone will eventually give up. “
Krueger says that Twitter won’t blink, but we’ll start to see a larger number of tweets that don’t load and accounts that pop in and out of what seems like a whim. “I would expect anything that writes data on the back end to likely have lags, timeouts, and more complex types of failure conditions,” he said. “But they are usually more cunning. And they also often take more work to track and resolve. If you don’t have enough engineers, that’s going to be a significant problem.”
Manual retweets and slowing follower counts are signs that this has happened. Twitter’s engineers designed buggy security systems that the platform could get back up and running so that functionality wasn’t completely offline, but cut-down versions were made available instead. That’s what we’re seeing, says Krueger.
Aside from the minor glitches, the Twitter engineer believes there will be significant outages to come, thanks in part to Musk’s efforts to offload Twitter’s cloud servers in an effort to save money. infrastructure costs up to 3 million USD per day. Reuters report that this project, which came out of Musk’s war room, was called “The Deep Cut Plan”. One of the Reuters sources called the idea an “illusion”, while Alan Woodward, a professor of cybersecurity at the University of Surrey, said that “unless they over-exploit the existing system, the potential risk is high. less power and availability seems to be a logical conclusion.”
Meanwhile, when things get tough, there’s no longer the institutional knowledge to quickly fix problems as they arise. The Twitter engineer said: “A lot of the people I see who left after Friday have been there 9, 10, 11 years, which is ridiculous for a tech company. When those individuals walked out of the Twitter office, decades of knowledge about how its system worked vanished with them. (Twitter insiders, and followers from the sidelines, have previously argued that Twitter’s knowledge base is excessive concentration in the minds of a few programmers, some of whom were fired.)