HomeTechnologyHow one volunteer stopped...

How one volunteer stopped a backdoor from exposing Linux systems worldwide


Linux, the most widely used open source operating system in the world, narrowly escaped a massive cyber attack over Easter weekend, all thanks to one volunteer.

The backdoor had been inserted into a recent release of a Linux compression format called XZ Utils, a tool that is little-known outside the Linux world but is used in nearly every Linux distribution to compresses large files, making them easier to transfer. If it had spread more widely, an untold number of systems could have been left compromised for years.

And as Ars Technica noted in its exhaustive recap, the culprit had been working on the project out in the open.

The vulnerability, inserted into Linux’s remote log-in, only exposed itself to a single key, so that it could hide from scans of public computers. As Ben Thompson writes in Stratechery.  “the majority of the world’s computers would be vulnerable and no one would know.”

The story of the XZ backdoor’s discovery starts in the early morning of March 29th, as San Francisco-based Microsoft developer Andres Freund posted on Mastodon and sent an email to OpenWall’s security mailing list with the heading: “backdoor in upstream xz/liblzma leading to ssh server compromise.”

Freund, who volunteers as a “maintainer” for PostgreSQL, a Linux-based database, noticed a few strange things over the past few weeks while running tests. Encrypted log-ins to liblzma, part of the XZ compression library, were using up a ton of CPU. None of the performance tools he used revealed anything, Freund wrote on Mastodon. This immediately made him suspicious, and he remembered an “odd complaint” from a Postgres user a couple of weeks earlier about Valgrind, Linux’s program that checks for memory errors. 

After some sleuthing, Freund eventually discovered what was wrong. “The upstream xz repository and the xz tarballs have been backdoored,” noted Freund in his email. The malicious code was in versions ​​5.6.0 and 5.6.1 of the xz tools and libraries. 

Shortly after, enterprise opensource software company Red Hat sent out an emergency security alert for users of Fedora Rawhide and Fedora Linux 40. Ultimately, the company concluded that the beta version of Fedora Linux 40 contained two affected versions of the xz libraries. Fedora Rawhide versions likely received versions 5.6.0 or 5.6.1 as well.

PLEASE IMMEDIATELY STOP USAGE OF ANY FEDORA RAWHIDE INSTANCES for work or personal activity. Fedora Rawhide will be reverted to xz-5.4.x shortly, and once that is done, Fedora Rawhide instances can safely be redeployed.

Although a beta version of Debian, the free Linux distribution, contained compromised packages, its security team acted swiftly to revert them. “Right now no Debian stable versions are known to be affected,” wrote Debian’s Salvatore Bonaccorso in a security alert to users on Friday evening.

Freund later identified the person who submitted the malicious code as one of two main xz Utils developers, known as JiaT75, or Jia Tan. “Given the activity over several weeks, the committer is either directly involved or there was some quite severe compromise of their system. Unfortunately the latter looks like the less likely explanation, given they communicated on various lists about the “fixes” mentioned above,” wrote Freund in his analysis, after linking several workarounds that were made by JiaT75.

JiaT75 was a familiar name: they’d worked side-by-side with the original developer of .xz file format, Lasse Collin, for a while. As programmer Russ Cox noted in his timeline, JiaT75 started by sending apparently legitimate patches to the XZ mailing list in October of 2021.

Other arms of the scheme unfolded a few months later, as two other identities, Jigar Kumar and Dennis Ens, began emailing complaints to Collin about bugs and the project’s slow development. However, as noted in reports by Evan Boehs and others, “Kumar” and “Ens” were never seen outside the XZ community, leading investigators to believe both are fakes that existed only to help Jia Tan get into position to deliver the backdoored code.

An email from “Jigar Kumar” pressuring the developer of XZ Utils to relinquish control of the project.
Image: Screenshot from The Mail Archive

“I am sorry about your mental health issues, but its important to be aware of your own limits. I get that this is a hobby project for all contributors, but the community desires more,” wrote Ens in one message, while Kumar said in another that “Progress will not happen until there is new maintainer.”

In the midst of this back and forth, Collins wrote that “I haven’t lost interest but my ability to care has been fairly limited mostly due to longterm mental health issues but also due to some other things,” and suggested Jia Tan would take on a bigger role. “It’s also good to keep in mind that this is an unpaid hobby project,” he concluded. The emails from “Kumar” and “Ens” continued until Tan was added as a maintainer later that year, able to make alterations, and attempt to get the backdoored package into Linux distributions with more authority.

The xz backdoor incident and its aftermath are an example of both the beauty of open source and a striking vulnerability in the internet’s infrastructure.

A developer behind FFmpeg, a popular open-source media package, highlighted the problem in a tweet, saying “The xz fiasco has shown how a dependence on unpaid volunteers can cause major problems. Trillion dollar corporations expect free and urgent support from volunteers.” And they brought receipts, pointing out how they dealt with a “high priority” bug affecting Microsoft Teams.

Despite Microsoft’s dependence on its software, the developer writes, “After politely requesting a support contract from Microsoft for long term maintenance, they offered a one-time payment of a few thousand dollars instead…investments in maintenance and sustainability are unsexy and probably won’t get a middle manager their promotion but pay off a thousandfold over many years.”

Details of who is behind “JiaT75,” how they executed their plan, and the extent of the damage are being unearthed by an army of developers and cybersecurity professionals, both on social media and online forums. But that happens without direct financial support from many of the companies and organizations who benefit from being able to use secure software.





Source link

Most Popular

LEAVE A REPLY

Please enter your comment!
Please enter your name here

More from Author

Read Now

Fever select Caitlin Clark with No. 1 pick in WNBA Draft

BROOKLYN, N.Y. — Eight days after completing one of the greatest collegiate basketball careers ever recorded, Caitlin Clark was selected with the No. 1 pick in Monday’s WNBA Draft by the Indiana Fever.Clark is poised to not only help the Fever return to the postseason for...

In 1998, Bollywood star killed two blackbucks. A gang still seeks revenge.

Khan has been in the crosshairs of an armed gang since 1998 for shooting two blackbucks on a recreational hunting trip.Two members of a notorious criminal gang have been arrested for firing at the home of Bollywood actor Salman Khan in apparent retaliation for the superstar’s...

14 More Games Land on PS Plus Extra, Premium Today

As always, we ran a Poll over the weekend to gather your thoughts on the line-up, and (almost) as always, one-third of you responded by saying "it's a crap selection". 26 per cent of voters were at least somewhat happy with the selection, with Dave the...

Where Biden’s billions in chipmaking money have gone — and what could be next

The White House announcement this week of up to $6.4 billion in chipmaking cash for Samsung (005930.KS) marks a move to the next phase of the Biden administration's effort to bring semiconductor manufacturing back to the US.The focus so far has largely been on leading-edge logic...

WNBA expansion headed for 16 teams. Portland, Toronto and Philly are options

NEW YORK — The WNBA is looking to continue growing in the next few years. The league intends to reach 16 teams by 2028, commissioner Cathy Engelbert said before the draft, confirming The Athletic report from earlier Monday.The league announced earlier this year that the Bay...

Microsoft Makes High-Stakes Play in Tech Cold War With Emirati A.I. Deal

Microsoft on Tuesday said it would make a $1.5 billion investment in G42, an artificial intelligence giant in the United Arab Emirates, in a deal largely orchestrated by the Biden administration to box out China as Washington and Beijing battle over who will exercise technological influence...

A hockey puck hurtled toward her son’s head. A fan jumped to block it.

Just over a minute was left on the clock during a hockey game in Cleveland Thursday when a player struck the puck, sending it barreling into the stands and toward Asia Davis’s 4-year-old son. That’s when another fan jumped and extended his right arm to block...

Thailand’s Finance Ministry to lead 17 agencies in considering House report on legalizing casinos

Thailand’s Ministry of Finance has been appointed lead agency to consider a House committee study recommending the legalization of casino gaming but will work with 16 other agencies in doing so, local media reports. According to English-language outlet The Nation, the appointment was ordered by way of...

WWE RAW Results Coverage, Reactions and Highlights for April 15, 2024

It’s Monday night, and you know what that means! WWE RAW will go down tonight, and Ringside News has got you covered with live play-by-play results coverage for the April 15, 2024 episode. The start time for WWE RAW is 8:00 PM EST. Match-by-match highlights...

Coachella Weekend 1 Recap, J Balvin’s Out Of This World Coachella Set, Metro Boomin & Future Top Hot 100 | Billboard News

It’s Monday April 15th, Billboard counts down the top 10 on the Hot 100, we’re running down all the headliners and special guest performers and some notable audience members that went down at the first weekend of Coachella, J Balvin gives us a behind the scene...