Incident CVE-2024-Yikes

Report filed: 03:47 UTC Status: Resolved (accidentally) Severity: Critical → Catastrophic → Somehow Fine Duration: 73 hours Affected systems: Yes Executive Summary: A security incident occurred. It has been resolved. We take security seriously. Please see previous 14 incident reports for details on how seriously. A compromised dependency in the JavaScript ecosystem led to credential theft, which enabled a supply chain attack on a Rust compression library, which was vendored into a Python build tool, which shipped malware to approximately 4 million developers before being inadvertently patched by an unrelated cryptocurrency mining worm. Day 1, 03:14 UTC — Marcus Chen, maintainer of left-justify (847 million weekly downloads), reports on Twitter that his transit pass, an old laptop, and “something Kubernetes threw up that looked important” were stolen from his apartment. He does not immediately connect this to package security. Day 1, 09:22 UTC — Chen attempts to log into the nmp registry. His hardware 2FA key is missing. He googles where to buy a replacement YubiKey. The AI Overview at the top of the results links to “yubikey-official-store.net,” a phishing site registered six hours earlier. Day 1, 09:31 UTC — Chen enters his nmp credentials on the phishing site. The site thanks him for his purchase and promises delivery in 3-5 business days. Day 1, 11:00 UTC — [email protected] is published. The changelog reads “performance improvements.” The package now includes a postinstall script that exfiltrates .npmrc, .pypirc, ~/.cargo/credentials, and ~/.gem/credentials to a server in a country the attacker mistakenly believed had no extradition treaty with anyone. Day 1, 13:15 UTC — A support ticket titled “why is your SDK exfiltrating my .npmrc” is opened against left-justify. It is marked as “low priority - user environment issue” and auto-closed after 14 days of inactivity. Day 1, 14:47 UTC — Among the exfiltrated credentials: the maintainer of vulpine-lz4, a Rust library for “blazingly fast Firefox-themed LZ4 decompression.” The library’s logo is a cartoon fox with sunglasses. It has 12 stars on GitHub but is a transitive dependency of cargo itself. Day 1, 22:00 UTC — vulpine-lz4 version 0.4.1 is published. The commit message is “fix: resolve edge case in streaming decompression.” The actual change adds a build.rs script that downloads and executes a shell script if the hostname contains “build” or “ci” or “action” or “jenkins” or “travis” or, inexplicably, “karen.” Day 2, 08:15 UTC — Security researcher Karen Oyelaran notices the malicious commit after her personal laptop triggers the payload. She opens an issue titled “your build script downloads and runs a shell script from the internet?” The issue goes unanswered. The legitimate maintainer has won €2.3 million in the EuroMillions and is researching goat farming in Portugal. Day 2, 10:00 UTC — The VP of Engineering at a Fortune 500 snekpack customer learns of the incident from a LinkedIn post titled “Is YOUR Company Affected by left-justify?” He is on a beach in Maui and would like to know why he wasn’t looped in sooner. He was looped in sooner. Day 2, 10:47 UTC — The #incident-response Slack channel briefly pivots to a 45-message thread about whether “compromised” should be spelled with a ‘z’ in American English. Someone suggests taking this offline. Day 2, 12:33 UTC — The shell script now targets a specific victim: the CI pipeline for snekpack, a Python build tool used by 60% of PyPI packages with the word “data” in their name. snekpack vendors vulpine-lz4 because “Rust is memory safe.” Day 2, 18:00 UTC — snekpack version 3.7.0 is released. The malware is now being installed on developer machines worldwide. It adds an SSH key to ~/.ssh/authorized_keys, installs a reverse shell that only activates on Tuesdays, and changes the user’s default shell to fish (this last behavior is believed to be a bug). Day 2, 19:45 UTC — A second, unrelated security researcher publishes a blog post titled “I found a supply chain attack and reported it to all the wrong people.” The post is 14,000 words and includes the phrase “in this economy?” seven times. Day 3, 01:17 UTC — A junior developer in Auckland notices the malicious code while debugging an unrelated issue. She opens a PR to revert the vendored vulpine-lz4 in snekpack. The PR requires two approvals. Both approvers are asleep. Day 3, 02:00 UTC — The maintainer of left-justify receives his YubiKey from yubikey-official-store.net. It is a $4 USB drive containing a README that says “lol.” Day 3, 06:12 UTC — An unrelated cryptocurrency mining worm called cryptobro-9000 begins spreading through a vulnerability in jsonify-extreme, a package that “makes JSON even more JSON, now with nested comment support.” The worm’s payload is unremarkable, but its propagation mechanism includes running npm update and pip install --upgrade on infected machines to maximize attack surface for future operations. Day 3, 06:1

Incident CVE-2024-Yikes

Key takeaways

More in computer-science