> During development of our public GitHub repository, xorl, an AI coding agent committed a credential that provided access to the private Hugging Face repository
My emphasis on the accountability laundering. No, "xorl" did not do this—you did. And, if "xorl" is https://github.com/togethercomputer/xorl and `togethercomputer` is together.ai, then you wrote "xorl" and are doubly responsible.
If five years ago I had said, "During development of our public GitHub repository, my computer decided to commit a credential", this would have been immediately recognized for the embarrassing abdication of duty that it is. So, too, should this be now.
You could have easily said "the company backup script" did something, and it wouldn't be embarrassing to phrase it that way. So I don't really see an issue here.
Am I inferring correctly here, in that Dragos basically had some sort of experiment set up and running for exactly this scenario, that effectively allowed them to trace the breach to the individual?
Zero Data Retention is supposed to mean something. In practice, these prompts were retained, shared with other companies, and read by people.
The first bit of advice is on point:
Host it yourself, if you can afford it. This is the only option where nobody else gets to read your prompts, but the hardware isn’t cheap. A Mac Studio with 512 GB of memory starts at $22,299, NVIDIA’s DGX Spark now costs $6,950, a DGX Station starts at about $105,000, and an eight-GPU B300 server lists for $570,000 or more. For most teams, this only makes sense for the most sensitive work.
Despite being an NLP researcher, I never used LLMs for coding until this summer. Glad I exclusively used locally hosted models. My M5 Ultra 512GB just arrived and I'm hoping it turns out to be a wise decision for my startup.
> During development of our public GitHub repository, xorl, an AI coding agent committed a credential that provided access to the private Hugging Face repository
My emphasis on the accountability laundering. No, "xorl" did not do this—you did. And, if "xorl" is https://github.com/togethercomputer/xorl and `togethercomputer` is together.ai, then you wrote "xorl" and are doubly responsible.
If five years ago I had said, "During development of our public GitHub repository, my computer decided to commit a credential", this would have been immediately recognized for the embarrassing abdication of duty that it is. So, too, should this be now.
You could have easily said "the company backup script" did something, and it wouldn't be embarrassing to phrase it that way. So I don't really see an issue here.
You don't quote the best part:
> Our automated scanning detected the exposed credential, and we reverted the commit.
These guys are total clowns. They knew the credential was exposed and they didn't rotate it. Then they "reverted the commit"!?!
Am I inferring correctly here, in that Dragos basically had some sort of experiment set up and running for exactly this scenario, that effectively allowed them to trace the breach to the individual?
You are inferring correctly.
https://www.dragos.com/blog/ai-supply-chain-security-canary-...
That's a truly ingenious and scary investigation.
The first bit of advice is on point:
Despite being an NLP researcher, I never used LLMs for coding until this summer. Glad I exclusively used locally hosted models. My M5 Ultra 512GB just arrived and I'm hoping it turns out to be a wise decision for my startup.
But their model wasn't
Related:
Data Processing Incidents on Inference Providers
https://news.ycombinator.com/item?id=50595554
The mass hacking continues…