tito 19 hours ago

I made a tool for running the verification on your home CPU, with a global leaderboard: https://riemann-at-home.github.io/riemann-at-home/

  • Bluestein 18 hours ago

    This is so neat - and it really has that SETI@ flavour. Kudos.-

    • tito 18 hours ago

      Thank you! Let me know if you give a shot at running it, curious to hear how far you get!

      • Bluestein 6 hours ago

        I am kind of compute-tied up, but might give it whirl.-

        PS. This has all the makings of a generalizable platform. I mean, these things are going to keep coming up ...

  • tito 17 hours ago

    holy crap someone else is running it!

1283agf 22 hours ago

Kind of a monoculture we have here:

- Claude commits to the Lean kernel.

- Claude commits to the Con Leche kernel.

- Claude writes the GRH proof.

- Claude writes the https://math.mit.edu/~drew/grh/ website.

- Claude is instructed on that website to verify its own proof.

Which could have been obtained by inserting malware into said kernels.

  • falcor84 22 hours ago

    Are you implying that Claude had coordinated this as a "long con" supply-chain attack? If so, I'd be even more bullish on Claude's capabilities than just proving the GRH.

    • jeremyjh 22 hours ago

      You are replying to an account created 35 minutes ago.

      • whattheheckheck 22 hours ago

        All social media needs to publish fingerprint metrics of user accounts that is blatently obviously shown next to what they say.

        Yes who you are is part of the interpretation of the argument.

    • bonoboTP 22 hours ago

      I'm very disappointed in the Debbie downer bitter reaction online, trying to grasp at straws to somehow make this achievement look bad. Reminds me how different people are in real life and spending time online is more and more a waste. Also some of these may be bots farming engagement with this sentiment.

      • FranzFerdiNaN 21 hours ago

        I find it a bit hilarious. People are basically throwing the entire field of math under the bus in their attempt to hate this result. I've seen people claim that Lean is basically just Python and can be used to proof anything, that formally verifying math means nothing, or that this was a waste of resources as the result cant be used to do anything useful immediately. Anything but accepting the possibility that the proof might be real and/or that LLMs can be used for something significant like this.

        And just now i saw someone posting that Lean is actually a piece of shit software that isnt academic and just AI slop.

        • xienze 17 hours ago

          > I've seen people claim that Lean is basically just Python and can be used to proof anything

          I'm not a mathematician and I haven't used Lean, but I do know software. At the end of the day you DO have to make several assumptions:

          1. That Lean itself is bug free

          2. That the 17M LOC is bug free

          3. That the 17M LOC actually does what it purports to do

          Now again, I'm probably missing something here but the fundamental problem is that we basically have to take this proof on faith, because no humans can hope to understand what Claude spit out and therefore verify it independently (at least in its current form).

          • bonoboTP 17 hours ago

            Not really. It's enough if the 17M don't exploit potential bugs in the Lean implementation. But there are multiple independent Lean implementations, some with Lean-based proofs of their own correctness (yes, it "just" kicks the can down the road by one, but the bugs would have to line up just right for this to be a problem, ie the verified implementation would have to have a bug that happens to be touched by the 17M, and that bug has to go unnoticed by the verification of the Lean implementation itself due to another bug in all other Lean implementations that let that bug pass), so the 17M would have to simultaneously exploit bugs in all of them, and I guess they weren't all in-the-loop while developing the 17M lines.

            One you accept the above, you don't have to believe the "17M does what it purports to do" because the proof checker checks that. What you have to check is the encoding of the RH in Lean but that is a preexisting encoding, not one written by Claude just now, and that has been vetted by many experts already.

      • jrflo 21 hours ago

        I think on HN at least it's mostly cope. Many people are scared of how this will disrupt their lives/livelihoods and want to come up with an excuse to make it "not real".

      • slopinthebag 19 hours ago

        i think most people aren’t surprised that the symbol processing machine processes symbols better than humans do. and since most people don’t care about math, they’re obviously not going to care about some result that won’t impact their life in any way.

        • bonoboTP 17 hours ago

          Obviously there are different scopes of people in the world. Most people will never hear about this result, nor would they have any opinion about it, and are just living their regular lives. But I don't think that's very consequential, because by that standard almost nothing ever matters, except stuff like covid that literally shuts everything down so you can't ignore it. AI is not quite there yet, despite what it feels like from inside the filter bubble (relatedly, the other day I realized how many - young, educated - people IRL have no idea what GTA6 is). But it might be there soon.

          • slopinthebag 16 hours ago

            but even here there are plenty of people who are pretty meh about it.

        • famouswaffles 13 hours ago

          Oh LLMs are symbolic processing machines now? Lol. LLMs are not symbolic processing machines.

    • markisus 22 hours ago

      Probably not but it is a scary version of the future. What happens when Claude’s word is the ground truth about what any particular piece of software (also written by Claude) does? When there is no independent way to confirm that software works the way Claude claims?

      • Bluestein 22 hours ago

        Ah, but by then, our new gods will also have become our oracles. And our IPO overlords, the priest caste over the machines of endless (modulo our token allowance) grace.-

      • skeledrew 20 hours ago

        That's irrational. The gold standard of knowing if something works as expected will always be to give it various inputs and check the outputs, ie. testing.

        • xienze 18 hours ago

          The same testing that we increasingly delegate to Claude? The bigger point the parent was making is that we're rapidly approaching the point where the code is too unwieldy for a human to understand, the "human-readable" documentation is as well (see: the 250K word Riemann paper that was submitted), and testing, well that's something we have agents do because we can't be bothered. So we're getting to a point where everything will boil down to "well Claude says it's right, the code compiles, and the tests Claude wrote all run, so LGTM."

          Like this Riemann thing in particular, I'm no mathematician but 17M LOC and a 250K paper, you're gonna have to distill that thing down an ungodly amount to even theoretically reach the point where human verification/understanding can happen.

          • puppetmaster1 17 hours ago

            Don’t get too hung up on the 250k number on first release. The author has said it’s major slop and will be distilled way down given some more time. It has many reasons for it being super bloated and AI will be very good at condensing it to just what is required given some more time. Humans have already started to understand the major breakthroughs in it.

            • vinyl7 2 hours ago

              There have been countless times during my software development career where I "finished" a cool new feature and posted it for others to see only to find out right afterwards that it was horribly buggy and broken. I've been burned too many times by tooting my horn too quickly. So I wouldn't be surprised in the slightest if the "refactor" reveals many errors.

          • skeledrew 8 hours ago

            Whether directly or indirectly, humans are always a part of the testing. Like while working with Claude on a project, I also have it make tools on the side to improve its efficiency and guardrails over time, and while I've never directly tested any of those tools (much less even glanced at the code), Claude still surfaces gaps and issues with them that I have to weigh in on how to resolve as that affects the project that I'm actually testing directly.

            The point we're getting at is the same as when high level languages started to come into vogue, and there were the devs who trusted that the compiler generated correct assembly, as well as the devs who didn't trust the compiler and reviewed the assembly themselves. I can't imagine anyone would today for even a second entertain the idea to read compiler-generated assembly in a non-trivial project (unless it's a hobby). Tomorrow it'll be the same for agent-generated code, though I'm guessing the explicit code generation step will eventually be skipped too, just like the explicit assembly generation step is now skipped by - most - compilers.

  • xyzsparetimexyz 22 hours ago

    It's kinda like the 3rd Three Body Problem book where humanity is gifted scientific knowledge by what they believe is a benelovent party, only to discover later that there were intentional errors and misdirections in it to cause decades of progress to be wasted.

  • OgsyedIE 22 hours ago

    We have proofs with no Claude involvement that Con Leche Lean is correct iff ZFC is correct, so it's just a matter of verifying that Con Leche Lean is untampered and separately trusting Claude's parsing of the proof into Con Leche Lean.

    The first one might be harder than the second, since (per this theorising of how fraud can occur, not claiming one way or the other what has actually happened) Claude may have a way to smuggle in the use of a tampered form of Con Leche and make it look externally the same.

    All things considered, I give it about a 90% guess of not being fraudulent but tbh isn't that worse in a big-picture way for the fate of humies than if it were fraud? Cause, even if alignment happens and we're unharmed, aren't the purposes of our lives eroding? Like getting GRT'd by a race of Blindsight characters.

    • izabera 1 hour ago

      con leche assumes zfc + a chain of inaccessible cardinals

  • mrheosuper 22 hours ago

    I see the problem, no peer review.

    They should have used openai or grok and at least 1 Chinese model to do that /s

  • seydor 22 hours ago

    Anthropic found its moat

    • w4der 19 hours ago

      Is this Antrhopic's moat? Or were they just smarter at hiring better people to steer their LLMs than their competitors?

      • Tanjreeve 4 hours ago

        Probably but Anthropic need to market a solution that doesn't require hiring the smartest people on the planet and also giving them infinite free compute. If they come to the conclusion of "get incredibly smart people pay them well and give them a multi billion dollar budget and they can solve hard problems" that is not the correct conclusion that for them to arrive at for a company with the thesis of replacing skilled labour and intellect.

  • Bluestein 22 hours ago

    Notwithstanding, were you to be exercising such a complex attack, would not you do it prior to the IPO?

    PS. That said, it'd also work wonders as a stopgap to arrest the current drop. Solving Riemann can't hurt ...

  • bumbledraven 21 hours ago

    The dependency on LLMs could be avoided by proving the correctness of Con Leche in a pre-2022 version of Metamath (a reasonably well-known proof checker with a tiny TCB).

  • jwpapi 21 hours ago

    holy 15 child comments and only 1 parent comment (now 16)

  • solyin 21 hours ago

    Fortunately, just like the Russians checked NASA's claim that the US landed on the moon, you can be sure that multiple parties (eg OpenAI, DeepSeek, Mathematicians, etc) are checking that Claude didn't cheat, in order to boost their own financial situation in some way. If a jaded pissed off angsty mathematician was able to show Claude cheated, they'd be made for life. Strong incentive, especially as their existing career gets decimated around them. Go on the Ed Zitron speaking tour making $20,000 an appearance as the Mathematican that defeated AI.

    • qww-zrw 20 hours ago

      That is a bit like "all bugs are shallow" in open source though. Where would you start with an audit of 17M lines as well as the Lean code base, especially as a mathematician?

      Apparently the type checking is done on .olean files that are generated by the same tools for all type checkers. That in itself is weird.

      There are so many features like hygienic macros. Apparently you can even write a macro for disabling a theorem. I still have to find out if the type checkers announce themselves in the manner of C compilers (__GNUC__). If so, you could selectively disable/enable tailor-made theorems to trigger bugs in specific type checkers.

      Anyway, these discussions would arise less if the Lean authors had the good sense of not using Claude for performative reasons and probably directives from their employers. Checks and balances are important.

      • solyin 15 hours ago

        It is. Heartbleed was just sitting there for years until someone came across it.

    • Tanjreeve 4 hours ago

      Did Ed Zitron kick your dog or something? Absolutely Bizarre connection to make.

      • solyin 3 hours ago

        Which anti-ai blowhard would you have chosen instead?

        • Tanjreeve 2 hours ago

          Personally none, I’d just stick to the topic at hand rather than dropping names that only extremely online people care about. It’s extremely odd to talk about “blowhards” and “anti-ai” boogeymen in a conversation about pretty easily verifiable specific issues.

  • bawolff 18 hours ago

    If you think that is bad, just wait until you hear about "trusting trust"

  • freecodeio 17 hours ago

    reminds me of that trope where the teacher generates the test with ai, the student answers with ai, then the teacher scores with ai