Does GIMP have a very restrictive AI policy or something? I get not wanting to ship a gigantic pile of UI tests without thorough review, but basically any model released in the last six months would be able to write UI tests covering every single control in an afternoon.
It’s frustrating to create tests with models, not sure if it’s everyone’s experience, but every time I asked it to create tests, it makes the tests conform to the sometimes broken code. Rather than calling out what the logic/code is wrong, it will make the tests conform to whatever is already broken.
It can’t magically know expectations. If the code is well documented or there’s decent associated documentation, I’ve found it has no issue writing tests.
Tests are usually the best documentation and often the only documentation. If humans bother to write anything it should be the tests, not ambiguous human language...beyond the high level 1-liner overviews.
I'm actually surprised at how well non-frontier/local models are able to infer the intent of code from the structure, method names and sparse random comments to the point that they are capable of writing reasonable tests.
Prodding an agent vs doing interesting things with my time? What should I choose?
To be fair, some people like to prod. But, for me, I'd rather take hours reading manuals than do trial and error prodding for 30 minutes (even if I knew it would be that short). Yes it's not optimal in terms of time, but it's better for my soul and well being.
It's a good use of your time to ask the agent to reflect and update it's skills if you intervene to manually steer it. Why wouldn't you want it to learn from your guidance?
> Yes it's not optimal in terms of time, but it's better for my soul and well being.
It is also optimal in terms of time in the long run. The prodder will have to prod over and over again because he has learned nothing. You on the other hand will actually understand what is going on and will be able to either do it yourself or give precise instructions to the machine next time. Don't let yourself become a meat proxy.
The problem I've had is even when it can explain back the issue, it still writes for itself awful instructions with extra fluff, strange points of focus, and borderline misinterpretations of what I said.
Maybe that's not a big deal, they can still be generally useful, but they seem important to commit with the project if they will be used going forward. I hate the idea of a massive amount of slop riding along with the code, and slop begets slop.
The idea I'm chasing now is writing the documents myself with the hope that enough examples and a document about writing documents will be enough to allow it to write clearly.
True. It's tough useful to build non-regression tests, that is build tests that test the current behavior, assuming it correct, to possibly spot bugs in the future.
Yeah, i'm sure they want to burn their own tokens, deny others to use AI, to hog all that foss money for themselves. Definitely not related to not wanting to review 10k ai generated prs no siree
thing is, we hav emuseums, well crafted historical relics.
Having projects with properly scaffolded code, et al, will be valuable when the FAA grounds all planes for a week because of <insert all the grusome things you know this administration turns a blind eye to>.
It absolutely is related to not wanting to review 10k AI-generated PRs. Those PRs are definitely going to be lower quality and more likely to contain backdoors than ones produced by your own AI. The back-and-forth during the review is also going to be much slower than the equivalent back-and-forth in the chat window.
If money/tokens are not an issue, never accepting external AI-made PRs makes perfect sense.
Weirdly that is likely an effective filter for low effort, drive-by PRs. Either you are telling the truth and didn't use LLMs, or you at least were paying enough attention to tell your agent to lie and check the box.
Not using AI to at least writes tests in this day and age is crazy work. I started a desktop application 2 months ago and it has almost 3,000 tests now, MANY of them UI tests.
AI is a bit too excellent at writing tests. The number of tests doesn't really convey any information on test coverage. At least I've found that agents that naively improve codebases will write tests for pretty much anything, including irrelevant behavior that has little value in ensuring stability, security or correct behavior. It would probably help to run a test review stage with a strong model, where agents attempt to correct and consolidate tests to guard behavior rather than ensuring that theoretical code paths are followed.
In my experience, designing tests cases to ensure that they're othogonal and verify your invariants (domain or technical) is very mentally taxing and not something you can truly delegate. The actual typing maybe, but testing is a lot of duplicate code and you can go far with just copy-pasting stuff around and building helpers.
I my opinion, AI only changes the ease of writing these tests, not really the quality.
I find that the human written tests on my project were the same. We were guarding not just the business behavior but also weird cases that have nothing to do with business, and also self-evident cases that basically boil down to x=x or assert(true).
Not to mention the stupid side-effect of not updating mocks when the APIs move but tests don't test that part so they are okay for a lot of time, and suddenly after a non-related change they are not.
I agree, some of my small projects have had an explosion of tests. It usually doesn't cause my any problems other than waiting for tests to run, but that isn't great.
Using AI to write tests in this day and age is crazy work. The machine is great at writing tests, but the quality of the tests is awful. It will write redundant tests, it will write tests that assert too many thing in one, it will not write tests that need to be tested. It's the part where I have to clean up the most every time. Quantity is not quality, and you really don't want to skip on the quality of your quality assurance. I'm writing a React + Django full-stack application, so there should be more than enough training data out there.
I created all the designs for flyers, announcements, invitations etc. in Affinity for my wife. I migrated to Linux and didn't like any of the apps. I was lost.
Then Affinity announced they'll add Linux support.
But for some time now my wife just uses Codex/Claude to do her day-to-day nothing special design work.
I feel, just like ArtCraft, Gimp will not be relevant in some years (months?).
Are you me?! I've also happily used GIMP for that long, just swap out Affinity for Paint Shop Pro or Adobe CS. I would probably feel just as lost going back to them now.
There are people who want to use Gimp (or Affinity etc), and there are people who want to have used Gimp. It’ll only lose relevance for the second group.
AI bros can't stand the thought that not everyone agrees with them about LLMs. It's like a religion to them, they feel compelled to get everyone else on board. Never mind that if they are right, it will become apparent to everyone without them having to preach.
It's funny now you instantly triggered someone lol.
Also, it was same exact thing with crypto. Same type of men who think that they'll somehow outsmart everyone and win a lottery with it, be better and richer than everyone else.
Wonder if gold rush people were just as insufferable.
Seriously. This comment thread is why I’m spending less time on HN and more on lobsters. All these fresh accounts losing their minds over Gimp not using ai, even when that’s not what the article is even about.
Funny how GIMP spawned the Gimp Toolkit, aka GTK, which is widely used in Linux, and yet GIMP's own UI seems to have long languished as a rather outdated design. Glad that they have tests, though, and hope they continue to improve it.
They made a blog post to announce that they’d written thirteen unit tests? Sure, there’s value in handmade software and all that but GIMP is not really a particularly excellent piece of software - if anything was to benefit from unleashing the agents it would be GIMP.
Lucky you. Making things testable seemed hard to me quite often and ensuring that the tests covered all the difficult bits without effectively mocking it to death so that one was testing almost nothing.
The blog article doesn't say they have written thirteen unit tests.
> *GIMP already has unit tests*. A unit test checks one small part of the code, for example how GIMP reads an XCF file. But our unit tests were not opening a real window. So it coult not tell us if a dialog opens, or if a menu item did its job.
> Note that GIMP *already had tests of GUI code*, usually interacting directly with the core code, and sometimes even simulating input devices’ events (pointer movements, clicks, drags…). But nothing can beat interacting with the real interface. Furthermore such legacy tests directly hijacking core code come with their own problems, such as breaking all too often (and not because they detected a bug), all while not being able to detect whole categories of bugs.
> UI tests check the program as a whole, in the same way as you use it.
It'd be really interesting to look back the comparison between projects with strict non-AI policies (e.g. GIMP) and ArtCraft in 2027.
My personal prediction is that ArtCraft will blow all the old non-AI projects in similar fields out of the water and GIMP will be remembered as a thing of the past. But if I got proven wrong I'd be happy too: at least we will know the value of non-AI policies.
GIMP is far from the best FOSS image editor for many use cases. IMO for many simple editing tasks that people usually need unless they're professional photo editors, even Inkscape, a primarily vector editor, is a better photo editor than GIMP.
Inkscape is what made me believe ArtCraft would be the clear winner, btw. It's very hard to imagine using Inkscape ever again since I tried VectorCraft. (And I'm fully aware that some Path Effects can only be done quickly in Inkscape.)
Timely, as I removed Gimp from my Macbook today after successfully testing PhotoCraft [1] for a week (the AI Photoshop clone written in open-source rust which has had an absured number of updates since you last tried it).
It makes me so HAPPY to remove the dinosaur Gimp, and to run "Adobe" software for free (it's good enough for me, maybe good enough for you). I expect to lose Internet points for saying that.
I have a long history with Gimp. I created Gimpshop... the Gimp maintainers did not appreciate that little stunt, I'll tell ya! :D
I have canceled my Adobe subscription as of last month and have fully moved over to Photocraft. Even if development of Photocraft stopped today, I would be able to do whatever I need to get done in Photoshop without paying Adobe or wrestling with Gimp. It's good enough for me as well. I had been using Adobe Photoshop for more than 30 years now and was bought in enough to spend a significant amount of effort making Gimp act like Photoshop. Bye Photoshop, bye Gimp, bye Gimpshop.
Same with VectorCraft. I spent over a decade looking for an Illustrator alternative. Muscle memory and specific workflows won out. VectorCraft is extremely close.
Components, sure. But not the UI. Building a toolkit is not the same as building a UI. You can use the toolkit to build a good UI and you can use the toolkit to build a bad UI.
The tests should simply take a screenshot and send it to an LLM with the question "Is the UI absolutely horrible?". If the answer is yes then the UI is working as expected.
> We have 13 tests at this time.
Does GIMP have a very restrictive AI policy or something? I get not wanting to ship a gigantic pile of UI tests without thorough review, but basically any model released in the last six months would be able to write UI tests covering every single control in an afternoon.
It’s frustrating to create tests with models, not sure if it’s everyone’s experience, but every time I asked it to create tests, it makes the tests conform to the sometimes broken code. Rather than calling out what the logic/code is wrong, it will make the tests conform to whatever is already broken.
It can’t magically know expectations. If the code is well documented or there’s decent associated documentation, I’ve found it has no issue writing tests.
Tests are usually the best documentation and often the only documentation. If humans bother to write anything it should be the tests, not ambiguous human language...beyond the high level 1-liner overviews.
I'm actually surprised at how well non-frontier/local models are able to infer the intent of code from the structure, method names and sparse random comments to the point that they are capable of writing reasonable tests.
I recommend iterating on your prompts.
Prod your agent first until it can explain back to you what's wrong with the tests.
Then ask your agent to write a better prompt for writing tests (and save it to an .md file)
Then reference that .md file and iterate on the tests again. It may take a couple of tries, but I've had good results with this strategy.
Ended up with a whole prompting framework.
Prodding an agent vs doing interesting things with my time? What should I choose?
To be fair, some people like to prod. But, for me, I'd rather take hours reading manuals than do trial and error prodding for 30 minutes (even if I knew it would be that short). Yes it's not optimal in terms of time, but it's better for my soul and well being.
It's a good use of your time to ask the agent to reflect and update it's skills if you intervene to manually steer it. Why wouldn't you want it to learn from your guidance?
Perhaps because they're more interested in learning something themselves.
> Yes it's not optimal in terms of time, but it's better for my soul and well being.
It is also optimal in terms of time in the long run. The prodder will have to prod over and over again because he has learned nothing. You on the other hand will actually understand what is going on and will be able to either do it yourself or give precise instructions to the machine next time. Don't let yourself become a meat proxy.
The problem I've had is even when it can explain back the issue, it still writes for itself awful instructions with extra fluff, strange points of focus, and borderline misinterpretations of what I said.
Maybe that's not a big deal, they can still be generally useful, but they seem important to commit with the project if they will be used going forward. I hate the idea of a massive amount of slop riding along with the code, and slop begets slop.
The idea I'm chasing now is writing the documents myself with the hope that enough examples and a document about writing documents will be enough to allow it to write clearly.
> Rather than calling out what the logic/code is wrong, it will make the tests conform to whatever is already broken.
It is the same like humans: if they say your code is wrong, you will hire someone else who doesn't find so many bugs. It doesn't look good.
True. It's tough useful to build non-regression tests, that is build tests that test the current behavior, assuming it correct, to possibly spot bugs in the future.
Turns out you're right! No AI allowed https://gitlab.gnome.org/GNOME/gimp/-/blob/master/.gitlab/me...
> for contributors without Developer role
From the blame PR:
> Of course, core developers don't need to check this monotomous checklist. This is only useful for the newcomers.
So this is saying core devs already know not to use AI, only new developers shouldn't.
It's quaint at this point. Embrace the future.
The future seems to be blocking external contributors and running the equivalent AI agents yourself.
Yeah, i'm sure they want to burn their own tokens, deny others to use AI, to hog all that foss money for themselves. Definitely not related to not wanting to review 10k ai generated prs no siree
thing is, we hav emuseums, well crafted historical relics.
Having projects with properly scaffolded code, et al, will be valuable when the FAA grounds all planes for a week because of <insert all the grusome things you know this administration turns a blind eye to>.
It absolutely is related to not wanting to review 10k AI-generated PRs. Those PRs are definitely going to be lower quality and more likely to contain backdoors than ones produced by your own AI. The back-and-forth during the review is also going to be much slower than the equivalent back-and-forth in the chat window.
If money/tokens are not an issue, never accepting external AI-made PRs makes perfect sense.
Weird, the author of the MR doesn't have a Developer role, he is a Contributor, and yet his commits look very much AI generated, look at this one for instance https://gitlab.gnome.org/GNOME/gimp/-/merge_requests/2909/di...
Or is it just me?
Huh, that's fascinating. I wonder if this policy is also meant to apply to the GIMP blog - if so, this very article is probably a violation.
RIP GIMP, Asahi, Codeberg et al
Weirdly that is likely an effective filter for low effort, drive-by PRs. Either you are telling the truth and didn't use LLMs, or you at least were paying enough attention to tell your agent to lie and check the box.
Not using AI to at least writes tests in this day and age is crazy work. I started a desktop application 2 months ago and it has almost 3,000 tests now, MANY of them UI tests.
AI is a bit too excellent at writing tests. The number of tests doesn't really convey any information on test coverage. At least I've found that agents that naively improve codebases will write tests for pretty much anything, including irrelevant behavior that has little value in ensuring stability, security or correct behavior. It would probably help to run a test review stage with a strong model, where agents attempt to correct and consolidate tests to guard behavior rather than ensuring that theoretical code paths are followed.
In my experience, designing tests cases to ensure that they're othogonal and verify your invariants (domain or technical) is very mentally taxing and not something you can truly delegate. The actual typing maybe, but testing is a lot of duplicate code and you can go far with just copy-pasting stuff around and building helpers.
Designing for testing helps as well. Pure functions for core logic, coding against interfaces etc.
I my opinion, AI only changes the ease of writing these tests, not really the quality.
I find that the human written tests on my project were the same. We were guarding not just the business behavior but also weird cases that have nothing to do with business, and also self-evident cases that basically boil down to x=x or assert(true).
Not to mention the stupid side-effect of not updating mocks when the APIs move but tests don't test that part so they are okay for a lot of time, and suddenly after a non-related change they are not.
I agree, some of my small projects have had an explosion of tests. It usually doesn't cause my any problems other than waiting for tests to run, but that isn't great.
Look, guys, you are literally allowed to fork and go ham on the code.
Yeah if people can make a better and more popular Gimp in an afternoon, go do it.
Gimp has millions of downloads this month. They are doing something right.
Using AI to write tests in this day and age is crazy work. The machine is great at writing tests, but the quality of the tests is awful. It will write redundant tests, it will write tests that assert too many thing in one, it will not write tests that need to be tested. It's the part where I have to clean up the most every time. Quantity is not quality, and you really don't want to skip on the quality of your quality assurance. I'm writing a React + Django full-stack application, so there should be more than enough training data out there.
Some people refuse to use AI for ethical, environmental, or social reasons. It isn't crazy to want your projects to be built by humans
> any model released in the last six months would be able to write UI tests covering every single control in an afternoon.
Microsoft also seems to use "models" to "test" its programs. Look how shitty they are.
Why "models" in quotation marks?
This raises questions of AI ethics; namely, is it conscionable to subject AI to GIMP’s UI?
Ha! We'll know that LLMs are truly intelligent when they start to despair at the moronic conversations we force them to have.
> So it coult not tell us if a dialog opens,
It feels good when I see spelling mistakes. It feels organic, human-sourced, non-AI. It feels like a good old village again.
BTW I love GIMP for everything it did to the world of graphics work. Such an awesome thing. Thank you folks.
This is really good.
So much of this tech like ui tests was built for the web, and it can be a pain to get working for plain old GUI apps
I never bothered with UI tests for my desktop apps. It always felt like it would be more trouble than it was worth.
I created all the designs for flyers, announcements, invitations etc. in Affinity for my wife. I migrated to Linux and didn't like any of the apps. I was lost.
Then Affinity announced they'll add Linux support.
But for some time now my wife just uses Codex/Claude to do her day-to-day nothing special design work.
I feel, just like ArtCraft, Gimp will not be relevant in some years (months?).
For some people GIMP is irrelevant for the last ~25 years. I was using it in 2000, I'm still using it in 2026.
It's a matter of getting used to things. I feel lost in Affinity tools, for example, despite having a license, I use them less and less.
Are you me?! I've also happily used GIMP for that long, just swap out Affinity for Paint Shop Pro or Adobe CS. I would probably feel just as lost going back to them now.
My son learned GIMP in school and refuses to learn anything else.
Maybe he will wise up with age
You switched to Linux, but you don't want to use Linux things. Why not just stick with Windows (or whatever you were using before)?
Because I do mainly other things for which Linux is suited better.
There are people who want to use Gimp (or Affinity etc), and there are people who want to have used Gimp. It’ll only lose relevance for the second group.
Use this: https://getartcraft.com/ It is open source, built with Rust and very fast and works great and is very similar to Adobe Photoshop
Tell me you didn't read the whole comment you replied to without telling me...
Great on GIMP for continuing on and attempting to improve things, you'd think the GIMP team killed people's dogs with some of these comments..
AI bros can't stand the thought that not everyone agrees with them about LLMs. It's like a religion to them, they feel compelled to get everyone else on board. Never mind that if they are right, it will become apparent to everyone without them having to preach.
It's funny now you instantly triggered someone lol.
Also, it was same exact thing with crypto. Same type of men who think that they'll somehow outsmart everyone and win a lottery with it, be better and richer than everyone else.
Wonder if gold rush people were just as insufferable.
Yeah they always get the flak, even though it's not like people HAVE to use Gimp.
I also hated the UI, coming from Photoshop, but what I found is that when I go back to Photoshop, I'm just as lost as I was before switching to Gimp.
I'm glad that they hadn't lost their courage over the years. Gimp is a solid project, and the FOSS ecosystem would lose a real gem if they stopped.
Seriously. This comment thread is why I’m spending less time on HN and more on lobsters. All these fresh accounts losing their minds over Gimp not using ai, even when that’s not what the article is even about.
Funny how GIMP spawned the Gimp Toolkit, aka GTK, which is widely used in Linux, and yet GIMP's own UI seems to have long languished as a rather outdated design. Glad that they have tests, though, and hope they continue to improve it.
Ah! I always thought it was Gnome Toolkit, didn't even bother to check. Thanks
It's GNOME Toolkit now, you're not wrong, but originally was GIMP.
Well it is, regardless of the name (I think "GTK" officially doesn't stand for anything now); to quote https://www.gtk.org/ -
> GTK is a free and open-source project maintained by GNOME and an active community of contributors.
and
> GTK and the GTK logo are trademarks of the GNOME Foundation.
It used to be called GTK+ to indicate that it's "more than the toolkit for GIMP".
Tests are a good first step, here's hoping it'll get a UI next :)
They made a blog post to announce that they’d written thirteen unit tests? Sure, there’s value in handmade software and all that but GIMP is not really a particularly excellent piece of software - if anything was to benefit from unleashing the agents it would be GIMP.
I’d go so far as to say that in general, adding tests for legacy software before a refactor or rewrite is the perfect time to put the agent to work.
> if anything was to benefit from unleashing the agents
AI can write tests now ? For its own software ? Can it evaluate the results ? Correctly ? /s
Writing tests never was the hard part of the job anyway.
Lucky you. Making things testable seemed hard to me quite often and ensuring that the tests covered all the difficult bits without effectively mocking it to death so that one was testing almost nothing.
Yup especially if you don't care if they actually show correctness or not
The blog article doesn't say they have written thirteen unit tests.
> *GIMP already has unit tests*. A unit test checks one small part of the code, for example how GIMP reads an XCF file. But our unit tests were not opening a real window. So it coult not tell us if a dialog opens, or if a menu item did its job.
> Note that GIMP *already had tests of GUI code*, usually interacting directly with the core code, and sometimes even simulating input devices’ events (pointer movements, clicks, drags…). But nothing can beat interacting with the real interface. Furthermore such legacy tests directly hijacking core code come with their own problems, such as breaking all too often (and not because they detected a bug), all while not being able to detect whole categories of bugs.
> UI tests check the program as a whole, in the same way as you use it.
They must have set up a reusable harness, and it's the most important part.
Did you read the article? These aren’t unit tests. They are UI tests which are pretty hard to do well with desktop apps.
Especially given it isn't as easy as using Cypress.
we forget, but before AI this would have been at least 6 months of work, at least 50 meetings, and at least 7 separate sprints.
It'd be really interesting to look back the comparison between projects with strict non-AI policies (e.g. GIMP) and ArtCraft in 2027.
My personal prediction is that ArtCraft will blow all the old non-AI projects in similar fields out of the water and GIMP will be remembered as a thing of the past. But if I got proven wrong I'd be happy too: at least we will know the value of non-AI policies.
GIMP is far from the best FOSS image editor for many use cases. IMO for many simple editing tasks that people usually need unless they're professional photo editors, even Inkscape, a primarily vector editor, is a better photo editor than GIMP.
Inkscape is what made me believe ArtCraft would be the clear winner, btw. It's very hard to imagine using Inkscape ever again since I tried VectorCraft. (And I'm fully aware that some Path Effects can only be done quickly in Inkscape.)
What is the point of image editors? A fleet of AI artists should be doing it.
It sounds like they are nearly end to end user interface (UI) tests, not just unit tests.
Timely, as I removed Gimp from my Macbook today after successfully testing PhotoCraft [1] for a week (the AI Photoshop clone written in open-source rust which has had an absured number of updates since you last tried it).
It makes me so HAPPY to remove the dinosaur Gimp, and to run "Adobe" software for free (it's good enough for me, maybe good enough for you). I expect to lose Internet points for saying that.
[1] https://getartcraft.com/apps/photocraft
Regular users would use a lot more open source software if more of it was like photocraft and less of it was like GIMP.
I have a long history with Gimp. I created Gimpshop... the Gimp maintainers did not appreciate that little stunt, I'll tell ya! :D
I have canceled my Adobe subscription as of last month and have fully moved over to Photocraft. Even if development of Photocraft stopped today, I would be able to do whatever I need to get done in Photoshop without paying Adobe or wrestling with Gimp. It's good enough for me as well. I had been using Adobe Photoshop for more than 30 years now and was bought in enough to spend a significant amount of effort making Gimp act like Photoshop. Bye Photoshop, bye Gimp, bye Gimpshop.
Same with VectorCraft. I spent over a decade looking for an Illustrator alternative. Muscle memory and specific workflows won out. VectorCraft is extremely close.
This and also and Affinity announced Linux version. Hopefully I'll never have to use GIMP again. The UI is soooo bad.
GIMP... didn't have ui tests before?
I'm surprised, tbh. I understand that UI is very much an afterthought in GIMP, but I figured it'd still at least have tests.
afterthought? they literally invented the most popular GUI toolkit used by most linux apps (GTK)
Components, sure. But not the UI. Building a toolkit is not the same as building a UI. You can use the toolkit to build a good UI and you can use the toolkit to build a bad UI.
GIMP doesn't need UI tests, but a more intuitive UI.
Oh, haven't heard from GIMP in a long time, has there been significant UX improvement and have they carved out a niche like Krita yet?
"GIMP now has UI" would be big news
The tests should simply take a screenshot and send it to an LLM with the question "Is the UI absolutely horrible?". If the answer is yes then the UI is working as expected.