Childish as in... A newish artist who is drawing a concept rather than light / forms (Which is something artists typically do as they understand drawing more and more).
The rose in the vase specifically - some models understood that there was supposed to be shading, reflections, the concept of refraction - others just drew "blue = glass" and "green = stem" and "red = rose".
Really odd to look at, considering if I saw any of these drawings from a human kid, I would say "good job buddy" and put it on the fridge. I'm expecting these to get better as models improve, and perhaps the artistic progression will be there along with it...
The Grok ones in particular gave me that thought. Most of them really look like what a kid would do when given the same tools, while the other models' output have distinctly more "AI-ness" to them (for lack of a better term.)
The fact that it has clearly iconified these concepts in its mind and is tracing the outlines of the things it thinks/expects to go where is very human.
How do we know the difference?
These SOTA LLMs aren't trying to mimic existing children's drawings, but interestingly they're following somewhat similar progression that human children do as they develop.
At work, even if Fable is technically better I much prefer Sol because it is so much faster and concise.
Seriously, what's going on there ? Why is it so different from others? Is it just behind technologically/training wise or it's using something fundamentally different?
It performs much better than composer2.5 (while being as fast). It's not Opus, but I think it's not that far off. Definitely better than sonnet for what I've been doing.
On the other hand, I think they probably heavily adapted the training data so that it really is extremely focused on code. I just recently ran my personal "poetry benchmark" on it (where I give it ~850 poems I've written over my life and ask it to comment the corpus as a whole), and it's whack. It tries to write in portuguese (most of the poems are portuguese) and code-switches constantly and mixes up words to the point of making what it writes almost unreadable (e.g. it writes stuff like "You can't QoS that that look for beast poems", in portuguese, all messed up). The quality of the analysis is also quite bad (I'd say it's definitely behind Sonnet).
So I really think they either threw away data that wasn't tied to coding so that they could fine-tune it to that, or somehow they've got such an unbalanced dataset that coding ends up dominating either way. To me, its disastrous performance in this drawing "competition" fits this narrative.
For some reason, OpenAI models, Gemini (and apparently Grok too), love to latch onto this and obsess over this idea that it's "programming poetry" or "poetry for the IT crowd". Often OpenAI and Gemini try to write the "equations of my poetry" (granted, I do write about a cyclical relationship between thinking, feeling and writing a lot, and I do have ONE poem which ends with a Q.E.D.).
I'm giving this context to say that it is very bizarre. It's as if they latch onto it and act as if it's a core or highly distinguished part of the poetry, when it really isn't. Anthropic models, on the other hand, absolutely do not do this, and have never done it.
I really don't understand why this happens. Maybe it's because it has a lot of portuguese, I don't know. And even though the "QoS" is clearly the wrong token being generated, I have had situations where gemini spoke of some phase of my poetry as the "Q&A part" (really, no joke...)
In any case, it's why it's my personal benchmark after all :D
Edit: I just don't see the point of redacting the Mona Lisa
The rectangular smudge tool is a weird tool in the first place, but it's cute to see the models try to use it.
https://www.tryai.dev/?error=server_error&error_code=unexpec...
- the best image related stuff I've seen is where the harness is constantly cropping and looking closer at things (likely helps a lot for computer vision in general) - also it would be interesting because the harness could almost have its own "palette" as if it could play with blank squares and different strokes over lapping or blending before applying to the main canvas
Citation needed - my company is paying more than ever for code generation. I have no reason to believe (given anecdotes) that anyone finds themselves in the opposite situation.
I get a lot more use (read: cheaper per interaction) and much better quality results from the $20 that I spend on this stuff every month than I did several years ago.
(And several years before that, it was all essentially unobtanium.)
.-""-.
/ \
| _ _ |
| (o)(o) |
\ /\ /
| -- |
| \/ |
| |
/ -- \
/ / \ \
/ / \ \
(__/ \__)