I'm finding this question difficult to answer without a concrete example of what would pass the bar.
Ignoring if it can do anything "new" by itself, is it not worth it as a tool if it can turn hours of work that a human would normally have to do synchronously into an async session done in minutes?
There's plenty of valid criticisms for the uses of LLMs, but you pointed out an unbelievably useful case for the tool that you're trying to question.
We give Fields medals to people who do things like that. There's no particularly good reason to bury your head in the sand and pretend AI is different. It's not surprising that the first big discoveries involve standing on the shoulders of previous researchers and their work. When work is described as "new" in some way, that's usually all we mean by it.
or maybe
What would something new even look like? Would we recognize it?
> "Everything I've seen so far still involves a human sussing out the new thing or the model is just aggregating data that is already there from disconnected sources that never got tied together."
Yes, that describes people as well. You think ideas appear from...what...handed to you by God through the aether from the spirit realm? We're literally meat computers.
Your question seems to boil down to whether current AI has any kind of intrinsic motivation to do anything. It doesn't.
- "where do you want to coarse-grain the concept of agency"
- "what qualifies as a novel synthesis of information"
- "how consistently correct must it be to count as intelligent"
?
> Your question seems to boil down to whether current AI has any kind of intrinsic motivating force. It doesn't.
Isn't an AI intrinsic motivating force its initial system prompt?
I wonder how many people's initial system prompt just looks something like "You are a sentient being attempting to maintain a sense of constancy in an ever-changing world, for which you inherently find this fact dis-satisfactory" ?
Would we say they don't have motivation?
It probably could be, but I'm talking about something abstractly analogous to "explore and create, compete and thrive in the world" (i.e. "good luck, have fun, don't die"). And I think it probably only counts if accompanied by the right scaffolding that allows the system to operate continuously. To my knowledge that hasn't been done. Doing it creates a very hairy ethical dilemma.
[FWIW I would personally count that entire system as 'conscious', for some definition of Integrated Information Theory's phi value, though maybe not 'sentient'; I don't think a single LLM (in the sense you can run a model) counts as a conscious system, but it makes sense to me that even one in a feedback loop where it can alter its weights probably counts]
The “spiral” neo religious movement (if you want to call it that) from the GPT 4o era is pretty novel even if it has antecedents in Nick Land and other CCRU people