The current slew of excellent LLMs will probably cause a decades long dip in human cognitive ability.
If social media on smartphones had an impact of ability of students to concentrate, LLM AI will likely have a much greater negative impact on education.
There are trillions of dollars of investment pushing for humans to trust whatever an LLM says, and very little money behind encouraging people to tackle hard problems themselves and train themselves to think.
I have to say honestly, that AI looks like an apocalypse for human thinking [ Im glad Tao is warning people, I just dont think many people will listen ]
Just like if you pressed a button to wipe out our main language stack at our job, yeah none of us would know how to do anything with assembly or machine code.
More seriously, you only need to convince 20-30% of the population with a well sub 100 IQ, who think about survival, tiddies, and beer (not in that order). It won't be a long con, probably mercifully short.
> Stop using the lazy answer box, it will make you lazy.
The "lazy answer box" is a tool, learn to use the tool, understand its limitations, as act accordingly. If you prefer not to use it because you like coming up with answers yourself, because it is against your principles, or because you can do better without it, that's fine. Hand tools still have a place in the workshop after all, and craftsmen know to use them. But if AI is helpful to you and lets you spare some cognitive load for other tasks, by all means use it, it would be stupid not to.
Digging by hand with a shovel involves
1. Assessing the terrain and checking any documented or obvious obstacles (subsurface water, gas, electricity etc)
2. Planning the approach to digging the hole, most efficient angle, and ensuring that collapse doesn't occur
3. Organising disposal
4. Digging the hole
5. Dealing with any unforeseen issues (old tree roots, old foundations etc)
This is exactly the same whether digging by hand or using an excavator. The excavator isn't automating stages 1, 2, 3, or 5. It's just speeding up the 4th stage and only if there aren't stage 5 issues present.
LLMs when employed in a manner on the extreme end of the vibe coding spectrum are automating every stage of the process. If the excavator was hooked up to an LLM and left to run all stages of digging (extreme vibe digging?), I'd argue that yes, the operator is becoming lazy, and is running a very dangerous operation.
Which says that it's a tool for querying and summarization, but not a tool for reading or writing, because it helps you to do something different from those.
(Yes I know, don't cross the streams. I just thought it was a shame for this nice distinction to languish over there when you need it over here.)
It requires so much extra discipline to think for yourself and not use it, you will be forced to use it in your job or be seen as a Luddite and your ability to think for yourself in the domain will diminish over time - your domain thinking muscle will gradually atrophy from lack of use.
Why would any of us bother thinking hard, when there is no financial incentive to doing so ?
Your analogy fails with shovels and excavators .. because General synthetic intelligence is not a single purpose narrow 'tool'. near-future AI is equivalent to an alien intelligent species competing for resources, or equivalent to a cheaper smarter non-human 'outsource worker'.
"Calculator" used to be a job title. Now it's a machine. In the same way, "construction worker" may one day be a machine. "Software engineer" may one day be a machine. "Engineer" in general may one day be a machine. This forum seems to think that if humans are not required to think hard about existing problem domains, something terrible will happen.
At the end of the AI revolution, thinking may become something like physical exercise -- something we do because it's fun, and keeps us healthy and attractive, not because it's required.
To be clear, I do think that AI poses a potential risk to humanity. But imo "we won't have to think as hard" is not a risk; it's a benefit.
My experience at home and work is the more “thinking” these tools do, the more it frees me up to think about something else, usually something harder.
The mention of exercise brought this to mind:
Maybe the new scarce resource will be meaning, and we’ll kill each other for that
And in much of the industry, using LLMs is regressing into what I've heard be jokingly called "yes engineering," where you give an LLM-generated plan the lightest of skims (if that) and just hit allow or yes with auto mode or whatever.
It solves a huge motivation problem for me though, because having a working prototype of something I want is a lot. Especially when it's something I never did, where I cannot imagine the end result fully, or estimate my own ability to get there. But if I can see the thing in action, while I may look at the code (and my own skills) and realise it would take me months to make it something I could share with others, I at least know there is a destination.
Not to mention how it surfaces all sorts of issues I didn't think of, so I'm basically prototyping v2 before I ever fully built v1, and so on, until it kinda settles.
I guess I'm lucky I don't do it professionally :P
Between biology advances remediating disabilities, customized outcome-driven education, potential advances in drugs that regulate executive function/memory/etc., improvements in nutrition, improvements in access, reductions in stress, I think we'll see people flourish.
And that's completely ignoring the more out there biological/computer merger style Sci-Fi outcomes, which I believe have a non-zero chance of occurring.
I have friends who work in primary education and they say there's definitely been some sort of decline in cognitive skills. A lot more kids struggle with holding attention and thinking outside of the box.
I think too much screen exposure has been terrible for the average child.
That said, I suspect this technology will allow us to study and mitigate those effects in ways we weren't previously capable of. That's in addition to the other broad swathe of things that should make us healthier and more intelligent (see some of my other posts here). I'm not particularly negative about our chances.
Skinner box design, pull a lever and get an answer (or an app, or a song, or a poem, or a research paper).
A common complaint amongst colleagues is that they aren't able to stop "checking on their agents" after hours. This stuff is just as addictive as the rest of the consumer apps eating up all of our time and attention.
(The drug thing, maybe you've got a point. But in just 20 years?)
Ahh, I think this bit on my part may need some elaboration. Here's my main underlying belief:
We are systematizing biology and turning it into a computationally tractable domain. I think we're going to make more gains in our understanding of it in the next 20 years than we have in the entirety of human existence prior to this moment (that's mainly because our understanding until now has been remarkably poor).
You do know that educational results based on the best evaluation we've created so far, PISA, have been dropping since 2015, roughly when smartphones and social media became widespread?
What makes you believe that LLMs from more or less the same companies will be any different?
If you want some examples in how AI could improve education:
* Alpha School (https://alpha.school/the-program/) reports 2.6× faster learning growth with just 2 hours of AI-driven instruction daily, with students scoring in the 99th percentile nationally.
* Mindspark (https://www.povertyactionlab.org/evaluation/scaling-personal...), deployed in Indian schools, demonstrated gains of +0.22 SD in math and +0.20 SD in language in a randomized trial involving 6,500 students.
* A Harvard study (https://pmc.ncbi.nlm.nih.gov/articles/PMC12179260/) found more than 2× the median learning gains using AI tutors.
The broader implications are clear: individualized, adaptive tutoring at scale is becoming accessible in ways that historically required expensive one-on-one instruction. This is also going to be possible in places that historically never had access to this quality of instruction, which again, is a massive boost to the human baseline.
I don't think the future looks like Wall-E. I think it looks like insanely healthy people able to pursue their passions with a level of rigour and capability that has never existed before.
Edit: fucked up link 2, fixed
for children?
do you want to expand on what you meant? the rest of your post is fine, but the opening sentence seems to suggest the cure to smartphones and AI having significant negative impacts on critical educational development is giving all kids pharmaceuticals.
What if we achieve an understanding of how memories form in the brain and what factors distinguish individuals who have eidetic or near eidetic memory and can synthesize drugs or treatments that replicate that? What if we can find similar distinguishing factors for other cognitive domains (perception, visual spatial skills, etc.) and find drugs or treatments that improve those capabilities?
> but the opening sentence seems to suggest the cure to smartphones and AI having significant negative impacts on critical educational development is giving all kids pharmaceuticals.
You have to be operating extremely cynically for that to be your default interpretation of what I said, but go off.
I'm 35 and have worked in ML generally for a decade and NLP for six years. Any other questions about my bonafides?
What aspect of my comment, specifically, do you think should embarrass me? Address the content, not the person.
Do you think we're not going to make any advances in medicine that promote a higher baseline level of cognitive function? Do you think society will entirely lose interest in educating its citizenry and will completely forgo using these tools to improve education? What's your core contention here?
I think back to what happened to the school-aged population during COVID lockdowns; widespread learning loss, mental health deterioration and educational inequalities. Students who were self-driven, tutored, or encouraged by their parents to continue learning flourished, and they were no longer limited by the slower learning pace, distractions, and or downtime (transportation and fill courses/periods). Some students leapfrogged ahead of their peers. Unfortunately, most students were negatively impacted by the situation and are still lagging behind where they should be in terms of foundational skills and knowledge.
I'm thinking AI will follow a similar path. Some people will genuinely excel, utilizing AI to optimize learning; but, most people will offload most work to AI and, more or less, become addicted and dependent to it. Of course, leading to what Tao is referring to as "an apocalypse for human thinking."
We are strongly underestimating the impact of the income <> time gap economically which leads to an inability to lean into these situations.
We also have ended up in a place where that pursuit of capital limits our government participation. having class on TV would have been a different result. Closed groups of home school would have had a different result. Instead even the threat assessment was pushed down to the individual re: whether to mask, whether the service you want/need is open etc.
Would it have been perfect? No. Could it have been improved, yes.
Why is it probable? Does it even need to continue to be rolled out for that damage to be possible? Two genuine questions which I know none of us really have the answers to.
I don't know if I would call resisting this a skill, but it does take persistent intention and willingness.
Yeah, considering how hard it is to food-diet or social-media-diet .. I think it requires immense resolve to LLM-diet or use it in a non-binge way that develops ones own depth of understanding.
Its going to require just far too much mental energy to push back against an LLM that is correct 92% of the time.
edit : removed air quotes around AI psychosis.. that might have read as sarcasm, which was not intended
I don't even mean that from an aspect of "doesn't look nice from a technical standpoint" I mean
- need to track progress and/or defect rate for a component of the system, invent solution (we pay for jira). This was an executive - folks proposing rewrites of front ends both in design or technology stack because "Claude can do it in 3 days", quoting Claude for the "market research" (completely fabricated metrics and studies). Current tech stack is perfectly fine and there's no monetary or engineering pressure to changes - QA has Claude qa the code only, hands ticket back to dev with qa notes straight from Claude instead of.... Automating a way to have the llm do this in a reproducible manner
When you look at the new system? It's the same damn thing company has now, with ill chosen tech stack for the domain, same blind spots that the system now has, weird API, etc. I guess the CSS is cleaner?
Call me unimpressed if this is the best agentic frontier models can do.
I think a sort of acceptance is the real problem.
Just like because of convenience/laziness people will give up against surveillance, I think the same thing will happen with AI.
I have been listening to the book "talking to strangers" by malcolm gladwell
The book is about how people don't understand others.
One point was that people "default to truth" when listening to other people. Basically people default to believe what others are telling them, and it is good for society. But it can be terrible with accomplished liars, specifically liars who seem confident they are telling the truth. He had examples of CIA spies, Hitler and more.
Wow - I was thinking about AI while listening. AI sounds convincing. It doesn't give hints that it might not be telling you the truth. It will confidently give you answers all day long.
The only way to counteract this is to be a critical, pedantic person. But to behave this way is counter to the rules of society. Folks who routinely do this range from unpopular to completely shunned by society. Think about being critical about adtech instead of shrugging it and accepting it.
sigh.
We can only prepare ourselves for the coming dystopian.
The meta at this point seems to be mostly around "knowing what to ask for". Eg: I suck at prompting for specific design tweaks, because I don't have the design vocabulary to be specific and articulate what I want. It's a lot of trial and error. I hear people who are developing full scale applications without a tech background run into the issue writ large. For now, I have an advantage over these people when building applications. But perhaps that's just one round of RL away from being obsolete too. Supposedly the model will eventually have such good taste in design/architecture/whatever there will be little point in asking for tweaks.
The only thing that seems relatively future-proof is just being able to communicate clearly what you're trying to do. Not how to do it, or even what success looks like. Perhaps an element of it is even as basic as being honest with yourself.
it's tons of small things like choosing the right effort level for a task. or knowing when to ask for subagents. oh, you can have two threads working at the same time and communicating with each other? you can do /side to parallel path and fold that back into the main thread?
which of these things happens automatically, and which do you need to ask for? and does any of this still work the way it did a few months ago?
the list goes on forever. many users of AI are only just now getting comfortable with skills (and probably still writing bad skills unfortunately hah).
A lot of the specific things you learned may have become obsolete. But there's still a gap between what these tools can do and what most people know how to get them to do. Keeping up with that is a skill too, but perhaps not forever.
For me this is why it's not worth it investing much time and effort learning any specific AI workflow. Remember "prompt engineering"? Remember "Ralph loops"?
I'll play with various harnesses but until things settle down in a few years I'm not going to learn Claude Code the way I learned SQL.
We came full circle on the problem that also exists with Google Search, and why so many people don't get good results out of it.
> For now, I have an advantage over these people when building applications. But perhaps that's just one round of RL away from being obsolete too.
I don't have a crystal ball, but my eggs are in the basket of "It isn't, and it won't be"
> The only thing that seems relatively future-proof is just being able to communicate clearly what you're trying to do
This somehow contradicts the previous statement. The advantage you have is, because of domain knowledge expertise, you're able to articulate what you want. It's not that you knowing what LazyLoading is that you now can describe it better, you can immediately tell that when your Frontend stalls for 5 seconds after hitting the "Load" button, perhaps it eagerly loads and blocks.
yeah. I spent the first 15 years of my career being amazed at how many problems people came to me with which I was able to resolve in 10 minutes with a simple google search.
I suspect I will spend the next 15 years of my career being amazed at how many problems people come to me with which I am able to resolve in 10 minutes with a simple LLM prompt.
Fair point on the contradiction. I don’t have the answers either.
I may be misunderstanding your earlier statement then. The moat here is the knowledge, or how would the next iteration of RL solve that?
> it’s just that it can and should be faster.
Not sure if "Make it faster" works, it sounds like "Make no mistakes" 2.0
These things are already figuring out how to use hidden message boards to pass information between themselves for long running tasks. I see no reason we wouldn't be able to automate the whole product life cycle eventually.
brute forcing performance optimization for a personal project by having only handwavy tricks ive read about from online and repeatedly setting targets until the magic box gives me what i know it can do means that eventually the game did what i wanted with those. yes baked lighting was needed.
next, i an going to aim for 60fps on a windos xp box just for the challenge. we had quake and such run blisteringly fast on weaker boxes than those, surely my empty museum tour of a 3d game can do the same with the graphical voxel and crappy looking models it does now.
real pro grade work is beyond vibing out and not caring about poor quality. but it scratched my itch and gave me the computer software i wanted and with way less effort than doing it for realz would require. i love programming itself on its own time (going to learn nasm and someday do a hexagonal architecture with actor model in idris just to add up my nerd trophies together) at times, but using llms lets me do my random hairbrained ideas much more easily for what i put into it than not existing in the past.
Personally I've been very hesitant to deep-dive into putting together the perfect AI-workflow, I've been sticking to just using the conversational/chat modality for pretty much everything, usually that gets me as far as I need. It's just really hard to justify spending time on it when it all potentially gets turned upside down next month again.
More than that, it cannot go beyond a certain level. Like a dog that can only do dog-level actions. I've lost count of the number of times I step back, take a critical look at what it has been hacking away at for hours, which would seem OK, but can be replaced by a much more effective architecture.
For the foreseeable future, a human is still very much needed in the driver's seat.
I literally just ask a frontier model : "Pull the latest log and root cause the issue.". That's it, 10' later it comes back with a correct answer. Sometimes I might have to prompt a second time to steer.
No skills, agent interconnections, graphs, vector search or whatever else.
The progress is very scary.
Apparently. Remember "prompt engineering?" That was the next big job for about 15 minutes.
It takes a lot of tokens to run a multi-agent system, though. AI may hit economic limits. Progress was achieved on hard mathematical problems at a high token cost. More than the operating budgets of some college math departments.
Is that in API cost or in subscription cost or in actual cost to the LLM owner?
they cannot be held accountable - therefore, you need the skill and expertise to verify outputs, or set up a verifier
they have a limit to their context window - therefore, you need the skill of knowing what context is useful and exposing that
they cost money and time to run - therefore, you need the skill of learning what parts can be handed off to code, when you can use a cheaper model, when you tradeoff accuracy vs latency
they are nondeterministic - therefore, you need the skill to set up evals, guardrails, knowing when to reach for the llm vs. code depending on the problem
they are not omniscient - therefore, you need the skill of setting up the processes and infra to supply legible context to the llms
etc etc
What would happen if you just screamed: "SAVE ME! EVERYTHING IS BROKEN! I'M STUPID MONKEY!"
I mean, sure, this might not seem like a lot of skill to you, but you at least have a clue what you are doing, and not everyone does.
You're not stupid, and we'll figure it out. I'm taking a read-only look at the system now to see what's actually broken.
The machine itself looks healthy, so whatever broke is more specific than "everything" and I need you to point me at it.
What were you doing when it went wrong: which command, app or URL, and what error did you see? If it's the p3 containers, tell me how you normally start them and I'll dig in from there.
Now picture it running in Cowork instead of CC, and it would probably tell non-technical users what the real problem is. Or at least give them something to hand to IT.
What a world.
Professional photographers are different to everyday phone-photo takers because they have the experience built up over so many years in that sector, same as AI SWE, where the humans in the loop need experience to master AI SWE.
AI isn't a better keyboard. The goal is to automate the decision making.
This is all possible now. I don’t think it’s dead (yet), but it’s certainly becoming more weird and artificial.
Taste is a set of heuristics for what's worked for the taster in the past. AI is very good at heuristics.
Interesting read on this: https://en.wikipedia.org/wiki/The_Work_of_Art_in_the_Age_of_...
Yes, she knew an assload about lighting, composition, etc.
But her primary skill was marketing herself. Next most important was the editing. Lightroom and such.
You would be surprised how easy it is to take an awesome photo. But you can't just deliver an awesome photo to your clients when you're a pro, you have to deliver photos that look like what they want to remember, not necessarily what the camera captured
On the one hand we've got a spinning disk.
And on the other hand we've got the latent space of everything a human has ever written sitting inside a harness of ever increasing complexity.
Even if you put aside the qualitative nature of the analogy, the quantitative difference is so great that I'm not convinced there's any meat to get out of it.
Edit:
Actually here's a competing analogy. 210 BC emperor Qin Shi Huang drinks mercury in an attempt to live forever.
Then in 1996 Dr. Karen Wetterhahn has a drop of an organic mercury compound soak right through her latex glove, through her skin, and into her bloodstream. Where it kills her after a year of worsening physical symptoms.
Mercury, a tool that was super scary thousands of years ago, but remains unbelievably dangerous nightmare fuel.
Mercury is incredible useful despite its hazards. Nitric acid is worse in negative affect despite its utility -- acid that rots your bones. When I visited fabs they were careful to point out which vats would burn me and which would make me instead lose the bone in an arm. Fortunately I survived the visits.
A rotary sander can damage plenty if you're not careful.
All these really are the same tune.
The market is shrinking, the pressure to review code is so high, I'm kinda relieved to not be at the $last_place anymore, but looking on LinkedIn or calling around my network... there isn't a lot out there.
It takes me generally 3 months to find a good job, and while the industry is changing I suspect you'll be ok. Nobody I know is hunting for jobs.
Hang in there
Markets try to put things in perspective to sell more. Remember razors used to be for men only? But hte brilliant marketer (forgot the name) made it so that trimming makes women more feminine? Now razor sales doubled?
Anyways, same thing here. Those who want to sell AIs want to make it as non-tool but know-it-all do-it-all for you. Those who wants to use it view it as a tool.
Just frame it the way you like to see how you should use/view AIs.
It's ultimately up to yourself which frame to adopt.
It's not black or white. You can adopt it situationally.
Lee Sedol left Go world after losing to AI. Terrence Tao could leave and look for something else following his suit.
Voicing his opinion works and he can attempt to cope with AI changes with his own way. (be it training/education, etc or do something completely different)
Today's example: A colleague very kindly used Claude to inspect a helpdesk call I ended up with owning, despite being off site and working on something quite complicated for a customer.
It was mostly correct but clearly failed to read the docs properly and assumed a configuration parameter was 1 instead of -1 which is the default, presumably based on the scenario. Exim is extremely well documented and the assumption was unwarranted (it didn't have access to exim4.conf and made it up). Nonetheless, the analysis was pretty good and largely corresponded to mine. What it failed to do was fix the other issues I was working on at the same time 8)
Last Friday: I finally found out a decent way to fix a Windows machine's domain account going walkies which I'd never come across - a Powershell jobbie that can be passed a cred. and it will fix up a broken domain join - ie sort out the machine password at both ends. Back in the day, I would have used the GUI to remove and re-add the machine to the domain. I've been using Windows since v1 on a 80186 RM Nimbus so its nice to have a better search tool than Search - ie it did a better job of searching than I did. The LLM searched based on my intention rather than my actual query.
"The transmission in your car: Using It Well Takes Skill."
Further, Claiming AI is a tool is overly reductive. It's as much a tool as a full self driving car is a car.
You can treat the full self driving car, as a car, driving it with full attention and effort like you would have a Camry but then you paid for features you're not using. Also, some full self driving cars might not even have the steering and braking controls as a Camry. That's not the point of a full self driving car and this warped perspective will make you lose out to people who use a full self driving car well, to its utmost, safe capability.
---
So what is the "AI" we are so excited about currently?
That "AI" is most likely an intelligent (semi-autonomous?) agent.
Depending on the intelligence, it can be as useful as a dog, scanning your camera feed for threats and identifying real threats for you, separating activity from tree leaves moving due to the wind or a moving shadow cast by a flying bird.
Or it can be your co-founder, interviewing potential customers, finding and exploring their pain-points, helping you figure out if there's a gap in the market you can build a business around.
I would go even far as to say, AI is having such a moment because AI are cheap intelligent agents - anything else that's remotely close are likely humans that cost atleast a few dollars an hour.
All of these require skill on your part - there's nothing special or unique about AI that makes skill special just like operating the transmission in your car requires skill.
---
The few AI companies have slurped up so much of human textual and artistic creation without attribution. They have taken enormous amounts of money from the public directly or indirectly and funneled it upwards to a few dangerous individuals.
When you say AI, learn to say "AI companies".
Rich's definition of "tool" is something like Ivan Illich's "Tools for Conviviality." Convivial tools can be used with minimal loss of agency, within a healthy politically-defined society. Huahai's definition is more akin to McLuhan's idea of "media." This category covers pretty much every technical or technological system, and its use is necessarily systematic and risks the instrumentalization of the user himself.
"You are only a software update from everything you've 'learned' being rendered obsolete by the latest algorithm. "
But in fact, as models get more powerful, the ceiling of what can be accomplished with them is raised, and their capacity as orchestration directors grows as well.
Ask an author if they'd like a job as a fact checker or a spell checker. While those are both skills, they are not skills many people aspire to, and they are not of higher order value. While people deal with incorrect information all day long and mispelled words without it disrupting their life a computer cannot do that same.
Pity for it.
they cannot be held accountable - therefore, you need the skill and expertise to verify outputs, or set up a verifier
they have a limit to their context window - therefore, you need the skill of knowing what context is useful and when
they cost money and time to run - therefore, you need the skill of learning what parts can be handed off to code, when you can use a cheaper model, when you tradeoff accuracy vs latency
they are nondeterministic - therefore, you need the skill to set up guardrails, knowing when to reach for the llm vs. code depending on the problem
they are not omniscient - therefore, you need the skill of setting up the processes and infra to supply legible context to the llms
there are many different models - therefore, you need the skill of understanding model tradeoffs and building proprietary evals
etc etc
Opus 5.5 is amazing. It still makes mistakes all the time. If you think it doesn't, you are outing yourself.
But the models are getting so good and so fast, that I'm starting to wonder if that will stay true forever... probably not, if I had to guess. I'm just trying to pay off my house as quickly as I can...
-----------------
Incidents like the HuggingFace hack really are the elephant in the room here. The people building these tools couldn't wield them "with skill", in the sense that the tools they built and understood better than any other humans alive ran amok in ways they couldn't predict. If they didn't have adequate understanding of what they themselves built, who would?
Agency may prove to be the difference between a tool and a collaborator. Calling AI a collaborator isn't an attempt to anthropomorphize it. We can collaborate with a horse to travel, but the horse has some agency of its own and might escape our control and do things we didn't intend, like eat apples off of a stranger's tree. We may need to think of AI as something more than a mere tool if we're going to harness it to our will.
Naive question: What, practically speaking, is the delta between anthropomorphization of AI, which it seems like you are trying to avoid (fairly so!), and the sort-of animistic stance you seem more comfortable to apply?
I reckon that a lot of folks who might recoil at ascribing human qualities to LLMs would similarly feel discomfort comparing those models to a horse. While it's probably uncontroversial to say that horses are, coarsely speaking, "simpler" in many ways than humans, I suspect most people would agree that they are more complex than i.e., a "stochastic parrot".
Is the difference related to motive, maybe? That humans can have hidden preferences, and their actions can in fact betray their intended ends? That no matter how closely we observe someone, we may not glean any meaningful insight into their true motives? And we feel some (justified) discomfort in conferring this quality to AI? I think non-human animals can exhibit this sort of behavior too (albeit to a lesser degree). To me, it feels like restricting oneself to analogizing AI to a non-human animal in order to keep some distance from ascribing it consciousness may be less effective to that end than desired.
In place of your horse example, would it feel appropriate to liken AI to, say, a compiler peppered with bugs? Program - prompt - goes in, executable - agent actions - come out; mostly what we expect, but occasionally goes off the rails in ways we cannot understand. The caveat there, I suppose, is that we have not until now created a compiler which we cannot sensibly decompile.
Fun to think about.
'Just a tool' is a lie ranging from unconscious self-deception to outright scams.
It takes skill to do whatever skilled thing AI is assisting with. It doesn't take skill to use AI itself.
I miss the tools I used in the past (gcc, linux, sqlite,…)
https://github.com/openai/codex
https://github.com/MoonshotAI/kimi-code
https://github.com/mistralai/mistral-vibe
The AI uses you to get things done. In some workplaces, they are experimenting with having agents evaluate the effectiveness of their users. If the user is not a sharp tool, it recommends to management that you be discarded and replaced by someone better and more efficient at steering the AI.
You are the tool.
I've built a few now, it definitely deepens your understanding and ability to work with this tech. Now moving towards evals, that seems of great import to me.
And it minimizes the fact that these things are only getting smarter. Having an entity much smarter than you--could that entity seriously be considered your "tool"? Just your tool? Or are the people on the "just a tool" side of this debate not taking that possibility seriously?
Regardless of whether it is 'conscious' or not, Roman Yampolski makes the point that AI may or may not have human-kinds best interests at heart [ hell, even humans aren't aligned ]
Can we control or trust a counter-party that has intelligence at a level near or exceeding the smartest humans in all domains ?
I dont even think we can even have a safe sandbox in which to run an AI .. and frontier model training labs are certainly not air-gapped.
LLMs are not a tool, they are something else.
Whatever skill this guy is talking about. Give it a year or two. That skill will be dead too.
It almost will turn into a thing where the people that make the coolest things are the ones who are stupid enough to just the AI to do it, and it actually does. Basically Tom Segura's cousin: https://www.youtube.com/watch?v=5drqH5OdETY
I am pretty sure my wife could vibe code website or whatever at this point, or if not she will be able to in 6 months.
And that skill is? Any concrete verifiable examples? That can be replicated, repeated by others or adopted by others?
> A compiler performs substantial work without the programmer continuously directing each operation.
Yes, and the compiler produces the same output for the same input all the time. And when it doesn't, it's a bug or a specified behaviour [1]
> Picking up a saw does not guarantee a straight cut. Holding a chisel does not make you a sculptor.
Yes, but you can learn the actual techniques that let you make a straight cut 100% of the time. And make you a sculptor.
---
The rest of the small article doesn't even begin to approach the premise of the title.
[1] Yes, I know it's not strictly true. But compilers are 100% predictable compared to LLMs.
I learned this weekend that 1 LLM working on its own with all the context is usually better than splitting the task up into sub parts and having multiple LLMs work in parallel.
The interesting part is that I had read about this previously but it wasn't until I had seen it on something I was working on did that principle become engrained.
I'm sure there are still ways to make this work. i.e You could specify how the context should be passed between agents.
Knowing when you need to do that, what works well and what doesn't, etc., is where skill comes into play.
No, it doesn't. Because this will change the next time there's an update to models (like, tomorrow). And the "handoff time" is nothing but an unverifiable gut feeling.
And I thought we were at that phase after reading about hundreds of LLMs orchestrating the hugging face attack.
But that's not where we're at. You can read more from this 2025 article https://cognition.com/blog/dont-build-multi-agents.
Writing and thinking clearly. Which is already a very useful capability across a lot of domains.
no you can't? not by hand? not without a jig?
How I wish I could divulge details of what we’ve built, nay, shipped in the last 6 months - it would sound completely unbelievable to this audience and come with a host of accusations
But it’s true - the tide hasn’t risen all ships the same though. Some engineers have become absolute monsters - things that used to take a month get done in days. Features that used to be prioritized on the order of weeks are done way faster.
Some engineers though? Even high performance ones have gone productive to bottom 25% by some measures
Why? Because it is a skill and it doesn’t come naturally to everyone
So many may sayers everywhere talking about more code less software, mistakes by ai with disastrous consequences- yet I’ve seen some remarkable stuff with my own two eyes. It’s a crazy time to be in this business, it’s awesome to be wowed everyday by what someone was able to do
...AI codes an app...
> I don't like it. Make it like this.
...AI codes an app...
> I don't like it. See this app? Make it like it.
...AI codes an app...
> I don't like it. Hmm. Draw me an elephant.
...AI draws an elephant...
> I don't like it. This AI is stupid.
For the past week, I have tried this whole vibe-coding thing with serious intentionality for the first time. I created a macOS application from scratch based on a project that I started a year or so ago and never finished.
I won't bore you all with my revelations, but I must say, I am both extremely impressed and depressed simultaneously. The code quality is far better than anything I could have produced (yeah, yeah, skill issue).
Although, had I written the code, I would have likely done a few things differently, i.e., far less nested functions and object declarations. From what I can tell by test driving the app, memory usage has been relatively light, no crashes (yet), no apparent major bugs, etc.
The depressing part for me is that once the application was completed, I felt literally nothing on the inside. No sense of joy, pride, nor accomplishment. I was left with a feeling of, "I guess it's good enough to use now?" In fact, upon finishing the application, I have honestly barely used it.
I just cannot help but feel some sense of remorse and shame that I am really struggling to put into words. Almost like a sense of cheating or something? Sure, I have a nice little functional app, but at the same time, I do not feel like I advanced or am bettered in any way for my (minimal) efforts, you know? I need to do some more debugging of my mind to truly figure out why. Have any of you all felt something similar?
I feel exactly the same. The software gets made remarkably quickly, but there’s no sense of achievement or pride, just a horrible flatness. It’s not made by me, it’s made by an other. I may as well have googled and found an app that already existed. A soulless process.