"Despite billions of dollars that firms around the globe have invested in AI over the past few years, major returns remain to be seen: 90% of executives say AI efforts have yet to translate to productivity gains at their companies"
1. Throw your entire 200k SLOC codebase in and ask for a bug fix.
2. Tokenize your codebase (AST -> SQLite) and have the LLM explore and reason about subgraphs, and then ask for a bug fix.
One takes expertise but yields a better signal to noise ratio. Though, at that level of expertise one encounters what’s effectively the library of babel: you get back what you write and / or already know, and coherent text outlining future events might be wrong until they transpire
1. Just because somebody tells you it's true and you really want it to be true, doesn't necessarily mean it's true. I see company ships crashing into the cliffs of reality.
2. An LLM is inherently a text-based system, would be great if we could drag thought and reason from the statistical relationship between tokens but that will only ever net us an average of what an association of concepts and words can ever mean, like a generated landscape of tiles rendered in a video game. Yes, each border is logically transitioning from one type to the next for a coherent render, and yet it will not automatically result in an accurate map of earth on these rules alone. To try to draw novel thought from this is like joining camp Orwell (and a bunch of others) in how language can define thought.
It is incredibly useful but nowhere close to the advertised abilities/ideas, and thus companies probably overcommitted on the shiny new tech which turned out to be shiny but not shiny enough.