It is NOT being used for this, instead we want faster “answers” that avoid due diligence. Same as always but lazier.
Wikipedia community bans AI-generated entries because verifying and removing AI hallucinations imposes even more workload on editors than writing articles from scratch [1, 2].
If you don’t agree that Wikipedia’s approach, one can always fork it like was done with SpaceX / Twitter did with its launch of Grokipedia on October 27, 2025, which hasn’t been really incorporating much improvement well with lots of digressions in its quality and reliability [3].
[1] https://en.wikipedia.org/wiki/Wikipedia:Writing_articles_wit...
[2] https://en.wikipedia.org/wiki/Wikipedia:Editing_policy#Artif...
[3] https://en.wikipedia.org/wiki/Grokipedia#Reliability
[Edited for better formatting]
To put it into words: a layer over an LLM that tracks provenance: the documents it has access to, the facts it has extracted, the mathematical expressions and queries it's run and finally the artifacts and results it's yielded complete with receipts for exactly how it arrived at it's findings and conclusions.
I assume legaltech products have to do it like this. Is the general pattern implemented somewhere on github?
we build a product called Veracity focused on citation and claim checking. legal is a major focus for us, also scientific and technical publishing, where we detect hallucinated citations in submitted works and all that jazz
planning to release something soon focused on the LLM harness layer
we've been working on the problem for years. gradually getting traction
all completely closed source atm, though we often revisit that discussion
i would contest a sibling comment - the solution is not RAG, at least not entirely. 1) because it doesn't necessarily involve 'generation' (e.g our grounding model doesn't depend on gen ai) and 2) RAG is just enriching the context. even if you can get the retrieval right (not a given) it doesn't guarentee that the LLM will faithfully or correctly use that context, or that all its claims are supported by sources. RAG is not the same as traceability and verification
- Groups focused strictly on model capability (i.e. benchmark scores)
- Groups really focused on governance, interpretability, control planes, etc
- A small but excitable minority of weirdos (me included) who think they're the same thing
A lot of the stuff that gets loosely clustered under neurosymbolic methods might be interesting if you're looking to dive in. The sort of pattern you're describing is something that a lot of people have taken stabs at from different directions, or at least have tangentially nipped at while working on related problems.
So AI being a source of truth and information is not a something that important. It's a generator and we should use it as such, getting it to map domains and repeating patterns. It should not be used as an encyclopedia or oracle.
Serious question - with something like a programming language where there is a sort of finite information available about the language and how to use it. I've found once you start moving into areas where this isn't the case like metaphysics where there is a lot more ambiguity, does the sourcing and information then become more valuable? Or are we going to finally be ok with ambiguity as humans?
I can see what you're saying and agree, but are there areas where sourcing does in fact become more important?
I think they do become more valuable. Who said something and who believe it is equally as important as what was said.
What you are describing is that a majority of humans are inherently not interested in that. There are are a subset of people who would like to do that - and certainly can now that they have the tooling.
Its just a vastly strong tool. If you want it, build it.
Most people think AI answers are carefully sourced, triple vetted, over-researched to death and can trust it with their life, despite the warnings, until they run into nonsense and then dismiss it as a one-off.
If there was a "always give the best possible, most researched and trusted answer" flag that was easily visible, or maybe "chatGpt" and "bestAnswersGpt", where the second one did all the verification and was considered much better overall, guess which one people would start using after a while and/or quote, because of improved accuracy?
It's not that people don't WANT honest information (well, most people) - they just don't KNOW they're not getting it right now. That's the problem with tech - they're trading on "computers were always right in the past" because there has been no generally available "fuzzy search" type application that needs handholding to give "good" results. Most software either gave the result of computations that are always the same, or enforced static rules (like spellcheck) which were perceived as "always right".
Well prompted LLMs with appropriate tools can do exactly what OP asked for. So AI can do this. That the AI-based products people are being given don’t is not a limitation of AI, it is a failure of humans.
bad (or even made up) statistics have been floating around meat space since forever.
Of course for anything of consequence I’d go and actually read the sources (but for plenty of every day stuff it’s fine). On the free plan btw
Don't forget to add "don't make mistakes!"
Ah, the good old magic trick!
LLMs are really weird sometimes. I had some issues using a coding harness where the agent was failing to use the correct tool call syntax (I was using some rando free model on OpenRouter). I sent a message “look closelier at the tool call format” then all the tool calls for the rest of the session succeeded.
Whether Perplexity is useful to someone, and how useful, will depend on how effective it is in giving answers that are (1) correct as often as possible and (2) checkable if they're sometimes wrong.
If Perplexity's answers often treat AI-generated blogspam as reliable then that will probably make them less often correct (because people generating AI blogspam generally don't need what they generate to be consistently correct) and harder to check (because people generating AI blogspam generally don't need what they have to be clearly sourced, and because even if they did it's an extra step of link-following).
So if Perplexity turns up a lot of AI-generated blogspam, that probably makes it less useful. Is it "up to Perplexity"? Again, I don't know what that means, but I don't see that it matters. What matters is how much value it provides, at what cost in time and money and effort and attention. If its output is mostly repeating claims from AI-generated blogspam and that makes the answer "not much, and at too much", then so much the worse for Perplexity even if there's some sense in which it's under no obligation to do better. If its output is reliable and checkable, then so much the better for Perplexity even if some of its deficiencies are in some sense moral failures.
(Or at least they used to, I think they changed everything recently, but I've mostly stopped using them.)
*That particular stat was of course made up, but not by me.
One of the original php2shell announcement posts (which was clearly AI written) claimed that 70% of websites use PHP. This number has now been repeated by every single article about php2shell, including in the most trusted tech news outlets.
I think it could have been approximately true in 2008, but not in 2026. The number is too round anyway. Trying to search for a source to the claim is now impossible.
If the basis of a reliable fact is consensus, then every AI model should now consider this fact to be true. In reality consensus is not useful of course, offhand facts are frequently generated by citogenesis. An article mentioning a stat offhand is not a source. A source should take some amount of epistemic responsibility for a claim.
My guess is the 70% figure comes from W3Techs[2], which currently puts PHP at 69.2%, but only of sites "whose server-side programming language we know." Given that PHP is uniquely trivial to fingerprint (via WordPress markup, .php paths, PHPSESSID cookies, etc), it is very likely that it is overrepresented in their dataset[3].
[2] https://w3techs.com/technologies/overview/programming_langua...
They do also say they don't count subdomains separately, so wordpress.com is one site.
I think you are interpreting the data on httparchive wrong. 'All technologies' does say 8.74m, but again I think that is only successfully fingerprinted sites? If we filter to top 10m ranked sites, it goes down to 6.47m, with PHP at 3.19m (so 32%). Desktop is even lower for some reason.
PHP apps don’t have to use the session; if an app does not start a session deliberately, there’s no cookie. Some PHP frameworks don’t use the built in session. Bespoke PHP code may simply never use it. Or perhaps they do but they rename the cookie. Laravel doesn’t use it, for example. Magento does, but it renames the cookie. Prestashop stopped using it.
Web servers don’t have to announce they use PHP at all.
It is no more uniquely trivial to fingerprint than ASP is/was, for the same reasons.
There’s really a lot of dark PHP out there; I maintain a bespoke app that you would not be able to easily establish was PHP.
One reason why the 70% figure is worth going with is that you barely need any PHP on a site to exploit php2shell anyway. It’s a runtime vulnerability. If it’s possible to provoke the server to hand a URL to a PHP runtime, it’s potentially possible to exploit. A hell of a lot of cheap hosting has ready-to-go support for PHP.
It lends credence to the theory that a smaller model without the ghost of information past, could do better with ability to search the web, although this is heavily reliant on the effectiveness of the search engine itself.
All this is to say, I fucking hate AI and enjoy research.
tangentbordskanin (keyboard bunny): keycaps for mechanical keyboards with bunny images on them
indianbokföring (Indian accounting): simplified accounting outside of legal norms
bokföringsindian (Indian accountant): informal, older term for an auditor
potatisbensinskämt (potatoe petrol joke): a type of classic absurd riddles or word plays
nagelfis (nail fart): bulging pain in fingers after having been outside in extreme cold without gloves
And it regularly lies about local stuff, e.g., it says that coffee shop A has lower prices than coffee shop B when it's the opposite in reality.
Being particularly critical of LLM output after using LLMs is not an acquired skill either; what is acquired is the erosion of trust.
Of course the reality that adults do trust AI uncritically because it's the path of least resistance means the traditional education I mentioned above wasn't sufficient. But that's all the more reason not to introduce a variable that makes things worse.
The salad is also pretty trash if you use the default dressing.