In the context of the paper, the entire book seems to go downhill from the definition of ontology for me.
There is no benefit of using Gruber's ivory tower definition. A simpler explanation (e.g., it describes a structured framework that defines and categorizes the entities within a specific domain and the relationships among those entities) would have sufficed, and easier to digest.
Palantir is doing nothing revolutionary or "paradigm shift" when it comes to data and information organization. Their secret weapon is not introducing ontology to information.
Ching (1000BC?) classified reality into binary ontological primitives, created trigrams and hexagrams a combinatorial ontology. Aristotle introduced categories, substance, properties, relations, etc. Thomas Aquinas systemized Aristotelian categories into theological knowledge systems, and used structured classifications.
I am becoming curmudgeony as I see more and more of these reverse-research papers. Write the paper, then find references that fit the statement and use weasel words ...
unbelievable scene unfolds, deep-rooted disease of silos, paradigm shift, fatal flaws, forged in these extreme environments, eliminated to the absolute limit...
Gag me.
The tech stack ontological model is flexible like Salesforce so that it can be jammed into any task or contract quicky. It isn't engineered, it's glued in.
They're able to do this fast because they have a flexible model and because they have the friendly relationships.
Their moat deepens every year with every new integration.
It's smart as hell, actually. That's why they're swimming in money. And government contracts are about as lucrative as you can get.
Engineers turn their nose at this, but look who has tapped into this wealthy revenue stream. While we preen about good architecture, they can retire for a thousand thousand lifetimes.
Is it a secret? I got an impression that it has been well known. How could you get any big number contracts without former secretaries or retired generals in your board or in your ‘consulting’ team?
They're a gov. contracting agency, with some re-usable components, that's it.
If they deliver stuff that works, good, if not, bad.
There's nothing interesting about 'ontology'
We've been using ontology well before RDF and the semantic web. It precisely describes their flexible engineering approach of using entities, definitions, and relationships.
This may be one of the most tone deaf, american imperialist sentiment, I’ve heard on HN for a while.
Engineers who have any sense of morality have a pretty good reason to turn their nose at this, and there is no but needed to follow that sentence.
It's more or less in the same vein as pointing out that WordPress powered a massive chunk of the Internet despite violating almost every good coding practice you can name, and that getting things done is what makes money, not building ivory towers.
The fact that you turned that argument into some sort of anti American screed says much more about you than the parent.
This engineer turned their nose at the bad architecture and glue code, but neglected to mention the total lack of morality from Palantir. I would argue that abandoning morality and aiding the American imperalist machine in its war against human rights and dignity, has been a much bigger reason for Palantir’s success then their lack of good engineering practice. They are willing to get paid for something most people morally object to. Lots of engineers are willing to abandon their craftsmanship if it pays well enough, few their morals.
Perhaps I read too much into this absence, in which case the post is only tone deaf, but I favor the read where this absence was intentional, in which case it is both tone deaf and American imperialist.
I've been tracking this project since 2018. It hasn't changed drastically since then, but man it was polished and robust library.
Perhaps gov contractor speak would be more accurate. I'd think a corp with an org sufficiently aligned to their business value and profit motive wouldn't stand for the fancy speak either
Also, what is UDF?
Also sometimes Lua, which is kinda a nice middleground between c++ efficiency and python ease of writing
Google will inmon for the info factory
Google data vault
Joe reis is the guy for tying it up with a modern bow recently.
Designing data intensive applications book
If performance becomes an issue, just turn it into an MV... and then consider some indexing on the JSON itself.
The paypal mafia are all about stories. They can attract talent and investor money with those stories, but they are just tall tales full of hype, and people are catching on (ok that last bit might be a hopium).
Their bread-and-butter is a few things
1) Willing to do dirty/harmful things no one else will touch
2) Making data and data analysis accessible to cops, dhs, anyone that is especially tech-averse (many police departments disqualify based on IQ test results measuring too high). You can type in a license plate, a name, an address, scan a face and it will show you every relevant information, but also contextualizes it and enriches it with any other data. You could to this in excel, postgresql, bigquery, etc.. but palantir gives these people simple text boxes, buttons, and links.
3) Their forward deployed engineers are great at what they do. They station their guys wherever Palantir is being used, and they'll work very closely to get things done. to make sure all problems are solved asap, and its users are very well educated on the usage of the platform.
This post looks like it's written by AI, but assuming it is in earnest, it isn't really ontology, at least no more than object oriented programming is ontology. Excel is all about numbers, palantir is all about people (or people-documents). It is simpler than excel and has BigQuery level analytical power behind it, and the human touch to make that interaction go over really well.
I said it's mid because you could do a lot more with just the dataset and queries. You could even possibly do more with command line tools and hoards of data files (minus the OCR and document scanning they do, as well as LLM/NLP). but that isn't accessible and takes a lot more time. Not to mention normalizing, extracting and structuring wildy unstructured data isn't easy. But with BigQ for example, it is done plenty, you just hire a team to do that for you typically.
Their ecosystem is basically google search (including image, reverse image,video,etc..) but much more targeted and oriented towards displaying collated data from hoards of structured and unstructured data (including pdfs, docx,etc..). I would prefer grep, bigquery,splunk myself. but for end users, palantir is unmatched in my experience.
But I'm not selling them here, I'm trying to communicate the power at the disposal of those who use palantir's platforms. Google could have crushed them any time, except even for Google the type of work required was too ghoulish and reputationally risky.
Even with MS copilot(lol), chatgpt, gemini,etc.. running as agents, they're not as simply as palantir's stuff is for searching your data. and you don't have specialists integrating all your data onsite either.
Ultimately, the bigger problem is that even in crowds like HN's, no one seems to have a good idea of what should be done about governments abusing datascience so efficiently. Every answer comes back to red-tapes and regulations, possibly criminal consequence. Are you willing to give up the liberties tech has enjoyed so that future generations can be well, and have shot at peace and prosperity? (ours is too far gone in my opinion)?
China is doing this too, but much more efficiently, much better and at a greater scale. but their society has accepted this, and traded certain liberties for social stability and economic prosperity. The west hasn't done that. lawmakers and the public at large need to be informed by those in tech about these things so informed decisions could be made.
The general-populace/crowd/mob has already lost this game. Govts/Companies (all of them irrespective of ethics/democracy/etc.) are doing what they want with data and datascience. The populace is easily propagandized/distracted from reality and can be easily cocooned.
The only recourse left for the individual is to learn and start playing the game himself. Fortunately the new tools are a great help in this asymmetric warfare. Organizations like EFF/OpenSource/GNU/etc. need to take the lead on this since most people are like sheep when it comes to uses/misuses of technology.
Palantir is just "Cambridge Analytica" redux but with more money/connections/data/breadth/depth/etc. Watch this old presentation by their then ceo Alexander Nix and extrapolate to today's AI world - https://www.youtube.com/watch?v=n8Dd5aVXLCc
Finally go and read the works of George Orwell, Edward Bernays, Jacques Ellul, Marshall Mcluhan, Noam Chomsky etc. on the whole subject of Propaganda/Manipulation to really understand where we are now.
Under rule of law, you need laws. Outside of rule of law, well..
The idea is to protect oneself by using various techniques of deception/reflexive control through stratagems like "poisoning the well", "borrow a knife", "misdirect/misinform" etc. to present different contradictory information/personas/overload-data to "The System" thus obfuscating/masking reality.
You mess-up the raw data and then use reflexive control techniques to allow them to draw inferences from a set that you have predetermined.
Technology, mass commoditized and given easy access to, is central here and should be spearheaded by the organizations i mentioned due to their reputation cachet eg; Tor/Tails/Proxies/Vpns/Secure OSes/Protocols/etc. Simultaneously they should also push for legislation for stricter govt. control over data usage/companies eg. GDPR/etc.
The populace also needs to educate themselves on techniques of Propaganda/Manipulation by reading the works of the authors i mentioned earlier. This will sharpen their critical thinking skills which is a must-have today.
Palantir is very much in the vein of Cambridge Analytica except that they have expanded vastly on its theme and gained "legitimacy". People need to study all about CA and then extrapolate it to today's AI capabilities if they want to understand the shape of things to come.
Some References:
Deception: The Invisible War Between the KGB and the CIA by Edward Jay Epstein; For history/ideas/themes/motivation for further study on a complex subject.
Propaganda: The Formation of Men's Attitudes by Jacques Ellul - https://en.wikipedia.org/wiki/Propaganda:_The_Formation_of_M...
Reflexive Control - https://en.wikipedia.org/wiki/Reflexive_control
Psychographics - https://en.wikipedia.org/wiki/Psychographics
This claim is unsupported by publicly available sources AFAICT.
https://www.google.com/search?q=verify+claim+many+police+dep...
What is this AI slop doing at the top of HN? Come on, you don't even have to click through to know it's slop! It even has an en dash right in the title!
[begin]
#### The Paradigm Shift Brought by Palantir: Ontology as an Operational Layer
The *"Ontology"* strategy by Palantir, explained in this book, is a paradigm shift that fundamentally breaks this deep-rooted disease of silos.
In the context of knowledge engineering and the semantic web, the widely cited academic definition of "ontology" is an "explicit specification of a conceptualization" by Gruber (1993).
Furthermore, Studer et al. (1998) expanded on this, proposing the definition of a "formal, explicit specification of a shared conceptualization."
This transition from "data just for viewing" to "data that directly drives the business" is the key to true digital transformation in the AI era.
[end]
Just gave me brain damage. Please for the love of god just go straight to the point. Just give me the prompts that wrote all of this.
> Link type: The relationships between object types, supporting 1-to-1, 1-to-many, and many-to-many relationships.
Seems incredibly naive in terms of symbolic representation of knowledge. Maybe I spent too much time with OWL.
OWL 1, for example, has stuff like transitive properties (the classical example is A ancestorOf B, B ancestorOf C, therefore I can infer A ancestorOf C if I annotate ancestorOf as a transitive property).
Union, equivalence, inversion, symmetries, cardinality. Those are all possible to represent symbolic in OWL ontologies.
They're also neatly separated in different types (OWL Lite, OWL DL, OWL Full). OWL Lite and DL for example are proven to be decidable (you won't get some halt when doing inference, no matter what).
I know there are plenty of database engines to store triples and graphs, and plenty of reasoners out there.
I haven't studied OWL 2 yet or newer stuff like SHACL, but I know it's supposed to be even better.
Ontology is one of those fancy words that sounds important but is basically, as another poster pointed out, a standardized vocabulary.
Watch these old presentations by CA's then ceo Alexander Nix and extrapolate to today's AI world - https://www.youtube.com/watch?v=n8Dd5aVXLCc and https://www.youtube.com/watch?v=6bG5ps5KdDo (note the Q/A at the end here)
Also watch this interview with Christopher Wylie the CA whistleblower and again extrapolate to today's AI world - https://www.youtube.com/watch?v=FXdYSQ6nu-M
Be afraid, Be very afraid.