Real neurons have some amazing properties that individual neurons in ANNs do not. It's still unclear how we can exploit those properties to improve AI.
I would image then, trying to imbue our neural networks with the properties of real neurons is a futile ambition?
Nobody knows how much would be gained by using a more biologically accurate model of a neuron. Much of the low level detail may be irrelevant, but the timing aspect of it, or some equivalent, might be helpful if we want to make ANNs capable of learning realtime pattern recognition and control.
> what appears to be
> Nobody knows how much would be gained by using a more biologically accurate model of a neuron.
Philosophical:
The neuron is an integral part of it's environment... so using models of much more primitive biological neurons ( less "processing" distance between env and self ) might yield something, I believe, but not evolved ones, where they are part of tens of thousands of systems in systems with only a few of them layered like onions. It's a completely different definition of distance and the neural nets are not part of anything.
> Much of the low level detail may be irrelevant, but the timing aspect of it
Tens of thousands of asynchronous contiguous and non-linear events firing at just the right timing in 10s of thousands different places at timings that create something even crazier than perfect synchrony ... That lower level detail is likely responsible for the timing ... it's a bit like why osmotic pressure inside and outside is important and implicit to all the ratios relevant in the different spots of the body/organ/tissue ....
I'm unimpressed by AI and genAI to this day. But I think it's cool. ... it can all be achieved without calling it "AI" at all. Just a bunch of nodes and well sorted and tagged input ... it's social proof laundered awe mostly ... no chance against that marketing machine, so LLM = AI. Every time you "think" it can be more than a dictionary and wikipedia I realize how stupid I am for thinking that briefly when I read the reply to my prompt. Same goes for code. It's holy shit, wow, ah wait, there's that open source code and that free frontend design bit and all the other things, snippets, assets I had already bookmarked somewhere or sometime in the past ...
In essence an ANN is just a complicated mathematical function that you can tweak to fit the training set. Originally the activation function of a neuron in an ANN resembled real neurons, but later it was found that other, very different, functions are much better, like RLE.
the neuron analogy has been there all along, since ANNs are entirely analogous to "BNNs" - for instance the NIPS conference.
the firing-rate-vs-activation-level difference isn't that big a deal.
Yet LLMs can achive miraculous results with so little. It really begs the question, where is the limit? How smart would a artificial human brain be if it would clock in gigahertz and had the somehow orders of magnitudes more efficient reasoning and detail recall of LLMs?
However LLMs are not really tiny compared to real brains, brains FLOP/s have been estimated to be around 1e15 to 1e21, while a single H100 SXM can already do 1e15 BF16 FLOP/s.
The brain also has about 1e15 synapses, each holding the equivalent of about 4.7 bits of information, which would be about 590 TB worth of parameters, equivalent to 294T BF16 parameters (would need 7.4k H100s to hold those, at 80GB per H100).
So in my opinion, it's more of a memory bandwidth problem than a computing power problem, as H100s have comparatively very little memory bandwidth per FLOP than the brain, which holds information directly inside the computing units (synapses).
And I also think that detail recall is a double-edged sword. Memorizing details can certainly be useful, but it can also compete with pattern generalization. If you can perfectly remember the training set (overfit) you don't need to learn to generalize.
By lowering the clock speed, power and heat drop exponentially.
I'm just hoping we figure out something logistically better than laminating chips together.
Where I think the really interesting parts missing are those in real neural circuits do, especially since the connections are more heterogenous, and the dynamics are much more fluid and time-dependent, whereas typical at-scale computer neurons obviously operate at fixed time steps, and don't exactly have bursters etc.
I wrote a very large scale distributed simulations for real neurons (straight from imaging datasets) when I was working in a TMS lab, and a longtime side project I always wanted to get going was some kind of pipeline for spitting out neuromechanical/cable-theoretic RTL tailored to specific morphologies and networking them according to their connectomes. I don't even think I would see any performance improvements, it just feels fun.
For a long time I've wanted to engineer a cell that can pulse like a fluorescent light by computing a taylor series approximation of a sine curve. But to do that you'd have to implement a reliable system and cells already can do this with fairly high accuracy using their internal existing systems.
I totally see the CheY as an intermediary layer that learned how the changes in attractant must affect mechanical activation.
in short, a continuously running graph that can rewire (instead of a feedforward network) with message passing; it would run training and inference simultaneously (inference would continuously retrain the network). Such a system running on the largest GPU clusters would be an interesting experiment in itself, although I don't think it would match the direct raw performance of today's AI models, and may never actually be better than just using feedforward networks trained by backpropagation, with access to external signals.
I'm also pretty sure people have been, and continue to, experiment with this. The current architecture is by no means the final form of LLMs. It's being tweaked continuously, though you don't hear about it as much anymore (unless it's DeepSeek dropping a bombshell every couple months).
Thing is, despite half the posters on HN and elsewhere arguing otherwise, LLMs are making such a massive, world-transforming progress, that humanity can't keep up with it. A good chunk of the world economy is now focused on AI R&D, and there's still not enough capacity to pick up all the low-hanging fruits found over those few years - there's still too much to gain from pushing the current paradigm.
Every now and then you see someone stumble over one of these low-hanging fruits, and it sparks a mini-revolution. Jev being the most recent example - a simple idea, arguably well-known in ML circles, but it's suddenly a big thing because someone stopped long enough to pick it up. There's many, many more such ideas still waiting for a minute of attention of someone with few million $ worth of GPUs to spare.
DNA, the molecule and associated enzymes, is a computer in and of itself. Every dimension - from its hydration to dimensional folding - is a computational manifold.
The cell itself is a system of distributed supercomputers, a machine assembly factory, a dizzying highway and information network, and a universe. Replete with defenses, power factories, intricate and robust signal processing, and self destruct mechanisms.
(That's true for computers (on semiconductor basis) as well, is the behaviour of an addition in the ALU or in the bit pattern to select the instruction? It's both, the ALU alone would exhibit all kind of operations (including unintended/not-meaningful ones), the bit pattern alone could trigger any kind of operation depending on the circuit it occurs in. But if both happen to occur at the same point in spacetime, the physical laws apply and the bit pattern emerges, that humans would call the addition.)
There are a very small number of DNA enzymes produced in labs, but there are no endogenous DNA enzymes (that anybody has found so far). Instead, it seems like life is much more based on RNA enzymes and proteins (the core of the ribosome is an RNA machine, decorated by proteins that enhance its speed and reliability).
It is cool, but not intelligence.
very simple organisms can't do that, at all. you really do need something with a lot of internal state and state-transition structure (ie, neurons) to do that.
Anyone who likes The Thought Emporium should watch his talk and branch out from there; you'll be amazed.
Your reductionist argument is as oversimplified as saying "Earth is just the cooled ejaculate of a supernova." True at some level, but only at a level without practical meaning for most purposes. It sounds cool and insightful. It isn't.
e.g. One reasonable definition of intelligence is ability to use prior experience to fruitfully guide future behavior, and if you look at the spectrum of animal intelligence it seems most useful not to have some arbitrary cutoff or mechanism attached to that. A simple animal like an insect, or a bacterium, has effectively encoded the prior experience of the species, in a relatively stable environment, into its DNA.
Animals with more complex nervous systems have ability to learn as individuals in their own lifetime, and display intelligence according to what they have individually experienced, although they too will have been born with some species-level intelligence as to how to survive & thrive in their environment.
If your 'problem' as a cell is to feed yourself, it is a clever apparatus that solves that problem, and can be depicted as intelligent.
If you understand the 'fully automatic mechanism' completely, you agree that it solves a problem, but you deny that this is intelligent. However, the denial depends on your understanding of the mechanism. If you assume that the world is deterministic (that means, every effect has a definite cause, to put it simply), we cannot be something other than a 'fully automatic mechanism', however a very, very complicated one that we do not fully understand. Indeed, in a deterministic world nothing can be different from a 'fully automatic mechanism', and, by your implied definition of intelligence, nothing can be intelligent. This conclusion leaves the definition of intelligence as something 'other than fully automatic mechanism' rather useless, as it does not distinguish anything. (If you do assume that the world is not deterministic, a similar argument still holds, as long as you do not imply that a person can determine the outcome of the non-determinism, which sounds strange to me to imply.)
I think what you want for something to call 'intelligent', is that you don't immediately understand how the mechanism works and the assumption of some kind of thought process and/or planning. The first property unfortunately depends on what you might consider your own intelligence, which makes it difficult as an objective measure of intelligence so that you can argue about it in a forum. The second property (thinking/planning) might be present in the cell in a way that you don't see (eg. see storage of the current value of 'now') and, similary, it might not be present in us humans in the way self-perceive it.
More than once did I experience a seemingly 'intelligent system' that I found fasciniating, which, after I studied it, I did not think of as intelligent anymore (eg. my first impressions of Weizenbaum's Eliza), which for the aforementioned reasons I think is kind-of silly of me.
I personally define it as more of an emergent property where you increase the complexity of the problem solving machine to the point where it begins to be more than just interconnected automatisms.
Maybe it could be said that processing information is not intelligence by itself. Intelligence, for me, requires the ability to adapt the way one processes information to the task at hand, either by changing it entirely of adapting and tuning to get more out of it.
A fixed mechanism that controls a variable can't be intelligent.
A mechanism that makes a rotor go one way or the other as a function of the time derivative of a variable would not qualify as such.
Now, if you have a trillion such mecanisms atuned and feeding one anothers inputs in a network... Well you just got yourself a brain and then we can start talking about intelligence.
It is more than just "cool". Furthermore, I would probe and inquiry what you actually mean by a positive definition of intelligence so we can get the claim better instead of a negative defintion.
If you have a trillion neurons, some doing basinc control stuff like the one before, with others forming systems that tweak or reroute the others depending on other, emergent potentials, then you have intelligence.
If the brain activity stops being only external information driven and starts being internal information driven (this is, it can change how it regulates stuff o even the stuff it regulates, at different levels of consciousness) then we start talking about something more than information processing.
Your eyes are just a series of mechanisms by which photons are converted into electric signals via proteins that react to the impact of a photon. "Seeing" is mechanical. "Feeling" is mechanical. Receiving and sending the messages is "mechanical". At what point is intelligence separate?
I can't control _how_ my intelligence works, and neither can e-coli.
My "intelligence" is the sum total emergent self-referential molecularly-mechanical signal-processing network that holds state. My DNA holds the state of all of my ancestors on top of which my own environmental changes are laid. My neurons hold the sum-total of my experiences, allowing me to introspect.
Here's my test: - I am intelligent. I have an internal representation of the world that shapes my behavior, and I react to my environment. I can choose to do "irrational" things.
- A mouse is "intelligent" but less than a human. It has a representation of the world that shapes its behavior, and it reacts to its environment. Mice can choose to do "irrational" things like care for their young, and form attachments with humans.
- A mosquito is "intelligent" but less than a mouse. It still has an internal experience of the world that shapes its behavior. It does not care for its young, nor does it form attachments (that we know of).
- Down, and down we go, one thing staying constant -- there is an internal representation of "the world" that guides an organism's behavior. It is an emergent quality of the interaction of mechanical sensory systems working at a molecular level where the goal is survival and reproduction if possible.
I think its intelligence. ymmv
There must be plenty of “versions” which don’t do anything but generations later are crucial to a function.
How does evolution not get stuck in a local optimum?
So in our case the starting conditions just lined up the way it did and no single step (or series of steps) easily flipped our wiring, so we just rolled with it (and we get by quite well).
Vertebrates for example had a at least 2 whole genome duplication events in the past giving us up to 4 copies of the same family of many genes. This enables mutations to alter one copy of a crucial gene to "invent" new functionality without killing the organism.
The most common version of this is probably duplication of G protein linked receptors for odor and taste. Mice and dogs have far more of these than we do.
Evolution does get stuck in a local optimum quite a bit. Think of how many vertebrates are now limited to 4 limbs compared to many insect or mollusk species.
Instead of having dexterous forelimbs around the mouth like crabs, elephants make due with prehensile noses.
Similarly, to get to running speeds while having opposable thumbs humans evolved to stand upright on the hindlimbs instead of simply becoming 6 limbed centaur like creatures. Humans can't outrun any major predators at short distances.
Could be wrong, but my understanding is that it does? Plenty of things in nature are not really evolving much at all (as in, variations are not selected) even over longer periods of time. What can happen to it is getting removed by something that evolved from somewhere else (taking the evolutionary niche, or making it unsurvivable)
Neutral drift and very large population sizes. Anything that preserves diversity in the population long enough to find the right basin. Most artificial evolution systems cannot support population sizes large enough to preserve diversity long enough to locate any solutions that matter.
Right now, evolution is running a live experiment with something like 10^30 candidates in one population alone. The computational requirements for this are effectively zero. The environment is the computer.
This is why some animals have barely changed in appearance for hundreds of millions of years, e.g. scorpions, while others have changed so much that nobody would guess the relationship between them and their ancestors, had we not discovered some intermediate steps in their evolution.
More recently, e.g. 70 million years ago, when dinosaurs still lived, the ancestor of humans looked more like a rat than like a human, but the ancestors of sharks, crocodiles and big constrictor snakes looked pretty much the same like today.
Now I'm wondering whether HN uses some sort of language model to infer semantic similarity and thus kill content automatically. Frankly, whether this is automatic or by consensus of certain users, there's not much of a difference. It is dystopian, especially given the mountains of meaningless one-line comments that abound here (probably “too little substance to be worth killing”).
Once upon a time the Internet was the realm of free thought and free speech. I get it that spam etc needs to be fought and suppressed, but frankly, none of my comments that have been killed (quite a couple of them already) come close to that category. Rather, it seems the ruling HN mindset is about as narrow-minded (if not more so in certain respects) than your typical mainstream forum.
https://ccsb.scripps.edu/goodsell
i've read and can highly recommend his book The Machinery of Life for a patient and visual explanation of these baffling machines!
This reminded me of Tim Davison’s LifeBrush, an interactive VR painting and simulation system explicitly inspired by Goodsell’s illustrations.
In his thesis, Davison describes using it to "structurally recreate, simulate and step inside" famous Goodsell illustrations -- the cytoskeleton demo is a striking example of a painting becoming a world you can enter and set in motion:
https://www.youtube.com/watch?v=6CsxADBpal0
https://hdl.handle.net/1880/111493
I first called LifeBrush "Emacs for Cytoplasm" in a Boids thread and asked Davison whether the molecular motion was slowed down. He replied that medium-sized proteins move around 5 m/s, and that LifeBrush deliberately slows and smooths Brownian motion for illustration. The code was open source; at the time it required a VR headset and had a clunky, research-grade interface.
https://news.ycombinator.com/item?id=22715199
https://news.ycombinator.com/item?id=22724602
https://github.com/timdecode/LifeBrush
The interaction is the magic: in the overview, a ribosome moves along mRNA as tRNA brings amino acids and a polypeptide chain grows. You can grab the ribosome and move it where you want, rather than wait for Brownian motion.
I described it as dragging the ribosome’s "lazy ass" through "tRNA pasta and mRNA meatballs." It’s a scientific illustration you can play with, not a literal molecular-dynamics visualization.
https://www.youtube.com/watch?v=5LBHRyGxLGI
https://www.youtube.com/watch?v=624F5iaCKco
https://www.youtube.com/watch?v=L482dpH9S8o
https://www.youtube.com/watch?v=bUnTEI9QZPE
Davison’s work has since grown into CellWalk, a guided, interactive tour through a bacterial cell, from the whole cell down to its atoms, for iPhone, iPad, and Apple Vision Pro.
And here the Goodsell connection is even more direct: Davison’s site credits David Goodsell, Ludovic Autin, and Martina Maritan with the CellWalk model and tour, based on their whole-cell paper. Davison is now CEO and cofounder of Ako Biotica, the company behind CellWalk.
https://www.youtube.com/watch?v=r8DOAPPjMw0
https://doi.org/10.1016/j.jmb.2021.167351
LifeBrush papers: the 2018 conference paper, followed by the 2019 Computers & Graphics article on painting, simulating, and visualizing dense biomolecular environments.
The baffling intelligence of a single cell: The story of E. coli chemotaxis - https://news.ycombinator.com/item?id=39777229 - March 2024 (213 comments)
>Kinases and phosphatases work ceaselessly like ants in a nest, adding phosphate groups to proteins and removing them again. It seems a pointless exercise, especially when you consider that each cycle of addition and removal costs the cell one molecule of ATP—one unit of precious energy.
>Indeed, cyclic reactions of this kind were originally labeled “futile.” But the adjective is misleading. The addition of phosphate groups to proteins is the single most common reaction in cells and underpins a large proportion of the computations they perform. Far from being futile, this cyclic reaction provides the cell with an essential resource: a flexible and rapidly tunable device.
...until you realize it's probably a game theory problem. If you can (effectively) think faster than other E coli, you can reach the food faster.
It's a Prisoners' Dilemma: the E coli can't trust the other E coli not to 'defect' and put stupid amounts of RPG video game points into faster food-seeking, so every E coli has to do it too. This drags up the metabolic penalty to "inefficient" levels.
We should expect the metabolic penalty is (somehow) dialed down when:
* food is abundant
* E coli population density is low
* food is homogenously distributed
...and then I read that social animals are noted for having unusually high intelligence. Maybe not such a joke after all!
"It's so complicated and unlikely, that it must have a designer."
OK, I'll bite: how unlikely?
How many times must the constituent chemicals bump together before they join to form RNA, divided by the number of times they must have bumped together? I'll wait for you to do the math...
If the odds are once in a quadrillion seconds over a quadrillion square meters of Earth-surface conditions, that means it probably happened two or three times already on Earth. Anything more common than 10^-30 per second per square-meter of "Goldilocks" area is fairly certain to have happened already on Earth. Anything a few orders of magnitude less likely is still going to happen frequently throughout the universe.
The frequencies and opportunities, however, are both unknown. "It's really unlikely!" is a stupid statement. It's really unlikely that someone hit by lightning has won a big jackpot, but it's still likely to happen, over all our civilization. Odds times opportunity is the number that no one can even make a reasonable SWAG for. Knowing that the odds are low tells you absolutely nothing.
I think this is one of the hints the Programmer gave us that once we reach the needed level of sophistication we'll understand. Similar to the ending in the book "Contact".
(So to state it differently, Christianity says that the God which you are talking about, does indeed not exist.)
[1] Transcendental meaning that it is not only the origin of specific things, but of space, time and causality itself. God is not before the big bang, because the meaning of before is in fact something that only exists in the universe. God is as much before, after and next to the big bang as he is to now.
This is why to me it seems pretty logical, that modern atheism occurred in a christian dominated culture. It's not that the atheists invented a new idea. They simply "forgot"(more like didn't agree) what Christianity says a God is, returned to the definition of God from the Antiquity and "simply" rephrased the criticism Christianity already voiced for millennia.
This assumes (your concept of) God is subject to time or/and causality, which makes it non-trancendental.
> God is transcendental by definition
Yes, but not by everyone's definition, which is the point.
It also could very well be the case that there's a limit not only to the complexity of things a human mind can grasp, but also to the simplicity thereof, and the concept of God may be located beyond the latter.
Bringing the concept of God into a biology discussion is sort of embarrassing. "Intelligent design" is for the unimaginative mind. We must always be open to being wrong, evolution fits what we know now, but I'm sure in a century we'll have a better understanding and it'll be a more complete understanding, but what it won't need is a divine eternal.
The world's not flat, there is no god. Can we get back to science now.
An Alien God (as a way to view evolution) from 2007 on lesswrong, isn’t comforting:
https://www.lesswrong.com/posts/pLRogvJLPPg6Mrvg4/an-alien-g...
"""the Shaper of Life is not itself a creature. Evolution is bodiless, like the Judeo-Christian deity. Omnipresent in Nature, immanent in the fall of every leaf. Vast as a planet's surface. Billions of years old. Itself unmade, arising naturally from the structure of physics. Doesn't that all sound like something that might have been said about God?
And yet the Maker has no mind, as well as no body. In some ways, its handiwork is incredibly poor design by human standards. It is internally divided. Most of all, it isn't nice.
In a way, Darwin discovered God—a God that failed to match the preconceptions of theology, and so passed unheralded. If Darwin had discovered that life was created by an intelligent agent—a bodiless mind that loves us, and will smite us with lightning if we dare say otherwise—people would have said "My gosh! That's God!"
But instead Darwin discovered a strange alien God—not comfortably "ineffable", but really genuinely different from us. Evolution is not a God, but if it were, it wouldn't be Jehovah. It would be H. P. Lovecraft's Azathoth, the blind idiot God burbling chaotically at the center of everything, surrounded by the thin monotonous piping of flutes."""
It sounds like you actually refer to pantheism, which is very much different from Christianity.
The name for this kind of random connection of disparate things, implying meaning, without providing any, should be "'if' abuse".
Cells are autopoetic. They write their own experience.
When we found out how "Space and time are connected", that actually meant we found mathematically how that relation works. It's not just some vague thought to ponder about. You can use space and time being connected to solve real problems about the world, like "Where will this particle be in the future and why is it somewhere you wouldn't expect?"
You can use "Space and time are connected" to understand the world, like why Mercury exists how it does. In fact, space and time being connected is necessary to solve that quandary.
The point of science is to advance our ability to predict the future from the current, not write high school poetry.
What would "Space is awareness" mean?
You don't need to do that. You are pursuing a goal that is not assigned to you by nature. This goal is assigned by the business and a perceived "human benefit".
This end-less understanding and influencing of the things should stop. Even if you say there are medical benefits to such investigations, you are going against nature and trying to control a few things while ignoring the full context that turns against you.
It's a zero sum game. It appears to get better only because you are only looking at the corner that was wiped clean, but not the other corners where the garbage went to. And for some reason the "wiped clean" state is considered a good thing. Other misconcieved "good" things are straight edges, flat surfaces, smooth curves, uniform colors and shapes, artificial uniformity in general, thinking hard, smartness, precision, excessive hardness or strength, extreme skills, any characteristics that reflects a lot of human/macine effort etc.
Human progress doesn't require too much of human agency to the extent of causing irreversible effects or a great risks. Right now the agency is completely out of pace with the nature, arrogantly driving things and crumbling down when nature stares back at.
Again this recourse to nature is just druidism. You have a particular metaphysics and morality, it's just wrong.
You are literally posting on the internet, likely from an air conditioned room. You can and should stop using these things.
That argument was used to argue against the morality of homosexuality. You haven't elevated it in this application.