It was my understanding that all this connectome-based research was largely a deadend, because it doesnt capture dynamics, nor a vast array of interactions. if you've ever seen neurones being grown (go search YT), you'll see it's a massive gelatinous structure which is highly plastic and highly dynamic. Even in the simplest brains (eg., of elgans), you get 10^x exponential growth in number of neurones and their connections as it grows.
Connectome-adjacent neuroscientist here. Definitely not a dead end! But also definitely not the whole picture.
One of the main open questions in neuroscience right now is how network structure, dynamics, and function are related in the brain. Connectomes provide tremendous insight into structure, but as mentioned this does not generically solve either the dynamics or function problem. For example, for many of these neurons we don't have a good understanding of their input-output relationship, and the nature of this relationship can strongly affect the dynamics that emerge in a highly connected network. Individual variability across connectomes, and how connectomes change over development are also a significant issue, but at least for the fly it's thought that many of the basic structures are pretty conserved across adult animals, even if many of the details could differ.
Modulo these caveats, knowing the physical network structure of the brain does still impose huge constraints on what kinds of models we should be using for gaining insight into dynamics and function. For example, there are well known areas (the "mushroom bodies") with specific feed-forward connectivity patterns that are very different from a random recurrent network. Further, there are at least some areas in the fly brain where we think there are indeed quite clean structure-function relationships, e.g. in the central complex of the fly brain, which contains a physical ring of neurons and is thought to support a "bump" of activity that acts as a sort of compass that helps flies navigate via a ring-attractor-like dynamical system. Thus, even though it has many missing pieces, a wiring diagram like this can be tremendously useful for generating hypotheses to guide more targeted experiments and theoretical studies.
How's Open Worm coming along? The connectome of C. Elegans has been known for years, and Open Worm tries to simulate it. [1] Not with enormous success.
Very Nice. --from a Connectome-Centric neuroimager :)
One technique that I am pursuing right now is information decomposition of timeseries to separate the mutual information of two timeseries into redundant and synergistic informational atoms (synnergystic here means the degree to which knowing both timeseries gives you more information than the individual parts give (more than sum of parts). The big limitation of the method is the geometric explosion in complexity of the decomposition as the number of time series grow, with most analyses being limited to two or three times series at a time. However, the scale of the data on which it is applied is not requisite, meaning the approach can equally be used on the mutual information between two regions of interest in rsfMRI , or two spiking timeseries from individual neurons.
https://en.wikipedia.org/wiki/Partial_information_decomposit...
Thanks for your insight! How repeatable are these structures between individual animals? Are they very similar or is it more like “here’s a feed forward kinda bit, here’s a toroidal bit, and over here it’s just a mess”?
This is done agaist an adult so all the neurons have already grown.
connectome isn't a dead end but it doesn't solve all known problems. It's like making a static map which you can then use to inspect all those cars driving around (the dynamics) and crashing (the interactions).
Giving scientists access to the connectome snapshot alone is very exciting. The first step to understanding why something is and how it came to be is seeing what it is.
There are systems at play that form the brain into what it is and we don’t know much about them. The individual neurons — we have a better understanding of, but not the emergent systems. Now that many more scientists will know what the target for these systems is — what is the brain they shape, we can start to understand the control and feedback loops that result in this snapshot state of the brain.
And that’s why it’s not a dead end. Just because it doesn’t immediately give some sort of a consumer product, doesn’t mean it’s not a step forward.
You don't get the dynamics from connectomes, but you absolutely need them. So it isn't that they are a dead end, it is that the dynamics by themselves are also insufficient and the connectome is insufficient, you need both. Further, if you want to actually be able to have anything to attach the dynamics to, you need the cellular anatomy, so connectomes are absolutely necessary. The fact that connectomes are insufficient does not mean that such research is a dead end, but rather that the prerequisites for understanding the nervous system are vastly more complex and demanding than some might have hoped.
It is like getting a static map of the country's roads with no cars on it.
You can not make it come alive with cars (activity), but you can infer where people need to drive but you don't know when and why they drive or what they are doing, but it is a major clue.
> It is like getting a static map of the country's roads with no cars on it.
I was thinking it was more like giving somebody iPhone schematics and die shots of all the chips and then asking them to figure out how Portrait Mode works in the Camera app.
The difference is that in the brain there's no real separation between hardware and software, so I'm your analogy, we also have the equivalent of the source code, but just maybe not the environment configuration needed to get it to run (nor would we at this stage have sufficient compute to fully run it).
Sort of, but mostly not. The critical distinction is that, given better data (the instruction set, the source code or binary of the OS and camera app), the schematics and die shots aren't necessary or even useful.
It's unlikely that brains have an abstraction layer like that, so work like this is a necessary precondition to understanding the rest of how it works. That actual understanding may be elusive for quite some time to come, but without a connectome, forget it, no change.
It would be really inefficient and neurons inherently provide a great deal of flexibility. Larger animals might use this kind of thing, but insects don’t have that many neurons to work with.
Luckily this is science so we can actually find out.
> So, to run the same [fMRI, NIRS,] stimulus response activation observation/burn-in again weeks or months later with the same subjects is likely necessary given Representational drift
Going to need a significant improvement in the software to get it to map a human. The fruit flu has 140,000 neurons and 54.5 million synapses and the AI that mapped it required a post process with humans checking it all with 3 million edits and they still have to identify every neuron type.
A human brain has about 86 billion neurons and quite likely many trillions of synapses and that is likely an underestimate. That 3 million edits will turn into 3 million * 10^6 at least manual edits, that doesn't seem feasible. The error rate on the fruit flu would have to come down into the single digits to be usable to map a human brain. So an improvement from about 6% of synapses to 0.000006%. That is one heck of a jump in improvement for an AI.
Did a rough calculation, it would be more like Edinburgh.
There's easily a century between the earliest accurate map of Edinburgh and the earliest accurate map of the world. And even at present, the accuracy of maps of Edinburgh is much greater than the accuracy of maps of the world.
So yeah, the whole world could be next. But the person you're replying to has a point when they say significant improvements are needed.
We did map a handful of brains yet, the more we do the better we will get at it.
I don't understand all this rushing and skepticism when such amazing science is being done. It's not like some AI company marking claims to sell a product, it's some researchers trying to accomplish something. Yes, they should (and probably will) do it better but that's not the goal here.
If 3 million manual edits are still doable then it's ok. And when the manual step is not feasible, a jump in the tech will be required.
It’s a neural network without weights. And it doesn’t have a body.
Figuring out the behaviour of the neurons could take decades, although I have no doubt that people will eventually. And simulating a whole fruit fly body seems like it’s going to be out of reach for a very long time.
no. turaga and co have some work where they constrain model network topologies with the connectome and train on visual data. this is imo a very silly line of research and they come to some very wrong conclusions about what neurons do what with it. but that's the closest to what you're asking for
Out there question: Do you have a hand crafted database of these setup or some sort of macro to take the output of the search api and form it like this, or are you hand editing these lists?
I don't think that drosophila are eutelic (https://en.wikipedia.org/wiki/Eutely) so no two flies have precisely the same cells at precisely the same locations (that's true for c. elegans, whose connectome is probably the best studied).
The large-scale architecture will be roughly the same between any two individuals. You would likely need some sort of mapping (like an embedding) to generalize. It's definitely an active area of research.
Fortunately, the whole chain of slicing, imaging, and analysis are now at least partially automated, so in theory you can repeat the process with nothing more than some time on the equipment and a bit of compute.
In practice, I suspect there's a fair bit of grad student manual labor that keeps the pipeline flowing...
I don't think that's correct- the nature article about the article says they don't, https://www.nature.com/articles/d41586-024-03190-y and drosophila are not eutelic (although I see that some insects do have "partial constancy"). Could you ask the author to clarify?
Looking in the paper more closely they say:
"""After matching, Schlegel et al.12 also compared our wiring diagram with the hemibrain where they overlap and showed that cell-type counts and strong connections were largely in agreement. This means that the combined effects of natural variability across individuals and ‘noise’ due to imperfect reconstruction tend to be modest, so our wiring diagram of a single brain should be useful for studying any wild-type Drosophila melanogaster individual. However, there are known differences between the brains of male and female flies46. In addition, principal neurons of the mushroom body, a brain structure required for olfactory learning and memory, show high variability12. Some mushroom body connectivity patterns have even been found to be near random47, although deviations from randomness have since been identified48. In short, Drosophila wiring diagrams are useful because of their stereotypy, yet also open the door to studies of connectome variation."""
i woudl expect the overall architecture to be the same, but not the cell identities or the connections. But as always, I'm happy to be shown wrong with facts.
I think that releasing the map on torrent would be a useful idea as well. This fly could end up like the lobsters in Accelerando. In that book the mapped animal is lobsters and they get first mover advantage on some post-scarcity type things. Getting them to the Internet would be a good first step, IMO.
Unfortunately, not. We get the graph of the connections, but there are tons of essential parameters that are not captured. Such as the synaptic weights, the complex non-linear dynamics of the real neurons, their intricate modulation by various chemicals, etc.
For example, after the connectome of the worm were finished, despite it being quite small, for many years it proved to be impossible to simulate the dynamics, because of so many unknown parameters.
This was one of the criticisms that the opponents of connectomics have always brought up. "You spend a lot of money that could have been used for other research, but in the end you do not get a true insight into how the brain really works." For the researchers who thought that knowing all the connections was important, it was an uphill battle to overcome such attitudes.
But one has to start somewhere -- like a genome, the connectome is not the whole story, but it is a very important part of it, on which many other advances can be built up.
> after the connectome of the worm were finished, despite it being quite small, for many years it proved to be impossible to simulate the dynamics, because of so many unknown parameters.
Apparently they have been able to simulate dynamics with the fruit fly connectome(?) [0]:
> researchers used the connectome to create a computer model of the entire fruit-> fly brain, including all the connections between neurons. They tested it by activating neurons that they knew either sense sweet or bitter tastes. These neurons then launched a cascade of signals through the virtual fly’s brain, ultimately triggering motor neurons tied to the fly’s proboscis — the equivalent of the mammalian tongue. When the sweet circuit was activated, a signal for extending the proboscis was transmitted, as if the insect was preparing to feed; when the bitter circuit was activated, this signal was inhibited. To validate these findings, the team activated the same neurons in a real fruit fly.
The researchers have taken a very simple idealized mathematical model of a neuron, assumed that all synaptic weights were the same, ignored modulation, ignored base level inhibitory inputs, and have shown that even in such a crude setting, for some important inputs (especially for a taste of sugar) the "logic" of how the inputs result in the activation of certain outputs still works, based on the connectome information alone.
This is certainly very cool. But as the authors themselves point out [1], much more work remains to be done to reproduce more subtle features of the dynamics of the system.
It's my (layman) understanding that it's more or less a wiring diagram. Synapse #8217492 connects neuron #27472 and neuron #27865. It's a graph with 140,000 nodes (neurons) and 54.5 million edges (synapses). And then some labels for them like neurotransmitter type, which class of brain operations they're associated with, its size and position in 3D, etc.
Depends what you mean by "modeled". You can probably create a visualization of it, but the data doesn't include any information about the dynamics of the system, how the neurons behave. So, you can't "simulate a brain" to any extent with this data, if that's what you were thinking.
Knowing what the individual neurons actually do. The connectome is like an electrical schematic but you don't even know which components are resistors, inductors, etc (let alone the resistances and inductances).
The connectome for the C. elegans nematode was mapped in the 1980s and the OpenWorm project has successfully simulated all non-neuronal cells. But they are very far from simulating the brain abd it will take decades of experimental work to understand C. elegans's brain - it's very difficult to observe a living brain in the required molecular detail.
Yeah, I think it's close to "what steps exist between observing the network topology of the internet and being able to emulate a Google search query?"
There's plenty of value to knowing where the datacentres are and which regions are active under which circumstances, but none of that is telling you what the internet is thinking...
I'm frankly not sure it will ever be possible. Forget about observing the inside of a running neuron. In spite of how confidently people on the Internet will tell you their body fat percentage, in reality we can't even accurately measure that without killing you first.
I wouldn't say never, at least for C. elegans: there's been quite a bit of progress on imaging its brain, and it's plausible we'll have a fairly complete picture in a few decades: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7801769/
But the challenges are substantial, and these imaging techniques (mostly optical, not MRI/etc) depend on the simplicity of C. elegans: its brain is essentially a thin disk, with only 300 neurons in its entire nervous system, and it is surrounded by a transparent membrane. I am not sure how these techniques could possibly extend to something with a thick exoskeleton like Drosophila. And there are great difficulties keeping track of just the 300 neurons in a moving nematode with its own unique brain; it seems completely intractable with current tools to extend the complexity 50x, especially since fruit flies move far more rapidly and have far more individual variation.
go look at a photo of a CPU and imagine what else you'd need to run Windows on it, and then imagine it's probably extremely exponentially more difficult
Do we have an accurate model of a single neuron or very small group of neurons? I understand the reality may be chaotic, but I would hope to have a simulation such that it mirrors the evolution of neurons to a reasonable extent.
"human brains could follow" feels like a few jumps ahead? a fruit fly has on the order of 100k neurons, a human brain has on the order of 100 billion neurons. that's 6 orders of magnitude larger. that's like saying "A map of San Francisco has been completed, the entire solar system could follow!"
The method used required 3 million manual human corrections. Even if Moore's Law actually still meant anything for compute power, this is still many orders of magnitude from scaling to a human brain.
Given that for a map, it is the sqkm which matters, 6 orders of magnitude from the map of San Francisco is a jump from 121 sqkm to 121 000 000 sqkm ... which is not even all dry land on Earth, much less in the Solar System.
Surely a daunting task, but depending on the tools used to create the smaller map, possibly a realistic one. Maybe with a bit of a less precision.
I don’t think the last one is right. Fully connected and CNN are part of “what neuron is connected to what neuron” (though in the case of CNN, a number of corresponding “neurons” have equal weights going to/from them ).
Also, “activation function” isn’t exactly the right thing for real biological neurons. They aren’t just functions of the current input or the like. Their behavior depends on their recent history. Some will like, by themselves iirc, periodically fire. Others will fire if enough input is sent within some amount of time (in some models of some of them there’s like, some accumulations of signal when receiving inputs, which gradually decays/leaks, and it fires (and depletes) if enough is accumulated).
But yes, the idea is that “what is connected to what” is obtained, but not more specific things about how the ones that are connected are connected (how the behavior of one relates to the behavior of the ones it is connected to).
1. bring this mapping into a AI framework for inferencing
2. We don't know the "OS" on how it runs. Just randomly triggering a neuron probably wouldn't work as there is a lot of other factors that trigger neurons.
> In one paper, for example, researchers used the connectome to create a computer model of the entire fruit-fly brain, including all the connections between neurons. They tested it by activating neurons that they knew either sense sweet or bitter tastes. These neurons then launched a cascade of signals through the virtual fly’s brain, ultimately triggering motor neurons tied to the fly’s proboscis — the equivalent of the mammalian tongue. When the sweet circuit was activated, a signal for extending the proboscis was transmitted, as if the insect was preparing to feed; when the bitter circuit was activated, this signal was inhibited. To validate these findings, the team activated the same neurons in a real fruit fly. The researchers learnt that the simulation was more than 90% accurate at predicting which neurons would respond and therefore how the fly would behave.
I doubt that's been done yet but I'd be surprised if it didn't happen soon using something like NEURON [1]. It would be telling to see how similar the simulation is to the living organism, since there is a lot going on inside the brain in addition to the neuron spiking.
If I understand what you're asking for correctly, then no, not in any meaningful sense. This is the gross structural anatomy of a dead brain, which is a small but important step towards understanding dynamics.
Inference from structure to dynamics in a brain is several orders of magnitude less plausible than inferring from a record of local weather reports to simulating actual weather patterns.
Maybe a better analogy would be inferring from Grey's Anatomy to the regulatory dynamics of proteins at the cellular level in vivo (although I think that might actually be easier?)
50 years from now I am dying in a hospital bed, the nurse informs me that my consciousness will be uploaded to a computer with all the other brains, a digital heaven if you will.
A big reason for my imminent-AGI Skepticism is the fact that our understanding of the currently existing, Biological intelligence is so, so shallow.
We're here at "Systems level sketch of a fruit fly brain". It's incredible work! But as other comments detail, there is far more to the function of a fly brain than this "map". It's quite a long way from "Deep understanding of a Human Brain, to the point where we can begin engineering a replica".
Maybe we'll get lucky, and find that "Neural Network" techniques really are a pathway to Intelligence in a broad sense. But without some mechanistic understanding of Biological Intelligence, it seems no better than betting on the Numbers in roulette.
I don't think you need to fully understand how the brain works to be able to create AGI. Did the invention of the wheel/cart/car require us to fully understand how we walk? Did we need to fully understand how fish swim before we could make a boat? The only caveat would be that the AGI we build would be entirely unlike human minds. In the sane way a car going 100 kph is different from a running person.
The paper published in Nature, which is open access: https://www.nature.com/articles/s41586-024-07558-y
https://archive.is/vBUjt
It was my understanding that all this connectome-based research was largely a deadend, because it doesnt capture dynamics, nor a vast array of interactions. if you've ever seen neurones being grown (go search YT), you'll see it's a massive gelatinous structure which is highly plastic and highly dynamic. Even in the simplest brains (eg., of elgans), you get 10^x exponential growth in number of neurones and their connections as it grows.
Connectome-adjacent neuroscientist here. Definitely not a dead end! But also definitely not the whole picture.
One of the main open questions in neuroscience right now is how network structure, dynamics, and function are related in the brain. Connectomes provide tremendous insight into structure, but as mentioned this does not generically solve either the dynamics or function problem. For example, for many of these neurons we don't have a good understanding of their input-output relationship, and the nature of this relationship can strongly affect the dynamics that emerge in a highly connected network. Individual variability across connectomes, and how connectomes change over development are also a significant issue, but at least for the fly it's thought that many of the basic structures are pretty conserved across adult animals, even if many of the details could differ.
Modulo these caveats, knowing the physical network structure of the brain does still impose huge constraints on what kinds of models we should be using for gaining insight into dynamics and function. For example, there are well known areas (the "mushroom bodies") with specific feed-forward connectivity patterns that are very different from a random recurrent network. Further, there are at least some areas in the fly brain where we think there are indeed quite clean structure-function relationships, e.g. in the central complex of the fly brain, which contains a physical ring of neurons and is thought to support a "bump" of activity that acts as a sort of compass that helps flies navigate via a ring-attractor-like dynamical system. Thus, even though it has many missing pieces, a wiring diagram like this can be tremendously useful for generating hypotheses to guide more targeted experiments and theoretical studies.
How's Open Worm coming along? The connectome of C. Elegans has been known for years, and Open Worm tries to simulate it. [1] Not with enormous success.
[1] https://openworm.org/assets/OpenWormPoster_Celegans_Glasgow_...
Like everything in science: we don't know until we know.
No need to treat research like a business.
Budgets are finite, and most science funding involves some decision making about how to allocate resources.
Very Nice. --from a Connectome-Centric neuroimager :) One technique that I am pursuing right now is information decomposition of timeseries to separate the mutual information of two timeseries into redundant and synergistic informational atoms (synnergystic here means the degree to which knowing both timeseries gives you more information than the individual parts give (more than sum of parts). The big limitation of the method is the geometric explosion in complexity of the decomposition as the number of time series grow, with most analyses being limited to two or three times series at a time. However, the scale of the data on which it is applied is not requisite, meaning the approach can equally be used on the mutual information between two regions of interest in rsfMRI , or two spiking timeseries from individual neurons. https://en.wikipedia.org/wiki/Partial_information_decomposit...
Thanks for your insight! How repeatable are these structures between individual animals? Are they very similar or is it more like “here’s a feed forward kinda bit, here’s a toroidal bit, and over here it’s just a mess”?
This is done agaist an adult so all the neurons have already grown.
connectome isn't a dead end but it doesn't solve all known problems. It's like making a static map which you can then use to inspect all those cars driving around (the dynamics) and crashing (the interactions).
[edit: I forgot to mention that neuron growth in adults (across many species) is still a controversial topic; see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7554932/ for some commentary on the challenge in fly; https://en.wikipedia.org/wiki/Adult_neurogenesis for commentary on the larger problem ]
Giving scientists access to the connectome snapshot alone is very exciting. The first step to understanding why something is and how it came to be is seeing what it is.
There are systems at play that form the brain into what it is and we don’t know much about them. The individual neurons — we have a better understanding of, but not the emergent systems. Now that many more scientists will know what the target for these systems is — what is the brain they shape, we can start to understand the control and feedback loops that result in this snapshot state of the brain.
And that’s why it’s not a dead end. Just because it doesn’t immediately give some sort of a consumer product, doesn’t mean it’s not a step forward.
You don't get the dynamics from connectomes, but you absolutely need them. So it isn't that they are a dead end, it is that the dynamics by themselves are also insufficient and the connectome is insufficient, you need both. Further, if you want to actually be able to have anything to attach the dynamics to, you need the cellular anatomy, so connectomes are absolutely necessary. The fact that connectomes are insufficient does not mean that such research is a dead end, but rather that the prerequisites for understanding the nervous system are vastly more complex and demanding than some might have hoped.
It is useful.
It is like getting a static map of the country's roads with no cars on it.
You can not make it come alive with cars (activity), but you can infer where people need to drive but you don't know when and why they drive or what they are doing, but it is a major clue.
> It is like getting a static map of the country's roads with no cars on it.
I was thinking it was more like giving somebody iPhone schematics and die shots of all the chips and then asking them to figure out how Portrait Mode works in the Camera app.
The difference is that in the brain there's no real separation between hardware and software, so I'm your analogy, we also have the equivalent of the source code, but just maybe not the environment configuration needed to get it to run (nor would we at this stage have sufficient compute to fully run it).
Yup, it is similar to that as well. It is a part of the puzzle definitely, but not at all the whole picture.
Sort of, but mostly not. The critical distinction is that, given better data (the instruction set, the source code or binary of the OS and camera app), the schematics and die shots aren't necessary or even useful.
It's unlikely that brains have an abstraction layer like that, so work like this is a necessary precondition to understanding the rest of how it works. That actual understanding may be elusive for quite some time to come, but without a connectome, forget it, no change.
Why exactly would it be unlikely?
It would be really inefficient and neurons inherently provide a great deal of flexibility. Larger animals might use this kind of thing, but insects don’t have that many neurons to work with.
Luckily this is science so we can actually find out.
the metaphor I've heard is it's like getting a map of the country's roads, but none of the signs are labelled.
Connectomes are like a static graph of a neural network.
But it's the flow of information as signals pass through nodes where everything actually happens.
> It was my understanding that all this connectome-based research was largely a deadend,
There's obviously something to it or implementing what we map in software wouldn't give results as accurately as they do.
You just need to supply your own training data.
Connectome is a necessary component to understanding dynamics.
it's a tool in the toolbox. useful for mapping things out when doing functional experiments
From https://news.ycombinator.com/item?id=35877402#35886145 :
> So, to run the same [fMRI, NIRS,] stimulus response activation observation/burn-in again weeks or months later with the same subjects is likely necessary given Representational drift
And isn't there n-ary entanglement?
Going to need a significant improvement in the software to get it to map a human. The fruit flu has 140,000 neurons and 54.5 million synapses and the AI that mapped it required a post process with humans checking it all with 3 million edits and they still have to identify every neuron type.
A human brain has about 86 billion neurons and quite likely many trillions of synapses and that is likely an underestimate. That 3 million edits will turn into 3 million * 10^6 at least manual edits, that doesn't seem feasible. The error rate on the fruit flu would have to come down into the single digits to be usable to map a human brain. So an improvement from about 6% of synapses to 0.000006%. That is one heck of a jump in improvement for an AI.
Cartographers have mapped Scotland. [random scribe muses that] The whole world could be next.
Did a rough calculation, it would be more like Edinburgh.
There's easily a century between the earliest accurate map of Edinburgh and the earliest accurate map of the world. And even at present, the accuracy of maps of Edinburgh is much greater than the accuracy of maps of the world.
So yeah, the whole world could be next. But the person you're replying to has a point when they say significant improvements are needed.
We did map a handful of brains yet, the more we do the better we will get at it.
I don't understand all this rushing and skepticism when such amazing science is being done. It's not like some AI company marking claims to sell a product, it's some researchers trying to accomplish something. Yes, they should (and probably will) do it better but that's not the goal here.
If 3 million manual edits are still doable then it's ok. And when the manual step is not feasible, a jump in the tech will be required.
> Did a rough calculation, it would be more like Edinburgh.
To put a fine point on the difference in scale:
Edinburgh[0]: 264 square km
Earth[1]: 510,000,000 square km
0 - https://www.britannica.com/place/Edinburgh-Scotland
1 - https://www.universetoday.com/25756/surface-area-of-the-eart...
You seem to have called it a fruit flu twice... Was that a typo or do you actually mean to call it a flu instead of a fly?!
[dead]
My (maybe very ignorant) question is: can this connectome be used to “run” simulations of a virtual fruit fly, a la MMAcevedo?
It’s a neural network without weights. And it doesn’t have a body.
Figuring out the behaviour of the neurons could take decades, although I have no doubt that people will eventually. And simulating a whole fruit fly body seems like it’s going to be out of reach for a very long time.
Microsoft Fly Simulator (tm).
no. turaga and co have some work where they constrain model network topologies with the connectome and train on visual data. this is imo a very silly line of research and they come to some very wrong conclusions about what neurons do what with it. but that's the closest to what you're asking for
There is this interesting past post:
Whole-brain connectome of the fruit fly (2023) https://news.ycombinator.com/item?id=36568609
Thanks! Macroexpanded:
Whole-brain connectome of the fruit fly - https://news.ycombinator.com/item?id=36568609 - July 2023 (94 comments)
The connectome of an insect brain by Winding et al. - https://news.ycombinator.com/item?id=35112234 - March 2023 (1 comment)
Map of an Insect’s Brain - https://news.ycombinator.com/item?id=35111371 - March 2023 (119 comments)
The Connectome of an Insect Brain - https://news.ycombinator.com/item?id=35094565 - March 2023 (1 comment)
The first wiring map of an insect's brain hints at incredible complexity - https://news.ycombinator.com/item?id=35089298 - March 2023 (5 comments)
Fruit Fly Brain Map - https://news.ycombinator.com/item?id=29672565 - Dec 2021 (1 comment)
Structure of Fruit Fly Brain (2018) - https://news.ycombinator.com/item?id=26474430 - March 2021 (7 comments)
Google publishes largest ever high-resolution map of brain connectivity - https://news.ycombinator.com/item?id=22124888 - Jan 2020 (1 comment)
Explore the the adult fruit fly brain - https://news.ycombinator.com/item?id=20015218 - May 2019 (1 comment)
To detect new odors, fruit fly brains improve on a well-known computer algorithm - https://news.ycombinator.com/item?id=18656016 - Dec 2018 (1 comment)
A Complete Electron Microscopy Volume of the Brain of Adult Fruit Fly - https://news.ycombinator.com/item?id=17590910 - July 2018 (50 comments)
Fruit Fly Brain Hackathon 2017 – Brain Circuit, Memory and Computation - https://news.ycombinator.com/item?id=13692166 - Feb 2017 (13 comments)
Neurokernel: Emulating the Fruit Fly Brain - https://news.ycombinator.com/item?id=9284802 - March 2015 (8 comments)
An open source platform for emulating the fruit fly brain - https://news.ycombinator.com/item?id=8377600 - Sept 2014 (17 comments)
Maybe also throw in:
Six Nobel prizes – what’s the fascination with the fruit fly? - https://news.ycombinator.com/item?id=15463522 - Oct 2017 (16 comments)
Fruit fly nervous system: new solution to fundamental computer network problem - https://news.ycombinator.com/item?id=2103668 - Jan 2011 (13 comments)
Another for you:
Map of an Insect’s Brain - https://news.ycombinator.com/item?id=35111371 - March 2023 (119 comments)
Inserted. Thank you!
Out there question: Do you have a hand crafted database of these setup or some sort of macro to take the output of the search api and form it like this, or are you hand editing these lists?
That question is so in there, there is also a list of many of the answers
https://news.ycombinator.com/item?id=35668525
Are all fruit fry brains the same? Does anyone know what has actually been mapped and why it would generalize from one fruit fly to the next?
I don't think that drosophila are eutelic (https://en.wikipedia.org/wiki/Eutely) so no two flies have precisely the same cells at precisely the same locations (that's true for c. elegans, whose connectome is probably the best studied).
The large-scale architecture will be roughly the same between any two individuals. You would likely need some sort of mapping (like an embedding) to generalize. It's definitely an active area of research.
The article describes it as slicing the fly brain into very thin slices, which are imaged by an electron microscope.
Then you analyze the slice images and determine the neurons and their connection. This is the hard part, and the breakthrough is an AI based method.
Pretty sure they've only mapped one brain so far.
Fortunately, the whole chain of slicing, imaging, and analysis are now at least partially automated, so in theory you can repeat the process with nothing more than some time on the equipment and a bit of compute.
In practice, I suspect there's a fair bit of grad student manual labor that keeps the pipeline flowing...
They crowdsourced three million manual corrections to the AI output, yeah.
Yes, they are apparently exactly the same, with exactly the same neurons and connections!
Happened to go for a walk with the corresponding author and made her repeat this fact for me.
I don't think that's correct- the nature article about the article says they don't, https://www.nature.com/articles/d41586-024-03190-y and drosophila are not eutelic (although I see that some insects do have "partial constancy"). Could you ask the author to clarify?
Looking in the paper more closely they say: """After matching, Schlegel et al.12 also compared our wiring diagram with the hemibrain where they overlap and showed that cell-type counts and strong connections were largely in agreement. This means that the combined effects of natural variability across individuals and ‘noise’ due to imperfect reconstruction tend to be modest, so our wiring diagram of a single brain should be useful for studying any wild-type Drosophila melanogaster individual. However, there are known differences between the brains of male and female flies46. In addition, principal neurons of the mushroom body, a brain structure required for olfactory learning and memory, show high variability12. Some mushroom body connectivity patterns have even been found to be near random47, although deviations from randomness have since been identified48. In short, Drosophila wiring diagrams are useful because of their stereotypy, yet also open the door to studies of connectome variation."""
i woudl expect the overall architecture to be the same, but not the cell identities or the connections. But as always, I'm happy to be shown wrong with facts.
No need to get angry and sarcastic.
highly stereotyped, definitely not identical
Datahoarder question but can I download the map of the fly?
See the FAQ: https://codex.flywire.ai/faq which leads to the API for access: https://codex.flywire.ai/api/download
You'd need to inspect the paper, the supplementals, and the website closely to determine exactly which files are interesting.
I think that releasing the map on torrent would be a useful idea as well. This fly could end up like the lobsters in Accelerando. In that book the mapped animal is lobsters and they get first mover advantage on some post-scarcity type things. Getting them to the Internet would be a good first step, IMO.
the raw data will be on the order of PBs
[flagged]
[flagged]
What does it mean "mapped". Does it mean we know what each nerve/axon does?
Unfortunately, not. We get the graph of the connections, but there are tons of essential parameters that are not captured. Such as the synaptic weights, the complex non-linear dynamics of the real neurons, their intricate modulation by various chemicals, etc.
For example, after the connectome of the worm were finished, despite it being quite small, for many years it proved to be impossible to simulate the dynamics, because of so many unknown parameters.
This was one of the criticisms that the opponents of connectomics have always brought up. "You spend a lot of money that could have been used for other research, but in the end you do not get a true insight into how the brain really works." For the researchers who thought that knowing all the connections was important, it was an uphill battle to overcome such attitudes.
But one has to start somewhere -- like a genome, the connectome is not the whole story, but it is a very important part of it, on which many other advances can be built up.
> after the connectome of the worm were finished, despite it being quite small, for many years it proved to be impossible to simulate the dynamics, because of so many unknown parameters.
Apparently they have been able to simulate dynamics with the fruit fly connectome(?) [0]:
> researchers used the connectome to create a computer model of the entire fruit-> fly brain, including all the connections between neurons. They tested it by activating neurons that they knew either sense sweet or bitter tastes. These neurons then launched a cascade of signals through the virtual fly’s brain, ultimately triggering motor neurons tied to the fly’s proboscis — the equivalent of the mammalian tongue. When the sweet circuit was activated, a signal for extending the proboscis was transmitted, as if the insect was preparing to feed; when the bitter circuit was activated, this signal was inhibited. To validate these findings, the team activated the same neurons in a real fruit fly.
[0]: https://www.nature.com/articles/d41586-024-03190-y
The researchers have taken a very simple idealized mathematical model of a neuron, assumed that all synaptic weights were the same, ignored modulation, ignored base level inhibitory inputs, and have shown that even in such a crude setting, for some important inputs (especially for a taste of sugar) the "logic" of how the inputs result in the activation of certain outputs still works, based on the connectome information alone.
This is certainly very cool. But as the authors themselves point out [1], much more work remains to be done to reproduce more subtle features of the dynamics of the system.
[1] https://www.nature.com/articles/s41586-024-07763-9
It's my (layman) understanding that it's more or less a wiring diagram. Synapse #8217492 connects neuron #27472 and neuron #27865. It's a graph with 140,000 nodes (neurons) and 54.5 million edges (synapses). And then some labels for them like neurotransmitter type, which class of brain operations they're associated with, its size and position in 3D, etc.
They have a cool website that lets you browse the data: https://codex.flywire.ai/
Is the data such that it can be modeled in software?
Yes. One example: https://www.biorxiv.org/content/10.1101/2024.03.11.584515v1
Depends what you mean by "modeled". You can probably create a visualization of it, but the data doesn't include any information about the dynamics of the system, how the neurons behave. So, you can't "simulate a brain" to any extent with this data, if that's what you were thinking.
https://github.com/Flowx08/Celegans-simulation
> Data available for download, programmatic access and interactive browsing and have been made interoperable with other fly data resources
Curious what a 'fly brain map' looks like - iss the download a 3D model, or a matrix with values for attributes?
What hard steps exist between mapping the physical structure of the brain and simulating a running one via software?
Knowing what the individual neurons actually do. The connectome is like an electrical schematic but you don't even know which components are resistors, inductors, etc (let alone the resistances and inductances).
The connectome for the C. elegans nematode was mapped in the 1980s and the OpenWorm project has successfully simulated all non-neuronal cells. But they are very far from simulating the brain abd it will take decades of experimental work to understand C. elegans's brain - it's very difficult to observe a living brain in the required molecular detail.
I think it's even more complex. The neurons are like individual raspberry pi. They have both complex logic and physical memory.
Yeah, I think it's close to "what steps exist between observing the network topology of the internet and being able to emulate a Google search query?"
There's plenty of value to knowing where the datacentres are and which regions are active under which circumstances, but none of that is telling you what the internet is thinking...
I'm frankly not sure it will ever be possible. Forget about observing the inside of a running neuron. In spite of how confidently people on the Internet will tell you their body fat percentage, in reality we can't even accurately measure that without killing you first.
I wouldn't say never, at least for C. elegans: there's been quite a bit of progress on imaging its brain, and it's plausible we'll have a fairly complete picture in a few decades: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7801769/
But the challenges are substantial, and these imaging techniques (mostly optical, not MRI/etc) depend on the simplicity of C. elegans: its brain is essentially a thin disk, with only 300 neurons in its entire nervous system, and it is surrounded by a transparent membrane. I am not sure how these techniques could possibly extend to something with a thick exoskeleton like Drosophila. And there are great difficulties keeping track of just the 300 neurons in a moving nematode with its own unique brain; it seems completely intractable with current tools to extend the complexity 50x, especially since fruit flies move far more rapidly and have far more individual variation.
go look at a photo of a CPU and imagine what else you'd need to run Windows on it, and then imagine it's probably extremely exponentially more difficult
Do we have an accurate model of a single neuron or very small group of neurons? I understand the reality may be chaotic, but I would hope to have a simulation such that it mirrors the evolution of neurons to a reasonable extent.
It thinks a lot about fruit.
Can we please stop perpetuating this racist stereotype?
Better source: https://www.nature.com/articles/d41586-024-03190-y
"human brains could follow" feels like a few jumps ahead? a fruit fly has on the order of 100k neurons, a human brain has on the order of 100 billion neurons. that's 6 orders of magnitude larger. that's like saying "A map of San Francisco has been completed, the entire solar system could follow!"
The method used seems like it would work as well on bigger brains.
The amount of data may mean we have to wait for Moore's Law to keep improving things for a while though.
The method used required 3 million manual human corrections. Even if Moore's Law actually still meant anything for compute power, this is still many orders of magnitude from scaling to a human brain.
Moores law ended.
I thought it was intended as more of a pun on questionable displays of human intelligence.
Given that for a map, it is the sqkm which matters, 6 orders of magnitude from the map of San Francisco is a jump from 121 sqkm to 121 000 000 sqkm ... which is not even all dry land on Earth, much less in the Solar System.
Surely a daunting task, but depending on the tools used to create the smaller map, possibly a realistic one. Maybe with a bit of a less precision.
Well assuming the same density it's "only" 100 times bigger in linear dimensions. Doesn't sound quite as crazy...
Isn't that just saying "if you take the cube root of the number, it's a smaller number"?
I don't mean to be facetious - I'm struggling to to see what other consideration this helps with.
The physical process of cutting. We're physically sectioning 3 dimensional blocks of tissue.
http://archive.today/vBUjt
Thanks
So can we run it on a computer now? That's the end goal, isn't it? Or maybe ask an LLM to look at the upload and figure out what makes it tick.
Why would an LLM in particular be good at this?
Is this correct?
It this like knowing only this:
which neuron is connected to which neuron
But you don't know:
the values of the weights (the value of the neuron, or the parameters)
the activation functions
what circuit do neurons implement (fully connected? CNN?)
I don’t think the last one is right. Fully connected and CNN are part of “what neuron is connected to what neuron” (though in the case of CNN, a number of corresponding “neurons” have equal weights going to/from them ).
Also, “activation function” isn’t exactly the right thing for real biological neurons. They aren’t just functions of the current input or the like. Their behavior depends on their recent history. Some will like, by themselves iirc, periodically fire. Others will fire if enough input is sent within some amount of time (in some models of some of them there’s like, some accumulations of signal when receiving inputs, which gradually decays/leaks, and it fires (and depletes) if enough is accumulated).
But yes, the idea is that “what is connected to what” is obtained, but not more specific things about how the ones that are connected are connected (how the behavior of one relates to the behavior of the ones it is connected to).
I believe they do know this.
However the real challenge would be:
1. bring this mapping into a AI framework for inferencing 2. We don't know the "OS" on how it runs. Just randomly triggering a neuron probably wouldn't work as there is a lot of other factors that trigger neurons.
> which neuron is connected to which neuron
yes. and you can get VERY roughly connection strengths by synapse count but that's as far as you can go
Does it matter ?
Openworm still hasn't succeeded.
Quite a leap, fruit fly to human....
Simulated too? I assume that if you can map it then you can simulate it. Am I correct?
Simulating it would require many orders of magnitude more compute. Biological neurons are not just a sigmoid function.
> In one paper, for example, researchers used the connectome to create a computer model of the entire fruit-fly brain, including all the connections between neurons. They tested it by activating neurons that they knew either sense sweet or bitter tastes. These neurons then launched a cascade of signals through the virtual fly’s brain, ultimately triggering motor neurons tied to the fly’s proboscis — the equivalent of the mammalian tongue. When the sweet circuit was activated, a signal for extending the proboscis was transmitted, as if the insect was preparing to feed; when the bitter circuit was activated, this signal was inhibited. To validate these findings, the team activated the same neurons in a real fruit fly. The researchers learnt that the simulation was more than 90% accurate at predicting which neurons would respond and therefore how the fly would behave.
https://www.nature.com/articles/d41586-024-03190-y
I doubt that's been done yet but I'd be surprised if it didn't happen soon using something like NEURON [1]. It would be telling to see how similar the simulation is to the living organism, since there is a lot going on inside the brain in addition to the neuron spiking.
[1] https://nrn.readthedocs.io/en/8.2.6/
If I understand what you're asking for correctly, then no, not in any meaningful sense. This is the gross structural anatomy of a dead brain, which is a small but important step towards understanding dynamics.
Inference from structure to dynamics in a brain is several orders of magnitude less plausible than inferring from a record of local weather reports to simulating actual weather patterns.
Maybe a better analogy would be inferring from Grey's Anatomy to the regulatory dynamics of proteins at the cellular level in vivo (although I think that might actually be easier?)
Paging mjg59, Matthew Garrett, Matthew Garrett to the white courtesy phone.
50 years from now I am dying in a hospital bed, the nurse informs me that my consciousness will be uploaded to a computer with all the other brains, a digital heaven if you will.
Get there and its full of flies.
No, beetles (JBS Haldane said god has an "inordinate fondness for beetles")
That may be so, but scientists have an inordinate fondness for flies.
heaven? not so fast. how about solving captchas at 100x speed for 100 years to aid the development of some ai vision project?
Well, the big question is if a human is "just" a mega-fly when it comes to brain structure.
Yeah but you can earn CPU cycles and egress bandwidth by sending bug reports
Keeps your virtual landlord happy... Landlord of the Flies, if you will.
A big reason for my imminent-AGI Skepticism is the fact that our understanding of the currently existing, Biological intelligence is so, so shallow.
We're here at "Systems level sketch of a fruit fly brain". It's incredible work! But as other comments detail, there is far more to the function of a fly brain than this "map". It's quite a long way from "Deep understanding of a Human Brain, to the point where we can begin engineering a replica".
Maybe we'll get lucky, and find that "Neural Network" techniques really are a pathway to Intelligence in a broad sense. But without some mechanistic understanding of Biological Intelligence, it seems no better than betting on the Numbers in roulette.
I think we've already done this with a certain flatworm.
I don't think you need to fully understand how the brain works to be able to create AGI. Did the invention of the wheel/cart/car require us to fully understand how we walk? Did we need to fully understand how fish swim before we could make a boat? The only caveat would be that the AGI we build would be entirely unlike human minds. In the sane way a car going 100 kph is different from a running person.
[dead]