My thoughts on Elon Musk’s Neuralink

By MariusNeurotalk
Pig Gertrude and her nose-boop-related potentials measured by Neuralink. Image taken from the press event video.

Estimated reading time: 16 min (plus viewtime of the press event…)

In this post I’ll first show you my “live” thoughts on everything you can see in the press  event video, then talk about what the neuroscience twitterverse thinks, and give my comments about how it may or may not relate to gaming.

Sooo. Neuralink, huh? Everyone and their mother in brain tech and research has heard about it, they had their second press event! I’m a bit late to the party because I was on vacation and instead of throwing a quick comment I thought I would discuss the things in more detail. It’s a long read but I hope you find it interesting, feel free to leave a comment here or in the discord:

But first, in case you don’t know what Neuralink is: It’s a company founded by Elon Musk which is supposed to create brain-machine interfaces (BMIs) with significantly higher fidelity than the state of the art, and ultimately even without a surgical procedure to open the skull. The core idea is that in a distant future artifical intelligence will be uncontrollable and thus we humans need to merge with it to not fall into the pit of irrelevance.

While we scientists may think that’s a relatively naive idea, honestly, this guy has proven a few times that incredible things are possible. I’ve been following the news since it started a few years ago, and while I generally think it doesn’t really affect me for now, I can very well imagine that it may become a super relevant thing for humanity in general, and of course also for us researchers specifically. If you want to dig deeper into the motivations of Neuralink and learn about BMIs, have a look at this Wait but Why article about Neuralink, which is actually endorsed by Elon Musk.

 

Live thoughts

I noted the following thoughts while watching and pausing, so I encourage you to do the same if you haven’t watched it yet.

  • Elon sounds anxious but that’s just him being him…
  • 2:00 goal to solve spine and general neurological problems, not for healthy. Ok they dialed down apparently…
  • 3:15 here we are. Solving anxiety and depression seems far-fetched. May or may not. Dangerous claim.
  • 4:30 agreed, Utah arrays are for science not really to be used.
  • 5:20 I didn’t know DBS was used with so many people!
  • 6:30 Finally an image of the link. Alright cool you can see the part that replaces the skull, it’s larger than anticipated but okay, and then the threads with the electrodes. Honestly, this is pretty neat, especially since it’s 1 kilochannels and all processing.
  • 7:00 alright. Flush with skull seems possible, yep, got a lot of additional sensors, could be useful for general health monitoring, megabit wireless is great and definitely necessary for the data, and all day battery life in such a small and potent device is impressive.
  • 9:00 procedure in one hour is very impressive, hard to imagine. Same day leave…? Bit crazy, I don’t know… general anesthesia is usually not used in brain surgeries anyways so that’s nothing special but sold as if it were.
  • 10:30 I’m curious if people trust a robot enough to do fully automated brain surgeries. I can imagine there will be issues, but then again surgeries are always problematic and human surgeons make mistakes more often that you’d probably like… if the robot really does everything and it’s been tested extensively, it’s probably the better approach.
  • 11:30 no noticeable neural damage is great and very important. That was already impressive in the last video. Cool approach.
  • 12:00 PIGS! I was thinking monkey but probably they first wanted to do general health hazard tests.
  • 12:45 you can see the data of pig 2 dropping signals a lot. The connector probably has to be really close.
  • 13:10 yes good point. Being able to remove the implant without issues is absolutely necessary.
  • 14:30 Neuropig, neuropig, does whatever a neuropig does…
  • 16:00 Love the idea of making neural spikes audible like this, with pitch. I’ve been thinking that one could make EEG audible as well, in the end it’s just a time-series… I wonder if a human classifier could hear high vs low workload.
  • 17:00 two months. Cool.
  • 17:30 multiple is probably necessary if you want left/right motor cortex, plus sensory and prefrontal information…
  • 18:45 casually introducing the fact that you can read out complete body movement from the brain, alright. The signal is neato.
  • 20:00 cool, nice visualization, but also cool in general. Probably brain stimulation will be crazily important at some point in the future, even for healthy humans.
  • 21:00 read & write on every channel, that’s new. Last time it was either read 1024, or write 64 iirc.
  • 20:00 Bluetooth Lowenergy for 1024 channels r/w, that’s really a lot of data. The Muse also runs on BTLE and I sometimes have trouble connecting depending how my BT USB Adapter is situated. As with the pig earlier, I’m not convinced of the 5-10m range. That’s probably a theoretical range, but practically speaking you’ll have to carry your phone with you at all times probably.
  • 21:45 FDA breakthrough device, what does that mean exactly? “The Breakthrough Devices Program is a voluntary program for certain medical devices and device-led combination productsthat provide for more effective treatment or diagnosis of life-threatening or irreversibly debilitating diseases or conditions.” I’m happy to hear that they take safety serious. Gonna remain cautious though of course.
  • 23:15 They really are more of an engineering company, clearly stated. That’s also what the beef with the scientists was about earlier this week I think… That’s what you get if you work for Elon! Move fast an break things, but at the same time please don’t break things that live, so yeah that’s not easy as a scientist.
  • 25:15 Interesting question and very important question about whether the spike detection is software or hardware and if it can be updated. Software updates for spike detection could be very important. Clearly they need to send out the spike only, but the question was smoothly unanswered. Probably not updateable but hardware implemented…? “You can configure what you’re looking for” but when? Only at the manufacturing state or also later by flashing the device?
  • 27:40 “Is there anything specifically from the robot that can be done to make this fast and safer and scalable…?” Question entirely unanswered. Like a politician… clearly they want to do that, but what exactly?
  • 28:00 apparently the robot does currently only do the thread stuff.
  • 29:00 classic. Paraplegics are always the first to be mentioned.
  • 30:00 API question: Super interesting and important! Is it going to be possible to write your own apps? Wait, you’re getting raw waveforms, but I thought they only send spikes?! And no mention of an API for stimulation. That’s gonna be absolutely critical.
  • 30:45 “Will this technology ever be used for gaming?” You bet! Well, “does it work well enough to play starcraft?” is nice for paraplegics, but the possibilities for gaming with such a device attached are… absolutely mind-blowing on the one hand, and actually not that mind blowing on the other. Gonna have to discuss that later in detail.
  • 31:15 Me must go deeper! Yeah but this is clearly meant to be cortical, but not only that, cortical surface, so no gyri and sulci, just what you can reach from outside. Definitely a limitation since many regions cannot be reached like this.
  • 32:00 Elon, you are really confident in your tech, but how much of a resolution could this device deliver? I say less than what the general population would infer from this statement…
  • 34:15 I think they might underestimate the amount on information you get from the brain and how difficult it is going to be to make sense of it. But yeah, materials are the current bottleneck probably
  • 38:30 Qestion about “read/write speed”: the question is really not well formulated… probably information I/O speed is what they meant, but that’s not the same as waveform data speed, or not even as spike speed. Time-resolution is one thing, but also information extraction which they haven’t really talked about yet. They just summed the spikes they got from the pig. But they were able to infer joint position, so they must do something more complex obviously… microsecond resolution is of course almost perfect, as a spike is in the milliseconds time-range. However, read/write sped in terms of information is essentailly a conceptually unanswerable question at this point.
  • 40:45 100 people at Neuralink right now, that’s already big, but 10k over time, yeah they really wanna do this. They’re gonna need cognitive neuroscientists, Elon, want to hire me?!
  • 41:45 good question about influences from outside! Both the spike detection and the data transmission will be affected.
  • 42:50 hey entertainment often was the driver of technical advancements! Electricity firs was just a partytrick where people would light drinks on fire with sparks and such. But yeah, clinical stuff will be first for Neuralink. Buuuut, if they already have the device implanted, why not make a VR neurogame with neuralink and have that as a fantastic PR statement? Elon, I could do that!
  • 44:45 no, that’s really oversimplified. The spinal cord is more than just wires, it’s part of the central nervous system. There is computation going on, and that’s computation the brain does not and cannot do, likely can’t replace at all. Also moving your limbs requires very precise sensory kinesthetic information, it’s not so simple. Without this immediate feedback, you can barely walk (these conditions exist, I forgot the name though, sorry). A BCI like Neuralink will not be able to fully restore walking for a long while.
  • 46:45 save and replay memories, yeah I guess that’s maybe gonna be possible in the far, FAR future, if you have access to all the neural activities. When I was a kid I wanted to record dreams… Spooky how close this is now. But Black Mirror is the right buzzword here, it can be very dystopian. Let’s hope Zuckerborg never gets his hands on this.
  • 47:45 hey pigs ARE smart! That’s nothing strange…
  • 50:15 “Yeah, but can it play Crysis?” LOL!
  • 50:50 It’s all about accomplishing things really quickly. I can see why working at Neuralink is stressful, especially when coming from a scientist’s perspective…
  • 51:15 Ohhhhh we’re going to consciousness now. Well that’s still all very much up for debate and simply stating “The hard problem will vanish very quickly” just means you haven’t understood what the hard problem of consciousness is. Even if you may be able to map states or contents of consciousness to neural activity 1 to 1, it still doesn’t explain how matter can create mind, or vice versa. That’s the problem: there is no bridge, only superposition. In that case I really think Elon was the cautious one and the Max was a bit out there. Consciousness and the philosophical issues of scientifically investigating it are complex and reductionism is not the answer to everything. I recommend the book “Consciousness: The Science of Subjectivity” by Antti Revonsuo. He was my master thesis supervisor, I learned a lot being there! Shoot me a message on discord if you want to read it, I’d love to discuss it!
  • 53:00 privacy and security, here we are. The I/O really must be 100% secure. If your brain stimulation gets hacked you’re gonna have a bad time. I think that is a thing they don’t talk about enough, just say that essentially this is a solved issue. I’m not convinced.
  • 57:30 price point of a few thousand dollars, that’s ambitious. But also interestingly, they seem to be fixated on the skull penetration like they presented it now. No more mention of neural lace or neural mesh or so, just plain old through the skull procedures, done very well with very well designed electrodes.
  • 1:00:30 good point about the active pigs and the fact that they do bang their head and so on, so apparently it’s pretty safe.
  • 1:04:15 oh this is another gross oversimplification… Speech is more than just thought compression and output. A linguists perspective on this conceptual telepathy thing would be interesting!
  • 1:08:30 the problem with limiting human suffering is that it can be very tempting and dangerous. Heroin also limits suffering… how dependent / addicted will people become of Neuralink? That’s an incredible ethical issue. Imagine the power Neuralink has over people using the device.
  • 1:09:00 Yep Elon’s vision is really something. Goes beyond anything the others mentioned… Check out the above mentioned Wait but Why article.
  • 1:10:00 Philosophy of consciousness as the core interest, but seems to be very limited in his view… I mean it’s great that they have this element in their press conference but it’s really unsatisfying if you ask me.
  • 1:12:00 Accessing memories when you’re feeling down can be super dangerous, to the point where people stop to live in the reality. All these positive thoughts, nobody discusses the vast number of potential issues…
 

Generally, all the stuff is impressive but does not really go super far beyond what is being done already. Refined, by a lot, yes, but nothing fundamentally new. Everyone has great positive thoughts, but not much talk about potential issues. I am personally very concerned about the ethical implications which seem to be rather underappreciated at Neuralink.

 
 

The neuroscience twitterverse

This seems to be also the general consensus in the neuroscience community. Check out a few tweets and the following discussion below, these happened right after the press conference:

My TL/DR from the discussions I have seen: The robot is cool, many ideas, not many facts, generally similar things have been done by renowned science labs, but it is impressive from an engineering standpoint that Neuralink can catch up relatively fast and surpass in parts. Neuroscience is missing, ethics are missing. Pigs are cute and the joint prediction is relatively interesting but not groundbreaking.

A bit later Elon encouraged people to apply for a job at Neuralink, but he did it in a way that didn’t exactly spark joy in the neuroscience community:

Now, I’d say the tweet comes off as pretty arrogant but understandable from an engineering point of view: In comparison to what the brain does, we don’t know very much about how it works exactly. The thing is: we do know things, but the brain is so vastly more complex that what we know pales in comparison to what we don’t know. So, “teaching on the job” is not exactly a great way to get a comprehensive understanding. But, and that remains true since the company has started, the bottleneck at this point for Neuralink is likely still the engineering part. So considering his goals, Elon makes the right call by attracting mainly engineers.
 
Whether this is the right call considering other goals like an ethically sound and
scientifically validated approach to BMIs is a different question…

 

Neuralink and gaming

But, with all this being said, let’s finally have a look at the juicy part, the most important aspect of Neuralink:

Jokes aside, gaming is going to be an important aspect of Neuralink. Gaming, or more specifically, entertainment, has always been a driver of technology. Especially the combination of BCI and VR is one that excites people and lets the imaginations run wild:

Before the Valve Index VR Headset was released, there were already (very wild) speculations, that Valve could be working with Neuralink. Now, although this was likely more of a joke, the discussion is interesting to follow. The entire topic was sparked not only by mere fantasy, though, as Mike Ambinder had already held a talk about BCI in gaming at the 2019 Games Developer Conference. A talk that I highly recommend watching by the way, as people at Valve seem to have a pretty decent grasp about what BCI can and cannot do (as opposed to other companies, for example Facebook, where people seem to be convinced that typing complete sentences with a BCI is a viable approach in the near to mid future). But the interest at Valve does not only come from the psychology department. On the contrary, Gabe Newell himself has stated that he’s convinced BCI will be part of gaming, which has sparked enthusiastic discussions in the community, and his son Gray Newell had a long conversation about the topic, too.

I don’t write about all this because of Valve specifically, but rather to show how much interest this topic finds in the community. Players want this, especially VR players.

Back to Neuralink: What can it do for gaming? Sadly, I’d say not much in the current state. The way it is designed so far is to be used in medical circumstances, for example the classic BCI for cummunication and control of paraplegics or patients with amyotrophic lateral sclerosis (ALS, like Stephen Hawking had it). The device will be first planted at the motor cortex, maybe left and right, and maybe also at the somatosensory cortex. Essentially, it is going to be used instead of muscles, to talk or to steer things. Now, for most of us who are fortunate enough to not be riddled with these diseases, that is not a functionality with a lot of value.

We already have well working Brain-World-Interfaces: our muscles. It certainly would not be worth the risk of opening the skull to get a BMI that replaces them. Musk said that they want  users to be able to play Star Craft II as good as or better than when using a regular mouse and keyboard. I find this highly unlikely with the stuff they showed us and also with the near future projections. It’s not impossible, but the amount of fine-grained control which is necessary here is pretty intense. A thousand or even ten thousand electrodes are not going to be specific enough to steer a mouse with that kind of precision, they’d have to be able to analyze tiny movements which don’t need a lot of mental effort to imagine or execute physically. If you ask me, there is one key point here:

BMIs won’t conquer the market to replace keyboard and mouse.

Why? Well, it simply is not necessary. I can already play games, just replacing the current control scheme is pretty lame. No way anyone is drilling holes in my head for that. In my opinion, the ideas need to go further. And these actually have me excited: With a BMI at the right place, we can tap into the mind of the player, in a way that is simply impossible without. That is the real deal.

I’m thinking of a device that would have to read out intentions (approach/withdrawal, friend/foe, build/destroy, that kind of stuff) and other covert aspects of user state like emotions or mental effort. If you think about what happens in a fast-paced game, it’s pretty intense: “Hit, block, drink potion, switch weapon, cast spell, block again” can easily happen in the time course of a second or two, and this assumes that movement happens more or less automatically meanwhile. Being able to execute these as soon as you even think about them, before your brain even starts the process of moving your fingers, with 100% precision, intuitively, that would be a real improvement. But these are extremely difficult abstract tasks, not something a BMI at the motor cortex can easily deliver.

 

You’d have to tap into the prefrontal cortex (left gif, or gif if you prefer that…),  which does a lot of rule understanding and planning and also has parts that are deeper and not directly at the skull. Or even entirely different parts like the Anterior Cingulate Cortex (right gif), which is involved in action monitoring and evaluation.

Prefrontal cortex (left) animation
Anterior cingulate gyrus animation

So, as of now I would say Neuralink does not go in the direction of a BMI that actually enhances gaming, they just don’t aim at the right cortex areas. Their current selling point of motor control replacement directly counteracts the idea of top-level control with a functional gain for healthy players.

However, if they start placing electrodes at areas in the brain involved in more abstract processing, things can get pretty cool indeed! I still think that they will have to find a way to do it without opening the skull, but hey, who knows what the crazy tech enthusiasts of the future will do to their brains?! A BMI that really includes most of the cortex would make gaming and especially multiplayer gaming an entirely new experience. Imagine really communicating on an intuitive level with your teammates, acting as a hive mind to win your game… Seeing things happen at the moment you intend to cause them… that would truly be magical. Alas, this is mere science-fiction for now. We’ll have to see what they come up with, and I will definitely keep you posted! Personally, I can at least imagine Neuralink succeeding in creating devices that go in this direcion.

Still, I will continue to work on the head surface with electroencephalography (EEG) until I find something better. The nice thing is that, unlike a bunch of intracranial electrodes at the motor cortex, a surface EEG does pick up signals from more abstract brain areas, which can be used to measure mental focus, surprise or evaluations of goals. It’s not very good at picking up fine-grained motor execution, but you can still do great things with EEG. For example, these are wonderful measurements to be used for magic in a VR game, so that’s what I do!

If you liked this text, please leave a comment here or in the RVM Discord community, where interested people from gaming and neuroscience can hang out and chat!

Join the Discord server!

Kommentar verfassen