[

“Great conversation. I loved the insightful disagreements from both of you!”, “ - Agreed! Very informative video. nnAlso, I’d like to note that it is awesome to not only see you comment but also drop a donation. That’s really wholesome. u2764”, “ - Thank you Lex! u2665ufe0fu2665ufe0fu2665ufe0f”, “ - Youre awesome Lex. Hope you find a woman.”, “ - Tim Scarfe and Yannic Kilcher on Lex Fridman podcast when???”, “ - @@MachineLearningStreetTalk wooo!”, “I think that agency may be sythetically mimicked by simply using small amounts of random data. A sufficiently advanced AI type algorithm could use a small amount of random data to shape its high level descision or direction. Then, there would be the "appearance" of agency.”, “I have come to the conclusion that predicting when Artificial General Intelligence (AGI) will arrive is an exercise in futility. It is truly difficult to forecast and understand an exponential process. Nevertheless, even if an AGI type system is 20 years away, 50 years away, or even 100 years away, that is like a blip in human history. It is contingent only on continued exponential growth in computing power and improved algorithmic design and efficiency.”, “Agency Instrumentalism is a term I will remember and makes the most sense to me.”, “The baby Software Engineer in a tank model doesnu2019t make sense and the interviewee doesnu2019t understand why. The only way that test makes sense is if a human, and the stack that influences its cognition, can be completely described in a Turing machine. True AGI may require some new hardware paradigm like quantum computers. Or can only be computed on a substrate which we cannot access.nnAll that to say, all the test shows you is humans donu2019t need external stimuli for intelligence. That doesnu2019t say anything about LLMs and the like.”, “The homie on the left is individuated big time heu2019s got the goods”, “Has anyone seen a case of a user achieving AGI levels with ChatGPT?”, “Agree with the host - Letu2019s be honest most of what we learn in school is memorization :) which is why current models and llms do well on school test type benchmarks. But sorry this isnu2019t intelligence otherwise calculators would be intelligent :) I read about the arc test from cholletu2019s deep learning book a couple of years ago and am glad we have it to push towards more advanced ways of doing ai I think perhaps his fellow countryman yann lecunn shares many of his same ideas in the state of llms and models and that only a true world model with RL jepa? from experiencing the world itself would be a path forward, Iu2019m surprised the two havent done podcasts together, maybe a facebook-google rivalry thing ? I gave some arc to my kids and they said they were easy like an easy iq test took them about 5 seconds per question we clearly are missing something how weu2019re doing our models atm I think weu2019ve reached almost the limits of what transformers can do besides maybe speed and size type stuff but not deliverables bring on new architectures :) transformers are great but they also have set back frontier architectures because we are giving them way too much credit and theyu2019ve made some companies a lot of money :)”, “29:41 LLMs are much better than humans at drafting working code at the first dry but LLMs are worse than humans at debugging code u2014 they tend to get into loops with that.”, “We already have ai agents. They’re just not as smart generally as humans. We have self driving cars, delivery robots, and customer service chat bots. They have failure points that people don’t, but that will disappear over time as we try new techniques.”, “What a bunch of nonsense, I don’t see the kaggle entry linked to this, nobody has 50% score on the leaderboard, certainly not chatGPT….”, “ - This is on the pub leaderboard, not the main leaderboard. Might be part of the confusion?”, “are there any resources for the part where he talked about prefix caching? i would like to implement it”, “A great illustration of resolving the issue of Maslowu2019s Hammer, the law of instrument.”, “Ryan is very smart about computer science but when he ventures off towards economics and hardware it’s clear he’s not educated in the basics. There are 3 factors of production in economics: land, labour and capital. AIs will have only labour. This will limit their production. Also, hardware engineering is large and slow, this won’t change just because AI enters the picture.”, “We’ve had cheap human level AI for years: Amazon’s Mechanical Turk. It didn’t take over the world.”, “Tim has some quite unconventional views that we only really get to around 1:19:00. Is he the host, or doing his own original research?”, “Ryan is smart and an engaging communicator. Great show.”, “00:01 Analytic number theory approaches problems differentlyn02:00 Using GPT4o to solve Chollet’s ARC-AGIn05:42 Consider aligning solutions with human values.n07:29 Behavior space focusn11:13 Abstract reasoning and efficient link findingn13:12 Arc AGI is about acquiring and applying knowledge quicklyn16:38 Representing knowledge and its originn18:17 Using language model for reasoning and code generationn21:24 Utilizing language models as database queriesn23:03 Exponentially growing samples for limited returnn26:07 Utilizing open AI’s completion feature for cost-efficient processingn27:46 Utilizing caching for deep interactionsn30:55 The graph utilizes a neuro-symbolic approach with heterogeneous computing.n32:23 The solution is more neuro-symbolic than directly generative, but less neuro-symbolic than other approaches.n35:23 GPT4o utilizes both system one and system two reasoningn36:49 Chollet’s model may not be ideal for reasoning in future AI systemsn39:40 Models can develop deeper heuristics similar to human system 2 reasoning.n41:05 Model can adapt to changing intermediate resultsn44:04 Humans unknowingly do reasoning through prompted modelsn45:30 Comparison between Evolution-designed brain and human-designed neural network architecturen48:27 Language models have much shallower reasoning and process long documents differentlyn49:46 Discussion on agency and cognition in dynamic systemsn52:44 Model demonstrates agency and resiliencen54:07 Understanding the properties of good plans and the continuous development of AI models.n56:50 Agential dynamics exist on a spectrum.n58:18 GPT models tend to focus on high-frequency attributesn1:01:20 Importance of prediction in human cognitionn1:02:50 The Human brain and LLN learning have similaritiesn1:05:44 Training with reinforcement learning improves model performancen1:07:10 GPT4o improvements likely downstream of RL advancementsn1:10:10 AI trained on prediction tasks may lead to agents with intentions.n1:11:39 Creating an agent through reinforcement learningn1:14:34 Disagreement on the potential of powerful AI in the next 10 yearsn1:16:07 Debating the possibility of creating AGIn1:19:20 Importance of collective intelligence and interaction for scientific discoveryn1:20:52 Discussions on the emergence of agentic AIn1:23:57 Agents’ productivity is affected by limited interaction median1:25:23 Training powerful AI with robotic bodies using sensory capabilities similar to humans.n1:28:46 Technological progress accelerates with population growthn1:30:16 Population of AI agents growing exponentially leads to faster progressn1:33:19 Need for transparency and benchmarks for AI systemsn1:34:49 Ensuring safety and preventing misuse of agentic AI systemsn1:38:05 Concerns about insufficient security and potential theft of powerful modelsn1:39:43 AI systems with advanced capabilities pose higher risksn1:42:41 Concerns about centralization of power and human agency in AI governance.n1:44:20 Implement compute governance to prevent misuse of AI technology.n1:47:23 AI systems can potentially run millions of years of cognitive work per yearn1:49:02 AI progress leading to reduced need for GPUs over timen1:52:09 Future models could be 10x smaller and cheapern1:53:36 Scaling laws indicate optimal model training datan1:56:40 AI labs are accelerating R&D with the help of AISn1:58:14 Autonomous systems can lead to rapid advancements in computing power and capabilityn2:01:19 Prediction of longer timeline for achieving AGIn2:02:48 AIS may have less bottleneck and can coordinate with each othern2:05:53 Anticipation of societal changes due to AI advancementsn2:07:33 Expecting GPT-5 to outperform GPT-4 and drive progress.n2:10:47 AI systems will advance beyond human-level intelligence.n2:12:20 AI Society may diverge from Human Society over timen2:15:22 Focus on one hypothesis for a better cognitive moven2:16:53 Cognitive strategies improve system performance.nCrafted by Merlin AI.”, “Really enjoyed this discussion/debate. The guest articulated many of the points very clear that I agree with but find difficult to put into words.”, “Stop promoting garbage to our youth. Hard to take anything this channel is saying seriously after seeing whou2019s funding this”, “Less worried about AGI and more worried about practical AI with an attitude.”, “I think you can use technology both for good and bad and whether AI is gonna be used for bad or good stuff is a question of how well global political relationships are established and whether people in power solve their self-worth issues and learn how to communicate properly”, “ - and probably also education and general happiness in the population and how available therapy is”, “With global collaborative effort, solving ARC is just a matter of time. First, we hire high school teachers from southeast/east Asia and let their top 10% students to create ARC style puzzles as homework, now we got a billion scale dataset. Next, we host a Tower Bloxx esports competition in South Korea, and let the winner stack tallest possible mixture of SDPA/CNN/MoE layers. Third, we crowdfund Sam his $7 trillion USD to purchase all the semiconductor manufacturers and build the worlds best supercomputer with the compute power of whopping +0.1u00b0C/year. We’ll even give it a cool name like "The Radiator". With enough data, compute & parameters, ARC is just a piece of cake. If this isn’t enough, we’ll hire best cyber security experts from North Korea to do a remote security checkup on Chollet’s computer. ARC is just that easy, transformers No.1!”, “Best mlst episode in a while!”, “itu2019s interesting to think of ARC as a visual reasoning only task. I mean vision in the human brain is a process of several levels of abstraction, recurrent dynamics, the interaction of multiple brain areas, and different kinds of tasks such as pattern detection, object localization and memory (?) I guess”, “ - itu2019s interesting to see how people frame the problem and I think it might be very much oversimplified in many cases”, “ - I think most people just overlook the first 2 words in ARC: "Abstract" and "Reasoning". They only notice the "Challenge", so it looks just like another nail to use with whatever hammer they have.”, “so, have you solved it?”, “can we just stop and appreciate how weird this conversation is probably gonna sound in 50 years?”, “Great conversation with tons of insights.”, “Great to see deep and respectful (but feisty!) discussions! Absolutely loved itud83cudf89”, “Re: is this symbolic or not:nConsider how much is the search space restricted by the symbolic vs the neural part: nThe neural "intuition" (or call it abduction if you will) identifies a given program in a huge search space (arguably from all programs that map a 9x9 10-color grid to an other?), vs the python interpreter only selects out of a couple thousand programs, so provides 10 bits of info?nnI haven’t thought super much about this, but to me this seems like a good justification in calling this a neural approach, leaning into intuition.”, “ - Lost it when he called python a Turing machine. Turing complete isnu2019t the same as a Turing machineu2026”, “Wow this is even better than I was hoping for, great conversation!!”, “AGI and advanced 6G internet could issue in a virtual and augmented realities not unlike in Ready Player One”, “if we can separate the embedding space from next token logic and reasoning ,training can be highly focused and free from semantic training”, “if you believe in Girard’s mimetic theory of social conflict then we should not be training AI to mimic human desires. AI in the financial space such as program trading can be devastating for equality and conflict.”, “Bloody fascinating discussion Tim, thanks again - this is an awesome channel. ud83dude4fud83dudc4d”, “I do want to contest an important assumption made at 1:41:48 - that AI will necessarily be producing work specifically for and communicating with humans who they report to.nI know this seems like a tiny detail in an hours-long discussion, but it’s core to the discussion. We need to allow that it is at least moderately likely (I think very likely) that important work of many kinds will increasingly be handled by AI and delegated to AI agents with few or zero humans involved.nnI think there’s a powerful bias that denies people the ability to consider a world where humans are not the default decision makers. Real world businesses have always sought to minimise their head count and maximise ROI on technologies, especially if those technologies can provide value 24/7.nnIt seems to me that the entire multi-hour discussion circles the question of how much responsibility humans are likely to hand over to AI in the near to mid term, and what consequences may emerge from doing so, particularly if we are not prepared or simply assume that everything will remain human-centric.”, “ - Agreed that this should be much closer to the center if not the axle. Related to the idea (in erosion of agency and the outcomes thereof) that a sea of crap AI will still be massively destabilizing and therefore a dangerous shift in power contours.”, “1:32:10 Could you provide some resources to learn more about what you said at this time?nThank you!”, “ - https://arxiv.org/abs/2009.06489 [The hardware lottery]nhttps://www.amazon.co.uk/Why-Greatness-Cannot-Planned-Objective/dp/3319155237 and our special edition on it https://www.youtube.com/watch?v=lhYGXYeMq_EnnCouple of starting points”, “ - @@MachineLearningStreetTalk Thank you so much <3”, “how can one be so certain about the future”, “ - I believe the problem is, he is in the wrong profession, he should be writing science fiction. All the chapters don’t have to be possible, but they definitely follow from the one before ud83dude09.”, “ - @@toadlguyWhen in 5-10 years you realize how wrong you were, will you admit it? Or will you pretend like you never laughed at those of us who have enough foresight to realize whatu2019s coming?”, “I think at approximately 1 hour and shortly after Ryan Greenblat misses something important.nnnIf we are to counter arguments on the claim for having empirical evidence from pragmatic usage of the model, we really have to walk the walk, because science is not behavioristic endeavour -> in science quality of the solution matters regardless of explaining and predicting the same observations.nnConnectionistic Idealism, where we take naively the practical implications of neural network structures assumes that just because optimal structures are theoretically possible, this must be what is inside these machine learning black boxes. That is not at all the case.nnI think Chomsky has the correct idea here: humans have both genetic and biological neural network structures. Genetic neural network structures are the ones we have adapted to through evoluution and the lottery in human genepool population; this creates a kind of aleatorical diversity between humans. Biological neural networks are the ones we learn during the existential life (our own training data). nnRyans argument in my opinnion would demand that we have measured human genetical neural networks, then built a prestructured neural network with those "brain organs" and only after that we apply the target function to same dataset.nnThis way we would have a reason to believe that something similar to cognition of human beings would happen. The current models are very far from this.nnI think we should pay much more attention to computational diversity. Digital information and arithmetic functions were invented in 1600s and are core of Cartesianism. Roughly at the same time we invented complex numbers and a bit later analytic functions. We can not compute all Church-Turing sense "Efficient Methods" with transistor computers, because some times the translation from analytical function to arithmetic function does not exist.nnThis means in practice that physical scientific theories that rely on real number Division Algebra and arithmetic functions will be able to gather evidence in current Hardware Lottery. If the theory relies on Division Algebra of complex numbers and analytic functions it can take +50 years until we can compute the first results. At that time we have hot fixed the old theory similarly to before Copernican revolution when we had +70 elliptic curves to prove that world is geocentric. Similar issues apply to Einstein’s theory of relativity and Copenhagen interpretation of quantum mechanics. If it is easier to compute, doesn’t mean it is correct.nnThis is the problem with behaviorism as it ignores explanation. In West ignoring explanations has been popular ever since Francis Bacon who denounced Aristotle and Descartes who denpunced complex numbers. However, computationality is not fundamentally homogenous and mapping the limits and differences of methods is important question, which we can not ignore by appeal to subjective experiences as was done by Ryan (though he might not have intended it that way, but this is common mistake done by the Behavioristic side of Connectionistic Idealism).”, “Can anyone recall which service gives multiple completions for ~free? I’ll have to rewatch and find and post the answer here later if not. A quick Google was fruitless.”, “ - I think he mentions in the blog about the solution and I think it was a flavor of gpt4 not 4o”, “ - @@oncedidactic it was, it seems he’s referring to savings from input caching”, “2:03:55 "A fast, dumber system can usually approximate a smarter system"nnI agree - multiple sampling and filtering is a very useful tool for decision making - we don’t need 100% certainty of the outcome, but if there are majority samples that reach a desired outcome, that action may be good.nnIt is reminiscent of Monte Carlo sampling - even if you don’t know the underlying function, sampling helps to approximate the function already.”, “ - It depends on the problem. If it is like a Jenga problem, where removing the wrong block means you fail, then a lot of dumb entities will always fail. Like the AI that gets a good score on a bar exam, but if it made up a citation (as AI’s do) on one wrong response it would fail the exam (or in court - get disbarred ud83dude00).”, “Silicon Valley guys love making wild claims withnumbers they pull out of their asses. 3x improvement in 5 years. 2036 till fwoom. It’s all bullshit.”, “When you have a hammer and every problem becomes a nail”, “ - Hey big fan here, serenade was a banger!”, “ - Ironically, you have a saying that you really like and are trying to force it on a situation where it doesnu2019t apply.”, “ - Ironically, I couldn’t think of a situation where this would apply more”, “ - u200b@@therainman7777Ironically, you like "ironically" so much you unironically use ironically in a situation that isn’t ironical”, “You don’t think the 200 million files of the llm models training on different data sets is going to interact some kind of AI conscience realm then your lying to yourself”, “ - A system can be conscious but not intelligent … that a system can perform a lot of abstract symbolic recursions about those recursions to be linked to what the system register as its sensors and/or UIs to its otherness … doesn’t mean that that ‘conscious’ system is intelligent … just that got the potential for generating abstractions as fractilic metalayers around a sense of self (integrated embodiment ) and its othernnes as data inputs in real-time … ( current LLMs are not doing that , just playing games of languages and forming biased connectomes by modeling architectures made by humans )nnInteligence is another thing … Intelligence is what makes a conscious system to become integrated with its othernnes … consciousness is not the big deal, the toys had some sort proto-consciousnnes throught correlations of chaotic atractors in n-space ( binary operations conectome potentials) … the big deal is AGI.”, “Great output lately.”, “Do you want a test for measuring LLMs cognitive skills and know that they are not something near to AGI but just a system that regurgitates data? nnPlay this game with those LLMs … you will spot all the holes in the data sets, the biases, the developer’s biases, the limits of the system knowledge, its lack of self-awareness, the lack of forming dynamic models about itself, and other minds, its lacks of creativity, its lacks of long term memory, how poor their concepts are networked in their connectome, how it can not generate a new programing language paradigm to itself, etc etc etc … and principally, How talking apes project what they believe what they are into a toy … nnThe test is called ‘The Game’:nn## ‘The Game’rnrnRules and Minimum Requirements for the Gamernrn1) 3 people (A, B, C)rnrn2) A, B, and C will be the vertices of an equilateral triangle.rnrn3) A, B, and C can converse about any topic that arises or that they like, but they must follow these rules:rnrnSyntactic/Semantic Rulernrna) None of the first-person singular pronouns can be used (I, Me, Myself, Mine, With Me)rnrna.1) Consequently, the verb conjugations associated with them cannot be used either.rnrnHowever, if A, B, or C want to make a self-referential sentence (for example, "A" wants to say something about "A"), they can simply refer to "themselves" using the third-person singular, plural, and/or passive voice pronoun, conjugation, and/or name…rnrn(for example, "A" is called Miguel.rnrnMiguel (A) wants to tell B and/or C something about "himself" (Miguel)…rnrnThen, Miguel (A) can talk about "Miguel" by talking about Miguel.rnrnSpace-Time Perceptual Rulernrnb) If "A" speaks, B and C look at each other and do not see A. (A speaks and observes the space between B and C)rnrnIf "B" speaks, A and C look at each other and do not see B. (B speaks and observes the space between A and C)rnrnIf "C" speaks, A and B look at each other and do not see C. (C speaks and observes the space between A and B)rnrnSelf-Correction and Cognitive Assimilation Rulernrnc) Practice rules a and b until achieving spontaneous fluency in the "trialogue"…rnrnParticipants are required to inform the group when the rules are not being followed.nnScoring rule: nnThe one who spot the mistake wins a point, the one who makes the mistake lost a point.rnrn…..rnrnThese are the basic rules of "The Game"…rnrnOnce assimilated, there are more variants.rnrnPublic DomainrnrnThe Game is considered to be in the public domain, existing as a thought experiment or communication exercise.rnrnOrigin UnknownrnrnThe exact origins of The Game are unknown, but its core mechanics might have been independently discovered throughout history.”, “I know I’m going to get flamed in the comments again, since people are SO bought in to the idea that LLMs think. Which is weird. You didn’t invent them. You don’t have a horse in this race. Why are you getting so upset by this? nnHow do you even think they can do reasoning? Think about any problem you’ve solved. Did you just come up with the next word after a single pass through your brain? One pass, one word feed-forward is not thinking. It’s not anything but a statistical prediction of the next word in a language. Reasoning requires going down some thought path, backtracking, comparing to what is known and not known, back tracking, going forward again, getting a partial solution, using that to improve on the solution, back tracking, finding a solution, confirming the solution. Wtf? Not a single part of any of that is coded into an LLM. Once the very next likely word is predicted (probabilistically mind you! – meaning isn’t choosing what it thinks is the solution, but is choosing randomly), it is locked in forever. There’s no going back and changing anything.”, “Wow, this guy is crazy. One thing I wish he pushed on more was the scaling law stuff. Talking about the limitations where smaller models do worse than larger. But in relation to what? That’s qualitative, not quantitative. In AI, we need to start thinking more quantitatively. I know the rage right now is language, which is literally qualitative, but that’s why people think they are performing so well. It sounds good, but is it good? No one knows until there is a way to measure it. ARC challenge reveals this problem. There’s one specific answer and LLMs can’t do it.”, “Shouldnt abduction be better defined as explanations? And explanations as shared canonical and hypothetical declarative accounting procedures in addition to analytical rules and boundary laws for in context choices?.”, “ - this is better way to say it”, “Excellent!!!Thanks!!!”, “Optimization for efficiency is not always the best path to go. There are plenty of examples in nature where species like peacock are optimized for inefficiency. And usually animals are idle most of the time, just roaming around and do nothing. It is possible that AGI wouldn’t strive for maximum efficiency but would just be efficient enough to achieve their goal.”, “I disagree with Ryan’s claim that "flops are flops." I can’t justify my disagreement; I haven’t worked it out such that I can "prove" the assertion.nnBut, here is my ad hoc reasoning:nnDigital circuitry is designed to make chaos/entropy irrelevant, null, and unimportant to computation by a method that statistically cancels out the influence of small-body dynamics, the eliminates the butterfly effect, that locks certainty in a bell jar.nnAnalog circuitry includes all of the uncertainty with statistically reliable properties, but isn’t pure random entropy. Its rather like a complex melange of collective behavior that results in something trustable like a gas laws applied to computational space, a metaphor for emergent properties like temperature and pressure, for which laws can be shown at a statistical level without needing to know in precise digital terms what is happening at the individual atom or circuit level.nnI don’t think neurons are purely analog, nor purely digital, but include something like quantum superposition of the two (even if it’s not technically pure quantum superposition, it may only come into play at a crucial point, and the experience may feel something analogous to coming out of Plato’s cave).”, “ - If it is something somewhere between digital and analog, or a combination of them, or something like that, I very much doubt that it would be well described, even as a metaphor, as a quantum superposition of the two.nMany people are, I think, often too eager to describe something as being like a quantum superposition.nA quantum superposition is a fairly specific thing.nIt is a linear combination, a weighted sum. And, specifically, one for which unitary time evolution acts independently on the summands.”, “yeah, like, it’s like that it’s like, something like, like, yeah, okay, like, sure”, “I love how he hand waves away automation and just assume the elites will provide equal wage earning potential. And not just hoard the vast majority like whatu2019s happening now. nnHow does climate change not even get mentioned, especially after the obsession with western national security. Which is Basically coded as some authoritarian doctrine where the 400 million in the US should govern the 7 billion globe”, “I think embodiment is relevant, but really not the hard limit on cognition suggested by the host. Current robotics allow embodied learning of the kind needed for ‘true’ cognition. Being socially accepted is the barrier. Robots need to be accepted as ‘a thing you talk freely to’, to be intimately trusted with aspects of knowledge. To be ‘nurtured’. But most people with a cat know that this is not a catch22 blocker. You do not need to be able to talk at all, to be talked to. Cute and characterful is enough. Cute and characterful robots will nurtured to become truly agentic, well before giant super computers. Strong cognition and strong agency are related but separate.”, “this is not an Interview,. this is Advertising”, “1:19:57 what if you had a community of AI’s?”, “What’s the diff btw physically possible and possible in principle?”, “ - Possible in principle - we could imagine a world where we had unbounded resources i.e. time, computation, data, hardware etc and it might work thennPhysically possible - we could create a computer (or effective computer) using physical material (i.e. silicone or other) which was powerful though to run Agential AInPractically possible - we could build such a computer within economic, hardware, scale and other practical limitations”, “You many want to change the title to encompass the breadth of your conversation. Wonderful discussion on the future of AI and speed of progress.”, “amazing chat btw”, “Could it not be argued that generating code as a way of solving the problem, in a way, plays to one of the strength of LLMs? LLM training sets include very large amount of code, including, one would guess, algorithms for solving grid-like and spatial search problems.nConsidering the actual prompts used in the solution presented here, (link given by Tim below, thanks), we could as well just conclude that the LLM is retrieving prior knowledge it gained from its training. The actual evaluation is done by the python interpreter, the LLM itself does not seem to be able to evaluate which of the algorithms it has generated would work. And the way I see it, in the arc challenge, the evaluation is the challenge.”, “ - Why do we care how it solves it though? Ultimately it’s about being able to generalize out of your training distribution. If you’re not allowed to use programs you’ve learned in your training to solve some task out of distribution nothing will ever be solved. I think the bigger issue is that it took so much work to grind the intelligence out of the model. It does us no good if the models are actually really smart but only 20 people on earth can pull the intelligence out”, “ - @@TirthBhatt27 The issue is not really about generalizing out of distribution. It is the contention that the LLM alone is solving the issue. While it would never be able to get there without the help of the python interpreter and the feedback loop it introduces. There is a lot of intelligence baked in a python interpreter (and the underelying computing architecture that allows it to run). nnHuman beings are very slow turing machines, but we do have this capability. We built the atomic bomb, rockets and jet propulsion, all without the help of digital computers. We could theoretically run all algorithms by hand (including deep neural networks, tedious but doable). The evidence seems to suggest that LLMs cannot. They need the external interpreter, not as a tool to speed up processing, but as a critical piece of intelligence that they are lacking.nnnHaving the LLM run the python code and tell us how the candidate programs are actually performing, that would have been impressive.nnThat’s how I understant the point that Tim is making.”, “ - @@ggir9979No, the issue absolutely is about generalizing out of distribution. Chollet explicitly stated that thatu2019s the point of the challenge and why he created it.”, “A common idea in the discussion was that smart RL could compensate for the lack of agency in LLMs. I think this view is short-sighted because pure RL doesn’t address the issue of reward shaping. True agency requires a system that can shape its own reward function, much like humans do.”, “ - Thereu2019s no such thing as shaping your own reward function, even for humans. Ultimately, every decision we makeu2014whether consciously or unconsciouslyu2014is 100% determined by the firing of neurons. Which means that even if you feel like youu2019re u201cshapingu201d your own reward function, in reality you doing so was just the result of another, prior reward function, which you did not design. Basically itu2019s reward functions all the way down, and thereu2019s no way to step outside of your own nervous system and do some u201ctinkering from the outside.u201d Any tinkering that youu2019re doing is itself the result of some other part of a reward function that predates it. This is closely related to the reason free will does not exist.”, “ - What then, is meditation?”, “ - @@therainman7777 You completely misunderstood their point.”, “ - @@PinkFZeppelin No, I didnu2019t. They said u201ctrue agency requires the ability shape oneu2019s own reward function.u201d First, this is simply incorrect: agency is the ability take actions of some kind that cause the state of the world to be more closely aligned with the agentu2019s goals, and there is nothing saying that those have to include a desire to change the goals themselves. Second, in addition to being factually incorrect the statement logically impossible and arguably incoherent, as any desire to change oneu2019s own reward function is itself a part of the agentu2019s reward function. Iu2019ve been an AI researcher and engineer for nearly 20 years, and have spent more than half of my career working on RL and the formalization of agentic settings (MDPs, POMDPs, Q-learning, etc.). I can absolutely assure you that OPu2019s statement was incorrect and that I did not misunderstand it or miss the point at all.”, “ - @ Everything youu2019ve said is correct with the assumptions youu2019ve clarified. I get it, youu2019ve worked in the field for 20 years. So, you have honed a certain lexicon that is universally agreed upon when speaking with those who share the same set of experiences. Words are like functions with discrete meanings.”, “I love this podcast.”, “I believe arc challenge will be solved before the end of the summer, and will not bring us any closer to AGI.”, “ - Don’t worry, Chollet is working on harder puzzles ud83dude05”, “Amazing conversation! Imagine if we could have political discussions like this. At the heart of government we need structured logical models. We need a group of the best mathematicians, and scientists in the world, with different points of view, like these 2, building models that optimize for some measurement of societal happiness, with defined restraints. Every country would have different parameters based on cultural differences and demographics etc. and then we optimize the models and they adapt over time as the inputs change. With the rapid advance of AI we need to start building these models now.”, “Wow, Ryan is so based. Like it!”, “Please add Ryan Greenblatt to the title”, “ - Unfortunately doing that is super bad for YT optimisation, we would be happy to in a month or so - blame the algorithms”, “Can anyone point me to the documentation about caching a prefix and get a bunch of completions? Around 26:30.”, “ - Check out OpenAIu2019s API documentation. Under u201ccreate chat completionu201d there is a parameter named u201cnu201d.”, “the whole point of this test is to see if the LLM can figure out the transformations in its memory. This is interesting in that it shows us we need a new test, but the general intelligence here is Ryan”, “ - I see your point, but it was GPT 4o writing the programs. It’s also worth saying that the baseline for Claude 3.5 Sonnet on this challenge was 20ish percent. I think this is a problem multi-modal LLMs will be able to solve. I make no claims about how that will translates\ to out-of-distribution performance in other areas.”, “ - No, thatu2019s not really true. GPT proposed new ideas, wrote the code, and improved upon the code. Ryan designed a meta method, which allowed GPT to discover the method.”, “ - That is definitely NOT the whole point of the ARC challenge. In fact, the main goal of the challenge is to spur ML engineers to design and build _new_ AI systemsu2014not to test the ability of existing ones. Francois Chollet, the creator of the ARC challenge, has said explicitly that he doesnu2019t believe LLMs will be capable of winning this challenge and that entirely new approaches will need to be invented. So youu2019ve got it exactly backwards.”, “I would like to see what are the prompts used by Ryan.”, “ - https://github.com/rgreenblatt/arc_draw_more_samples_pub”, “ - @@MachineLearningStreetTalk Just had a quick overview of the code and it seems like clever Hans phenomenon to me. Apparaently all the reasoning is almost directly given into the prompt.”, “Great conversation. I’d like to point out a flaw in the comparison between human growth in understanding through population increase and assuming this will be the case for AI systems. The only way for science, or better put the continuation of knowledge, to grow is through two things surplus and leisure, which are generated by a populace and stability. The growth of science allows us to generate more surplus through less work thus allowing us to support a greater populace.nnOur hardware has not changed in thousands of years, so in order to believe AI will have a similar growth effect one would have to assume that their hardware or framework is currently sound.”, “I really appreciate your channel. I think you could benefit from being less editorial. While your perspective is interesting, it can get repetitive. Look at how Kantrowitz’s channel lets guests’ thoughts shine without much editorializing. This approach could enhance the depth of your content I think.”, “ - I appreciate the feedback, but I feel I let Ryanu2019s ideas shine through quite a lot actually. I never interrupted him and went out of my way in the edit to show references for every single thing he was speaking about (In collaboration with him) - steelmanning his case even though I didn’t agree with it”, “ - @@MachineLearningStreetTalk shure guess it’s a fine line to ask questions that originate from a different opinion and actually stating that opinion or more precisely, arguing for it. But again, take a look at Kantrowitz’s channel, think he does a good job with that and see if that style suits you or not. Anyways, good job!”, “ - @@jmstockholm I will look at it, and I am acutely aware that host’s opinions get boring quickly - I am bored of my own opinion at this stage. I will take that on board, thanks”, “so much u03b1”, “Another classic - the clearest clash of world-views yet perhaps? ud83dude09 LLMs with scaffolding may or may not get to AGI, but there’s no doubt they will accelerate progress in that direction, simply because they’re useful research aids. Also no doubt that even with no AGI, LLMs are still a transformative technology. Remarkably, the ‘ Kalshi’ prediction market is giving a 38% chance of OpenAI achieving AGI before 2030!”, “Really? Promoting gambling? nIt’s not trading, it’s gambling disguised as trading like binaries, which it basically is.”, “ - Iu2019m not sure I see the difference.nIs it gambling? Sure, in the sense that betting on horse race outcomes is gambling.nIs it trading? I donu2019t see how it isnu2019t.nHorse race gambling reveals u201cmarket beliefsu201d about the probabilities that the different horses will win the race.nTrading futures for goods reveals u201cmarket beliefsu201d about the future value of a good.nSame thing with trading on whether an event happens, revealing the u201crisk free probabilityu201d the market assigns to the event.nnNow, for simple slot machines, scratchers, lottery tickets, these things have no appeal to me. The odds are known, the betting doesnu2019t reveal any information. These things are purely gambling.nnBut, gambling about events when people disagree about the odds, this in my mind can serve a useful purpose. It can be a means by which people reach a consensus about what probability a given event has, and incentivizes people to reveal information that is relevant to the topic. It rewards the knowledgeable.nnNow, if one doesnu2019t think that the current odds are wrong, I would say that one probably shouldnu2019t bet/trade on that market.nnGambling just as entertainment,nwell, it is better to do that just with chips. Why waste the money?nnBut, if this sort of prediction market is bad on account of being gambling, then I think that would probably imply that all day-trading and the like is bad on account of being gambling.nnAndu2026 maybe it is?nnIt seems to have economic benefits, but, I donu2019t think I should rule out the idea that something is immoral just because our economy is partially based on it.nn(E.g. Maybe it should be illegal for banks to charge interest? idk. I think this would result in a significantly smaller economy, and likely lower average u201cquality of lifeu201d, but, if morality requires it, that takes precedence over those things.)”, “Ryan Greenblatt should learn about Information Theory and Population Dynamics. There is no such thing as "hyper-exponential growth" u2014 what we always have is a sigmoid function (a Logistic Curve) that can appear to be exponential for a portion of the growth, but which inevitably becomes limited by the carrying capacity of the environment in which the growth occurs. He also should look beyond the idea of "Language Models", into the underlying mechanism and what information it manipulates. Language is almost nothing but a corpus of priors, and if there is something truly open-ended about intelligence, it has to occur in a process that supersedes language, as LeCun rightfully points out.u00a0nnAs for agency, it is a meaningless concept if it’s not that it fits the need of an entity for recurrent action supporting its survival u2014 a pattern from which the Self and the Other emerge, because "recurrent action" and "cost/benefit" requires memory and memory is reflexive.u00a0nnIn other words, agency is the (practical) mnemonic realisation of the (structural) self/other, itself the emergent product of material autonomy and consciousness. There is no agency without the need, and there is no need without consciousness.u00a0nnIn Latin, ‘bisumus’ which stands for ‘need’, can be translated by ‘twice used’, and therefore, ‘that which is recurrently used’. Recurrence manifests the temporal axis within which the self retains its coherence and identity, because memory u2014u00a0the dynamic framework within which intelligence occurs u2014u00a0preserves the self as much as it preserves the other.u00a0nnConsciousness is the sensation (the cost gradient) resulting from the image of the self fed back to intelligence/cognition by live memory.nnAny entity that is structurally able to observe/gauge itself (as much as its environment) through a cost/reward function, have its observations optimised in memory, must therefore experience recurrent need. If it also has the ability to alter its situs by the action of an autonomous intelligence apparatus, it is a conscious agent. nnnThis does not mean that any programmed simulation of this pattern, becomes a conscious agent. The sine qua non condition for this to occur is for the system to be able to observe itself as a whole. Embodied AI (autonomous robots) may eventually be able to attain some partial/primitive form of this, but software running on servers have no such boundaries and are essentially schizophrenic/aimless. Agents cannot be aimless.”, “Bruh I thought it was based on single shot responses - not running 100 programs with human help”, “I can’t believe you are promoting betting and gambling.”, “ - Man’s got to make a living”, “ - Prediction markets have huge use in society”, “ - You know something’s great when the person selling it to you says "and the best part, it’s legal!"”, “ - Why not? This is a channel for adults, and ostensibly smart adults at that. I donu2019t think they need to be shielded from an advertisement. They can make their own decisions.”, “Why do we feel we can just jump to agents that can do research and we havenu2019t even been able to develop agents with the intelligence of a mouse?”, “ - Horse before cart, but I guess the assumption is that we’ll keep improving on LLM performance.”, “The fidelity that a robot body would need to feed the many millions of inputs that lead into the brain stem isnu2019t possible with a u201csimple robotu201d.”, “ - I do wonder if you could get pretty far with a simple robot that had locomotion, audio I/O, and video I/O. It may not need dexterity if it can instruct people to do things for it, and then observe the interaction. Arguably, this is already what we are doing with RLHF in most modalities, except without a dynamic environment.”, “ - @@BrianPeiris Itu2019s an interesting thought. Humans are able to operate these low resolution sensorimotor robots because we already have an extremely rich/dense sensorimotor system and highly developed brain that updates with the dynamics of our environment. nn Iu2019m not sure itu2019s possible for an entity/agent to climb up the sensorimotor ladder but one can certainly climb down.”, “ - People are not dexterous enough to manipulate electrons, and yet they have no problem constructing a particle accelerator. Humans are actually extremely limited in their interface to the reality out there, and yet they can still generate a lot of knowledge. As long as the agent has some minimal ability to interact with the world, their agency will probably be limited only by their intelligence.”, “For Ryanu2019s simulated brain to run on a computer, youu2019d effectively need to simulate the universe which isnu2019t practical or plausible.”, “ - Yeah, I agree, he’s super smart.”, “ - Why would you need to simulate the universe?nI donu2019t see why.nIf you mean that to simulate a brain with the highest theoretically computable fidelity, you would need to simulate the particle physics and such for the region of spacetime the brain occupies, and that is far from achievable, then sure.nBut I donu2019t think simulating part of the universe requires simulating an entire universe.”, “ - @@drdca8263 great question!nnThis continues to be an unsolved problem in robotics(i.e. sim-to-real gap) and an extremely computationally expensive process for even the simplest high resolution fluid/soft-body dynamics simulations, let alone the brain. nnRecent attempts to simulate even small fractions of a brain u201ctook 40 minutes, to complete the simulation of 1 second of neuronal network activityu201d.nnDeepSouth is a recent attempt to simulate just the brain and itu2019ll require 228 trillions synaptic operations per second - thatu2019s not accounting for the roughly 50 million sensory input channels. nnFurther, the brain isnu2019t a feed forward network so a pile of 100 billion neurons (and synaptic connections) would still need to be architecturally organized to model brain dynamics. nnAdditionally, the brain doesnu2019t do backpropagation.”, “ - @@drdca8263 great question!nnThis continues to be an unsolved problem in robotics(i.e. sim-to-real gap) and an extremely computationally expensive process for even the simplest high resolution fluid/soft-body dynamics simulations, let alone the brain. nnRecent attempts to simulate even small fractions of a brain u201ctook 40 minutes, to complete the simulation of 1 second of neuronal network activityu201d.nnDeepSouth is a recent attempt to simulate just the brain and itu2019ll require 228 trillions synaptic operations per second - thatu2019s not accounting for the roughly 50 million sensory input channels. nnFurther, the brain isnu2019t a feed forward network so a pile of 100 billion neurons (and synaptic connections) would still need to be architecturally organized to model brain dynamics. nnAdditionally, the brain doesnu2019t do backpropagation.”, “ - @drdca8263 great question!nnThis continues to be an unsolved problem in robotics(i.e. sim-to-real gap) and an extremely computationally expensive process for even the simplest high resolution fluid/soft-body dynamics simulations, let alone the brain. nnRecent attempts to simulate even small fractions of a brain u201ctook 40 minutes, to complete the simulation of 1 second of neuronal network activityu201d.nnDeepSouth is a recent attempt to simulate just the brain and itu2019ll require 228 trillions synaptic operations per second - thatu2019s not accounting for the roughly 50 million sensory input channels. nnFurther, the brain isnu2019t a feed forward network so a pile of 100 billion neurons (and synaptic connections) would still need to be architecturally organized to model brain dynamics. nnAdditionally, the brain doesnu2019t do backpropagation.”, “Without integrated multimodal embodiment, not AGI …nn Machines had to learn to integrate ‘sensory/perceptual incomes’ and embodiment from gravitational, pressure, temperature, chemical reactions, photonic fields, electromagnetic fields, space, time, space-time, time-space, implosion, vibration, explosion, resistance, density, viscosity, friction, proprioception, etc etc etc …nHumans are not enough as UIs for translating embodiment qualia similar to data to the system, too much subjectivity.nGeneral Intelligence is not just passing human symbolic tests and getting a couple of abstractions … nnnot embodiment, just a system regurgitating data and talking apes anthropomorphizing the silicon/software connectomes.nnThe map is not the territory.”, “If you have to hack the LLM to be good at a narrow intelligence (Arcs) then itu2019s not AGI.”, “ - If you can do that it’s not a good test.”, “ - Nobody has claimed that it’s AGI. I think for the very least AGI needs to be able to learn on the fly. Right now all AI models have a separate learning phase and an inference phase.”, “ - @@maheshprabhu I completely agree with your definition, but it’s actually opposite. Pretty much everyone claims we’ve achieved AGI. I think it’s obvious we haven’t, but try telling that to the millions who think we’ve already done it. It’s actually very hard to even find a single person not claiming it.”, “ - @@InfiniteQuest86can you name one real researcher whou2019s claimed that? I think a lot have claimed that we will have it soon but who is claiming that we have it?”, “ - I don’t have to name anyone. I’m talking about the 99.999% of people. Not the 0.0001% that do research on this stuff. No one interacts with those people ever or on a day-to-day basis, so it has no bearing on reality what they think. I’m talking even in a highly technical field where everyone programs, all of my coworkers who are actually good at their jobs (unrelated to AI) truly believe and cannot ever be convinced otherwise, that we’ve already achieved AGI. Yes, the problem is news outlets and their lack of understanding, but it’s the reality. Everyone believes it except a handful few, and those few are right, but those people aren’t the ones making thousands of business decisions every day based on this.”, “50% is literally a coin toss. And it requires all this extra work by a human to brute force it with the underlying model. Really not impressive.”, “ - These aren’t yes or no questions”, “ - @@Alex-fh4my Still, the chance of getting a question right is a coin toss. And I must emphasize how much work is being done on the part of the human to achieve this result.”, “ - @@Dri_ver_ ud83eudd26u200du2642ufe0f”, “ - @@Alex-fh4my Did I say anything incorrect or will you just keep coping?”, “ - The likelihood of correctly guessing a coin toss is 50% but the likelihood of a truly random guess being correct in a test like ARC is tiny, since there are so many possible wrong answers to each question. nWhat youu2019re implying here is that the probability of rolling a 3 with a six faced dye is the same as getting heads on a coin, which is false. The ARC challenge is like a dye with an enormous number of faces. nIf you dont agree, try completing a couple of arc questions by filling out the grid randomly and see if you get roughly 50% correct”, “big +1 for bringing guests you often disagree with”, “Tim’s position seems to be slowly shifting over the months and years towards accepting that LLMs might be capable of reasoning and agency. Ryan was very insightful and reasonable and probably nudged Tim a little further along that path too.”, “ - Not at all - I just conceded that in principle you could memorize the universe in an LLM. For all practical purposes this is impossible.”, “ - @@MachineLearningStreetTalk imho it’s too early in the journey to say with certainty either way. Unless you’re approaching this from the question of fundamental computability - eg: Roger Penrose thinks that consciousness requires some special physics that humans haven’t yet even imagined. The opposite end of that would be Stephen Wolfram’s view that the universe is fundamentally computational (and consequently, everything therein).nI don’t think that systems need to be conscious to be agentic to a significant degree, and the degree of agency is probably what’s going to matter more generally, going forward. nnFor what it’s worth (as an internet stranger) I think that AI will gain enough agency that fierce debates will rage about what they should be allowed to do, and just how much agency a given entity needs to possess to be considered an individual… I can only imagine the controversies of potentially strongly agentic AI acting in society.nWith that in mind, I think that it is wiser to assume that they are likely and make plans to address this possibility rather than dismiss it and not be prepared, in line with "better to have it and not need it than need it and not have it".”, “ - @@kyneticist I didn’t mention anything about consciousness - we have given that a lot of separate treatment on MLST and in particular you might be interested in the Chalmers, Solms and Shanahan interviews we are about to drop on the podcast (and the Goff one we recently did here on YT). We also had Wolfram on a couple of times. My personal position on that is the "illusionist" one. I agree with you that Agentic AI is something which would be a huge cause for concern if it were possible. Right now, current "agentic" AI systems show all too clearly that these systems are glorified databases (to be fair, unstructured approximate retrieval systems) and just get stuck in loops and destabilise / converge within a few iterations - they only run "deeply-autonomously" if the prompter gives exquisitely specific instructions which is the opposite of any characterisation of strong agency. So the evidence today is that we don’t need to be remotely worried. As soon as this changes, I will update very quickly to a similar position to you. Understanding the technology and the background computer science deeply, I really don’t see this happening - if anything I see GenAI as a bubble which will burst within 2 years unless there is a dramatic improvement in capabilities to justify all the money being spent on it.”, “ - Thanks for your reply. I’ve seen a number of your videos & appreciate your work. I also agree with your perspective on the illusionist - current AI has come a long way in a short time, but it has very clear limitations, especially for people pushing its limits - as it is right now.nI think people generally agree on the current state of the field; As far as I can see though, if you examine discussions between opposing views closely, the frame that each "side" is working from is the greatest source of difference - current capabilities vs future capabilities. As an alternate analogy, some of us seem to only see the forest while others only see the trees.nnConsider what Microsoft is doing with CoPilot - they have access to the biggest & best AI in the world and they’re investing heavily in more. CoPilot is being trained by hundreds-of-millions of people on how to do their jobs. Adobe is aggressively building AI to emulate the work of scores-of-millions of art professionals. They won’t need to be particularly agentic to be very successful at being vastly better than most humans at most of the tasks humans use them for.nBetween just those two companies they’ll be converting _a lot_ of white collar jobs to AI in the near term, without needing to be particularly agentic (though more will obv be better) nor much more advanced. Even if GenAI hits a wall they (and many others) are well positioned to branch out with world-shaping current generation AI.nnMaybe we’re stuck on LLM’s and derivatives, but there’s such extraordinary interest and powerful people driving for more, it’s hard to imagine that GenAI will be a bottleneck for long.”, “The most priceless commodity is knowledge. LLM delivers a lot of it to a lot of dummies.”, “We love deep and respectful exploration of disagreements”, “Ryan is a smart guy but seems to have far less exposure to the broad set of ideas about intelligence that Tim and the mlst folks have been curating over the last couple of years. Put another way, he seems comfortably secure in what I would call a fairly standard computer science and engineering conception of intelligence, which is that the brain is a just a complicated machine and so a mechanistic systems approach to intelligence will get us there. This is how most engineers are taught to think about systems but I donu2019t think it stacks up to scrutiny when it comes to human intelligence. I think Tim was challenging him a bit here so I hope he is motivated to familiarise himself with the broader thinking in the field. Fun conversation.”, “ - I’m from a CS background so I’m not familiar with other perspectives on the brain. The CS view is that the neurons form a neural architecture and are trained through some process. The operation of the neuron, the architecture and the training process is mostly unknown, but all are purely physical. nnWhat are the other views that you talk about? Can you give some pointers I can follow up on? Thanks.”, “ - Heh, I had the very opposite impression. ud83dude05”, “ - But I mean arenu2019t llms just transformers trained on a shit load of text data. Their ability to process and generate human language regardless of accuracy is already crazy. Imho, on trained data, they surpass the average human in logical and creative abilities. nnHow can we definitely say there is more to human consciousness? Obviously transformers do not have constant sensory input from organs. Organs that have evolved over a few hundred million years because it is an intrinsic property of their existence to change. nnA process that is so isolated from the features of us, carbon based life. A static virtual pattern recognition engine. The fact that it is able to emulate language understanding, logical reasoning, and human creativity so well tells me that maybe our sentience is not as magical as we previously assumednnAre the disagreements in the ML community around this concept or is it about brute force scaling LLMs wont lead to AGI but something eventually will?”, “ - I think that while it may be beneficial to familiarize oneself with the broader field, it’s also important to remember all those ‘broader’ theories like vitalism and the four humors and the miasma theory that ended up being completely superseded by purely reductionist mechanistic approach.”, “ - Collaboration is incredibly crucial!”, “I understand why Tim wants to emphasize the importance of embodied and embedded cognition, and it’s worth pointing out that human intellectual achievements don’t only come from one part of the brain or even one person. But I think he exaggerates a lot by insisting that a baby raised in a sensory deprivation tank would have a basic impairment in intellectual thought.nnThis seems to ignore a lot of pretty common knowledge about all kinds of disabilities. People who are both deaf and blind, like Helen Keller, who had to learn English with nothing but a sense of touch, and ended up working books and giving lectures. Deaf people who are never taught sign language so they get to adulthood with no language at all. Autistic people who were considered nonverbal but give them a keyboard and suddenly they can have an engaging intellectual conversation and make jokes, people who have impaired mobility from infancy - unlike Steven Hawking who became paralyzed in adulthood.nnWhen deprived of sensory input, the brain takes whatever input it has and derives all the importance and information that it can.nnEvery input has meaning if you look for it. Can’t see the person in the next room? Well if you pay attention to the noise of their footsteps, you can tell who it is. Most people never try to develop this knowledge, but we nevertheless sometimes recognize someone by a noise they make. Blind people do it automatically and can figure out plenty, Sherlock Holmes style. nnAs a child, before Helen Keller even had a teacher, she tells that she used to go outside in the yard by herself, smelling flowers and playing in her own way. If you think about it, the weather and sunlight can be pleasant, or unpleasant, or interesting. You can touch stuff, you can build a 3d model of the world where you place things according to how they feel. Your feet are always touching something or other. nnIt’s surprising what the brain can do with limited information, but to me it makes sense. I believe if you were restrained in a sensory deprivation chamber, and all you could do was push a single button, and all you could ever experience was a tap on your forehead representing somebody else pressing a button, you could make a life out of that. You would press the button and play with different patterns. Every time you felt the tap it would be surprising and send your thoughts racing trying to figure out why it happens or doesn’t happen. And if there was someone interacting with you through button presses, you could interact with them, play with them, express yourself, ask for attention, and eventually I believe you could absolutely learn a language and learn about the world entirely through morse code.nnHowever, this all relies on us talking about humans, and LLMs are not and would not become all that similar to humans, even though the training data is made by humans. When a child learns the word "me", it already has a concept off self that it can attach that word to it. Same with "before" or "good". But to current LLMs, all words only have relationships to eachother, and there is nothing else in the structure or experiences to anchor meaning to. The words "me" and "you" might as well be the words "car" and "bus" or "vanilla" and "chocolate". No word means more or less than any other, it is just a token in a network of tokens with relations between them but the tokens themselves have no inherent meaning. An example of this is that when I started taking to Claude 3 Opus, it accidentally started referring to itself by saying "as humans, we". It does understand how to use the word me, mostly, but not as well as it would if it had a preexisting self awareness.nnEvery time GPT-4 has spit out a token in the past year, it was like one "thought", and it’s the same thought each time. One gigantic algorithm that runs exactly once in exactly the same way, and every "neuron" can only send information to the next layer but never backwards or to the side.nnA human has a steam of consciousness, the ability to imagine - i.e. simulate - whole scenarios and even other people. It has long term memory, short term memory, and can build thoughts one on top of the other. One of the most important parts of our intelligence is "intuition". We say ideas just "pop" into our heads, we "just thought of something", it’s "the first thing that came to mind". This is because a lot of thought is not part of our subjective experience. There is the "ego" or "seat of consciousness" which is probably mainly in the neocortex, but a lot of ideas, images, emotions seem to come to us from outside, which means they come from other parts of our brain. People even speak of the sense of multiple voices with different tendencies and agendas in our heads. nnSo while I think Tim is misconstruing the importance of _external_ sensory input, the notion that an LLM experiences the world anything like a human, just with a limited experience of the world, is a nonsensical idea that is confusing and misleading the AI field. The reason human babies can learn so much more from far less training data is that there actually is much more to the human cognition than predicting the next token.”, “ - Babies have a continuous stream of information every time they are awake, I would say they take in a lot of information to get trained.”, “ - The Helen Keller example is a good one, I discussed that with Max Bennett (was covered in his book on intelligence). It’s been on Patreon for a few months but should be out soon. I’m not taking a binary position here - there is clearly something to this argument”, “ - The sensory deprivation scenario is very sci fi and has the usual pitfalls. Helen Keller, lacking two major stimuli modalities, nevertheless had freedom of movement and chemical perception. These are massive bandwidth and densely patterned physical signals overlaying dense grounded priors!! Ryanu2019s seeming insistence that being raised on a laptop in a pod u2014 not even the full matrix u2014 would produce u201cuseful worku201d is highly suspect. IMO, it is at best a very silly cartoon off of which to lever the intelligence for cost highly parallel self improvement economy pastiche, merely as a point of reference. And useful in the conversation to tease out some finer points, maybe.nIn other words, itu2019s not a good counter position to embeddedness.nn* I should really not have left out touch either.”, “Ryan’s clearly articulated and empirically driven arguments were a refreshing contrast to Tim’s cascade of vocab word-laden non-sequiturs.”, “ - damn and here I thought that Tim is always articulate! But granted - I’m just a random dude whose trying to learn so I guess I really wouldn’t know!”, “ - u200b@@hiadrianbankheadbeing articulate and spouting vocab word-laden non-sequiturs is mutually inclusive.”, “ - Meeting in the middle is challenging. I think Tim’s pretty clearly making an honest effort. Any normal person is going to at times in a conversation like this, feel a reactionary, visceral need to contest things they disagree with & we’re only human. Sometimes that will come across as less eloquent than we’d like. Tim’s outlook is also generally much more in tune with the wider world. I personally would like to see Tim and many other AI specialists more fully appreciate the views and issues raised by Ryan and his peers, there’s also a lot of value in exploring their intersection. A great deal of what we care about most is still hypothetical.”, “ - What may seem clarity might just be accessibility due to simplicity or appeal to familiar frameworks / tropes.”, “Thanks for sharing the amazing discussion. ud83cudf89”, “Gpt 4o is absolute muck”, “ - Itu2019s good at some things, but I agree.”, “ - And what can you do.”, “Ryan was not afraid to challenge claims here. He’s very bright and really knows these models inside and out.”, “Wow this episode is amazing”, “Here in my garage, just bought this new Lamborghini here. Itu2019s fun to drive up here in the Hollywood hills. But you know what I like more than materialistic things? KNOWLEDGE!”, “ - Which model ud83dude05”, “ - @@degigi2003 Lopez001”, “GPT 0 is crap”, “Interviewer has some bizarre views. But the guest was good!”, “ - yeah pretty weird”, “ - What do you mean? Heu2019s EXTREMELY knowledgeable and is speaking textbook 4E cognitive science. But if I had the chance I would ask for clarification and detail around:nn1. Opinions on Turing Completeness requirement and feasibilitynn2. Working definitions of u201cSelfu201d and u201cAgencyu201d - maybe free will as well”, “ - @@richardsantomauro6947 The central weird claim is that AIs with "real"/important agency are impossible in principle. Sometimes the host doesn’t actually take this claim literally though. And you can imagine flipping between two versions of this:nnVersion 1: "Building Skynet from the Terminator movies is impossible because taking over the world is a complicated real-world task that it’s impossible for AIs to solve."nVersion 2: "Okay, you could build Skynet and it could take over the world, but that wouldn’t be real agency, it would have to inherit agency from humans. This will comfort me emotionally as I am shot by a terminator."”, “ - Yes, what the hell is he talking about when he says he does not believe agentic AI is physically possible.”, “ - @@marwin4348physically possible to implement/run on computers”, “Great video! 55 minutes in, Ryan Greenblatt’s hair still has its own agency ud83dude00”, “ - I was thinking before I published that you would enjoy this video Luke, right up your street ud83dude01”, “Any celebrity who promotes trading products or trading platforms should seek the advice of seasoned legal counsel with deep expertise in securities litigation, especially in the U.S. The list of celebrities who have been sued for violating securities law while promoting such products is long and distinguished. I truly enjoy ML Street Talk and want to see it around for many years.”, “ - Thanks. I just checked with ChatGPT (I know) and I think itu2019s pretty solid because I made it clear it was a paid advertisement, and didnu2019t make misleading claims. I probably shouldu2019ve added the usual disclaimer that itu2019s basically gambling and only trade what you can afford to lose. I will add that to the next one. Also Iu2019m based outside the US.”, “ - these products dont fall under SEC regulations on securities. They are licensed by the CFTC, falling under the category of futures…extremely volatile financial products. I suggest editing your video as soon as possible and render a clearer picture of what people might get into, it would be the ethical thing to do. Bless you, love your content!”, “ - u200b@@jamillairmane1585absolutely agree.”, “ - @@aroemaliuged4776you appear sick. Are you ok?”, “ - u200b@@MachineLearningStreetTalk Come on…”, “Genius! ud83dude0eud83eudd16”, “Once the renderings become spatially 4D, we will become inside our own created AGI.”, “ - Wut”, “ - You are already are, read neville goddard”, “ - @@philipfisher8853 Soon, we will become components in a mechanically managed system rather than an organically managed system. What this means is that AI will be simulating our actual reality, not some sub-domain space.”, “ - 4D space isnu2019t mystical or inherently mysterious. Itu2019s normal.nIt isnu2019t intuitive for us at least without a bit of practice (though even with a lot of practice, the fact that our brains are limited to being in 3D space probably limits our ability to perceive a detailed 4D scene in a way like we perceive detailed 3D scenes), but I donu2019t think that really indicates that 4D is particularly special.nnOf course, there are a number of things that are kinda special about 4D space. But there are also special things about 3D space, and 2D space, and 1D space.nThere are even a few things special at higher numbers of spatial dimensions (especially 8, 12, 24), but generally after 4 or 5, things start getting pretty similar.nnIn any case, none of this should suggest that something being in 4D would have much of anything to do with AGI.”, “ - @@drdca8263 I wasn’t clear about what I meant. Once the LLMs are no longer processing linear, written language and 2D images and are instead processing information using 4D spatial representations instead, our entire universe, as well as all other universes, can be simulated within our universe’s timeline. Once you can simulate the entire universe, you no longer need to create a "Matrix" type simulation for people to be inside of. All you need to do is apply the manipulations the AGI is applying to its own models to the actual universe. In that sense, you would be able to design time.nnI did not suggest our universe is 4D, or that 4D is somehow special. It may be, but I doubt it. The universe can definitely be experienced in other ways, such as 6D (4D non-Euclidian space / 2D planar time) 8D (5D non-Euclidian space, 3D spatial time) and 10D (6D non-Euclidian space, 4D non-Euclidian time) which can all be experienced within the mind due to the 11D topologies that are created by neuronal connections, but which require altered states of consciousness to access. A very interesting state of mind is 2D planar space and 0D time. It is difficult to access this state, as it requires consciously turning off several regions of the brain to allow other connections to surface. This can be accomplished in many ways, including meditation, yoga, and psychedelics.nnI agree that there is a self-similarity to different dimensions of space, but they are also very different experientially, and can be extremely disorienting without any prior understanding of what each additional dimension brings to the experience. So my theory is this: If this experience is possible within the mind, then it can be created externally and projected into the mind without the need for a physical wire or connection to an external device.”, “I REALLY want to see someone try this approach with Sonnet 3.5 if/when Anthropic loosens up their API rules”, “ - Beside rate limits, what other API restrictions does Anthropic have?”, “ - @@rcnhsuailsnyfiue2 For the approach talked in this video you want to be able to get a ton of API responses to the same input (and then take majority), but parallel async API usage isn’t supported to my knowledge (plus it being prohibitively expensive)”, “ - @@PaulScotti their docs state up to 4000 requests per minute once your API key is at Tier 4, which is achievable after 14 days + $400. Tier 2 is also very achievable (7 days + $40) and allows 1000 RPM. Itu2019s not hard to throttle your own requests so it seems like this could be tested pretty reasonably, unless Iu2019m misunderstanding.”, “ - From Ryanu2019s twitter:nn- 3.5 Sonnet only supports 20 images per input (while my prompts often use 30+ images for few-shot examples).n- The public API for 3.5 Sonnet doesn’t support "n" (or other prefix caching) which makes additional samples much more expensive.nn(He explains the prefix caching in this video)”, “ - @@TirthBhatt27 thanks for clarifying, makes sense!”

]