Tim urban là ai

PDF: We made a fancy PDF of this post for printing và offline viewing. Buy it here. (Or see a preview.)


We have sầu what may be an extremely difficult problem with an unknown time khổng lồ solve it, on which quite possibly the entire future of humanity depends. — Nichồng Bostrom

Welcome khổng lồ Part 2 of the “Wait how is this possibly what I’m reading I don’t get why everyone isn’t talking about this” series.

Bạn đang xem: Tim urban là ai

quý khách hàng sẽ xem: Tlặng urban là ai

Part 1 started innocently enough, as we discussed Artificial Narrow Intelligence, or ANI (AI that specializes in one narrow task lượt thích coming up with driving routes or playing chess), and how it’s all around us in the world today. We then examined why it was such a huge challenge khổng lồ get from ANI lớn Artificial General Intelligence, or AGI (AI that’s at least as intellectually capable as a human, across the board), and we discussed why the exponential rate of technological advancement we’ve seen in the past suggests that AGI might not be as far away as it seems. Part 1 ended with me assaulting you with the fact that once our machines reach human-level intelligence, they might immediately vì this:





This left us staring at the screen, confronting the intense concept of potentially-in-our-lifetime Artificial Superintelligence, or ASI (AI that’s way smarter than any human, across the board), and trying to figure out which emotion we were supposed khổng lồ have on as we thought about that.11← open these

Before we dive inlớn things, let’s remind ourselves what it would mean for a machine to lớn be superintelligent.

A key distinction is the difference between speed superintelligence unique superintelligence. Often, someone’s first thought when they imagine a super-smart computer is one that’s as intelligent as a human but can think much, much faster2—they might picture a machine that thinks like a human, except a million times quicker, which means it could figure out in five sầu minutes what would take a human a decade.

That sounds impressive, and ASI would think much faster than any human could—but the true separator would be its advantage in intelligence quality, which is something completely different. What makes humans so much more intellectually capable than chimps isn’t a difference in thinking speed—it’s that human brains contain a number of sophisticated cognitive modules that enable things lượt thích complex linguistic representations or longterm planning or abstract reasoning, that chimps’ brains vị not. Speeding up a chimp’s brain by thousands of times wouldn’t bring hlặng to lớn our level—even with a decade’s time, he wouldn’t be able lớn figure out how to use a mix of custom tools to assemble an intricate Mã Sản Phẩm, something a human could knoông xã out in a few hours. There are worlds of human cognitive sầu function a chimp will simply never be capable of, no matter how much time he spends trying.

But it’s not just that a chimp can’t vì chưng what we bởi vì, it’s that his brain is unable lớn grasp that those worlds even exist—a chimp can become familiar with what a human is và what a skyscraper is, but he’ll never be able to lớn underst& that the skyscraper was built by humans. In his world, anything that huge is part of nature, period, & not only is it beyond hyên to build a skyscraper, it’s beyond hyên to realize that anyone can build a skyscraper. That’s the result of a small difference in intelligence quality.

And in the scheme of the intelligence range we’re talking about today, or even the much smaller range aao ước biological creatures, the chimp-to-human unique intelligence gap is tiny. In an earlier post, I depicted the range of biological cognitive sầu capacity using a staircase:3


To absorb how big a khuyễn mãi giảm giá a superintelligent machine would be, imagine one on the dark green step two steps above sầu humans on that staircase. This machine would be only slightly superintelligent, but its increased cognitive ability over us would be as vast as the chimp-human gap we just described. And like the chimp’s incapathành phố to ever absorb that skyscrapers can be built, we will never be able lớn even comprehend the things a machine on the dark green step can vì, even if the machine tried lớn explain it to lớn us—let alone vì it ourselves. And that’s only two steps above sầu us. A machine on the second-to-highest step on that staircase would be to lớn us as we are lớn ants—it could try for years lớn teach us the simplest inkling of what it knows & the endeavor would be hopeless.

But the kind of superintelligence we’re talking about today is something far beyond anything on this staircase. In an intelligence explosion—where the smarter a machine gets, the quicker it’s able khổng lồ increase its own intelligence, until it begins khổng lồ soar upwards—a machine might take years to rise from the chimp step lớn the one above it, but perhaps only hours khổng lồ jump up a step once it’s on the dark green step two above us, and by the time it’s ten steps above us, it might be jumping up in four-step leaps every second that goes by. Which is why we need khổng lồ realize that it’s distinctly possible that very shortly after the big news story about the first machine reaching human-level AGI, we might be facing the reality of coexisting on the Earth with something that’s here on the staircase (or maybe a million times higher):

And since we just established that it’s a hopeless activity to try to lớn understand the power of a machine only two steps above us, let’s very concretely state once và for all that there is no way to know what ASI will vị or what the consequences will be for us. Anyone who pretends otherwise doesn’t underst& what superintelligence means.

Evolution has advanced the biological brain slowly & gradually over hundreds of millions of years, and in that sense, if humans birth an ASI machine, we’ll be dramatically stomping on evolution. Or maybe this is part of evolution—maybe the way evolution works is that intelligence creeps up more và more until it hits the level where it’s capable of creating machine superintelligence, & that level is like a tripwire that triggers a worldwide game-changing explosion that determines a new future for all living things:

And for reasons we’ll discuss later, a huge part of the scientific community believes that it’s not a matter of whether we’ll hit that tripwire, but when. Kind of a crazy piece of information.

So where does that leave us?

Well no one in the world, especially not I, can tell you what will happen when we hit the tripwire. But Oxford philosopher & lead AI thinker Nick Bostrom believes we can boil down all potential outcomes inkhổng lồ two broad categories.

First, looking at history, we can see that life works lượt thích this: species pop up, exist for a while, & after some time, inevitably, they fall off the existence balance beam và land on extinction—

“All species eventually go extinct” has been almost as reliable a rule through history as “All humans eventually die” has been. So far, 99.9% of species have sầu fallen off the balance beam, and it seems pretty clear that if a species keeps wobbling along down the beam, it’s only a matter of time before some other species, some gust of nature’s wind, or a sudden beam-shaking asteroid knocks it off. Bostrom calls extinction an attractor state—a place species are all teetering on falling into & from which no species ever returns.

And while most scientists I’ve come across acknowledge that ASI would have the ability to sover humans lớn extinction, many also believe that used beneficially, ASI’s abilities could be used lớn bring individual humans, và the species as a whole, to lớn a second attractor state—species immortality. Bostrom believes species immortality is just as much of an attractor state as species extinction, i.e. if we manage to lớn get there, we’ll be impervious to extinction forever—we’ll have sầu conquered mortality & conquered chance. So even though all species so far have sầu fallen off the balance beam & landed on extinction, Bostrom believes there are two sides lớn the beam & it’s just that nothing on Earth has been intelligent enough yet to lớn figure out how to fall off on the other side.

If Bostrom and others are right, và from everything I’ve read, it seems like they really might be, we have sầu two pretty shocking facts to absorb:

1) The advent of ASI will, for the first time, open up the possibility for a species lớn lvà on the immortality side of the balance beam.

2) The advent of ASI will make such an unimaginably dramatic impact that it’s likely khổng lồ knock the human race off the beam, in one direction or the other.

It may very well be that when evolution hits the tripwire, it permanently ends humans’ relationship with the beam & creates a new world, with or without humans.

Kind of seems like the only question any human should currently be asking is: When are we going to lớn hit the tripwire and which side of the beam will we land on when that happens?

No one in the world knows the answer lớn either part of that question, but a lot of the very smardemo people have sầu put decades of thought inlớn it. We’ll spend the rest of this post exploring what they’ve come up with.


Let’s start with the first part of the question: When are we going to lớn hit the tripwire?

i.e. How long until the first machine reaches superintelligence?

Not shockingly, opinions vary wildly and this is a heated debate among muốn scientists & thinkers. Many, like professor Vernor Vinge, scientist Ben Goertzel, Sun Microsystems co-founder Bill Joy, or, most famously, inventor and futurist Ray Kurzweil, agree with machine learning expert Jeremy Howard when he puts up this graph during a TED Talk:

Those people subscribe khổng lồ the belief that this is happening soon—that exponential growth is at work and machine learning, though only slowly creeping up on us now, will blow right past us within the next few decades.

Others, lượt thích Microsoft co-founder Paul Allen, retìm kiếm psychologist Gary Marcus, NYU computer scientist Ernest Davis, and tech entrepreneur Mitch Kapor, believe sầu that thinkers like Kurzweil are vastly underestimating the magnitude of the challenge & believe that we’re not actually that cthua thảm lớn the tripwire.

The Kurzweil camp would counter that the only underestimating that’s happening is the underappreciation of exponential growth, & they’d compare the doubters to lớn those who looked at the slow-growing seedling of the internet in 1985 and argued that there was no way it would amount to anything impactful in the near future.

The doubters might argue bachồng that the progress needed khổng lồ make advancements in intelligence also grows exponentially harder with each subsequent step, which will cancel out the typical exponential nature of technological progress. And so on.

Xem thêm: Tiểu Sử Lưu Ánh Loan Là Ai ? Tiểu Sử Ca Sĩ Lưu Ánh Loan Tiểu Sử Ca Sĩ Lưu Ánh Loan

Still others, lượt thích philosopher Hubert Dreyfus, believe sầu all three of these groups are naive sầu for believing that there even is a tripwire, arguing that it’s more likely that ASI won’t actually ever be achieved.

So what bởi you get when you put all of these opinions together?

In 2013, Vincent C. Müller và Nick Bostrom conducted a survey that asked hundreds of AI experts at a series of conferences the following question: “For the purposes of this question, assume that human scientific activity continues without major negative sầu disruption. By what year would you see a (10% / 1/2 / 90%) probability for such HLMI4 lớn exist?” It asked them to lớn name an optimistic year (one in which they believe there’s a 10% chance we’ll have AGI), a realistic guess (a year they believe there’s a một nửa chance of AGI—i.e. after that year they think it’s more likely than not that we’ll have AGI), and a safe guess (the earliest year by which they can say with 90% certainty we’ll have AGI). Gathered together as one data mix, here were the results:2Median optimistic year (10% likelihood): 2022Median realistic year (một nửa likelihood): 2040Median pessimistic year (90% likelihood): 2075

So the median participant thinks it’s more likely than not that we’ll have AGI 25 years from now. The 90% median answer of 2075 means that if you’re a teenager right now, the median respondent, along with over half of the group of AI experts, is almost certain AGI will happen within your lifetime.

A separate study, conducted recently by author James Barrat at Ben Goertzel’s annual AGI Conference, did away with percentages và simply asked when participants thought AGI would be achieved—by 2030, by 2050, by 2100, after 2100, or never. The results:3By 2030: 42% of respondentsBy 2050: 25%By 2100: 20%After 2100: 10%Never: 2%

Pretty similar to Müller and Bostrom’s outcomes. In Barrat’s survey, over two thirds of participants believe AGI will be here by 2050 & a little less than half predict AGI within the next 15 years. Also striking is that only 2% of those surveyed don’t think AGI is part of our future.

But AGI isn’t the tripwire, ASI is. So when do the experts think we’ll reach ASI?

Müller & Bostrom also asked the experts how likely they think it is that we’ll reach ASI A) within two years of reaching AGI (i.e. an almost-immediate intelligence explosion), and B) within 30 years. The results:4The median answer put a rapid (2 year) AGI → ASI transition at only a 10% likelihood, but a longer transition of 30 years or less at a 75% likelihood.

We don’t know from this data the length of this transition the median participant would have sầu put at a 50% likelihood, but for ballpark purposes, based on the two answers above, let’s estimate that they’d have sầu said 20 years. So the median opinion—the one right in the center of the world of AI experts—believes the most realistic guess for when we’ll hit the ASI tripwire is = 2060.

Of course, all of the above sầu statistics are speculative, và they’re only representative of the center opinion of the AI expert community, but it tells us that a large portion of the people who know the most about this topic would agree that 2060 is a very reasonable estimate for the arrival of potentially world-altering ASI. Only 45 years from now.

Okay now how about the second part of the question above: When we hit the tripwire, which side of the beam will we fall to?

Superintelligence will yield tremendous power—the critical question for us is:

Who or what will be in control of that power, và what will their motivation be?

The answer khổng lồ this will determine whether ASI is an unbelievably great development, an unfathomably terrible development, or something in between.

Of course, the expert community is again all over the board & in a heated debate about the answer to this question. Müller và Bostrom’s survey asked participants to assign a probability lớn the possible impacts AGI would have sầu on humanity & found that the mean response was that there was a 52% chance that the outcome will be either good or extremely good and a 31% chance the outcome will be either bad or extremely bad. For a relatively neutral outcome, the mean probability was only 17%. In other words, the people who know the most about this are pretty sure this will be a huge khuyễn mãi giảm giá. It’s also worth noting that those numbers refer lớn the advent of AGI—if the question were about ASI, I imagine that the neutral percentage would be even lower.

Before we dive much further into lớn this good vs. bad outcome part of the question, let’s combine both the “when will it happen?” & the “will it be good or bad?” parts of this question into lớn a chart that encompasses the views of most of the relevant experts:

We’ll talk more about the Main Camp in a minute, but first—what’s your deal? Actually I know what your giảm giá is, because it was my giảm giá too before I started researching this topic. Some reasons most people aren’t really thinking about this topic:

Humans have a hard time believing something is real until we see proof. I’m sure computer scientists in 1988 were regularly talking about how big a deal the mạng internet was likely khổng lồ be, but people probably didn’t really think it was going khổng lồ change their lives until it actually changed their lives. This is partially because computers just couldn’t bởi vì stuff lượt thích that in 1988, so people would look at their computer and think, “Really? That’s gonmãng cầu be a life changing thing?” Their imaginations were limited khổng lồ what their personal experience had taught them about what a computer was, which made it very hard khổng lồ vividly picture what computers might become. The same thing is happening now with AI. We hear that it’s gonna be a big giảm giá, but because it hasn’t happened yet, and because of our experience with the relatively impotent AI in our current world, we have sầu a hard time really believing this is going lớn change our lives dramatically. And those biases are what experts are up against as they frantically try lớn get our attention through the noise of collective daily self-absorption.Even if we did believe sầu it—how many times today have sầu you thought about the fact that you’ll spkết thúc most of the rest of eternity not existing? Not many, right? Even though it’s a far more intense fact than anything else you’re doing today? This is because our brains are normally focused on the little things in day-to-day life, no matter how crazy a long-term situation we’re a part of. It’s just how we’re wired.

One of the goals of these two posts is lớn get you out of the I Like lớn Think About Other Things Camp và into one of the expert camps, even if you’re just standing on the intersection of the two dotted lines in the square above, totally uncertain.

During my retìm kiếm, I came across dozens of varying opinions on this topic, but I quickly noticed that most people’s opinions fell somewhere in what I labeled the Main Camp, và in particular, over three quarters of the experts fell into lớn two Subcamps inside the Main Camp:

We’re gonna take a thorough dive sầu into lớn both of these camps. Let’s start with the fun one—

Why the Future Might Be Our Greademo Dream

As I learned about the world of AI, I found a surprisingly large number of people standing here:

The people on Confident Corner are buzzing with excitement. They have sầu their sights phối on the fun side of the balance beam and they’re convinced that’s where all of us are headed. For them, the future is everything they ever could have sầu hoped for, just in time.

The thing that separates these people from the other thinkers we’ll discuss later isn’t their lust for the happy side of the beam—it’s their confidence that that’s the side we’re going lớn l& on.

Where this confidence comes from is up for debate. Critics believe it comes from an excitement so blinding that they simply ignore or deny potential negative sầu outcomes. But the believers say it’s naive lớn conjure up doomsday scenarquả táo when on balance, giải pháp công nghệ has & will likely end up continuing to lớn help us a lot more than it hurts us.

We’ll cover both sides, và you can khung your own opinion about this as you read, but for this section, put your skepticism away và let’s take a good hard look at what’s over there on the fun side of the balance beam—& try khổng lồ absorb the fact that the things you’re reading might really happen. If you had shown a hunter-gatherer our world of indoor comfort, technology, và endless abundance, it would have sầu seemed like fictional magic khổng lồ him—we have sầu lớn be humble enough lớn acknowledge that it’s possible that an equally inconceivable transformation could be in our future.

Niông xã Bostrom describes three ways a superintelligent AI system could function:6As an oracle, which answers nearly any question posed khổng lồ it with accuracy, including complex questions that humans cannot easily answer—i.e. How can I manufacture a more efficient car engine? Google is a primitive sầu type of oracle.As a genie, which executes any high-cấp độ command it’s given—Use a molecular assembler khổng lồ build a new & more efficient kind of car engine—and then awaits its next command.As a sovereign, which is assigned a broad và open-ended pursuit và allowed lớn operate in the world freely, making its own decisions about how best lớn proceed—Invent a faster, cheaper, & safer way than cars for humans to privately transport themselves.These questions & tasks, which seem complicated to lớn us, would sound khổng lồ a superintelligent system lượt thích someone asking you khổng lồ improve sầu upon the “My pencil fell off the table” situation, which you’d vày by picking it up và putting it back on the table.

Eliezer Yudkowsky, a resident of Anxious Avenue in our chart above sầu, said it well:

There are no hard problems, only problems that are hard lớn a certain cấp độ of intelligence. Move the smallest bit upwards , & some problems will suddenly move from “impossible” khổng lồ “obvious.” Move a substantial degree upwards, and all of them will become obvious.7There are a lot of eager scientists, inventors, & entrepreneurs in Confident Corner—but for a tour of the brighthử nghiệm side of the AI horizon, there’s only one person we want as our tour guide.

Ray Kurzweil is polarizing. In my reading, I heard everything from godlike worship of him và his ideas lớn eye-rolling contempt for them. Others were somewhere in the middle—author Douglas Hofstadter, in discussing the ideas in Kurzweil’s books, eloquently put forth that “it is as if you took a lot of very good food và some dog excrement & blended it all up so that you can’t possibly figure out what’s good or bad.”8Whether you like his ideas or not, everyone agrees that Kurzweil is impressive sầu. He began inventing things as a teenager & in the following decades, he came up with several breakthrough inventions, including the first flatbed scanner, the first scanner that converted text khổng lồ speech (allowing the blind to read standard texts), the well-known Kurzweil music synthesizer (the first true electric piano), & the first commercially marketed large-vocabulary speech recognition. He’s the author of five national bestselling books. He’s well-known for his bold predictions and has a pretty good record of having them come true—including his prediction in the late ’80s, a time when the mạng internet was an obscure thing, that by the early 2000s, it would become a global phenomenon. Kurzweil has been called a “restless genius” by The Wall Street Journal, “the ultimate thinking machine” by Forbes, “Edison’s rightful heir” by Inc. Magazine, and “the best person I know at predicting the future of artificial intelligence” by Bill Gates.9 In 2012, Google co-founder Larry Page approached Kurzweil and asked hyên khổng lồ be Google’s Director of Engineering.5 In 2011, he co-founded Singularity University, which is hosted by NASA và sponsored partially by Google. Not bad for one life.

Xem thêm: To Turn Up Là Gì ? Nghĩa Của Từ Turn Up Trong Tiếng Việt Nghĩa Của Từ Turn Up

This biography is important. When Kurzweil articulates his vision of the future, he sounds fully like a crackpot, & the crazy thing is that he’s not—he’s an extremely smart, knowledgeable, relevant man in the world. You may think he’s wrong about the future, but he’s not a fool. Knowing he’s such a legit dude makes me happy, because as I’ve learned about his predictions for the future, I badly want hlặng to be right. And you vày too. As you hear Kurzweil’s predictions, many shared by other Confident Corner thinkers like Peter Diamandis & Ben Goertzel, it’s not hard to lớn see why he has such a large, passionate following—known as the singularitarians. Here’s what he thinks is going lớn happen:


Kurzweil’s depiction of the 2045 singularity is brought about by three simultaneous revolutions in biotechnology, nanocông nghệ, &, most powerfully, AI.

Before we move sầu on—nanocông nghệ comes up in almost everything you read about the future of AI, so come inkhổng lồ this xanh box for a minute so we can discuss it—