Computational Thinking - Учебное пособие, страница 10

Technical failure is when you try tobuild an AI and it doesn’t work the way you think it does —you have failed to understand the true workings of your owncode. Philosophical failure is trying to build the wrong thing,so that even if you succeeded you would still fail to helpanyone or benefit humanity. Needless to say, the two failuresare not mutually exclusive.In place of laws constraining the behavior of intelligentmachines, we need to give them emotions that can guide theirlearning of behaviors.

They should want us to be happy andprosper, which is the emotion we call love. We can design52intelligent machines so their primary, innate emotion isunconditional love for all humans. First we can buildrelatively simple machines that learn to recognize happinessand unhappiness in human facial expressions, human voicesand human body language.

Then we can hard-wire the resultof this learning as the innate emotional values of morecomplex intelligent machines, positively reinforced when weare happy and negatively reinforced when we are unhappy.Machines can learn algorithms for approximately predictingthe future, so we can program intelligent machines to learnalgorithms for predicting future human happiness, and usethose predictions as emotional values.A technical failure occurs when the code does not do whatyou think it does, though it faithfully executes as youprogrammed it.

Suppose we trained a neural network torecognize smiling human faces and distinguish them fromfrowning human faces. Would the network classify a tinypicture of a smiley-face into the same attractor as a smilinghuman face?This form of failure is especially dangerous because it willappear to work within a fixed context, then fail when thecontext changes.Suppose we wish to develop an AI of increasing power. TheAI possesses a developmental stage where the humanprogrammers are more powerful than the AI, smarter, morecreative, more cunning than the AI.

During thedevelopmental period we suppose that the programmerspossess the ability to make changes to the AI’s source codewithout needing the consent of the AI to do so. However, theAI is also intended to possess postdevelopmental stages,including superhuman intelligence. An AI of superhumanintelligence surely could not be modified without its consent.At this point we must rely on the previously laid-down goalsystem to function correctly, because if it operates in asufficiently unforeseen fashion, the AI may actively resist ourattempts to correct it — and, if the AI is smarter than ahuman, probably win.53Trying to control a growing AI by training a neural networkto provide its goal system faces the problem of a huge contextchange between the AI’s developmental stage andpostdevelopmental stage. During the developmental stage, theAI may only be able to produce stimuli that fall into the“smiling human faces” category, by solving humanly providedtasks, as its makers intended.

Thus the AI appears to workfine during development, but produces catastrophic resultsafter it becomes smarter than the programmers.One of the most critical points about Artificial Intelligence isthat it might increase in intelligence extremely fast. Theobvious reason to suspect this possibility is recursive selfimprovement. The AI becomes smarter, including becomingsmarter at the task of writing the internal cognitive functionsof an AI, so the AI can rewrite its existing cognitive functionsto work even better, which makes the AI still smarter,including smarter at the task of rewriting itself, so that itmakes yet more improvements.Human beings do not recursively self-improve in a strongsense. To a limited extent, we improve ourselves: we learn, wepractice, we hone our skills and knowledge.

To a limitedextent, these self-improvements improve our ability toimprove. New discoveries can increase our ability to makefurther discoveries— in that sense, knowledge feeds on itself.But there is still an underlying level we haven’t yet touched.We haven’t rewritten the human brain. The brain is,ultimately, the source of discovery, and our brains today aremuch the same as they were ten thousand years ago.An Artificial Intelligence could rewrite its code from scratch.The key implication for our purposes is that an AI mightmake a huge jump in intelligence after reaching somethreshold of criticality.An often heard reaction “We don’t need to worry aboutFriendly AI because we don’t yet have AI,” is misguided ordownright suicidal. The mathematics and techniques ofFriendly AI will not materialize from nowhere when needed;it takes years to lay firm foundations. And we need to solve54the Friendly AI challenge before Artificial GeneralIntelligence is created, not afterward.The possibility of sharp jumps in intelligence also implies ahigher standard for Friendly AI techniques.

The techniquecannot assume the programmers’ ability to monitor the AIagainst its will, rewrite the AI against its will, bring to bearthe threat of superior military force; nor may the algorithmassume that the programmers control a “reward button”which a smarter AI could wrest from the programmers; etc.Indeed no one should be making these assumptions, to beginwith. The indispensable protection is an AI that does notwant to hurt you.ConclusionModern civilization occupies an unstable state. Pragmaticallyspeaking, our alternatives boil down to becoming smarter orbecoming extinct.

Nature is not cruel, but indifferent; aneutrality which often seems indistinguishable from outrighthostility. Reality throws at you one challenge after another,and when you run into a challenge you can’t handle, yousuffer the consequences. Often Nature poses requirementsthat are grossly unfair, even on tests where the penalty forfailure is death.

Nature does not match her challenges to yourskill, or your resources, or how much free time you have tothink about the problem. And when you run into a lethalchallenge too difficult for you, you die. It may be unpleasantto think about, but that has been the reality for humans, forthousands upon thousands of years. The same thing could aseasily happen to the whole human species, if the humanspecies runs into an unfair challenge.To survive any appreciable time, we need to drive down eachrisk to nearly zero.

But the human mind is not the limit ofthe possible. Homo sapiens represents the first generalintelligence. We were born into the uttermost beginning ofthings, the dawn of mind. With luck, future historians willlook back and describe the present world as an awkward inbetween stage of adolescence, when humankind was smartenough to create tremendous problems for itself, but not quite55smart enough to solve them.Yet before we can pass out of that stage of adolescence, wemust, as adolescents, confront an adult problem: thechallenge of smarter-than-human intelligence.

This is the wayout of the high-mortality phase of the life cycle, the way toclose the window of vulnerability; it is also probably the singlemost dangerous risk we face. Artificial Intelligence is one roadinto that challenge; and it is the road we will end up taking.All scientific ignorance is hallowed by ancientness. Each andevery absence of knowledge dates back to the dawn of humancuriosity; and the hole lasts through the ages, seeminglyeternal, right up until someone fills it.

It is possible for merefallible humans to succeed on the challenge of buildingFriendly AI. Intelligence must cease to be any kind ofmystery whatever, sacred or not. We must execute thecreation of Artificial Intelligence as the exact application ofan exact art. And maybe then we can win.In April 2000, Bill Joy, the technologists' technologist, wrotean article entitled “Why the Future Doesn't Need Us” forWired magazine.

“Our most powerful technologies – robotics,genetic engineering, and nanotech – are threatening to makehumans an endangered species,” he wrote. In August 2011Jaron Lanier, another widely respected technologist, wroteabout the impact of AI on the job market. In the non-too-farfuture, he predicted, it would just be inconceivable to put aperson behind a wheel of a truck or a cab. Machines will becapable of doing any work a man can do. AI has been provento be much more difficult than early pioneers believed. BillJoy's question deserves not to be ignored.

Does the futureneed us?Notes:IEEE Computational Intelligence Society – a professionalsociety that focuses on theory, design, application anddevelopment of neural systems, genetic algorithms,bioinformatics, and hybrid intelligent systems.John McCarthy – an American computer and cognitive56scientist who coined the term “Artificial Intelligence”.Swarm intelligence – the collective behavior of decentralized,self-organized systems, natural or artificial (ant colonies, birdflocking, bacterial growth, etc.).Anthropomorphism – an attribution of a human form orother characteristics to anything other than a human being.Wired (magazine) – a full-color monthly American magazine,published in both printed and online editions, that reportshow emerging technologies affect culture, economy, andpolitics.Exercises1. Give Russian equivalents to the following words andphrases and explain them in your own words:To overlap; trainable systems; fuzzy systems; an acuteproblem; to stem from...; stellar astronomy; to take forgranted; to anthropomorphize; an anthropomorphic bias; toput oneself in one's shoes; to underestimate; oft-committedfallacy; to overwhelm human resistence; the vast majority; apalatable region; a beneficial effect; prediction and realitydiverged; to rationalize ways to produce nice results; aninnate emotion; to hard-wire the result; a goal system.2.

Translate the sentences into Russian paying attention toprefixes and suffixes:1. Neural networks are trainable systems with very strongpattern recognition capabilities.2. Human beings often “anthropomorphize”— expecthumanlike properties of that which is not human.3.

Underestimating the power of intelligence implies aproportional underestimate of the potential impact ofArtificial Intelligence, which is symmetrical around potentialgood impacts and potential bad impacts.4. There is a fallacy oft-committed in discussion of ArtificialIntelligence, especially AI of superhuman capability.5. We need relational understanding to manipulate the future,steer it into a region palatable to humankind.576. The AI is intended to possess postdevelopmental stages,including superhuman intelligence.7. Artificial Intelligence might increase in intelligence throughrecursive self-improvement.8. Nature is not cruel, but indifferent; a neutrality whichoften seems indistinguishable from outright hostility.3.

Give English equivalents to the following words andphrases:Основатели ИИ; испытывать радость; дружественныйИИ; цепочки рассуждений; противостоять искушению;направлять будущее; естественный отбор; предсказания иреальность разнятся; изобретательный программист;сформулировать цель; саморазвитие; переписать с нуля;достичь критического порога; прочное основание;совершенно необходимая защита; изложить требования.4. Answer the following questions:1. What are the subjects in AI?2. What is the most acute problem in AI?3. Why do people often “anthropomorphize”?4. How should natural selection and evolution be consideredby scientists?5. What are the potential failures of attempted Friendly AI?6. Is there a conflict between developmental andpostdevelopmental stages of AI improvement?7.