Toarriveattheedgeoftheworld'sknowledge,seekoutthemostcomplexandsophisticatedminds,puttheminaroomtogether,andhavethemaskeachotherthequestionstheyareaskingthemselves.
Untilfairlyrecently,artificialintelligencedidn’tlearn.Tocreateamachinethatlearnstothinkmoreefficientlywasabigchallenge.Inthesamesense,oneofthethingsthatIwonderaboutishowwe'llbeabletoteachamachinetoknowwhatitdoesn’tknowthatitmightneedtoknowinordertoaddressaparticularissueproductivelyandinsightfully.Thisisahugeproblemforhumanbeings.Ittakesawhileforustolearntosolveproblems,andthenittakesevenlongerforustorealizewhatwedon’tknowthatwewouldneedtoknowtosolveaparticularproblem.
~
Thetragedyofthecyberneticrevolution,whichhadtwophases,thecomputersciencesideandthesystemstheoryside,hasbeentheneglectofthesystemstheorysideofit.Wechosemarketablegadgetsinpreferencetoadeeperunderstandingoftheworldwelivein.
HOWTOBEASYSTEMSTHINKER
Atthemoment,I’maskingmyselfhowpeoplethinkaboutcomplexwholesliketheecologyoftheplanet,ortheclimate,orlargepopulationsofhumanbeingsthathaveevolvedformanyyearsinseparatelocationsandarenowre-integrating.Tothinkaboutthesethings,Ifindthatyouneedsomethinglikesystemstheory.So,Iwentbacktothinkingaboutsystemstheorytwoorthreeyearsago,whichIhadn’tforquitealongtime.
Whatprompteditwasconcernaboutthestateoftheworld.Oneofthethingsthatwe’reallseeingisthatalotofworkthathasbeendonetoenableinternationalcooperationindealingwithvariousproblemssinceWorldWarIIisbeingpulledapart.We’reseeingtheprogresswethoughthadbeenmadeinthiscountryinracerelationsbeingreversed.We’reseeingthepartialbreakup—wedon’tknowhowfarthatwillgo—ofaunitedEurope.We’removingourselvesbackseveralcenturiesintermsofthinkingaboutwhatitistobehuman,whatitistosharethesameplanet,howwe’regoingtointeractandcommunicatewitheachother.We’regoingtobestartingfromscratchprettysoon.
Twoorthreeyearsago,IstartedgettinginvitedtodothingswiththeAmericanSocietyforCybernetics.IkeptsayingthatIhadn'tdoneanythingorthoughtaboutthatforyears,buttheypersisted.IwasinvitedtowriteachapterforahugehandbookcalledTheHandbookofHumanComputation.Basically,whattheymeantbyhumancomputationishuman-computercollaborationofvarioussorts.ItoldthemIdidn'tknowanythingaboutthat,andtheysaid,"Sinceyoudon’thavetimetowriteachapter,pleasewritethepreface."IaskedhowIwoulddothatifIcouldn'twriteachapter,andtheysaid,"We'llsendyoualltheabstracts."Ibecamequitecrankyandtoldthemtogetsomeoneelsetodoit,buttheykeptsendingmethingstoread.First,IsearchedGoogleforwhathumancomputationwas,andIfoundthatIdidknowaboutsomecornersofthefield.So,IwroteeverythingIknewabouthumancomputation,sentitin,andsaidtothem,"See,Idon’tknowanythingaboutit."Theypublishedit.
ThenIwenttotheconferenceandIstartedtogetbackintheconversationwithpeopleworkingonAI.IhadrealizedthatI’dlearnedanawfullotasquiteayoungperson,evenasachild,frommyparents,whowereinvolvedintheCyberneticMacyConferencesrightthroughthe‘50s.Theyandotherfiguresthatwereinvolved,likeWarrenMcCullochormanyotherpeople,weredriftingthroughthehouseandhavingconversationsallthetime,andIwaslistening.
Ididn’tgostraighttoAI;Iwasnibblingatedgesofit.Ihadrealizedthatourcapacitytothinkaboutcomplexinteractivesystemsseemedtobefallingapart,thatagreatmanyeffortstowardsinternationalcooperationwerefallingapart;statesthatinvolvedmultipleethnicsystemsordialectswerebreakingup;and,indeed,societiesliketheUnitedStates,withmanyethnicgroupsandracialgroups,werehavingaprogressivelyhardertimetryingtocooperate.
Weallthinkwithmetaphorsofvarioussorts,andweusemetaphorstodealwithcomplexity,butthewayhumanbeingsusecomputersandAIdependsontheirbasicepistemologies—whetherthey’reaccustomedtothinkinginsystemicterms,whetherthey’remainlyinterestedinquantitativeissues,whetherthey’reusedtousinggamesofvarioussorts.AgreatdealofwhatpeopleuseAIforistosimulatesomepatternoutsideintheworld.Ontheotherhand,peopleuseonepatternintheworldasametaphorforanotheroneallthetime.
Americansareinclinedtotalkaboutthe"waragainstdrugs,"orthe"waragainstpoverty,"orthe"waragainstcancer,"withoutquestioningwhether"war"isanappropriatemetaphor.It’sawayoftalkingaboutcomplexity,butifitdoesn’tfit,itwillcauseyoutomakeerrorsinhowyoudealwithyourproblems.Thewaronpovertyfailedpartlybecausepovertyisnotsomethingyoucandefeat,andthatmakeswarfareaninappropriatemetaphor.Thesameistruewiththewarondrugs,whichhasgottenusintosomeuglysituations.
Oneoftheproblemswhenyoubringtechnologyintoanewareaisthatitforcesyoutooversimplify.Thatis,thepossibilitiesofAIhavebeentherefromtheverybeginningofthinkingaboutcomputers,butthere'salwaysthisfeelingofdisappointmentthattherearelimitationstowhatyoucando.Wekeepattemptingtodomorecomplexthings.
Untilfairlyrecently,computerscouldnotbesaidtolearn.Tocreateamachinethatlearnstothinkmoreefficientlywasabigchallenge.Inthesamesense,oneofthethingsthatIwonderaboutishowwe'llbeabletoteachamachinetoknowwhatitdoesn’tknowbutthatitmightneedtoknowinordertoaddressaparticularissueproductivelyandinsightfully.Thisisahugeproblemforhumanbeings.Ittakesawhileforustolearntosolveproblems.Andthenittakesevenlongerforustorealizewhatwedon’tknowallthatwewouldneedtoknowtosolveaparticularproblem,whichobviouslyinvolvesalotofcomplexity.
HowdoyoudealwithignoranceIdon’tmeanhowdoyoushutignoranceout.Rather,howdoyoudealwithanawarenessofwhatyoudon’tknow,andyoudon’tknowhowtoknow,indealingwithaparticularproblemWhenGregoryBatesonwasarguingabouthumanpurposes,thatwaswherehegotinvolvedinenvironmentalism.Weweredoingallsortsofthingstotheplanetweliveonwithoutrecognizingwhatthesideeffectswouldbeandtheinteractions.Although,atthatpointwewerethinkingmoreaboutsideeffectsthanaboutinteractionsbetweenmultipleprocesses.Onceyoubegintounderstandthenatureofsideeffects,youaskadifferentsetofquestionsbeforeyoumakedecisionsandprojectionsandanalyzewhat’sgoingtohappen.
Thesamethingistrue,forinstance,withdrugtesting.Thefirstquestionpeopleaskis,"Doesthedrugwork"Butthenextquestionshouldbe,"Whatelsedoesthedrugdobesidesdealingwiththepathology"Acertainnumberofdrugsgetpulledoffthemarketeveryyearwhenpeoplerealizethatthelong-sideeffectsmaybemoreseriousthanwhatthey’retryingtocorrect.
Whattheanalogtothatinthecomputerworldis,Idon’tknow.Whatwedoistrytosetupprocessesforproblemsolvingandsupplythedataforanalysis,butwedon’tgivethemachineawayofsaying,"WhatelseshouldIknowbeforeIlookatthisquestion"Therehasbeensomuchexcitementandsenseofdiscoveryaroundthedigitalrevolutionthatwe’reatamomentwhereweoverestimatewhatcanbedonewithAI,certainlyasitstandsatthemoment.
Oneofthemostessentialelementsofhumanwisdomatitsbestishumility,knowingthatyoudon’tknoweverything.There’sasenseinwhichwehaven’tlearnedhowtobuildhumilityintoourinteractionswithourdevices.Thecomputerdoesn’tknowwhatitdoesn’tknow,andit'swillingtomakeprojectionswhenithasn’tbeenprovidedwitheverythingthatwouldberelevanttothoseprojections.HowdowegetthereIdon’tknow.It’simportanttobeawareofit,torealizethattherearelimitstowhatwecandowithAI.It’sgreatforcomputationandarithmetic,anditsaveshugeamountsoflabor.Itseemstomethatitlackshumility,lacksimagination,andlackshumor.Itdoesn’tmeanyoucan’tbringthosethingsintoyourinteractionswithyourdevices,particularly,incommunicatingwithotherhumanbeings.Butitdoesmeanthatelementsofintelligenceandwisdom—Ilikethewordwisdom,becauseit'smoremulti-dimensional—aregoingtobelacking.
Asachild,Ihadtheearlyconversationsofthecyberneticrevolutiongoingonaroundme.Icanlookatexamplesandrealizethatwhenoneofmyparentswastryingtoteachmesomething,itwasdirectlyconnectedwithwhattheyweredoingandthinkingaboutinthecontextofcybernetics.
Oneofmyfavoritememoriesofmychildhoodwasmyfatherhelpingmesetupanaquarium.Inretrospect,Iunderstandthathewasteachingmetothinkaboutacommunityoforganismsandtheirinteractions,interdependence,andtheissueofkeepingtheminbalancesothatitwouldbeahealthycommunity.Thatwasjustatthebeginningofourlookingatthenaturalworldintermsofecologyandbalance.Ratherthanitemizingwhatwasthere,Iwaslearningtolookattherelationshipsandnotjustseparatethings.
Blesshisheart,hedidn’ttellmehewasteachingmeaboutcybernetics.IthinkIwouldhavewalkedoutonhim.Anotherwaytosayitisthathewasteachingmetothinkaboutsystems.Gregorycoinedtheterm"schismogenesis"in1936,fromobservingthecultureofaNewGuineatribe,theIatmul,inwhichtherewasalotofwhathecalledschismogenesis.Schismogenesisisnowcalled"positivefeedback";it’swhathappensinanarmsrace.Youhaveapointoffriction,whereyoufeelthreatenedby,say,anothernation.So,yougetafewmoretanks.Theylookatthatandsay,"They’rearmingagainstus,"andtheygetalotmoretanks.Thenyougetmoretanks.Andtheygetmoretanksorairplanesorbombs,orwhateveritis.That’spositivefeedback.
Thealternativewouldbeifyousawthemgettingtankstosay,"I’dbettergetridofmytanks.Let’scoolthearmsrace,insteadofmutuallyescalating."Gregorywastalkingaboutthatanddidn’treallyhaveatermforit,soheinventedthetermschismogenesis.Genesistomeanbringingintobeinggreaterandgreaterschisms,conflicts.Thatwasbeforetheconceptofpositivefeedbackhadbeencoined.That’swhathewastalkingabout,thekindoffeedbackthatacceleratesaprocessratherthancontrolsit,whichisaveryimportantconcept.
IwouldsaythatthegreatmajorityofAmericansstillbelievethat"positivefeedback"iswhensomeonepatsyouonthebackandsaysyoudidagoodjob.Whatpositivefeedbackissayingis,domoreofthesame.So,ifwhatyou’redoingistakingheroinorquarrelingwithyourneighbor,thisisjustgoingtoleadtotrouble.Negativefeedbackcorrectswhatyou’redoing.It’snotsomebodysaying,"Thatwasalousyspeech."It’ssomebodysaying,"Reversecourse.Stopbuildingmorebombs.Stoptakinginmorealcoholfaster.Slowdown."Negativefeedbackiscorrectivefeedback.
GregorythenwroteapaperaboutanarmsraceandmadethemovefromthinkingabouttheNewGuineatribetothenatureofarmsracesinthemodernworld,whichwestillhaveplentyof.
Atthebeginningofthewar,myparents,MargaretMeadandGregoryBateson,hadveryrecentlymetandmarried.TheymetLawrenceK.Frank,whowasanexecutiveoftheMacyFoundation.Asaresultofthat,bothofthemwereinvolvedintheMacyConferencesonCybernetics,whichcontinuedthenfortwentyyears.Theystillquotemymotherconstantlyintalkingaboutsecond-ordercybernetics:thecyberneticsofcybernetics.TheyrefertoGregoryaswell,thoughhewasmoreinterestedincyberneticsasabstractanalyticaltechniques.Mymotherwasmoreinterestedinhowwecouldapplythistohumanrelations.
Myparentslookedatthecyberneticsconferencesratherdifferently.Mymother,whoinitiallyposedtheconceptofthecyberneticsofcybernetics,second-ordercybernetics,cameoutoftheanthropologicalapproachtoparticipantobservation:HowcanyoudosomethingandobserveyourselfdoingitShewassaying,"Okay,you’reinventingascienceofcybernetics,butareyoulookingatyourprocessofinventingit,yourprocessofpublishing,andexplaining,andinterpreting"OneoftheproblemsintheUnitedStateshasbeenthatpiecesofcyberneticshaveexplodedintotremendouseconomicactivityinallofcomputerscience,butmuchofthesystemstheorysideofcyberneticshasbeensortofastepchild.Ifirmlybelievethatitisthesystemsthinkingthatiscritical.
Atthepointwhereshesaid,"Youguysneedtolookatwhatyou’redoing.Whatisthecyberneticsofcybernetics"whatshewassayingwas,"Stopandlookatyourownprocessandunderstandit."Eventually,Isupposeyoudorunintotheinfiniterecursionproblem,butIguessyougetusedtothat.
HowdoyouknowthatyouknowwhatyouknowWhenIthinkabouttheexcitementofthoseearlyyearsofthecyberneticconferences,therehavebeenseverallosses.Oneisthattheexplosionofdevicesandmanufacturingandthehugeeconomiceffectofcomputertechnologyhasovershadowedtheepistemologicalcuriosityonwhichitwasbuilt,ofhowweknowwhatweknow,andhowthataffectsdecisionmaking.
Ifyouusetheword"cyber"inoursocietynow,peoplethinkthatitmeansadevice.Itdoesnotevokethewholemysteryofwhatmaintainsbalance,orhowasystemiskeptfromgoingoffkilter,whichwasthekindofthingthatmotivatedthequestioninthefirstplace.It’sprobablynotthefirsttimethat’shappened,thatatechnologywithaverywidespectrumofuseshasbeensoeffectiveforcertainproblemsthatit’sobscuredtheotherpossibleuses.
Peoplearenotusingcyberneticmodelsasmuchastheyshouldbe.Inthinkingaboutmedicine,forinstance,wearethinkingmorethanweusedtoaboutwhathappenswhenfiftyyearsagoyouhadchickenpoxandnowyouhaveshingles.WhathappenedHowdidthevirussurviveItwentintohiding.Ittookadifferentform.We’refindingexamplesofproblemsthatwethoughtwe’dsolvedbutmayhavemadeworse.
Wehavetallersmokestacksonfactoriesnow,tryingtopreventsmogandacidrain.Whatwe’regettingisthatthefumesaretravelingfurther,higherup,andstillcomingdownintheformofacidrain.Let’slookatthat.Someonehastriedtosolveaproblem,whichtheydid—theyreducedsmog.Butwestillputsmokeupthechimneyandthinkitdisappears.Itisn'tgone.It’sgonesomewhere.Weneedtolookattheentiresystem.WhathappenstothesmokeWhathappenstothewash-offoffertilizerintobrooksandstreamsInthatsense,we’reusingthetechnologytocorrectaproblemwithoutunderstandingtheepistemologyoftheproblem.Theproblemisconnectedtoalargersystem,andit’snotsolvedbythequickfix.
Ifyoulookbackatthecyberneticsconferences,you’dfindalotofexamplesthatcouldbeappliedtosocialandhumanproblemsthathavenotbeen.Mostpeopledon’tlearnaboutcybernetics.Theybuydevices.Cybernetics,becauseitdevelopedawholebranchofcommunicationtheory,isawayofthinking,notanindustry.Inourrelationswithothernations,forinstance,wegetcaughtinschismogenesis—armsraces,competitions,escalationsofvarioussorts—withoutpeoplebeingawarethatthat’swhat’shappening,withoutthemthinkingthroughwhatneedstobeattendedtoinordertosolveaproblem.
Wethinkthatwecansolvedrugaddictionbypunitivepoliceenforcement.Doesn’twork.Infact,itmakesmorejobsforpolicemenandprisonguards.Wearenotusingsystemstheorytothinkaboutsocialproblemsmostofthetime.Businessproblems,yes.Therearespecialists.Businessschoolseventeachsystemstheory.Butwe’renotraisingourchildrentobesystemsthinkers.That’swhatweneedtodo.
Youdon’thavetoknowalotoftechnicalterminologytobeasystemsthinker.OneofthethingsthatI’vebeenrealizinglately,andthatIfindfascinatingasananthropologist,isthatifyoulookatbeliefsystemsandreligionsgoingwaybackinhistory,aroundtheworld,veryoftenwhatyourealizeisthatpeoplehaveintuitivelyunderstoodsystemsandusedmetaphorstothinkaboutthem.TheexamplethatgrabbedmewasthinkingaboutthepantheonofGreekgods—ZeusandHera,ApolloandDemeter,andallofthem.Isuddenlyrealizedthatinthemythologythey’remarried,theyhavechildren,thesunandthemoonarebrotherandsister.Therearequarrelsamongthegods,andmarriages,divorces,andsoon.SoyoucanusetheGreekpantheon,becauseitisbasedonkinship,totakeadvantageofwhatpeoplehavelearnedfromtheirobservationoftheirfriendsandrelatives.
ItturnsoutthattheGreekreligioussystemisawayoftranslatingwhatyouknowaboutyoursisters,andyourcousins,andyourauntsintoknowledgeaboutwhat’shappeningtotheweather,theclimate,thecrops,andinternationalrelations,allsortsofthings.Ametaphorisalwaysaframeworkforthinking,usingknowledgeofthistothinkaboutthat.Religionisanadaptivetool,amongotherthings.Itisaformofanalogicthinking.
TheotherthingthatIliketotalkaboutisthatwecarryananalogmachinearoundwithusallthetimecalledourbody.It’sgotallthesedifferentorgansthatinteract;they’reinterdependent.Ifoneofthemgoesoutofkilter,theothersgooutofkilter,eventually.Thisistrueinsociety.Thisishowdis-easespreadsthroughacommunity,becauseeverythingisconnected.
Thereareacoupleofotherthingsthatareverystriking.IfyoulookattheOldTestament,theHebrewScriptures,whatyousee—whichyoucanalsoseeinyoungchildren—isthattheystartfromthedifferencesbetweenthings.Mommy’snotthesameasDaddy.Daddy’snotthesameasbrother.Icanremembermydaughterlearningtheword"Goggy,"whichobviouslywas"Doggy."Butthenshesaidthatthecowisa"Goggy,"becauseithadfourlegs,Iguess.Butthenyouhavetolearntodistinguishthecowfromthedog.Whenwethinkaboutachilddeveloping,youhavetolearntodistinguishbetweenthings—thisisthisandthatisthat.StartingwiththeBookofGenesis,eachthingiscreatedseparately.Theydon’tevolvebydifferentiation.Godseparatesthedayfromthenight,thelightfromthedark,thedrylandfromthewater.Andthenyouendupwithalargenumberofrulesofthingsthathavetobekeptseparate.Youcan’tweavetwodifferentkindsoffibersintothesamefabric.Youcan’tplowwithanoxandanass,butmustusetwooxen.
Whatyouhaveisthisprocessofdifferentiation,whichisintellectuallyprofoundbutonlyabeginning.Taxonomyisanessentialbasisforallweknowaboutthenaturalworld.Wehavelearnedtoclassify.Abeeisnotabutterfly.Youcanseethatstageinmanyformsofreligionandmythology.Andtheninsomelaterforms,theswitchisfrommakingdistinctionstorecognizingrelationships.
WhatcomesalongifyoulookattheNewTestamentisJesuskeepsviolatingalltherulesaboutkeepingthingsseparate,whichmakespeopleangry,becausethat’swhatthey’vebeentaught.He’sconstantlyposingthequestion,"What’stheconnection"Andnot,"What’sthedifference"Youcanseethatthisconstantnecessityofrecognizingthatthingsareseparateanddifferentandcanbeusedindifferentways,andthenseeingthateverythingisconnected,andhowit’sconnectedandinterdependent,thatthisisasortofpermanentbalanceinhumanintellect.Ifyoulookatthehistoryofmythology,youcanseepeoplemovingslowlyforward.Youcanlookatthehistoryofscience—thingsthatwereonceequatedwenowseeasseparate.Wecanonlygosofarinbreakingdownmoreandmoreelementaryparticles,butwe'restillfindingparticles.We’restillinterestedintheseparationofthings,butwe’realsostilldiscoveringrelationships.
I’vebecomeverymuchinvolvedinissuesaroundclimatechange.Climatechangecomesfromproceedingononepathwithoutrecognizinghowthatwillaffectotheraspectsofourreality.Takeitanotherstep,oneofthethingsthat’shardtogetacrosstopeopleisthatwhenhumanbeingsareuncomfortable,theyfight,ormove.Atthispointwehavearefugeecrisis,migrations,peopleleavingareaswheretheirwaysofmakingalivingdon’tworkanylongerbecauseofclimatechange.Wealsohaveconflicthappeningasonecountrywantstocontrolmorearableland—Lebensraum.So,peoplearefightingaboutland,oraboutfishingrights.
Mostpeopledon’trealizeit,butamythhasbeenputtogetherabouttheso-calledArabSpringofafewyearsago,wheremanyAmericanssaid,"Oh,good,they’rerebellingagainsttheirauthoritariangovernmentsandthey’regoingtobecomedemocratic."Well,theydidn’t.ThecauseoftheArabSpringwasafive-yeardrought,withalotofpeoplehavingdifficultyfeedingtheirfamilies,sotheymigratedfromthevillagestothecities,lookingforjobswheretheywouldbepaidmoneyandcouldbuyfoodfortheirfamilies.Buttherewerenojobsinthecities,sotheyhadrevolutions.