Machinelearningisthescienceofgettingcomputerstoactwithoutbeingexplicitlyprogrammed.Inthepastdecade,machinelearninghasgivenusself-drivingcars,practicalspeechrecognition,effectivewebsearch,andavastlyimprovedunderstandingofthehumangenome.Machinelearningissopervasivetodaythatyouprobablyuseitdozensoftimesadaywithoutknowingit.Manyresearchersalsothinkitisthebestwaytomakeprogresstowardshuman-levelAI.Inthisclass,youwilllearnaboutthemosteffectivemachinelearningtechniques,andgainpracticeimplementingthemandgettingthemtoworkforyourself.Moreimportantly,you'lllearnaboutnotonlythetheoreticalunderpinningsoflearning,butalsogainthepracticalknow-howneededtoquicklyandpowerfullyapplythesetechniquestonewproblems.Finally,you'lllearnaboutsomeofSiliconValley'sbestpracticesininnovationasitpertainstomachinelearningandAI.
Thiscourseprovidesabroadintroductiontomachinelearning,datamining,andstatisticalpatternrecognition.Topicsinclude:(i)Supervisedlearning(parametric/non-parametricalgorithms,supportvectormachines,kernels,neuralnetworks).(ii)Unsupervisedlearning(clustering,dimensionalityreduction,recommendersystems,deeplearning).(iii)Bestpracticesinmachinelearning(bias/variancetheory;innovationprocessinmachinelearningandAI).Thecoursewillalsodrawfromnumerouscasestudiesandapplications,sothatyou'llalsolearnhowtoapplylearningalgorithmstobuildingsmartrobots(perception,control),textunderstanding(websearch,anti-spam),computervision,medicalinformatics,audio,databasemining,andotherareas.
WelcometoMachineLearning!Inthismodule,weintroducethecoreideaofteachingacomputertolearnconceptsusingdata—withoutbeingexplicitlyprogrammed.TheCourseWikiisunderconstruction.Pleasevisittheresourcestabforthemostcompleteandup-to-dateinformation.
5videos,4readings
WhatisMachineLearningTwodefinitionsofMachineLearningareoffered.ArthurSamueldescribeditas:"thefieldofstudythatgivescomputerstheabilitytolearnwithoutbeingexplicitlyprogrammed."Thisisanolder,informaldefinition.
TomMitchellprovidesamoremoderndefinition:"AcomputerprogramissaidtolearnfromexperienceEwithrespecttosomeclassoftasksTandperformancemeasureP,ifitsperformanceattasksinT,asmeasuredbyP,improveswithexperienceE."
Example:playingcheckers.
E=theexperienceofplayingmanygamesofcheckers
T=thetaskofplayingcheckers.
P=theprobabilitythattheprogramwillwinthenextgame.
Ingeneral,anymachinelearningproblemcanbeassignedtooneoftwobroadclassifications:
SupervisedlearningandUnsupervisedlearning.
SupervisedLearningInsupervisedlearning,wearegivenadatasetandalreadyknowwhatourcorrectoutputshouldlooklike,havingtheideathatthereisarelationshipbetweentheinputandtheoutput.
Supervisedlearningproblemsarecategorizedinto"regression"and"classification"problems.Inaregressionproblem,wearetryingtopredictresultswithinacontinuousoutput,meaningthatwearetryingtomapinputvariablestosomecontinuousfunction.Inaclassificationproblem,weareinsteadtryingtopredictresultsinadiscreteoutput.Inotherwords,wearetryingtomapinputvariablesintodiscretecategories.
Example1:
Givendataaboutthesizeofhousesontherealestatemarket,trytopredicttheirprice.Priceasafunctionofsizeisacontinuousoutput,sothisisaregressionproblem.
Wecouldturnthisexampleintoaclassificationproblembyinsteadmakingouroutputaboutwhetherthehouse"sellsformoreorlessthantheaskingprice."Hereweareclassifyingthehousesbasedonpriceintotwodiscretecategories.
Example2:
(a)Regression-Givenapictureofaperson,wehavetopredicttheirageonthebasisofthegivenpicture
(b)Classification-Givenapatientwithatumor,wehavetopredictwhetherthetumorismalignantorbenign.
UnsupervisedLearningUnsupervisedlearningallowsustoapproachproblemswithlittleornoideawhatourresultsshouldlooklike.Wecanderivestructurefromdatawherewedon'tnecessarilyknowtheeffectofthevariables.
Wecanderivethisstructurebyclusteringthedatabasedonrelationshipsamongthevariablesinthedata.
Withunsupervisedlearningthereisnofeedbackbasedonthepredictionresults.
Example:
Clustering:Takeacollectionof1,000,000differentgenes,andfindawaytoautomaticallygroupthesegenesintogroupsthataresomehowsimilarorrelatedbydifferentvariables,suchaslifespan,location,roles,andsoon.
Non-clustering:The"CocktailPartyAlgorithm",allowsyoutofindstructureinachaoticenvironment.(i.e.identifyingindividualvoicesandmusicfromameshofsoundsatacocktailparty).
Thisoptionalmoduleprovidesarefresheronlinearalgebraconcepts.Basicunderstandingoflinearalgebraisnecessaryfortherestofthecourse,especiallyaswebegintocovermodelswithmultiplevariables.
6videos,7readings
MatricesandVectorsMatricesare2-dimensionalarrays:
Theabovematrixhasfourrowsandthreecolumns,soitisa4x3matrix.
Avectorisamatrixwithonecolumnandmanyrows:
Sovectorsareasubsetofmatrices.Theabovevectorisa4x1matrix.
Notationandterms:
RunthecellbelowtogetfamiliarwiththecommandsinOctave/Matlab.Feelfreetocreatematricesandvectorsandtryoutdifferentthings.
AdditionandScalarMultiplicationAdditionandsubtractionareelement-wise,soyousimplyaddorsubtracteachcorrespondingelement:
SubtractingMatrices:
Toaddorsubtracttwomatrices,theirdimensionsmustbethesame.
Inscalarmultiplication,wesimplymultiplyeveryelementbythescalarvalue:
Inscalardivision,wesimplydivideeveryelementbythescalarvalue:
ExperimentbelowwiththeOctave/Matlabcommandsformatrixadditionandscalarmultiplication.Feelfreetotryoutdifferentcommands.Trytowriteoutyouranswersforeachcommandbeforerunningthecellbelow.
Matrix-VectorMultiplicationWemapthecolumnofthevectorontoeachrowofthematrix,multiplyingeachelementandsummingtheresult.
Theresultisavector.Thenumberofcolumnsofthematrixmustequalthenumberofrowsofthevector.
Anmxnmatrixmultipliedbyannx1vectorresultsinanmx1vector.
Belowisanexampleofamatrix-vectormultiplication.Makesureyouunderstandhowthemultiplicationworks.Feelfreetotrydifferentmatrix-vectormultiplications.
Matrix-MatrixMultiplicationWemultiplytwomatricesbybreakingitintoseveralvectormultiplicationsandconcatenatingtheresult.
Anmxnmatrixmultipliedbyannxomatrixresultsinanmxomatrix.Intheaboveexample,a3x2matrixtimesa2x2matrixresultedina3x2matrix.
Tomultiplytwomatrices,thenumberofcolumnsofthefirstmatrixmustequalthenumberofrowsofthesecondmatrix.
Forexample:
MatrixMultiplicationPropertiesMatricesarenotcommutative:A\B≠B\AMatricesareassociative:(A\B)\C=A\(B\C)Theidentitymatrix,whenmultipliedbyanymatrixofthesamedimensions,resultsintheoriginalmatrix.It'sjustlikemultiplyingnumbersby1.Theidentitymatrixsimplyhas1'sonthediagonal(upperlefttolowerrightdiagonal)and0'selsewhere.
Whenmultiplyingtheidentitymatrixaftersomematrix(AI),thesquareidentitymatrix'sdimensionshouldmatchtheothermatrix'scolumns.Whenmultiplyingtheidentitymatrixbeforesomeothermatrix(IA),thesquareidentitymatrix'sdimensionshouldmatchtheothermatrix'srows.
InverseandTransposeTheinverseofamatrixAisdenoted\(A^{-1}\).Multiplyingbytheinverseresultsintheidentitymatrix.
Anonsquarematrixdoesnothaveaninversematrix.Wecancomputeinversesofmatricesinoctavewiththepinv(A)functionandinMatlabwiththeinv(A)function.Matricesthatdon'thaveaninversearesingularordegenerate.
Thetranspositionofamatrixislikerotatingthematrix90°inclockwisedirectionandthenreversingit.Wecancomputetranspositionofmatricesinmatlabwiththetranspose(A)functionorA':
Inotherwords:
%InitializematrixAA=[1,2,0;0,5,6;7,0,9]%TransposeAA_trans=A'%TaketheinverseofAA_inv=inv(A)%WhatisA^(-1)*AA_invA=inv(A)*A
Thiscourseincludesprogrammingassignmentsdesignedtohelpyouunderstandhowtoimplementthelearningalgorithmsinpractice.Tocompletetheprogrammingassignments,youwillneedtouseOctaveorMATLAB.ThismoduleintroducesOctave/Matlabandshowsyouhowtosubmitanassignment.