OutlineConceptionofdeeplearningDevelopmenthistoryDeeplearningframeworksDeepneuralnetworkarchitecturesConvolutionalneuralnetworks
IntroductionNetworkstructureTrainingtricksApplicationinAestheticImageEvaluationIdea
OutlineConceptionofdeeplear1DeepLearning(Hinton,2006)Deeplearningisabranchofmachinelearningbasedonasetofalgorithmsthatattempttomodelhighlevelabstractionsindata.Theadvantageofdeeplearningistoextractingfeaturesautomatically
Eachunitisbinary(0or1).
Everyvisibleunitconnectstoallthehiddenunits.
Everyhiddenunitconnectstoallthevisibleunits.
Therearenoconnectionsbetweenv-vandh-h.HintonGE.Deepbeliefnetworks[J].Scholarpedia,2009,4(6):5947.Fig1.RBM(restrictedBoltzmannmachine)structure.Fig2.DBN(deepbeliefnetwork)structure.IdeaComposedofmultiplelayersofRBM.Howtowetraintheseadditionallayers
UnsupervisedgreedyapproachDBN(DeepBeliefNetwork,2006)H6RNN(RecurrentNeuralNetwork,2013)WhatRNNaimstoprocessthesequencedata.RNNwillrememberthepreviousinformationandapplyittothecalculationofthecurrentoutput.Thatis,thenodesofthehiddenlayerareconnected,andtheinputofthehiddenlayerincludesnotonlytheoutputoftheinputlayerbutalsotheoutputofthehiddenlayer.MarhonSA,CameronCJF,KremerSC.RecurrentNeuralNetworks[M]//HandbookonNeuralInformationProcessing.SpringerBerlinHeidelberg,2013:29-65.ApplicationsMachineTranslationGeneratingImageDescriptionsSpeechRecognitionHowtotrain
avoids
thecomplexpre-processingofimage(etc.extracttheartificialfeatures),wecandirectlyinput
theoriginalimage.
Basiccomponents:ConvolutionLayers,PoolingLayers,FullyconnectedLayersConvolutionalNeuralNetworks(11ConvolutionlayerTheconvolutionkerneltranslates
ona2-dimensionalplane,andeachelementoftheconvolutionkernelismultiplied
bytheelementatthecorrespondingpositionoftheconvolutionimageandthensumalltheproduct.Bymovingtheconvolutionkernel,wehaveanewimage,whichconsistsofthesumoftheproductoftheconvolutionkernelateachposition.localreceptivefieldweightsharingReduced
thenumberofparametersConvolutionlayerTheconvoluti12PoolinglayerPoolinglayeraimstocompresstheinputfeaturemap,whichcanreducethenumberofparameters
intrainingprocessandthedegreeof
over-fitting
newfunctionalunitintegration19801998198920142015ImageNetILSVRC(ImageNetLargeScaleVisualRecognitionChallenge)20132014201520152014,2015201520122015BN(BatchNormalization)RPNCNNStructureEvolutionHinton16LeNet(LeCun,1998)LeNet
isaconvolutionalneuralnetworkdesignedbyYannLeCunforhandwrittennumeralrecognitionin1998.Itisoneofthemostrepresentativeexperimentalsystemsinearlyconvolutionalneuralnetworks.LeNetincludestheconvolutionlayer,poolinglayer
SimonyanK,ZissermanA.VeryDeepConvolutionalNetworksforLarge-ScaleImageRecognition[J].ComputerScience,2014.Why3*3filtersStackedconv.layershavealargereceptivefieldMorenon-linearityLessparameterstolearnVGG-Net(OxfordUniversity,201420Network-in-Network(NIN,ShuichengYan,2013)Networkstructure:4Mlpconvlayers+GlobalaveragepoolinglayerFig1.linearconvolutionMLPconvolutionFig2.fullyconnectedlayerglobalaveragepoolinglayerMinLinetal,NetworkinNetwork,Arxiv2013.Fig3.NINstructureLinearcombinationofmultiplefeaturemaps.Informationintegrationofcross-channel.ReducedtheparametersReducedthenetworkAvoidedover-fittingNetwork-in-Network(NIN,Shuich21GoogLeNet(InceptionV1,2014)Fig1.Inceptionmodule,naveversionProposedinceptionarchitectureandoptimizeditCanceled
SqueezeNet:AlexNet-levelaccuracywith50xfewerparametersand<0.5MBmodelsizeSqueezeNet
SqueezeNet:AlexNet30XceptionXception31R-CNN(2014)Regionproposals:SelectiveSearch
Resizetheregionproposal:Warpallregionproposalstotherequiredsize(227*227,
AlexNetInput)
ComputeCNNfeature:Extracta4096-dimensionalfeaturevectorfromeachregionproposalusingAlexNet.
GirshickR.Fastr-cnn[C]//ProceedingsoftheIEEEInternationalConferenceonComputerVision.2015:1440-1448.FastR-CNN(2015)AFastR-CNNn34FasterR-CNN(2015)FasterR-CNN=RPN+FastR-CNN
ARegionProposalNetwork(RPN)takesanimage(ofanysize)asinputandoutputsasetofrectangularobjectproposals,eachwithanobjectnessscore.
advantagesrectifiedSimplifiedcalculationAvoidedgradientdisappearedEnhancedthefunctionalityof39BatchNormalization(2015)Intheinputofeachlayerofthenetwork,insertanormalizedlayer.Foralayerwithd-dimensionalinputx=(x(1)...x(d)),wewillnormalizeeachdimension:IoffeS,SzegedyC.Batchnormalization:Acceleratingdeepnetworktrainingbyreducinginternalcovariateshift[J].arXivpreprintarXiv:1502.03167,2015.Internal
Covariate
Shift
BatchNormalization(2015)Inth40ApplicationinAestheticImageEvaluationDongZ,ShenX,LiH,etal.PhotoQualityAssessmentwithDCNNthatUnderstandsImageWell[M]//MultiMediaModeling.SpringerInternationalPublishing,2015:524-535.LuX,LinZ,JinH,etal.Ratingimageaestheticsusingdeeplearning[J].IEEETransactionsonMultimedia,2015,17(11):2021-2034.WangW,ZhaoM,WangL,etal.Amulti-scenedeeplearningmodelforimageaestheticevaluation[J].SignalProcessingImageCommunication,2016,47:511-518.ApplicationinAestheticImage41PhotoQualityAssessmentwithDCNNthatUnderstandsImageWellDCNN_Aesthtrainedwellnetworkatwo-classSVMclassifierDCNN_Aesth_SPoriginalimagessegmentedimagesspatialpyramidImageNetCUHKAVADongZ,ShenX,LiH,etal.PhotoQualityAssessmentwithDCNNthatUnderstandsImageWell[M]//MultiMediaModeling.SpringerInternationalPublishing,2015:524-535.PhotoQualityAssessmentwith42RatingimageaestheticsusingdeeplearningSupportheterogeneousinputs,i.e.,globaland
localviews.AllparametersinDCNNarejointlytrained.Fig1.GlobalviewsandlocalviewsofanimageFig3.DCNNarchitectureFig2.SCNNarchitecture
SCNNDCNN
Enablesthenetworktojudgeimageaestheticswhilesimultaneouslyconsideringboththeglobalandlocalviewsofanimage.LuX,LinZ,JinH,etal.Ratingimageaestheticsusingdeeplearning[J].IEEETransactionsonMultimedia,2015,17(11):2021-2034.Ratingimageaestheticsusing43Amulti-scenedeeplearningmodelforimageaestheticevaluationDesignasceneconvolutionallayerconsistofmulti-groupdescriptorsinthenetwork.Designapre-trainingproceduretoinitializeourmodel.Fig1.Thearchitectureofthemulti-scenedeeplearningmodel(MSDLM).Fig2.TheoverviewofproposedMSDLM.ArchitectureofMSDLM:4
convolutionallayers+1sceneconvolutionallayer+3fullyconnectedlayersWangW,ZhaoM,WangL,etal.Amulti-scenedeeplearningmodelforimageaestheticevaluation[J].SignalProcessingImageCommunication,2016,47:511-518.Amulti-scenedeeplearningmo44Example-Loadthedatasetdefload_dataset():url=filename=
if
print("DownloadingMNISTdataset...")
urlretrieve(url,filename)
withgzip.open(filename,'rb')asf:data=pickle.load(f)X_train,y_train=data[0]X_val,y_val=data[1]X_test,y_test=data[2]X_train=X_train.reshape((-1,1,28,28))X_val=X_val.reshape((-1,1,28,28))X_test=X_test.reshape((-1,1,28,28))y_train=y_train.astype(np.uint8)y_val=y_val.astype(np.uint8)y_test=y_test.astype(np.uint8)
returnX_train,y_train,X_val,y_val,X_test,y_test
X_train,y_train,X_val,y_val,X_test,y_test=load_dataset()plt.imshow(X_train[0][0],cmap=cm.binary)Example-Loadthedatasetdefl45Example–Modelnet1=NeuralNet(layers=[('input',layers.InputLayer),
('conv2d1',
layers.Conv2DLayer),
('maxpool1',
layers.MaxPool2DLayer),
('conv2d2',layers.Conv2DLayer),
('maxpool2',layers.MaxPool2DLayer),
('dropout1',layers.DropoutLayer),
('dense',layers.DenseLayer),
('dropout2',layers.DropoutLayer),
('output',layers.DenseLayer),
],
#inputlayerinput_shape=(None,1,28,28),#layerconv2d1conv2d1_num_filters=32,conv2d1_filter_size=(5,5),,
#layermaxpool1maxpool1_pool_size=(2,2),#layerconv2d2conv2d2_num_filters=32,conv2d2_filter_size=(5,5),,
#layermaxpool2maxpool2_pool_size=(2,2),
#dropout1dropout1_p=0.5,
#densei.e.full-connectedlayerdense_num_units=256,
#dropout2dropout2_p=0.5,
#outputoutput_num_units=10,
Basiccomponents:ConvolutionLayers,PoolingLayers,FullyconnectedLayersDataAugmentationConvolutional53Overfeat(2013)SermanetP,EigenD,ZhangX,etal.OverFeat:IntegratedRecognition,LocalizationandDetectionusingConvolutionalNetworks[J].EprintArxiv,2013.Overfeat(2013)SermanetP,Eige54InceptionV2(2015)IoffeS,SzegedyC.Batchnormalization:Acceleratingdeepnetworktrainingbyreducinginternalcovariateshift[J].arXivpreprintarXiv:1502.03167,2015.InceptionV2(2015)IoffeS,Sze55FractalNetFractalNet56FasterR-CNN(2015)FasterR-CNN=RPN+FastR-CNN
RenS,HeK,GirshickR,etal.Fasterr-cnn:Towardsreal-timeobjectdetectionwithregionproposalnetworks[C]//Advancesinneuralinformationprocessingsystems.2015:91-99.Figure1.FasterR-CNNisasingle,unifiednetworkforobjectdetection.Figure2.RegionProposalNetwork(RPN).FasterR-CNN(2015)FasterR-CNN57Example-Loadthedatasetdefload_dataset():url=filename=
X_train,y_train,X_val,y_val,X_test,y_test=load_dataset()plt.imshow(X_train[0][0],cmap=cm.binary)Example-Loadthedatasetdefl58OutlineConceptionofdeeplearningDevelopmenthistoryDeeplearningframeworksDeepneuralnetworkarchitecturesConvolutionalneuralnetworks
OutlineConceptionofdeeplear59DeepLearning(Hinton,2006)Deeplearningisabranchofmachinelearningbasedonasetofalgorithmsthatattempttomodelhighleve