deeplearningGraphEmbedding学习笔记（3）：GraphConvolutionNetworks数据实验室

478 818

从Graph的视角看CNN，上图左右两个部分是等价的。左子图每个网格的通道，对应右子图每个节点的一个属性，左子图卷积核的参数相当于右子图边的权重。所谓3x3卷积就是右子图9个节点属性的加权平均（注意右子图中中间节点自己指向自己的边）。

CNN目前主流的应用场景都是规则的空间结构数据，比如图像是2Dgrids，语音和文本是1Dgrids。

在Graph上应用CNN，面临了较大的挑战：空间结构不规整，每个节点的邻接节点数量不同，如何将变长的空间结构转化为定长的空间结构。

目前，将变长转化为定长的思路主要有两个：一个方法比较直接，也比较暴力，就是对节点之间的关系进行排序，强行按照预设好的参数取出排序最前面的几个节点作为邻接节点，从而将Graph的不定长结构转化为定长结构，典型代表Patchy-SAN；另一个方法稍微优雅一些，也属于典型的不定长变定长的思路，本质是加权求和，典型代表是Kipf&Welling(ICLR2017)，目前GCN讨论的都是该作者提供的思路。

上图讲解了PATCHY-SAN方法的总体流程。第一步是nodesequenceselection，输入数据是最底层的Graph，将Graph节点按照度进行排序，取出前w个节点。第二步是NeighborhoodAssembly，即对于选出来的每个节点逐个构建邻域，方法很简单就是BFS，先搜索直接相邻的一级邻域，再搜索下一级邻域，直到找到超过k个邻居位置。第三步是NeighborhoodNormalization，因为BFS搜索出来的邻域有可能超过k个，也有可能不足k个，需要normalize为k个，具体方法是以选出来的每个节点为root，从邻域中找出前k个与root最接近的节点。

上图讲解了neighborhoodnormalization后怎么转化为标准的CNN问题进行求解。对于每个节点，normalizedneighborhoods可以理解为CNN中receptivefields，节点和边的属性attributes可以理解为CNN中的channel。扁平状的长方体表示节点attribute，正方体表示边的attribute（可以理解为边是通过邻接矩阵k*k的方式表示）

对于选出的每个节点，normalize后的数据维度是[wk,n]，使用M个一维的receptivefields为k，stride为k的卷积核，对n个通道的节点属性进行卷积，就可以得到上图所示的效果，输出数据维度是[w,M]。同样的，对于每条边，normalize后的数据维度是[wk^2,n]，使用M'个一维的receptivefields为k^2，stride为k^2的的卷积核，对m个通道的边属性进行卷积，输出数据维度是[w,M']。之后就可以使用任意的神经网络结构对Graph进行建模了。

问题定义：学习一个函数，输入数据是节点的属性和图的邻接矩阵，输出是一个NXF的特征矩阵，F是每个节点的输出特征数。

Forthesemodels,thegoalisthentolearnafunctionofsignals/featuresonagraphG=(V,E)whichtakesasinput:

andproducesanode-leveloutputZ(anN×Ffeaturematrix,whereFisthenumberofoutputfeaturespernode).Graph-leveloutputscanbemodeledbyintroducingsomeformofpoolingoperation

先看一个简单的函数定义：

上面这个式子有两个局限性，一是A只考虑了邻接节点，缺少自身节点的信息，可以用A+I来替换A，以保持与传统卷积定义一致。二是加权平均没有归一化处理，可以用节点的出入度来归一化，我们用度矩阵D表示。综上，作者做了以下调整，得到了图卷积操作：

看一下作者搭建的GCN结构。第一层，输出的每个节点只包含了一级邻接节点的信息；第二层，输出的每个节点包含了上一个隐层的一级邻接节点的信息，相当于输入层多级邻接节点的信息。层级越高，信息越宏观。

本人年少时在欧洲三国边境小城Aachen游学，瞻仰了两位机械泰斗的风采，然未继承任何技能，终日游手好闲四处转悠。归国后，有感于在机械行业难有建树，遂投身互联网，遇大牛周公，授我以Python和Hadoop大法，立足...

THE END

深学习综述讨论简介deepLearning课件.pptx

Neuralnetworksanddeeplearning

SpringBoot集成DeepLearning4j实现图片数字识别1.什么是DeepLearning4j？Deep

Halcon深度学习2标注工具DeepLearningTool下载安装

吴恩达deeplearning.ai五项课程完整笔记了解一下？

deeplearningGraphEmbedding学习笔记（3）：GraphConvolutionNetworks数据实验室

ClassifyTimeSeriesUsingWaveletAnalysisandDeepLearning

1.InstallationsupportforDeepLearningFrameworksfortheOnce you've installed the deep learning libraries, you can use the Deep Learning Tools to train geospatial deep learning models. You can also find out more about the capabilities of the arcgis.learn module which provides specialized access to many geospatial models beyond those directly available https://github.com/Esri/deep-learning-frameworks?tab=readme-ov-file
2.DeepLearning8元学习Few-Shot Learning和传统的监督学习有所不同,它的目标不是让计算机识别训练集里的图片,并且泛化到测试集,而是让计算机自己学会学习,我拿一个很大是数据集来训练神经网络,学习的目的不是让计算机知道什么是大象什么是老虎,不是让计算机识别没见过的大象和老虎,学习的目的是让计算机理解事物的异同,学会区分不同的事物,https://blog.csdn.net/zzqingyun/article/details/136940636
3.深度学习概述Learn 发现 产品文档 开发语言 主题 登录 搜索 ML.NET 概述 模型生成器和 CLI API 新增功能 教程 概念 ML.NET 任务 数据转换 算法 模型评估指标 提高模型准确性 操作指南 参考 资源 下载PDF 使用英语阅读 保存 添加到集合 添加到计划 通过 Facebookx.com 共享LinkedIn电子邮件 https://learn.microsoft.com/zh-cn/dotnet/machine-learning/deep-learning-overview
4.DeepLearning.AILearn and build diffusion models from the ground up, understanding each step. Learn about diffusion models in use today and implement algorithms to speed up sampling. Deep LearningDiffusion ModelsGenAI ApplicationsGenerative Models Building Systems with the ChatGPT API https://learn.deeplearning.ai/
5.DiveintoDeepLearning—DiveintoDeepLearning1.0.3Interactive deep learning book with code, math, and discussions Implemented with PyTorch, NumPy/MXNet, JAX, and TensorFlow Adopted at 500 universities from 70 countriesFollow @D2L_ai [Feb 2023] The book is forthcoming on Cambridge University Press (order). The Chinese version is the best https://www.d2l.ai/
6.DeepLearning[Book]Dive into machine learning concepts in general, as well as deep learning in particular Understand how deep networks evolved from neural network fundamentals Explore the major deep network architectures, including Convolutional and Recurrent Learn how to map specific deep networks to the right problem https://www.oreilly.com/library/view/deep-learning/9781491924570/
7.DeeplearningMathOnlineTomCircle1. Learn Everything in 《Deep Learning》: Math (eg. Gradient Descent – by French GrandMaster Cauchy 1847), Linear Algebra (eg. Matrix, Eigen-decomposition), Probability (eg. Bayesian, etc), Key Deep Learning techniques. Note: Available at Singapore National Library (LKC Reference #006.31).https://tomcircle.wordpress.com/tag/deeplearning/
8.DEEPLEARNING·DeepLearningINSTRUCTORSYann LeCun & Alfredo Canziani LECTURESWednesday 9:30 – 11:30, Zoom PRACTICATuesdays 9:30 – 10:30, Zoom FORUMr/NYU_DeepLearning DISCORDNYU DL MATERIAL2021 repo 2021 edition disclaimer Check the repo’sREADME.mdand learn about: https://cds.nyu.edu/deep-learning/
9.EclipseDeeplearning4jTheEclipseFoundationThe goal of theEclipse Deeplearning4jproject is simple: To enable artificial intelligence (AI) applications to run everywhere, for everyone. We’reThe media picks up on researchers’ ideas and the industry gets left behind. Most of the information people learn about AI through the media ishttps://www.eclipse.org/community/eclipse_newsletter/2019/december/2.php
10.综合DeepLearning的学习实践系列1基础介绍里面还有Hinton自己讲解Deep Belief Nets的视频：http://videolectures.net/jul09_hinton_deeplearn/ 这里https://developer.huawei.com/consumer/cn/forum/topic/0201113529329500003
11.RegressionandClassificationCourse(DeepLearning.AI)Learn the fundamentals of machine learning with Andrew Ng in this updated 3-course Specialization by DeepLearning.AI and Stanford Online. Build and train models using Python, NumPy, and scikit-learn for real-world AI applications. Ideal for beginners.https://www.coursera.org/learn/machine-learning
12.DeepLearningforNLPdeep learning is to explore how computers can take advantage of data to develop features and representations appropriate for complex interpretation learn structured tree outputs as well as vector representations for phrases and sentences. We cover both equations as well as applications. We showhttps://nlp.stanford.edu/courses/NAACL2013
13.deeplearningfromscratch斋藤康毅Python Machine Learning Machine Learning And Deep Learning From Scratch Illustrated With Python, Scikit-Learn, Keras, Theano And Tensorflow by Moubachir Madani Fadoul (z-lib.org).pdf 上传者:weixin_39397839时间:2020-06-22 Linux_From_Scratch.rar_From Scratch_linux_linux from scratch_scr https://www.iteye.com/resource/mahoon411-14055605
14.hands?Explore various possibilities with deep learning and gain amazing insights from data using Google's brainchild-- TensorFlow ?Want to learn what more can be done with deep learning? Explore various neural networks with the help of this comprehensive guide https://bbs.pinggu.org/jg/kaoyankaobo_kaoyan_5925473_1.html
15.CrazyStoneDeepLearningCrazyStone DeepLearning 关注 3 厂商 Unbalance Corporation 游戏介绍 上次更新于2023/06/30 策略 简介The strongest, the best Go app in the world based on Crazy Stone employing Deep Learning technology ! New Features!! -IAGA Rating Certification Tests Challenge the Dan/Kyu tests provided by the https://www.taptap.cn/app/68160
16.Deeplearning:Dropout,DropConnect我使用的是Matlab的Deeplearning 的工具包https://github.com/rasmusbergpalm/DeepLearnToolbox, 我只使用的是简单地单隐层的感知机,数据是MNIST手写数字识别,该数据一共有60000个训练样本和10000个测试样本。图片大小是28 * 28,网络结构的层数是[784 512 10],100次迭代,minibatch大小是100,我做了在没有dropouthttps://www.jianshu.com/p/b349c4c82da3
17.UsingdeeplearningsurrogatestoimprovenewproductInstead, researchers have proposed that deep learning models could be trained using real-world data from existing products in the field. That could transform the way companies improve their products, with design and optimization tools that learn automatically from the performance of previous product https://www.mckinsey.com/capabilities/operations/our-insights/deep-learning-in-product-design
18.吴恩达深度学习编程练习–AndrewNg’sDeepLearning仅仅是为记录学习吴恩达老师 Deep Learning 课程,会不断更新。 第门课《神经网络和深度学习》 第周练习题答案 1. Basics with Numpy (optional assignment) importmath import numpy as np ### Building basic functions with numpy ## 1.1 - sigmoid function, np.exp() # GRADED FUNCTION: basic_https://aiti.fun/475.html
19.专知继Pytorch教程后,我们推出面向Java程序员的深度学习教程DeepLearning4J。Deeplearning4j的案例和资料很少,官方的doc文件也非常简陋,基本上所有的类和函数的都没有解释。为此,我们推出来自中科院自动化所专知小组博士生Hujun与Sanglei创作的-分布式Java开源深度学习框架Deeplearning4j学习教程,第六篇,用卷积神经网络CNN进行https://cloud.tencent.com/developer/article/1089085
20.StartHerewithMachineLearningDeep Learning (Keras) Deep learning is a fascinating and powerful field. State-of-the-art results are coming from the field of deep learning and it is a sub-field of machine learning that cannot be ignored. Here’s how to get started with deep learning: http://machinelearningmastery.com/start-here/
21.DeeplearningforirregularlyandregularlymissingdataDeep learning (DL) is a powerful tool for mining features from data, which can theoretically avoid assumptions (e.g., linear events) constraining conventional interpolation methods. Motivated by this and inspired by image-to-image translation, we applied DL to irregularly and regularly missing datahttps://www.nature.com/articles/s41598-020-59801-x
22.[DeeplearningAI笔记]第三章2.72.8多任务学习/迁移学习[DeeplearningAI笔记]第三章2.7-2.8多任务学习/迁移学习 觉得有用的话,欢迎一起讨论相互学习~ 吴恩达老师课程原地址 2.7 迁移学习 Transfer Learninig 神经网络可以从一个任务中习得知识,并将这些知识应用到另一个独立的任务中.例如:你已经训练好一个能够识别猫的系统,你利用这些知识或者这些知识的部分去完成更好的https://www.cnblogs.com/cloud-ken/p/7794250.html
23.机器学习MachineLearning集智百科多线性子空间学习算法 Multilinear Subspace Learning Algorithms旨在直接从多维数据的张量表示中学习低维的表示,而不是将它们重塑为高维向量。[41]深度学习算法 Deep Learning Algorithms发现了多层次的表示,或者是一个特征层次结构,具有更高层次、更抽象的特征,这些特征定义为(或可以生成)低层次的特征。有人认为,一个https://wiki.swarma.org/index.php/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0_Machine_Learning