收藏 | 深度學(xué)習(xí)到深度推理
↑↑↑點(diǎn)擊上方藍(lán)字，回復(fù)資料，10個(gè)G的驚喜
僅做學(xué)術(shù)分享，如有侵權(quán)，聯(lián)系刪除 轉(zhuǎn)載于 ：專知
【導(dǎo)讀】ACM SIGKDD（國際數(shù)據(jù)挖掘與知識(shí)發(fā)現(xiàn)大會(huì)，簡(jiǎn)稱 KDD）是世界數(shù)據(jù)挖掘領(lǐng)域的最高級(jí)別的學(xué)術(shù)會(huì)議，由 ACM 的數(shù)據(jù)挖掘及知識(shí)發(fā)現(xiàn)專委會(huì)（SIGKDD）主辦，被中國計(jì)算機(jī)協(xié)會(huì)推薦為 A 類會(huì)議。自 1995 年以來，KDD 已經(jīng)連續(xù)舉辦了26屆，今年將于2021年8月14日至18日舉辦，今年的會(huì)議主辦地在新加坡。
來自deakin大學(xué)的研究人員在KDD2021上將給出關(guān)于深度學(xué)習(xí)推理的教程，非常值得關(guān)注!
大數(shù)據(jù)和大計(jì)算的興起給數(shù)字生活的許多領(lǐng)域帶來了現(xiàn)代神經(jīng)網(wǎng)絡(luò)，這要?dú)w功于構(gòu)建與現(xiàn)實(shí)世界相適應(yīng)的大型模型相對(duì)容易。Transformers的成功和對(duì)海量數(shù)據(jù)的自監(jiān)督預(yù)訓(xùn)練讓一些人相信，只要我們有數(shù)據(jù)和計(jì)算資源，深度神經(jīng)網(wǎng)絡(luò)幾乎可以做任何事情。然而，情況可能并非如此。雖然神經(jīng)網(wǎng)絡(luò)可以快速地利用表面統(tǒng)計(jì)，但它們?cè)诜夯叫碌娜蝿?wù)上卻失敗得很糟糕。目前的神經(jīng)網(wǎng)絡(luò)并不執(zhí)行刻意推理——即從上下文數(shù)據(jù)中有意地推導(dǎo)出新知識(shí)的能力。本教程回顧了最近的發(fā)展，將神經(jīng)網(wǎng)絡(luò)的能力擴(kuò)展到從數(shù)據(jù)“學(xué)習(xí)推理”，其中的任務(wù)是確定數(shù)據(jù)是否包含一個(gè)結(jié)論。這種能力開辟了新的途徑，通過使用自然語言進(jìn)行任意查詢，從數(shù)據(jù)中生成見解，而不需要預(yù)先定義一組狹義的任務(wù)。
本教程由三個(gè)主要部分組成。A部分涵蓋了學(xué)習(xí)-推理框架，解釋了神經(jīng)網(wǎng)絡(luò)如何通過綁定、注意力和動(dòng)態(tài)計(jì)算圖等自然操作作為推理的強(qiáng)大支柱。我們還將展示神經(jīng)網(wǎng)絡(luò)如何學(xué)習(xí)執(zhí)行組合算法。第二部分將更詳細(xì)地介紹神經(jīng)網(wǎng)絡(luò)如何在非結(jié)構(gòu)化和結(jié)構(gòu)化數(shù)據(jù)上進(jìn)行推理，以及跨多種模態(tài)。將解釋集合、關(guān)系、圖和時(shí)間的推理。C部分回顧了更高級(jí)的主題，包括帶有外部記憶的神經(jīng)網(wǎng)絡(luò)，學(xué)習(xí)用有限的標(biāo)簽進(jìn)行推理，以及用心智理論進(jìn)行遞歸推理。我們將特別關(guān)注神經(jīng)記憶作為支持實(shí)體、關(guān)系甚至神經(jīng)程序推理的基本機(jī)制。如有可能，將提供文本理解和視覺問答方面的個(gè)案研究。
https://truyentran.github.io/kdd2021-tute.html
目錄:
Part A: 推理學(xué)習(xí)框架 Learning to reason framework (60 mins)
Reasoning as a prediction skill that can be learnt from data.
Question answering as zero-shot learning.
Neural network operations for learning to reason:
Concept-object binding.
Attention & transformers.
Dynamic neural networks, conditional computation & differentiable programming.
Reasoning as iterative representation refinement & query-driven program synthesis and execution.
Compositional attention networks.
Neural module networks.
Combinatorics reasoning.
Part B: 結(jié)構(gòu)化與非結(jié)構(gòu)化數(shù)據(jù)推理 Reasoning over unstructured and structured data (60 mins)
Cross-modality reasoning, the case of vision-language integration.
Reasoning as set-set interaction.
Query processing.
Context processing.
Dual-attention.
Conditional set functions.
Relational reasoning
Query-conditioned dynamic graph constructions
Reasoning over knowledge graphs.
Graph embedding.
Graph convolutional networks.
Graph attention.
Message passing.
Relation networks
Graph neural networks
Temporal reasoning
Video question answering.
Part C: 高級(jí)主題 Advanced topics (60 mins)
Reasoning with external memories
Memory of entities – memory-augmented neural networks
Memory of relations with tensors and graphs
Memory of programs & neural program construction.
Learning to reason with less labels:
Data augmentation with analogical and counterfactual examples
Question generation
Self-supervised learning for question answering
Learning with external knowledge graphs
Recursive reasoning with neural theory of mind.
講者
參考文獻(xiàn)
Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. “Neural module networks”. In CVPR, pages 39–48, 2016.
Dzmitry Bahdanau, Shikhar Murty, Michael Noukhovitch, Thien Huu Nguyen, Harm de Vries, and Aaron Courville. “Systematic generalization: what is required and can it be learned?”, In ICLR, 2019.
Bottou, Léon. “From machine learning to machine reasoning”. Machine learning 94.2 (2014): 133-149.
---------?---------
推薦閱讀
（點(diǎn)擊標(biāo)題可跳轉(zhuǎn)閱讀）
老鐵，三連支持一下，好嗎？↓↓