自動機器學(xué)習(xí)：方法、系統(tǒng)與挑戰(zhàn)（AutoML）

目 錄

自動機器學(xué)習(xí)方法

第1章 超參優(yōu)化 ··································2

1.1 引言 ··············································2

1.2 問題定義 ·······································4

1.2.1 優(yōu)化替代方案：集成與邊緣化 ·············5

1.2.2 多目標優(yōu)化 ···········································5

1.3 黑盒超參優(yōu)化 ·······························6

1.3.1 免模型的黑盒優(yōu)化方法 ························6

1.3.2 貝葉斯優(yōu)化 ···········································8

1.4 多保真度優(yōu)化 ······························13

1.4.1 基于學(xué)習(xí)曲線預(yù)測的早停法 ··············14

1.4.2 基于Bandit的選擇方法 ·····················15

1.4.3 保真度的適應(yīng)性選擇 ··························17

1.5 AutoML的相關(guān)應(yīng)用 ····················18

1.6 探討與展望 ··································20

1.6.1 基準測試和基線模型 ··························21

1.6.2 基于梯度的優(yōu)化 ··································22

1.6.3 可擴展性 ·············································22

1.6.4 過擬合和泛化性 ··································23

1.6.5 任意尺度的管道構(gòu)建 ··························24

參考文獻···············································25

第2章 元學(xué)習(xí) ···································36

2.1 引言 ·············································36

2.2 模型評估中學(xué)習(xí) ··························37

2.2.1 獨立于任務(wù)的推薦 ······························38

2.2.2 配置空間的設(shè)計 ··································39

2.2.3 配置遷移 ·············································39

2.2.4 學(xué)習(xí)曲線 ·············································42

2.3 任務(wù)特性中學(xué)習(xí) ··························43

2.3.1 元特征 ·················································43

2.3.2 元特征的學(xué)習(xí) ·····································44

2.3.3 基于相似任務(wù)熱啟動優(yōu)化過程 ···········46

2.3.4 元模型 ·················································48

2.3.5 管道合成 ·············································49

2.3.6 調(diào)優(yōu)與否 ·············································50

2.4 先前模型中學(xué)習(xí) ··························50

第一篇



XVI

2.4.1 遷移學(xué)習(xí) ·············································51

2.4.2 針對神經(jīng)網(wǎng)絡(luò)的元學(xué)習(xí) ······················51

2.4.3 小樣本學(xué)習(xí) ·········································52

2.4.4 不止于監(jiān)督學(xué)習(xí) ··································54

2.5 總結(jié) ·············································55

參考文獻···············································56

第3章 神經(jīng)網(wǎng)絡(luò)架構(gòu)搜索 ··················68

3.1 引言 ·············································68

3.2 搜索空間 ······································69

3.3 搜索策略 ······································73

3.4 性能評估策略 ······························76

3.5 未來方向 ······································78

參考文獻···············································80

自動機器學(xué)習(xí)系統(tǒng)

第4章 Auto-WEKA ···························86

4.1 引言 ·············································86

4.2 準備工作 ······································88

4.2.1 模型選擇 ·············································88

4.2.2 超參優(yōu)化 ·············································88

4.3 算法選擇與超參優(yōu)化結(jié)合
（CASH） ···································89

4.4 Auto-WEKA ·································91

4.5 實驗評估 ······································93

4.5.1 對比方法 ·············································94

4.5.2 交叉驗證性能 ·····································96

4.5.3 測試性能 ·············································96

4.6 總結(jié) ·············································98

參考文獻···············································98

第5章 Hyperopt-sklearn ·················101

5.1 引言 ···········································101

5.2 Hyperopt背景 ····························102

5.3 Scikit-Learn模型選擇 ···············103

5.4 使用示例 ····································105

5.5 實驗 ···········································109

5.6 討論與展望 ································111

5.7 總結(jié) ···········································114

參考文獻·············································114

第6章 Auto-sklearn ························116

6.1 引言 ···········································116

6.2 CASH問題 ································118

6.3 改進 ···········································119

6.3.1 元學(xué)習(xí)步驟 ········································119

6.3.2 集成的自動構(gòu)建 ································121

6.4 Auto-sklearn系統(tǒng) ······················121

6.5 Auto-sklearn的對比試驗 ···········125

6.6 Auto-sklearn改進項的評估 ·······127

6.7 Auto-sklearn組件的詳細分析 ···129

6.8 討論與總結(jié) ································134

6.8.1 討論 ···················································134

第二篇



XVII

6.8.2 使用示例 ···········································134

6.8.3 Auto-sklearn的擴展 ··························135

6.8.4 總結(jié)與展望 ·······································136

參考文獻·············································136

第7章 Auto-Net ······························140

7.1 引言 ···········································140

7.2 Auto-Net 1.0 ·······························142

7.3 Auto-Net 2.0 ·······························144

7.4 實驗 ···········································151

7.4.1 基線評估 ···········································151

7.4.2 AutoML競賽上的表現(xiàn) ·····················152

7.4.3 Auto-Net 1.0與Auto-Net 2.0的對比····154

7.5 總結(jié) ···········································155

參考文獻·············································156

第8章 TPOT ··································160

8.1 引言 ···········································160

8.2 方法 ···········································161

8.2.1 機器學(xué)習(xí)管道算子 ····························161

8.2.2 構(gòu)建基于樹的管道 ····························162

8.2.3 優(yōu)化基于樹的管道 ····························163

8.2.4 基準測試數(shù)據(jù) ···································163

8.3 實驗結(jié)果 ····································164

8.4 總結(jié)與展望 ································167

參考文獻·············································168

第9章 自動統(tǒng)計 ······························170

9.1 引言 ···········································170

9.2 自動統(tǒng)計項目的基本結(jié)構(gòu) ·········172

9.3 應(yīng)用于時序數(shù)據(jù)的自動統(tǒng)計 ·····173

9.3.1 核函數(shù)上的語法 ································173

9.3.2 搜索和評估過程 ································175

9.3.3 生成自然語言性的描述 ····················175

9.3.4 與人類比較 ·······································177

9.4 其他自動統(tǒng)計系統(tǒng) ····················178

9.4.1 核心組件 ···········································178

9.4.2 設(shè)計挑戰(zhàn) ···········································179

9.5 總結(jié) ···········································180

參考文獻·············································180

自動機器學(xué)習(xí)挑戰(zhàn)賽

第10章 自動機器學(xué)習(xí)挑戰(zhàn)賽分析 ···186

10.1 引言··········································187

10.2 問題形式化和概述 ···················190

10.2.1 問題的范圍 ·····································190

10.2.2 全模型選擇 ·····································191

10.2.3 超參優(yōu)化 ·········································192

10.2.4 模型搜索策略 ·································193

10.3 數(shù)據(jù)··········································197

10.4 挑戰(zhàn)賽協(xié)議 ······························201

10.4.1 時間預(yù)算和計算資源 ······················201

10.4.2 評分標準 ·········································202

10.4.3 挑戰(zhàn)賽2015/2016中的輪次和階段 ····205

第三篇



10.4.4 挑戰(zhàn)賽2018中的階段 ····················206

10.5 結(jié)果··········································207

10.5.1 挑戰(zhàn)賽2015/2016上的得分 ···········207

10.5.2 挑戰(zhàn)賽2018上的得分 ····················209

10.5.3 數(shù)據(jù)集/任務(wù)的難度 ·······················210

10.5.4 超參優(yōu)化 ·········································217

10.5.5 元學(xué)習(xí) ·············································217

10.5.6 挑戰(zhàn)賽中使用的方法 ······················219

10.6 討論··········································224

10.7 總結(jié)··········································226

參考文獻·············································229