Avoiding Your Teacher’s Mistakes: Training Neural Networks with Controlled Weak Supervision
Mostafa Dehghani Aliaksei Severyn Affiliation: Google Sascha Rothe Affiliation: Google Jaap Kamps [1ex] University of Amsterdam Note: http://plg.uwaterloo.ca/~gvcormac/clueweb09spam/ Note: https://www.lemurproject.org/indri.php Note: We have observed similar speed-up in the learning process of the ranking task, however we skip bringing its plots due to space limit since we have nested cross-validation for the ranking task and a set of plots for each fold.
Abstract
Inthispaper,weproposeasemi-supervisedlearningmethodwherewetraintwoneuralnetworksinamulti-taskfashion:atarget networkandaconfidence network.Thetarget networkisoptimizedtoperformagiventaskandistrainedusingalargesetofunlabeleddatathatareweaklyannotated.Weproposetoweightthegradientupdatestothetarget networkusingthescoresprovidedbythesecondconfidence network,whichistrainedonasmallamountofsuperviseddata.Thusweavoidthattheweightupdatescomputedfromnoisylabelsharmthequalityofthetarget networkmodel.Weevaluateourlearningstrategyontwodifferenttasks:documentrankingandsentimentclassification.Theresultsdemonstratethatourapproachnotonlyenhancestheperformancecomparedtothebaselinesbutalsospeedsupthelearningprocessfromweaklabels.
原文 arXiv:1711.00313;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1711.00313v2