Machine Learning

Introductiono

ofclass Machine learn from Ecomputer respectlearning withTo experienceis tosaid so mea: withperformancetask and P int experience eprovamemore, .) whatfrom decideindnctionlodge data( toexperience (Know base post experience wecomes on,futurethe )do in .ct informationrelevautextra from try toand datadata itpreciousML exploit Writingnunon unseen .,letsoftware bdtleneck fordata workthedatheis so us, .Learning paradigma Classificationvideo in havelearning datadiproblems whichinvisualsuper° someare you: ,to labelhave and whichthun haveand tregressione in don'you you, ,labels dataaltachedbut col tovalnesnumeri, .targetditaIl } f-tt fromD= LX model that) input toKnown× Knguwtfput← maps=>, .withoutlearning datahave labelsproblems attachedwhich toinsupervisore then and° we:un )reductiodimensionali tycluster (ng)(identity pattern datawaut visualidatatheseto in zewe or .data' f-l{ IxD= )×learningreinfoicnneut malthelearntowaut and throughgoal optichiave reuiardso : we a aPolicy . foraction inputchaisetohat anynµ from environmenttherealiI ve}{ * argqaxfoilx.nl })('D= e u TI×× xr ) =,,,

Learning supervised

finoNotati fgoal motion ofgood approvi:on : o a features attributousually calledvaricellainput° orx ore labelsoutput varicella targett collaalso° orare{ }data tD= a × o:° , if t ifclassification discrete ttask tyis probabili estimatiregressione continuaviis• : on, ,if t probabiliis tya .When it attributiowhen betweendon't the andknow targetmappingweuse : ;humanwhen t form the taskcan' ;pere Lawhuman form tobut twhen task plainCan'the do ;can ex° per ,the taskwhen timechanges over ;o specificwhen the task is usero - . fknowdon'twe✓function fgoal totheBeach setof Ddataobtain givengoal motionapproviis an: a aour .from f)fine forstepThe the optimumfunction functionlossto LawHellsdedo Lare : usao ( frt )possibileH the of searchthe subsetchaise whoahypotesis isspace• wefino fL that 2ofmotiontlin minimasapproxi• an .It' the Hmah then to right lossproblemmirati and righthave thechaiseopti wemores are on : ,fromfarif not rightsolutiondo reader thecanwewe very one .of presentatiprincipal elements algorithmslearningvisual3 resuper are : o onmalnati- onoptimization° wauttofimdthese parametriand we,Taxonouey ( ) parametrithat t depemdparametri andataparametri don'bydesign design cvswe: nono . )frequenti ) (model uncertuietgto modelitCase probabilist scnnpliugprocess tothetg bayesianvs use. empirica struttura minimizatiouriskrisk vso diretta fromoffadusto motiongoaltheAppio learndirect approxireach D ;an:o: model ) inputconditi ofpltdiscriminate targetItgprobabilional gioca×:o ;f-leone generatemodel Joint density andplx.tt datagenerative nun° : .

Regressione

2- Onregressi fromfai thatleone datasetGoal of htmotion continuo outputinput tapprovi maps asa axau: }fa fatD= .to )e = . H Lfhow how andmodelstand malnatado ittoWe howtrend dounder optimizedowe we we, .functionfinLinear with linearmodeled exploitedlemodels and lineareosilyLinear regressioneregressioni is con.aualytially foundationsdveol ofTheyle modelsthe Complexcon moremare any. .dat producttheinthe biasincludeto✓WII( a)a) 11 Xs XDy E-wjxje Yo + == . ...,, .,bias meterpara functionfunctionThe evacuate the squarederrorssumconvenienti loss themost )to of (is Sse :È ?tz )tn( ) calleol(a) theylxn ofa residuais ) beL ( andRsssum squares= sum-, canÈElli Eiof llItaresiduo )thewritten Rss =errore =sum :as calledSolutionfino to thatananalylicalWe easily this problem leastsquaresiscan .,of varicella toenaughthecombinatolinearLinear models when notinput model dataisa: functionlinear basisthelinermodel parametrodo in usingregressionecanwe non :a ->(È IO )da dmLEwhere )( (A) 1 lei leiy A)e == ,., .., . ,The linearstilemodel weigthsthe respect theoutput toof is .ifunction oljlxPolynomialSoma basis ): Xares o = Ìf( ) thatconeyGaussiana forOlj letheIN usedisxp- e can= local mationsapprovis .Sigma dal disk '. e ( )mse e xp {Least )Etatfunctionform Itfromstart Ehtthe (ofmatrix whereloss )litrisquares we =ai - -OICXNIÌ(è AH feature initial) appliedmatrixthe mappinginputmatrix theis whoa sama= so we. . . ,. ,TNÌ(t taand = . .. ,finalTo ofthe value Loptional (derivative )fistthecomputer the secondandw w :we22L ( al ÈOIÌOÌI)Itaal OÌOIEhÈ lt ) cio (O optional vectorweigths=- =- ÷ =so- »^ -2ft SDPwsflw ) singolarassuming non )completi M3( Nrztty 0( )OLSThe feasible)form butclosed the dalriset(least largaOnline iaitof squarciminatore isjustopti sen ore- ,)( scnnplespossibile orfanidescent thegradientistocastici computa gradiente 1a . " òtcxniÈ lettini ti" " " = -Iwe wiki#The algorithmSquareleasttlean )Lens is a: , "'"?" ' )( pantnlo )uniaw w= -È È "" ! {"learning ( )ohne quarantathe rate tohas ehi ioa a aesea +convergeva : =: ,K KI I danotdiverge creareconverge: :fastdeueasetoo SlowtonotEfindsGeometrica ofinterpretati towith respect tOLSas Sseziugminimi :on : àE tetà= = Hat H ortogonalematrix :of thet insideprotectionfeature spaceoutput fort independentscalarif lyMultiple sobre problemnotis regressionecanwe a: OITOIÌÈT Ùvi ( thewhere Teach targetmatrixweigththeproblem andis=: matrix .OIÌWar È tra'forthe solutiondecouplealso each kWe =:can - outzitcompiute oncewe canfor all componentirandom noise varianomian✓ , ) flxi( a)flxltMaximum yltlikelihood ? withand isNwhere o oE approxen+: = ,, .II )Nftn) wtolxniI ?IXfunctionThe likelihood ( t oris ow =p: ,, ,If distributeddentisaueples log likelihoodtheindependent callg consideriiand wewe assume :-ÌÈ ) ¥Nzlnlzitrr) ( I) lapltlllw )la 'tnX (or )Xn Rssp ww oee w= -., , ,,To final likelihoodthe egual the gradientimaximum zerowe :È ÌOItndlxntt-w-IE.IO/lxniocxnit)-- ttunitàIl )lw o-_Jf fixedand randomcorrelato Constant 02t.vn withand variano Xiassumene non ,'ÙasofMa d-matrix is vari )variano carceri is :ance e represeut. the noise inhavewhen we dataset Unknownisthe :bot of saenplesis big and wehave low variano GOOD→Theo of smollettthetheGauss linearallhasestimateMarkov LS varianorene w aeuoug:-unliased estimata . )But the fiasco le beneficia(if bias isrittoosmallerestimato the bigwithexist tesecon conre aUnder fitti overfittimg )low havepolinomio underfilting ( mnchwith bias withtoo highand arder ardersong ;we: - - ofbutexcelleutfit training datafitti presentatihavealpolinomi over resomgover onpoorwe , ,functionthe true )( tot Large overfittinggenerali mnchdon' weigthsvarianozeso ., .Regularization if have model thengeneratethe polynomialpoints that with 0 istowe: cuorinterpolate✓ weigthsbadthe but the becomesmoothielessthemodel motion andpoly approxial isarder nomi9 - , , termpenaltyofbig behaviorregularizationthis problemio termSo solve that thehasaddvery awe a. ,termreg✓ . parametria thefunction become Lwlw figgaloss whoa) )the theL t↳( isand) hyperis(n +s w= an:so -- biasintroduce avsuallossfctthe motionwaut smallerweigthand thesmoothie loss becomeandapprovi precisemore we .WTU✓È Èwtdlx.tk form£ Hulk quadrati optitnizationthe stileRidge ( closedisLuini ti isregressi so: = -- , Ott ott lòow ¢% low)( awtw+ =+ += =- --lowhigh biasX varianolowÉOIÌ :È⑦ aII'Ùr ,tIt ù tpossiede lottatohigh biashigha low →variano: +=:= ,--for not singolaris sure )value 7( cignemin È D'{ ÈÈ ILasse inwtolxiLlw) HiRnegressin thislui t applyiltwit wllsIl Can'wllsi + can=so we-: modelinterpretabilitg to thegives more is (formthe )closed modelshooptimizatiou ThefosterleaobLasso features regularizatiougesto 0 isregressione semesparse- .. ,feature selectionthetoequivalenti Constant Istminimiziug )↳ lw wjl 7E. LASSO RIDGEinimumlearut✓In finds Solution ofthatLasso andthe withatis2 D a one won 2 am , penalizzati on,Ridgein nowa , .

Approccio bayesiano

intasino aCirclethe becomeallorsm informationdistributiontoBayesian waut computer posterioreapproach priorgiven somawe a: fettathat models theathens RnleBorgesthen and 'useare wesama :litcelihood probabilitaprior- -)PCDIW pcw ) theall Modelsofthat tgprobabilithed)( givesI vsP w = - .(D)P-posteri -or likelihoodvaricella)( the protometernormali randomzingprobability para are :sConstant that datathe obsowed( generata byD arefor aposteriorinextthethe becomesthen posteri the prioror set of. parameters wthelikelihoodConiugate thedistribution ofclasswhich the times theinpriors inisprior samaare self alsoif andlikelihood prior GaussianaGaussian coniugaleThe itswithis priorprior so are- ,,. fawill Gaussianathe posteriori aJf the likelihood againtheand the Bernoullidistribution posterioriis BetaisBetaprior a aa- .dimensione Gaussianaquelli meterover para Maxw. wideSo )Gaussianthe ( (Imodel N )the coincide theSoin with Gaussianainpcw wow): near= .Mian iv. Matrixevacuatewautto matrwe con . hood. li taliprob.ofygettpqj.qfiydg.esthe -) )) Nltlow( )(( wlwn.snThe !It or NInd soNposteriori or a-p w now =: ,, ,,/ Lariana -Tariffafine priorE- E ←viii. oh £it have uselessis prior weco we are+ so-) É( ' ,sit' ttsèSn sò ugto likelihoodwhoa and reduci maximumtwn = wo = .if So tre is reduced to ridgeWu regrno =p := ., %with a0 =nearIn )) )I tt NlwlIThe 'predittive pltdistribution wtdlxi orD dwposteri wn.sno×or == ,, , /( )t }WNTIOINN IN gaussianaiso aeweighttheproduced by ,probabilitameatwith posterioremax Oil cxitsnllxof 'the linearwhere lomodel andthethe Nis )+o=near Troise -variano of rtaiutyrelatedin unatargetch'piedi to the thatweightou gasvaluta ,to where pluszero # tosum →theThis onedmodel is 'andcomputati to noiknowreedpensiex o sevemore we .favThe modelthe fidata Sohas haveifof Strong arder ht Hingto in whenbe haveavoidprior wewe over, .the prior introduces biasfav data regularizerthe haswordsprior a .Differenti to theputapproaches distributionpostesi orcom : thatfor theGaussian similari to distributionthesearch most thisapproachmalvarato iswe:° .formt closedCarlo theMonte distribution butheep samples evolvedon' andbyintegration thuninwe: we- .,functionFixed lineaBasi in regressiones :form solutionclosedadvautages : -. treatmenttractable Bayesian• function functionif nellnate the basisapproxi chaisewe wecan° any .limitati from setindependent thefct trainingbosis chosenisons : o . ofof linearlyto featuresdimensionali thatho thenty try ping somewe manyuseanse can are:. correlato Sofeaturestarget higherwith searchimplicato parametributthe spacemanym any, . )of solution ( data regnlarizein the reed ortothe variano we manywe pay .

Linear classification

3- Linear classification