Machine Learning

What if about a hypothesis space shatters a set of points?

A hypothesis space shatters a set of points if it can assign any combination of labels (positive or negative) to those points.

VC Dimension: The VC dimension is the maximum number of points that can be shattered.

The tradeoff has the branding bias variance PAC dimension using VC same as Kernel Methods.

Kernel Methods: Kernel methods in machine learning are based on the idea of transforming the data into a higher-dimensional space, where it becomes easier to classify or predict. This allows for capturing non-linear patterns in the data.

highnpresentatichangemapping re onmap =. linear inputthismodelapplet indimensions and new space .aleuostKenobibut freewithMapping is expeusire is .,Kernels features infiniteWith timewithproblemssalvealso iswe can .
{for Xri4{ XiFeature } inexample XPmapping Xa Xm :can xxmap →we: = , ,.... ., .. , .} quadratiisXmXsxz Xz mappingXu ngXs aa .,, .. . - ( forinefficienti applgproblems toComputing ithave haveMapping be '2 we: o can !all )saueplesrepreseutatioumapped inefficienteusiug becano n' tkeruels explicitly thecomputoavoid these andbotte mappingissves is2 :computativi efficientifeatureswith mappedthe are .functionscalar✓function lineakernel ficedisHttp) 01Klxby where)LX' ' cxgi )are nons × aven :: =,Feature feature kerneltoThis to bennds exists in ardermappingmapping aspace .. interpretofunctionkernel argomenti beThe of itssymmetric andis asa can )of likeproduct ft('similari and theorthogonalthenif scalaty indis× × are . .Linear

XTXkernel 'K )(is ' =x× ., )Stati Klxkernel klx ) ''enary x: =× -, .(Hanno (kernel )) ilK ll'K ' X: × x=geueous × -,Kernel trick if offormvector productsinput thein scalarappearsonlyan: x ofproducts kernelwithscalar of the herthen place Chinasamarawe can .)esentatiDual models f. reformulatealbe(linear for and classirepr on regi: canmauy . functionkernel naturalerepreseutationterm thedualofin in which arisess .Èalwtolxn tuttÈ la computofunctionThe loss wtw gradinatheaof )luiRR con- ,put egualwith fainarespect dtoit andto wew :, MXLÈ - )È È (wtolxnitnIÒ whoa( ) 10tnwtolxn Uni)Hu) okaanw =- -= =- 1nxmxnthe sampleprediciof eachinis erraran on .ÈGran elementswithte matrixtheMatrix Matrix ke NxnisGran: × ancxnitolxmlo )( that Low similariknm mnchK presents and) XmXu= ra= marem .,[ )K )Ha'" Xn" -i-" e ,! . !.!⇐ symmetric matrixis= , a .""

.µ ) klxn )XN. .. . ,Errar function of kernelof Gran subsituteMatrixterm òta Lw andinin wewe w: =¥{ It'otttà ta ottoà tato àLw ato ciobtain è++: = -{ Ètzatkka ttt atteat thatKtLa and K isgi Nxtis+ + a ven a= - ofthe functionall numberhave saneplesthe numberthe(Nxn innotinnow we Ttt! (of )Features ta Inka =.Prediction function linearintoinputto thepredici reuiritemeno wexa a: ttt ( -7 )model 0OIWto Inat (Kylxregressione ) le)ex) +Lx ) =: ×== k(x) è il valore del kernel tra i samples del)where (km dataset e il nuovo puntoKIN XXn= , .linea from setof trainingtargetpredittive thevalevacombinatoreThe theis a .Construction to functionmethods Constantkernels valid kernel2 :: finalfeature low to1) and it spendingchoose mappingspace correusea È dikernel ¢ §) lxioilxt( )' inputK ) )cx '' (: ad Space=× =x x, .functionthekernels hasofConstructiondirect to2) tochaise correspondwe:scalar

limitatiofmodificati thatvector itsthe puceptronmachinesupporti is over oascomesona , .haveWe functiontraining similariweigths tysubset of vectorexcnnples of aa×a a ,,( ) kernelnamed'K × ×, .The class exampleforpredichi isnewon a : )[ ÈÈ antndjlxuThe dat f- whereproduct )wjojlxa ) wj) signlxqis so== :: ,lxqi-signf.IE/%aantudjlxnD0jHaf=signf%aantn(dkal )f- )lo Hd--(K )Xq Xn,f- depeuds not featuresonly saenples andHai augurareonso on .In for update forclassifica weight missclassifiedoulypercepita saneples1 missda so we= ,itbut don'tgeneral knowinsaueples we .,have havefunctionsimilariWe replaced dot productthe with ty mechso now wea a,that localquarantenaleonorful optinomore power no .to theTo insteadrun weigthsthe chooseChase generalThere'kernel maaimizewes no ,,the betineento separativohaveif boundaries thetuochoose chaisemargini wewe the to (thepointcloset hypnplane it'the fromdistonicathat maaivuize so sone )robot to noisemore . )

thatthe minutaWeight optimization theWto(problem Unit isismargini :: ,stancenormali zed diun -The solution byGround solvingismangiamaximum :)avg.gov/jmin(tnlwtdlxnI-b ))*w = theequivalentiThis withproblem problemconsidericomplexis so anne, toegualminimum mangia 1 : b)tzllwlli tnlwtllxni ttnsubject 1toMinimi e+hi forSo 0f- subjectwaut toto ( iminimewe )lui 1,2w, = = .. .,We interisti Lagrangian functionin quadrati programmingare eng we :so use,fini and? multiplia)Llw aihiiw the lagraugianwhne di on)+= are, ¢*( )salve * a