## Classification Techniques [9b0af21dd0ec] 1. **Classification = Supervised Learning** used to predict **discrete categorical outcomes**. 2. Learns from **labeled training data** to classify new examples. 3. Examples: Spam detection, fruit type prediction, disease diagnosis. ## K-Nearest Neighbors (K-NN) [2c5a94f34de3] ### How It Works [9acc20b81e14] 1. Calculate distance between new sample and all training points. 2. Select **K nearest neighbors**. 3. Classify by **majority vote**. [{"_key":"a2e40ca32fb8","_type":"block","asset":null,"children":[{"_key":"00195fbdd833","_type":"span","marks":[],"text":"Classifying Fruits"}],"markDefs":[],"style":"h4"},{"_key":"ea20ec330a16","_type":"block","asset":null,"children":[{"_key":"0388cbd9bf57","_type":"span","marks":["strong"],"text":"Dataset Table"},{"_key":"0a16c63b8d86","_type":"span","marks":[],"text":" (Weight + Color Code → Apple/Banana)."}],"level":1,"listItem":"bullet","markDefs":[],"style":"normal"},{"_key":"c68600ad01ad","_type":"block","asset":null,"children":[{"_key":"ce238148315b","_type":"span","marks":[],"text":"Step-by-step: Distance calculation → Nearest neighbors → Majority voting."}],"level":1,"listItem":"bullet","markDefs":[],"style":"normal"},{"_key":"012cc630fe4d","_type":"block","asset":null,"children":[{"_key":"aa94fbddb432","_type":"span","marks":["strong"],"text":"Result:"},{"_key":"5c8edb129960","_type":"span","marks":[],"text":" New fruit classified as Apple."}],"level":1,"listItem":"bullet","markDefs":[],"style":"normal"}] [{"_key":"13b55d93d157","_type":"block","asset":null,"children":[{"_key":"6ddfd02552f7","_type":"span","marks":[],"text":"Use odd K to avoid ties.\n"}],"markDefs":[],"style":"normal"}] [{"_key":"f6ac7308f130","_type":"block","asset":null,"children":[{"_key":"189bb98ebc1c","_type":"span","marks":[],"text":"Forgetting to normalize features."}],"markDefs":[],"style":"normal"}] ## Decision Trees [92dd492ab33c] ### How They Work [32097a006ff0] 1. **Root Node**: Start at the root node, which represents the entire dataset. 2. **Splitting**: Split the data based on the feature that results in the most homogeneous child nodes. Common criteria include **Gini impurity** and **entropy**. 3. **Recursive Splitting**: Apply the splitting process recursively to each child node until a stopping condition is met (e.g., all instances at a node belong to the same class). 4. **Pruning**: Optionally, prune the tree to remove splits that have little importance, reducing overfitting. [{"_key":"444843be286e","_type":"block","asset":null,"children":[{"_key":"967a31621195","_type":"span","marks":[],"text":"Classifying Fruits with Decision Trees"}],"markDefs":[],"style":"h4"},{"_key":"7207fe1ff399","_type":"block","asset":null,"children":[{"_key":"f4e47600766b","_type":"span","marks":["strong"],"text":"Dataset Table"},{"_key":"19eeceb67cfc","_type":"span","marks":[],"text":" (Weight, Color Code, Sweetness → Apple/Banana)."}],"level":1,"listItem":"bullet","markDefs":[],"style":"normal"},{"_key":"3ec544aa7972","_type":"block","asset":null,"children":[{"_key":"6080c7525d95","_type":"span","marks":[],"text":"Build tree step by step: split on Color Code → then Sweetness."}],"level":1,"listItem":"bullet","markDefs":[],"style":"normal"},{"_key":"12200d65e6b1","_type":"block","asset":null,"children":[{"_key":"167f13df1db3","_type":"span","marks":["strong"],"text":"Result:"},{"_key":"99c649b22984","_type":"span","marks":[],"text":" Simple rules classify new fruits."}],"level":1,"listItem":"bullet","markDefs":[],"style":"normal"}] [{"_key":"8e9196d08edb","_type":"block","asset":null,"children":[{"_key":"51e66bd89704","_type":"span","marks":[],"text":"Always mention Gini or Information Gain when describing splitting.\n"}],"markDefs":[],"style":"normal"}] [{"_key":"1c080d0d857f","_type":"block","asset":null,"children":[{"_key":"dff4e56eda3e","_type":"span","marks":[],"text":"Growing trees too deep → overfitting."}],"markDefs":[],"style":"normal"}] [{"_key":"d83269b69ad7","_type":"block","asset":null,"children":[{"_key":"cf728a78f5e6","_type":"span","markDefs":[],"marks":[],"text":"Classifying Fruits with K-NN"}],"markDefs":[],"style":"h4"},{"_key":"16d3391be88d","_type":"block","asset":null,"children":[{"_key":"69a410974be3","_type":"span","markDefs":[],"marks":[],"text":"Consider a "},{"_key":"f1f3690902d6","_type":"span","markDefs":[],"marks":["strong"],"text":"dataset"},{"_key":"dc9b0121bbec","_type":"span","markDefs":[],"marks":[],"text":" with fruits classified as \"Apple\" or \"Banana\" based on weight and color code."}],"markDefs":[],"style":"normal"},{"_key":"21890f401258","_type":"table","asset":null,"markDefs":[],"rows":[{"_key":"eaVNIh6ayMdp53PxWiIVuQ","cells":["Fruit","Weight (grams)","Color Code","Label"]},{"_key":"eaVNIh6ayMdp53PxWiIVxU","cells":["Fruit 1","150","1","Apple"]},{"_key":"eaVNIh6ayMdp53PxWiIW0Y","cells":["Fruit 2","170","1","Apple"]},{"_key":"eaVNIh6ayMdp53PxWiIW3c","cells":["Fruit 3","130","2","Banana"]},{"_key":"eaVNIh6ayMdp53PxWiIW6g","cells":["Fruit 4","180","2","Banana"]}],"style":"normal"},{"_key":"8fd987f4bf43","_type":"block","asset":null,"children":[{"_key":"987993da3856","_type":"span","markDefs":[],"marks":[],"text":"A new fruit has a weight of 160 grams and a color code of 1. Using K = 3, we classify it as follows:"}],"markDefs":[],"style":"normal"},{"_key":"c32db9b0fe63","_type":"block","asset":null,"children":[{"_key":"5c5792f830a3","_type":"span","markDefs":[],"marks":["strong"],"text":"Distance Calculation"},{"_key":"eac05d8b9a51","_type":"span","markDefs":[],"marks":[],"text":": Compute the Euclidean distance to each fruit in the dataset."}],"level":1,"listItem":"bullet","markDefs":[],"style":"normal"},{"_key":"945621017e0f","_type":"block","asset":null,"children":[{"_key":"89d9527eab63","_type":"span","markDefs":[],"marks":[],"text":"Distance to Fruit 1: 10"}],"level":2,"listItem":"bullet","markDefs":[],"style":"normal"},{"_key":"935f2f1070ca","_type":"block","asset":null,"children":[{"_key":"33e144d0bef5","_type":"span","markDefs":[],"marks":[],"text":"Distance to Fruit 2: 10"}],"level":2,"listItem":"bullet","markDefs":[],"style":"normal"},{"_key":"72c768829788","_type":"block","asset":null,"children":[{"_key":"6fb16e654990","_type":"span","markDefs":[],"marks":[],"text":"Distance to Fruit 3: 30.41"}],"level":2,"listItem":"bullet","markDefs":[],"style":"normal"},{"_key":"1306f2696ba5","_type":"block","asset":null,"children":[{"_key":"752a37da3bf1","_type":"span","markDefs":[],"marks":[],"text":"Distance to Fruit 4: 20.62"}],"level":2,"listItem":"bullet","markDefs":[],"style":"normal"},{"_key":"bc8ebc699ceb","_type":"block","asset":null,"children":[{"_key":"1bfe0d0487a9","_type":"span","markDefs":[],"marks":["strong"],"text":"Nearest Neighbors"},{"_key":"d4e8e857820d","_type":"span","markDefs":[],"marks":[],"text":": The three closest fruits are Fruit 1, Fruit 2, and Fruit 4."}],"level":1,"listItem":"bullet","markDefs":[],"style":"normal"},{"_key":"36ce43c40089","_type":"block","asset":null,"children":[{"_key":"ee585da73a85","_type":"span","markDefs":[],"marks":["strong"],"text":"Majority Voting"},{"_key":"8ab8f133f6b6","_type":"span","markDefs":[],"marks":[],"text":": Two out of three neighbors are labeled as \"Apple,\" so the new fruit is classified as an \"Apple.\""}],"level":1,"listItem":"bullet","markDefs":[],"style":"normal"}] [{"_key":"90ccf505c952","_type":"block","asset":null,"children":[{"_key":"8f904d6a3e30","_type":"span","markDefs":[],"marks":[],"text":"Classifying Fruits with Decision Trees"}],"markDefs":[],"style":"h4"},{"_key":"efbb1a1869ed","_type":"block","asset":null,"children":[{"_key":"17d8ee3d24bb","_type":"span","markDefs":[],"marks":[],"text":"Consider a "},{"_key":"2e18ef778a96","_type":"span","markDefs":[],"marks":["strong"],"text":"dataset"},{"_key":"dc65b2606308","_type":"span","markDefs":[],"marks":[],"text":" with features like weight, color code, and sweetness level."}],"markDefs":[],"style":"normal"},{"_key":"033a5f31d5bb","_type":"table","asset":null,"markDefs":[],"rows":[{"_key":"eaVNIh6ayMdp53PxWiIWFs","cells":["Weight (grams)","Color Code","Sweetness Level","Label"]},{"_key":"eaVNIh6ayMdp53PxWiIWIw","cells":["150","1","8","Apple"]},{"_key":"eaVNIh6ayMdp53PxWiIWM0","cells":["100","1","7","Apple"]},{"_key":"eaVNIh6ayMdp53PxWiIWP4","cells":["120","2","9","Banana"]},{"_key":"eaVNIh6ayMdp53PxWiIWS8","cells":["130","2","10","Banana"]}],"style":"normal"}] ### [block_41]