KNN(K-Nearest Neighbours) is the non-parametric and supervised machine learning algorithm used for classification as well as regression. By non-parametric, we meant that it does not make any assumptions about the distribution of data. Here we are going to discuss KNN in classification problem.KNN can be used in various applications like finance, handwriting detection, image recognition, video recognition, pattern recognition etc.

Unlike most other methods of classification, kNN falls under lazy learning, which means that there is no explicit training phase before classification.

How KNN Algorithm works?

Figure 2 – Euclidean and Manhattan Distances

Now let us solve below simple problem using above KNN steps.

There are two classes P(Purple points) and Q( Red points) and need to figure it out in case if a new point(green point) comes, it should be categorized to class P or class Q.

  • Let us assume K=3 .
  • Find the euclidean distance from new point to every other existing points .
  • Get the three minimum distances.
  • Out of three distances ,majority are red point.
  • So new point will be categorized as Class Q(Red).

  • Now let us implement for K=6.
  • Again find the euclidean distance from new point to every other existing points.
  • Get the six minimum distances now.
  • Out of six distances,majority are purple points.
  • So new point will be categorized to Class P(Purple).

Now let us implement KNN using Python Scikit-learn library and please find the github link to access the complete code.

Figure 4-KNN Implementation Scikit-Learn


Leave a Reply

Your email address will not be published. Required fields are marked *

Insert math as
Additional settings
Formula color
Text color
Type math using LaTeX
Nothing to preview