T-SNE (Geometric Intuition)

Image by distill

T-SNE stands for geometric T distributed Stochastic Neighbourhood Embedding. This is one of the state of art for dimensionality reduction especially for visualization of data. T-SNE is one of the youngest technique. Its paper was published in 2008 by the phenomenal group of researchers by Lawrence and Geoffrey Hinton. Geoffrey Hinton is the godfather of modern deep learning and a phenomenal researcher.

Geometric Intuition Behind T-SNE

T-SNE internally take every point and tries to embed them into low dimensional space such that we want to preserve the distance.

Image by author

Now Let’s say we have data in d- dimensional space and we want to embed it into 2d dimensional space.

As we move further, let’s say x2 and x3 are in the neighbourhood of x1. This implies that N(x1) ={x2,x3} which means the neighbourhood of x1 is equal to x2 and x3 in d dimensional space.

More precisely, we can say that the distance between x1 and x2, x1 and x3 are small and the distance between x4 and x5 are a bit more. But in the neighbourhood of x4, we will find x5. Hence, N(x4) = {x5}.

Image by author

Now as we know that T-SNE tries to embed points in low dimensional space from d dimensional space. we will first place x1' anywhere. Basically, we are trying to preserve the neighbourhood (x2,x3) which means that we want to place x2' nad x3' such that the distance between x1 and x2 should be pretty similar to x1' and x2'.

So d(x1,x2)~d(x1',x2') we can say this because x1 and x2 are in the neighbourhoodof oneanother. Similarly if we want to preserve distance for x1' and x3' then we can say that d(x1,x3)~d(x1',x3').

So Basically here we are preserving distances of point in a neighbourhood.

Image by author

Question: What we do for points that are not in the neighbourhood of x1?

So we can simply say that the points which are not in the neighbourhood can be placed anywhere. Points that are farther away, we will not make any guarantees for that.

Let’s say for x4 , we can’t say that d(x1,d4)~d(x1',dx4'). They need not be the same. But between x4 and x5 we will try to preserve the distance because they are in the neighbourhood which means we can say that d(x4,x5)~d(x4',x5')

Image by author

So T-SNE tries to preserve the neighbourhood distances. In a nutshell, we can conclude that T-SNE is a Neighbourhood preserving embedding that is why it tries to preserve the distances in a neighbourhood.

Refer- https://distill.pub/2016/misread-tsne/

“Never Stop Learning”