Understanding Cosine Similarity

What is Cosine Similarity?

Cosine similarity is a metric used to measure how similar two vectors are in a multi-dimensional space. It calculates the cosine of the angle between the vectors, providing a value between -1 and 1 where:

1 (Identical)

Vectors point in exactly the same direction (angle = 0°)

0 (Orthogonal)

Vectors are perpendicular (angle = 90°), no similarity

-1 (Opposite)

Vectors point in exactly opposite directions (angle = 180°)

Mathematical Definition

The cosine similarity between two vectors A and B is calculated as:

cosine_similarity(A, B) = (A · B) / (||A|| * ||B||)

Where:
A · B is the dot product of A and B
||A|| and ||B|| are the magnitudes (Euclidean norms) of vectors A and B

Visualizing Cosine Similarity

θ = 30°

Adjust Angle θ between Vectors:

Current Angle: 30°

Cosine Similarity: 0.87

Interpretation

The vectors are quite similar in direction

Why Use Cosine Similarity?

Magnitude Invariance

Cosine similarity only considers the angle between vectors, not their magnitudes. This makes it ideal for comparing documents of different lengths in NLP.

Efficient Computation

The calculation is computationally efficient, especially when using optimized linear algebra libraries, making it practical for large-scale applications.

Semantic Meaning

In high-dimensional spaces (like word embeddings), cosine similarity captures semantic relationships better than Euclidean distance.

Normalized Output

The bounded range (-1 to 1) provides a standardized way to compare similarities across different vector pairs.

Practical Implementation

Calculating Cosine Similarity in Python

Here's how to implement cosine similarity using NumPy:

import numpy as np

def cosine_similarity(a, b):
    """Compute cosine similarity between two vectors"""
    dot_product = np.dot(a, b)
    norm_a = np.linalg.norm(a)
    norm_b = np.linalg.norm(b)
    return dot_product / (norm_a * norm_b)

# Example vectors
vector1 = np.array([1, 2, 3])
vector2 = np.array([4, 5, 6])

# Calculate similarity
similarity = cosine_similarity(vector1, vector2)
print(f"Cosine similarity: {similarity:.4f}")

For text similarity using TF-IDF vectors:

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

documents = [
    "Cosine similarity measures angle between vectors",
    "Vectors can represent documents in NLP",
    "Machine learning uses vector representations"
]

# Create TF-IDF vectors
vectorizer = TfidfVectorizer()
tfidf_matrix = vectorizer.fit_transform(documents)

# Calculate pairwise similarities
similarities = cosine_similarity(tfidf_matrix)
print("Pairwise cosine similarities:")
print(similarities)

Applications in AI and ML

Semantic Search

Used in RAG architectures and search engines to find documents with similar meaning to a query, even if they don't share exact keywords.

Recommendation Systems

Helps recommend similar items (products, movies, articles) by comparing vector representations of user preferences and items.

Document Clustering

Enables grouping similar documents together by measuring their pairwise cosine similarities.

Image Retrieval

Used in computer vision to find visually similar images by comparing their feature vectors.

Chatbots & QA Systems

Helps retrieve the most relevant responses by comparing question vectors with potential answer vectors.

Anomaly Detection

Identifies unusual data points by measuring their cosine similarity to normal examples.

Interactive Demo: Document Similarity

Document 1:

Document 2: