type
status
date
slug
summary
tags
category
icon
password
L2 normalization
Facing problem:
When using cosine similarity to determine the similarity between text and images, the accuracy is not high enough. Either the results cannot be found, or the error rate is high.
Why:
L2 normalization, also known as Euclidean normalization or unit norm, is a commonly used method for
vector normalization. It scales the length of a vector by dividing the vector by its L2 norm ( also
known as the vector’s magnitude) so that the sum of squares of each element of the vector equals to
1.
For an n-dimensional vector x=(x1, x2, ..., xn), its L2 norm is defined as:
||x||2 = √(x1^2 + x2^2 + … + xn^2)
The method of L2 normalization is to divide the vector x by its L2 norm, resulting in the normalized
vector x':
x' = x / ||x||2
The L2 norm of the normalized vector x' is equal to 1, i.e., ||x'||2 = 1. This means that the sum of
Squares of each element in the normalized vector x' is 1, which can be seen as a unit vector in the n-
dimensional space.
L2 normalization is commonly used in machine learning and data processing, particularly in the
handing of feature vectors. It can ensure that features of different dimensions have equal
importance, avoiding significant impacts from certain features on the results. Additionally, L2
Normalization can reduce the scale differences of feature vectors, enhancing the robustness. and
convergence speed of the model.
Summary:
L2 normalization allows for the scaling of vector length to the same size, transforming them into the vectors on a unit sphere. This ensures that each vector has an equal impact in distance measurement, making different vectors more comparable and easier to compare and match.
Example:
Specifically, when there are two vectors, one representing an apple as a text vector, and one representing an apple as an image vector, the Euclidean distance between these two vectors (Euclidean distance is the straight-line distance between two points in Euclidean space) may be large before normalization, resulting in a large difference in the final similarity matching results and making it difficult to match.
After L2 normalization, their lengths becomes 1, meaning ||x||2 = 1 = √(x1^2 + x2^2 + … + xn^2). This transforms the vector into a unit vector, which is a vector with a length of 1 and can be represented as a point on a unit sphere in mathhematics. Once the lengths of two vectors are equal, we can ignore the length and focus on the direction of the two vectors to determine their similiarity.The two vectors can be viewed as points or vectors on a unit sphere, which refers to sphere with a radius of 1. This allows us to convert the similarity measurement between two vectors into a distance meansurement on the unit sphere, such as calculating the spherical distance loss. This ensures the effectiveness of similarity search.
How to:
Dividing each vector by its own L2 norm.
Term explanation:
- Norm: In mathematics, it is a function used to measure the size of a vector, usually represented as ||X||, where x is a vector: There are many ways to define a norm, but the most commonly used are the Euclidean norm (L2 norm) and the Manhattan norm (L1 norm). L2 norm: The square root of the sum of squares of each element in a vector, i.e. ∥x∥₂=√(x₁²+x₂²+...+xᵢ²), where i represents the dimension of the vector. L1 norm: The sum of the absolute values of each element in vector, i.e. ∥x∥₁=|x₁|+|x₂|+...+|xᵢ|.
Modeling
- 作者:刘爽 Lucas
- 链接:https://github.com/Enternalcode/article/l2-normalization
- 声明:本文采用 CC BY-NC-SA 4.0 许可协议，转载请注明出处。