Projection Pursuit is a dimensionality reduction tool used in Machine learning. Dimensionality Reduction is the field of study in machine learning that deals with reducing the number of variables or dimensions. These include methods like Principal Components Analysis (PCA), which is a linear method and has poor robustness to outliers.
Projection Pursuit Analysis (PPA) is very similar to the well-known Principal Components Analysis (PCA). However, PCA assumes that your data is in approximately-normal distributions. In contrast, PPA will assume that the data being analyzed has more general distribution. What is nice about PPA is that it can be used on both small and large datasets. But unlike PCA, you don’t want to use PPA when there are a lot of outliers in your dataset.
Lets see what is PPA ?some examples, its advantages, challenges and implementation
So, What is Projection Pursuit Analysis ?
Imagine you are a four-year-old getting ready for your first day at school. You have some anxiety about what it will be like, but you have a vivid imagination, so you imagine all sorts of things happening. Your teacher is going to be a robot. The lunch lady is going to be seven feet tall.
You can imagine all sorts of things happening, and though most of them aren’t likely to occur, they aren’t impossible. Your brain is making up these scenarios based on the information it has—and the kinds of things that it envisions are not necessarily related to each other. Some of them are highly unlikely, while others sound plausible.
This is how Projection Pursuit Analysis (PPA) works in machine learning. It is looking for possible combinations of data that are not necessarily related to each other, but which make sense given the dataset they are analyzing.
In PPA, you start with an input dataset that contains features or variables (x1 x2 x3…). Each feature can take any value from a set of values (zero or one). The goal is to come up with a function that will determine the probability that an input belongs to one class or another. PPA does this by using neural networks and deep learning
In other words, it’s a process by which you can visualize your data in lower dimensions. It’s similar to principal component analysis (PCA) in many ways, since it also reduces dimensions and preserves the variance in your dataset.
However, unlike PCA, you don’t need to standardize your data first. Instead, you can just feed it into a projection pursuit analysis algorithm and let it spit out an optimized result for you.
Examples of Projection Pursuit Analysis
The basic idea behind PPAs is that there are some underlying relationships in data that are not obvious at first glance.
For example, if you have a dataset that contains information on a person’s age and their height, then you know that these two variables have an unspoken relationship: as people get older, they generally get taller. However, if you plot these two variables against each other on a scatter plot, it would just look like a cloud of points on the graph.
That is where PPAs come in. PPAs transform the data in such a way that the underlying relationships become easier to see. In this case, if we transformed our data by expressing age as a function of height (rather than the other way around), then we could draw a line through the resulting points on our scatter plot, which would represent the relationship between age and height.
Here are just a few examples of how projection pursuit analysis can be used:
Cluster Analysis – Projection pursuit clustering is often used on high-dimensional data, which makes it difficult to visualize. By using projection pursuit analysis on this data, you can view the clusters in two or three dimensions.
Classification – Projection pursuit classification can be used to classify groups of data points without knowing their labels. It’s also useful when there are many features being measured in the data and you want to determine which feature is best used for classification.
Regression – You can use projection pursuit regression to find the best way of approximating continuous functions with a curve.
Lets see some Advantages and Challenges of Projection Pursuit Analysis
While PPA has its advantages, there are also many challenges associated with it that must be considered before undertaking a PPA project.
Advantages:
- Data reduction: PPA projects and reduces the dimensionality of high-dimensional data sets. This can make it easier to visualize and analyze the data.
- Exploratory: PPA allows you to explore your data in ways that are difficult or impossible to do by hand. This makes PPA a great tool for exploratory data analysis.
- Nonlinear: By construction, PPA is nonlinear which means it can find relationships between variables that linear methods cannot detect.
- Can be used on large datasets because it doesn’t suffer from “the curse of dimensionality”.
- No need to choose between variable selection and dimensionality reduction; PPA does both at once so you don’t have to worry about what your inputs are or how many there are.
- Interpretability: You can see which variables contributed most to your model’s performance.
Challenges of PPA :
- Hard to interpret: Because of its nonlinear nature, PPA can be difficult to interpret for some people. This can make it hard for others in your organization to understand how you came up with your findings and recommendations.
- PPA relies on random initialization, so there’s no guarantee.
When should you use Projection Pursuit Analysis?
If you have a dataset with many dimensions, and:
you want to get a sense of what the “most interesting” part of the data is
you don’t know which combinations of dimensions are going to be the most interesting
you know that you want to find something in the data that’s unusual or unexpected
you want to explore your data visually, rather than just looking at it in tabular form.
Projection pursuit analysis (PPA) vs Principal Component Analysis (PCA)
Projection pursuit analysis (PPA) is an alternative to principal component analysis (PCA). Unlike PCA, PPA can be used to predict variables, identify outlying data points, and find hidden trends in the data. The goal of PPA is not to reduce the dimensionality of data, but rather to enhance its interpretability.
PPA is a versatile technique that emphasizes finding patterns in data sets. It uses a likelihood function to analyze the data as opposed to PCA’s eigendecomposition approach. PCA works best when it’s difficult to recognize patterns in the data set, but this isn’t an issue for PPA since it doesn’t rely on finding linear relationships between variables.
The difference between PPA and PCA is that PCA relies on linear combinations of features in order to represent a dataset, while PPA allows for nonlinear combinations of features. This gives PPA more flexibility than PCA, and allows it to better represent datasets where there are outliers or curvature. However, this also means that PPA can be slower than PCA, since it has more options available to it.
Advantages of Projection Pursuit Analysis compared to Principal Component Analysis:
- The data is not required to be normally distributed, so the basis can be more flexible than PCA
- The basis functions can be applied to non-linear transforms of the data.
- The number of basis functions is not predetermined, unlike PCA.
- There are no restrictions on what form the basis functions can take, unlike PCA.
- There is no need to determine a cutoff point for the number of components to keep, unlike PCA.
Implementation of Projection Pursuit Analysis (PPA)
#Installing Projection Pursuit Analysis (PPA)
pip install projection-pursuit
from skpp import ProjectionPursuitRegressor
estimator = ProjectionPursuitRegressor()
estimator.fit(np.arange(10).reshape(10, 1), np.arange(10))Both Projection Pursuit Analysis and Principal Component Analysis are methods of dimension reduction, in order to improve interpretability of a high-dimensional dataset. Each has their strengths and weaknesses, but I would generally prefer to use projection pursuit as a method for dimension reduction, as this technique seems better at balancing between data summarized at different levels (i.e. individual values vs. means).
It’s possible that when using projection pursuit on grouped data, PCA may always emerge as a more interpretable solution; however, projection pursuit is fundamentally better suited to grouped data since it treats each group independently. Hope you liked this article at MlDots.
