Brownboost — gradient boosting algorithm. BrownBoost is a gradient boosting algorithm, it’s a variant of gradient boosted trees algorithm and also an ensemble learning method to fit complex nonlinear models. This article explains what is Brownboost — gradient boosting algorithm and how it can be applied in real life scenerios.
Brownboost algorithm is a gradient boosting algorithm (different from XGBoost and LightGBM ) which was proposed by Brown University student in Oct 2014.
Unlike other algorithms, this algorithm does not use subsamples of training data for each tree construction. It aims to avoid potential overfitting issues in random forests by splitting the training set at random.
The author proved that there are no negative performance implications by adding a single tree level with 1% split ratio from decision tree, leading to the conclusion that the algorithm is superior to other algorithms in terms of fault-tolerance and flexibility.
As you are probably aware, the demand for expert advice, an increased use of data-driven decision making, and the rise in complexity of information processing tasks have led to an exponential growth in the need for advanced machine learning algorithms. Boosting is a powerful family of supervised machine learning techniques that has been successfully utilized in many complex real-world applications.
The essence of boosting consists in iteratively building a classifier by stacking weak learners on top of each other. The most powerful weak learner is trained first: it learns from a small subset of training examples and produces a good predictor.
Each next weak learner is trained on the residuals (errors) made by previous classifiers and tries to correct them. By iteratively building classifiers on top of each other starting from weak ones, we obtain an ensemble (a strong overall classifier) that has high accuracy on test data.
Brownboost is one of the gradient boosting algorithms alongside Adaboost, Stochastic Gradient Descent (SGD), LGBM, and so on. However, unlike the other popular algorithms, Brownboost does not rely on the gradient descent optimization technique but on a very different formulation that makes it more efficient to compute and more versatile to use in practice.
If you’re using machine learning to figure out how best to present your content to site visitors, you might be overlooking a powerful tool: the Brownboost gradient boosting algorithm. It helps you identify which elements of your site are most important and allows you to increase the prominence of those elements.
Naturally, this is hugely useful for e-commerce sites that want to drive their sales. But many brands use it in other ways as well. For instance, political action groups have used it to help make change happen on a national level. And fashion magazines have used it to help readers get a better idea of how clothing will look on them before they buy anything.
Brownboost is more effective and robust for noisy datasets than any other boosting classification algorithms
The formula for Brownboost is:
g(x) = β1 g(x) + β2 x + β3 g(x)^2
Now let’s see Brownboost implementation using python, using Intel DAAL:
#Training set:
from daal.algorithms.brownboost import prediction, training
from daal.algorithms import classifier
trainAlg = training.Batch()
trainAlg.input.set(classifier.training.data, X_train)
trainAlg.input.set(classifier.training.labels, y_train)
trainAlg.parameter. maxIterations = 1000
trainingResult = trainAlg.compute()
Python (Intel DAAL), BrownBoost (prediction):
#Prediction set:
predictAlg = prediction.Batch()
predictAlg.input.setTable(classifier.prediction.data, X_test)
predictAlg.input.setModel(classifier.prediction.model, trainingResult.get(classifier.training.model))
predictAlg.compute()
predictionResult = predictAlg.getResult()
test_predictions = predictionResult.get(classifier.prediction.prediction)
More implementation can be found here.
Boosting is also known as additive boosting. This is because it uses and adds many individual signals together to form one large, composite signal for output or classification of the data. The BrownBoost model was created by Yoav Freund at JPL which is a combination of Gaussians and decision stumps. Know about must known other 8 boosting algorithm at MLDots.
