1

Topic: Re: Gradient boost to whom generally it is necessary?

Hello, Tyomchik, you wrote: Those> In what of its advantage? As far as I know, it is one of the most popular and powerful algorithms of a regression / classifications at present. If to look at competitions on Kaggle a decision top very often use xgboost (extreme gradient boosting) in this or that type. From advantages: * High performance * Big kol-in parameters for tuning (everyones , a cross connect  etc.) * the Autochoice of features (i.e. it is possible to transfer to an input  with big kol-vom variables and during training algorithm itself selects the most significant variables) * Hands-off processing of empty values * Still plus in my judgement -  models (to understand as easier works the decision tree than to interpret coefficients of any neural network) As a matter of fact gradient boost is a dial-up of more simple decision trees and at the expense of their combination is reached gain (boosting) results. It is curious that sometimes combining a little gradient boost trees in ensemble, it is possible to refine result of model even more.

2

Re: Re: Gradient boost to whom generally it is necessary?

Hello, Tyomchik, you wrote: whether Those> the gradient, a path, the price Can  forecast the weather? I would tell that it is a philosophical question which can be applied to any method/model. And whether the neural network can predict the price? And whether the linear regression can predict the price? The general answer - if the phenomenon which we  to predict, has objective laws of development and if the data which we  for a prediction, are relevant and describe pacing factors which influence the phenomenon the model (including gradient boost) can predict such phenomenon.  to practice gradient boost proves to be on the average, as more powerful and productive standalone model for a prediction, than remaining (regressions, decision trees and ). I wrote standalone because if to look at the same Kaggle it is possible to see that decisions of winners are a combination of different models more often. Often any difficult architecture of model (for example), yields hardly the best result than usage of one xgboost in a forehead. And still I would distinguish a prediction as prediction (i.e. something like classification or a regression on a dial-up input to steams) and forecasting (i.e. predictions of the future value time series on last). It seems to me that gradient boost approaches for prediction, than for forecasting more. Those> the Gradient as though hints at local extrema, and after all it angrily. How the gradient  bypasses this restriction? Yes, there under a cowl the method of graded-index descent for training is used. There are many techniques - for example, cross validation, random subsampling, to set different learning rate, whether besides to use ensembles of models etc. Those> the gradient  reinforcement learning Can? Looking whether what consider under "can". As I understand, reinforcement learning is as a matter of fact model adjustment under the new data. Nobody hinders to retrain model anew when fresher data arrives. If to look, can be even there are methods to update coefficients  models by means of a new portion of the data, but anew to train model in me it will seems easier.

3

Re: Re: Gradient boost to whom generally it is necessary?

Hello, Tyomchik, you wrote: I Will add Jeffrey Tyo> the Gradient as though it hints at local extrema, and after all angrily. Machine training also does not assume search of a global extremum. Almost always "the retrained model" becomes it.

4

Re: Re: Gradient boost to whom generally it is necessary?

Hello, Tyomchik, you wrote: Those> Hello, Jeffrey, you wrote: whether Those> the gradient, a path, the price Can  forecast the weather? People from Yandex state that their weather works on Matrix Net which is very close to that they expose on audience as the implementation graded-index  (catboost) About a price prediction - with time series it too works not worse many other approaches, there is a nuance that not so is able to see trends, sometimes for this purpose interpose  from a trend received in another way (the linear regression, as a simple example) as one of features on an input. Whether those> the gradient  reinforcement learning Can? Did not hear, that it there applied.

5

Re: Re: Gradient boost to whom generally it is necessary?

Hello, Jeffrey, you wrote: whether Those>> the gradient  reinforcement learning Can? Looking whether what consider under "can". As I understand, reinforcement learning is as a matter of fact model adjustment under the new data. Nobody hinders to retrain model anew when fresher data arrives. If to look, can be even there are methods to update coefficients  models by means of a new portion of the data, but anew to train model in me it will seems easier. No, reinforcement learning is the code model studies without knowing initially, for example, playing game, or the robot,  about hindrances (those place model in the environment and put to it prizes and punishments for actions,  consists in maximization of the received prize, for example, a maximum of points in game) Doobuchenie on the new data is about another

6

Re: Re: Gradient boost to whom generally it is necessary?

Hello, Tyomchik, you wrote:> I would tell that it is a philosophical question which can be applied to any method/model. And whether the neural network can predict the price? And whether the linear regression can predict the price? Those> I so understood that was not present, cannot. This piece allows  some qualifiers in one, more exact, so? I.e. in the theory it is possible to feed 2  in a gradient  and to receive more exact model? No, can, if such regularities are in learning samplings and its capacity is proportional to complexity of the predicted phenomenon. It is impossible to take two neural networks and to receive the best final algorithm. But it is possible to take one neural network and everyone subsequent to train, so it compensated errors of the previous. Idea  in it, instead of in simple join of several qualifiers. > The general answer - if the phenomenon which we  to predict, has objective laws of development and if the data which we  for a prediction, are relevant and describe pacing factors which influence the phenomenon the model (including gradient boost) can predict such phenomenon. Those> Talk is cheap. Whether can gradient boost predict price USD/EUR for the next day? Look, I can repeat Jeffrey words, or a question on a question: and on what learning sampling?> Yes, there under a cowl the method of graded-index descent for training is used. There are many techniques - for example, cross validation, random subsampling, to set different learning rate, besides to use ensembles of models etc. Those> I.e. from a box cannot struggle, so? It from a box is not required to it. The method guarantees that the magnification of number of basic qualifiers increases its accuracy. All these technicians - optimization called for smaller number of steps to construct more exact model. And partly to compensate an inaccuracy of the selected loss function. These methods generally are applicable to all methods of machine training. Whether those>>> the gradient  reinforcement learning Can?>> Looking whether what consider under "can". As I understand, reiforcement learningn is as a matter of fact model adjustment under the new data. Nobody hinders to retrain model anew when fresher data arrives. If to look, can be even there are methods to update coefficients  models by means of a new portion of the data, but anew to train model in me it will seems easier. Those> I not about conversion training anew, and about  models to a changing surrounding, aspiring to minimize an error. It is possible to add final algorithm in the portions of qualifiers expressing new precedents, but the error imported by prior qualifiers will collect and to grow computing complexity. Therefore it is possible, but it is not practical.