Today when we construct a classifier we are limited to learning machines and data transformation methods that are already known. Typically, we learn few or more learning machines and choose the best one, or additionally, we test some data transformations in cooperation with classifiers (or ensemble of classifiers). This article presents how to construct a learning machine on-the-fly (during learning), basing on the experience from learning of some base machines. This can be seen as a kind of advanced meta-level learning in which positive experience from learning is extracted from the machines and composes new elements in the construction of new machine configurations. This is much more sophisticated than just selecting a transformation machine and a classification machine.