We introduce a novel ensemble learning approach which combines random partitions models through Potts clustering with a non-parametric predictor such as shallow feedforward neural networks (S-SPNNR). Neural network are known as universal approximators, and are very well suited to explore others learning methods. We combine them with Potts clustering models to create a bagging-like learning framework where several estimates from each random partition are aggregated into one prediction. Our approach carries out the balance between overfitting and model stability in presence of small datasets with high dimensional features. We found that S-SPNNR is really effective in \emph{multivariate multiple regression} task and present more predictive power than Multi-layer feedforward neural network and the Multi-layer Multi-target Regression (MMR) model given some datasets from the Mulan Multi-label learning project.