This paper develops an efficient hybrid algorithm to solve the credit scoring problem. We use statistical mathematical programming to develop new classification models for the discriminant analysis problems. The novelty of our approach can be summarized by the following: the combination of two objective functions, the first minimizing the sum of misclassified points' distances using linear programming and the second minimizing the number of misclassified points using an efficient variable neighborhood search heuristic based on jackknife resampling technique to improve the classification performance. Our proposed scoring systems are often just as accurate as the most powerful black-box machine learning models, but transparent and highly interpretable. The obtained results prove the effectiveness of the proposed approach on real benchmark datasets.