Drawing on statistical learning theory, we derive out-of-sample and optimality guarantees about the investment strategy obtained from a regularized portfolio optimization model which attempts to exploit side information about the financial market in order to reach an optimal risk-return tradeoff. This side information might include for instance recent stock returns, volatility indexes, financial news indicators, etc. In particular, we demonstrate that a regularized investment policy that linearly combines this side information in a way that is optimal from the perspective of a random sample set is guaranteed to perform also relatively well (i.e., within a perturbing factor of \(O(1/\sqrt{n})\)) with respect to the unknown distribution that generated this sample set. We also demonstrate that these performance guarantee are lost in a high-dimensional regime where the size of the side information vector is of an order that is comparable to the sample size. We further extend these results to the case where non-linear investment policies are considered using a kernel operator and show that with radial basis function kernels the performance guarantees become insensitive to how much side information is used. Finally, we illustrate our findings with a set of numerical experiments involving financial data for the NASDAQ composite index.