This paper presents a framework in which many structural credit risk models can be made hybrid by randomizing the default trigger, while keeping the capital structure intact. This produces random recovery rates negatively correlated with the default probability. The approach is implemented on a firm-by-firm basis using maximum likelihood and the unscented Kalman filter (UKF) on each of the 225 companies of the CDX NA IG and HY indices using weekly CDS data from December 2007 to January 2012. Adding the surprise element and the time-varying distribution of recovery rates has a large impact on credit spreads as it modifies both the level and shape of the curves. When a bond portfolio is considered, the presence of dependence among firm leverage ratios and between default probabilities and recovery rates produces clusters of defaults with low recovery rates. It has a major impact on standard risk measures such as Value-at-Risk and conditional tail expectation.