In demand-response programs, aggregators balance the needs of generation companies and end-users. This work proposes a two-phase framework that shaves the aggregated peak loads while maintaining the desired comfort level for users. In the first phase, the users determine their planned consumption. For the second phase, we develop a bilevel model with mixed-integer variables and reformulate it as a single-level model. We propose an exact centralized algorithm and a decentralized heuristic. Our computational results show that the heuristic gives small optimality gaps and is much faster than the centralized approach.