The crew rostering problem (CRP) for pilots is a complex crew scheduling task assigning pairings, or sequences of flights starting and ending at the same airport, to pilots to create a monthly schedule. In this paper, we propose an innovative solution method for the CRP that uses a windowing approach. First, using a combination of machine learning (ML) and combinatorial optimisation (CO), we quickly generate an initial solution. The solution is obtained with a sequential assignment procedure (seqAsg) based on a neural network trained by an evolutionary algorithm. Then, this initial solution is reoptimized using a branch-and-price algorithm that relies on a windowing scheme to quickly obtain a CRP solution. This windowing method consists of decomposing the optimization horizon into several overlapping windows, and then optimizing each one sequentially. Although windowing has been successfully used in other airline applications, it had never been implemented for the CRP, due to its large number of horizontal constraints involving the whole planning horizon. We test our approach on two large real-world instances, and show that our method is over ten times faster than the state-of-the-art branch-and-price CRP solver GENCOL while providing solutions on average less than 1% away from optimality. We show that our windowing approach greatly benefits from being initialized with good-quality ML-based solutions. This is because the initial solution provides reliable information on the following windows, allowing the solver to better optimize the current one. For this reason, this approach outperforms other naive heuristics, including stand-alone ML or windowing.