In this work, we propose a non-intrusive and training free method to detect behind-the-meter (BTM) electric vehicle (EV) charging events from the data measured by advanced metering infrastructure (AMI) such as smart meters. By leveraging the contextual information of EV charging, we formulate a mixed-integer convex quadratic program (MICQP) to detect EV charging events from customers' daily meter data. No labelled training data or hyperparameter tuning are required, and the MICQP can be efficiently solved. By collecting information about the start time, the charging duration, and the power level of each detected charging event, we infer customers' charging patterns in terms of probabilities of charging profiles through a data-driven approach using one year's meter data. In a numerical case study, we use the proposed approach to extract EV charging events from a test dataset of customers' meter data, and we demonstrate that similar detection accuracy is achieved as that of other learning-based approaches which use high-solution meter data. Finally, impacts of EV charging on the IEEE-8500 test feeder are presented in the case study by using the inferred charging patterns.