Motivated by our collaboration with one of the largest fast-fashion retailers in Europe, we study a two-echelon inventory control problem called the One-Warehouse Multi-Store (OWMS) problem when the demand distribution is unknown. This system has a central warehouse that receives an initial replenishment and distributes its inventory to multiple stores in each time period during a finite horizon. The goal is to minimize the total expected cost which consists of shipment costs, holding costs, lost-sales costs, and end-of-horizon disposal costs. The OWMS system is ubiquitous in supply chain management, yet its optimal policy is notoriously difficult to calculate even under the complete demand distribution case. In this work, we consider the OWMS problem when the demand is censored and its distribution is unknown a priori. The main challenge under the censored demand case is the difficulty in generating unbiased demand estimation. In order to address this, we propose a primal-dual algorithm in which we continuously learn the demand and make inventory control decisions on the fly. Results show that our approach has great theoretical and empirical performances.