With the ongoing energy transition, electric power and energy systems are becoming increasingly multi-dimensional and complex with higher levels of uncertainty. Obtaining an accurate system model or multiple predictions in such environments is becoming increasingly problematic. Such features are challenging aspects to the classical power system control approaches which are essentially model-based and rely on accurate predictions. Reinforcement learning is promising a data-driven alternative that can efficiently tackle the raising complexity of controlling such systems with no prior system dynamics modeling or predictions. This work proposes a multi-agent deep reinforcement learning-based control framework for optimally solving multi-dimensional power dispatch problems in systems featuring multiple uncertainties. Learned control strategies are based on centralized training decentralized execution to promote an efficient and robust coordination among different dispatchable assets with no communication burden. Experimental results of various typical power dispatch scenarios with significant integration of low-carbon generation demonstrate the effectiveness of such control strategies.