The existing technologies that update geostatistically simulated models of mineral deposits cannot self-learn from incoming new information generated in operating mines and do not account for high-order spatial statistics. This work proposes a novel self-learning artificial intelligence algorithm that learns from incoming new information and accounts for high-order spatial statistics, in order to update the geostatistically simulated models of mineral deposits in real-time. The proposed algorithm uses deep policy gradient reinforcement learning with an actor and a critic agent. The actor and critic agents visit each grid node of the geostatistically simulated model sequentially in a random path, interact with an environment that generates states, and feeds the states to the actor and critic agents that respectively predict and evaluate the updated property of the grid node. The data is stored in a replay memory, which is sampled at regular intervals to train the agents. The trained agents are then used for further rounds of self-learning. An application of the proposed algorithm at a copper mining operation with incoming drilling machine sensor data (collected spatially), and processing mill sensor data (collected over time), demonstrates its applied aspects in updating the geostatistically simulated models of copper grades of the mineral deposit in real-time, while also reproducing spatial patterns and high-order spatial statistics.