In this study we introduce a general definition of an equilibrium concept (called "strong equilibrium") for both discrete and continuous time dynamic games and under varying (symmetrical and asymmetrical) modes of play. The underlying system is stochastic, with structural and modal uncertainties determined by a finite state jump process. The new equilibrium concept encompasses both the feedback Nash and feedback Stackelberg solution concepts for the special cases of deterministic discrete-time games with symmetrical and asymmetrical modes of play, respectively, and it also provides a convenient framework for the introduction of a feedback Stackelberg solution concept in deterministic differential games. For the general class of stochastic nonzero-sum games with structural and modal uncertainties, and under the feedback closed-loop information, we obtain the optimality conditions in both discrete and continuous time. Certain special cases are also studied, and the intrinsic relationship between information patterns and possible definitions of value in nonzero-sum differential games is clarified.