This talk will show some of the recent advances we are doing at Microsoft Research towards agents capable of modeling complex environments and human behaviors, which is a key goal of artificial intelligence research. My team focuses on applications of AI in video games, as this has the potential of empowering game developers to realize new creative visions. The talk is split in two parts. In the first one, I will focus on diffusion models as generative models of human behavior. Previously shown to have impressive image generation capabilities, I present insights that unlock applications to imitation learning for sequential decision making. In the second part of the talk, I will discuss a recent project extending modern language modeling with new modalities to build a world and action model of an Xbox game.