As computing systems continue to advance, there is a push towards autonomous agents as team members alongside humans to improve capabilities of existing human-operated systems in many real-world applications. We develop an artificial intelligence framework for training autonomous agents to effectively collaborate with humans, capturing aspects of intelligence that produce collaborative teaming, and providing the opportunity to quantitatively investigate the effects of heterogeneity on task allocation in multi-agent systems.