Remaining committed to a joint goal in the face of many enticing alternatives is challenging. Doing so while cooperating with others under uncertainty is even more so. Despite this, agents can successfully and robustly use bootstrapping to con-verge on a joint intention from randomness under the Imagined We framework. We demonstrate the power of this model in a real-time cooperative hunting task. Additionally, we run a suite of model experiments to answer some of the potential challenges to converging that this model could face under imperfect conditions. Specifically, we ask what happens when (1) there are increasingly many equivalent choices? (2) I only have an approximate model of you? and (3) my perception is noisy? We show through a set of model experiments that this framework is robust to all three of these manipulations.

Visual experience involves not only physical features such as color and shape, but also higher-level properties such as animacy and goal-directedness. Perceiving animacy is an inherently dynamic experience, in part because agents’ goal-directed behavior may be frequently in flux—unlike many of their physical properties. How does the visual system maintain and update representations of agents’ animate and goal-directed behavior over time and motion? The present study explored this question in the context of a particularly salient form of perceived animacy: chasing, in which one shape (the “wolf”) pursues another shape (the “sheep”). Here the participants themselves controlled the movement of the sheep, and the perception of chasing was assessed in terms of their ability to avoid being caught by the wolf—which looked identical to many moving distractors, and so could be identified only by its motion. The wolf’s pursuit was frequently interrupted by periods in which it was static, jiggling in place, or moving randomly (amidst distractors that behaved similarly). Only the latter condition greatly impaired the detection of chasing—and only when the random motion was grouped into temporally extended chunks. These results reveal (1) how the detection of chasing is determined by the character and temporal grouping (rather than just the brute amount) of “pursuit” over time; and (2) how these temporal dynamics can lead the visual system to either construct or actively reject interpretations of chasing.

Although our world is hierarchically organized, the perception, attention, and memory of hierarchical structures remain largely unknown. The current study shows how a hierarchical motion representation enhances the inference of an object’s position in a dynamic display. The motion hierarchy is formed as an acyclic tree in which each node represents a distinctive motion component. Each individual object is instantiated as a node in the tree. In a position inference task, participants were asked to infer the position of a target object, given how it moved jointly with other objects. The results showed that the inference is supported by the context formed by nontarget objects. More importantly, this contextual effect is (a) structured, with stronger support from objects forming a hierarchical tree than from those moving independently; (b) degreed, with stronger support from objects closer to the target in the motion tree; and (c) directed, with stronger support from the target’s ancestor nodes than from its descendent nodes. Computational modeling results further indicated that the contextual effect cannot be explained by correlated and contingent movements without an explicit causal representation of the motion hierarchy. Together, these studies suggest that human vision is a type of intelligence, which sees what are in the dynamic displays by recovering why and how they are generated.

Human vision supports social perception by efficiently detecting agents and extracting rich information about their actions, goals, and intentions. Here we explore the cognitive architecture of perceived animacy by constructing Bayesian models that integrate domain-specific hypotheses of social agency with domain-general cognitive constraints on sensory, memory, and attentional processing. Our model posits that perceived animacy combines a bottom-up, feature-based, parallel search for goal-directed movements with a top-down selection process for intent inference. The interaction of these architecturally distinct processes makes perceived animacy fast, flexible, and yet cognitively efficient. In the context of chasing, in which a predator (the “wolf”) pursues a prey (the “sheep”), our model addresses the computational challenge of identifying target agents among varying numbers of distractor objects, despite a quadratic increase in the number of possible interactions as more objects appear in a scene. By comparing modeling results with human psychophysics in several studies, we show that the effectiveness and efficiency of human perceived animacy can be explained by a Bayesian ideal observer model with realistic cognitive constraints. These results provide an understanding of perceived animacy at the algorithmic level—how it is achieved by cognitive mechanisms such as attention and working memory, and how it can be integrated with higher-level reasoning about social agency.