How imagining can set us free
Imagining is the source of our perception of free will, and imagining more can expand our freedom.
Imagining is the root of our freedom. Or so I argue in this essay, which aims to describe the neural basis of imagination and its role in free will perceptions. First, I review imagination on all three of Marr’s levels of analysis: its computational function, its algorithmic structure, and its neural implementation (Marr, 1982). Then, I argue that the capacity to imagine alternative possibilities is essential to perceiving oneself as acting freely. I also show that the imagination is not free and unconstrained, but has systematic constraints, and these limit our ability to act volitionally and choose among possibilities. Further, expanding the imagination results in a greater perception of free will. By imagining we can make ourselves even more free.
1. What is Imagination?
The term “imagination” presents a challenge for researchers, since it is used colloquially in a variety of ways, its meaning is the subject of intense debates in philosophy and other fields, and “imagination” does not have an agreed-upon formalized definition in mathematics, computer science, or cognitive science. However, a minimal shared concept of imagination allows us gloss over the differences between the many sub-types of imagination, from mental imagery of visual scenes to propositional supposing, and focus on the commonalities.
Specifically, imagination is mental simulation: the ability to simulate non-occurrent possibilities, representing something in the mind without aiming to capture things as they actually are in the moment. In other words, imagination is to “represent without aiming at things as they actually, presently, and subjectively are” (Liao and Gendler, 2011). Thus, imagination can be understood as a form of “attention to possibilities,” where potential realities are projected, simulated, and operated upon internally (Williamson 2016, 4). In computational terms, imagination refers to a system’s processing or manipulation of information that is not directly present to the system’s sensors (Marques, 2009). This ‘imaginative’ processing occurs offline (when the agent is not receiving new sensory data or is not connected to its environment), covertly (without immediate consequences for the agent’s actions), and/or internally (occurring within the agent’s own latent models). Generally, the defining feature of imagination is the simulation of non-occurrent sensory states, making use of internal representations or mental models.
The human capability to imagine is vital to a wide range of cognitive processes, including memory, predicting the future, conjuring alternate worlds, simulating and empathizing with other minds, spatial navigation, and inventing novel combinations of images and objects (Mullaly & Maguire, 2014). As we would expect of a process with so many diverse roles, imagination varies on several dimensions. Specifically, imaginings can be voluntary (e.g. creative generation) or involuntary (e.g. daydreaming). Some even apply the dual-process framework to imagination, dividing imaginative processes into the (1) unconscious, uncontrolled, spontaneous, non-volitional, and effortless and (2) the conscious, controlled, volitional, and effortful (Stuart, 2021). Further, imagination can be sensory, as in mental imagery, or cognitive and non-sensory, as in imagining yourself with alternative beliefs or traits (Dokic, & Arcangeli, 2014). This sensory aspect can implicate any modality, from vision and touch to smell and sound. Finally, the complexity of the imagined object can vary dramatically. Imagination often involves representing a complete situation, “a configuration of objects, properties, and relations” rather than a single isolated object (Berto, 2018). It can even involve constructing an entire imaginary world. This paper will touch on imagination in all its aspects but will focus especially on the voluntary generation and manipulation of imagined possibilities.
2. The Computational Role of Imagination
At the computational level, we ask what imagination does, and why: the role imagination plays in the human information system and its adaptive function in our environment (Bechtel & Shagrir, 2013). As stated above, imagination’s primary role is world-simulation: it generates a model of the world or a part of the world and simulates it in internal experience. Imagination takes sensory data, memories, and our implicit and explicit models of the world as input, and it outputs an imagining – most often, a consciously experienced internal simulation.
Imagination has myriad adaptive functions. Critically and perhaps most obviously, imagining is crucial for decision-making. It allows us to simulate the consequences of actions, allowing our imaginings to ‘die for us’ so that we do not have to make choices by costly trial-and-error alone (Kielak, 2019). Further, the imagination can facilitate escaping local maxima in a decision environment: situations where no single choice could put you in a better position, but a series of choices (potentially through a long ‘desert’ of low reward) could improve your situation significantly. If we were unable to imagine the high-value oasis at the end of the low-reward desert, it would be far more difficult to escape these sub-optimal, least-bad situations. Gaesser (2013) also shows that imagination has a crucial role in enabling empathy and social cognition, supporting theory of mind, and in encouraging prosocial behavior, where more vivid and detailed imaginings of a person more effectively promoted altruistic actions toward them. Earlier in human evolution, imagination was likely crucial to creating novel tools and traps, mentally planning attack strategies, and improving tribal cohesion with religion, myth, and art (Vyshedskiy, 101). Imagination is therefore vital to human behavior.
More passive modes of imagining, like daydreaming and dreaming, may allow us to draw connections and integrate disparate information, incubate potentially creative ideas, and move experiences to long-term memory (Malinowski & Horton, 2015). Further, they may even serve a ‘defensive activation’ role: dreaming keeps visual areas in the brain active and engaged to prevent them from being reduced or replaced (Eagleman & Vaughn, 2021). Among the many functions of imagination, this paper focuses on two that are especially vital to making choices and perceiving oneself as free: imagination’s role in creativity and modal cognition.
1.2 Imagination and creativity
While imagination is closely connected to creativity, it is a separate process. Creativity is the process of producing ideas, artifacts, or concepts that are both novel and valuable. Imagination is the ability to produce and/or simulate new objects, sensations, or ideas in the mind, and can be understood as both a sub-process within creativity and as a semi-separate capacity that supports creative generation. As the initial generative step in creativity, imagination produces the creative possibilities that are then considered, evaluated, and implemented by other systems. Imagination produces internal representations that will not necessarily be novel or useful (creative), but that can provide a fertile starting point for the creative process. Ellamil et al (2012) finds that the two phases of generation and evaluation involved in creativity implicate distinct neural systems: creative generation recruits primarily medial temporal lobe regions like the hippocampus, while evaluation co-recruits the default mode and executive control networks. Further, these networks are competitive: “the more successfully [participants] were able to engage in creative generation while avoiding evaluative processes, the more they recruited MTL regions associated with creative generation” (p. 6). This neuroscientific evidence supports the hypothesized computational role of imagination in creativity: it generates loose, unrefined ideas to be evaluated, modified, and polished by other cognitive processes. Free choice relies on this relatively divergent, unstructured initial step to produce creative options, which can then be winnowed down and selected from in a convergent process.
1.2 Imagination and modal cognition
Imagination is closely tied to modal cognition – thinking about possibilities. Modal cognition relies on imagination to represent situations and generate potential alternatives. Just as in creativity, imagination is the initial step in modal cognition, as it generates the possibilities for consideration. The possibilities in the generated consideration set can then be partitioned into a more limited set of relevant possibilities, and ordered based on some criteria, like value or probability. Considering the ways a captain could have prevented a ship from sinking, for instance, requires mentally simulating this scenario and varying its features to produce alternative possibilities. If we were unable to generate and represent the alternative possibilities for a given situation, it would be difficult or impossible to see ourselves as free. Section 4 expands on the importance of imagination in modal cognition for free will perceptions.
3. The Algorithm of Imagination: Generative Models
The representational or algorithmic level asks how information is organized, encoded, and processed in the imagination, transforming representations into an imagined output. I argue that generative models serve as the fundamental algorithm of imagination. Imagination uses rules and implicit models of the world learned through perception to generate a limitless variety of possibilities. A generative model estimates the probability distribution of an observed variable given a target variable, in contrast to discriminative models that estimate a target variable’s probability distribution based on observed variables. In other words, a generative model simulates the interactions among unobserved variables that might generate the observed variables. Rather than just creating input-output mappings or categorizing a signal, generative algorithms attempt to figure out how the data was generated to classify it, asking which target category is most likely to have produced the observation. By understanding the methods of generation, these models can also create new data similar to the observed data.
Williams (2020) provides detailed arguments to show that imagination and perception are best described as generative models. Discriminative models are unable to explain top-down effects in perception (where higher-level representations impact processing of early info) or endogenously generated percepts like mental imagery and dreams (which have no clear inputs for classification). Generative models correlate to the widely-accepted predictive processing framework in neuroscience, as they are prolific expectation-generators that allow continuous predictions of incoming sensory information based on estimates of their external causes. The brain likely uses temporal generative models, which use current observations and perceptual history to make inferences and find dependencies in input patterns that appear in timed order, to predict its future sensory stream.
The imagination co-opts this predictive capacity of perception and re-uses its core representational architecture, modifying our implicit, learned representations of the dynamics of the real world to generate imagined worlds with new or altered dynamics. Extensive evidence supports the theory that the brain uses a hierarchical generative model to “minimize prediction error in the cascade of cortical processing,” and higher-level areas can use these generative models to drive lower neural populations into predicted patterns and produce internal perception (Clark, 2013). Thus, the cortex likely implements a generative model to explain, predict, and learn about sensory data, and then cross-applies this model to synthesize rich visual representations without external input.
Treating imagination as a generative model is valuable for a few additional reasons. First, imagination is generally governed by principles of generation: a set of (implicit or explicit) rules that guide our imaginings (Walton, 1990, p. 53). For example, in Harry Potter, “Latin words and wands create magic” is a principle of generation that readers can consistently use to simulate the imagined world. The imagination generates a set of possibilities guided by context-relevant principles, like graphics rendering algorithms that unfold an artificial world procedurally using algorithmic rules. Treating imagination as a generative model also explains imaginative mirroring: our imagination defaults to follow the rules of the real world unless prompted otherwise by principles of generation (Leslie, 1994). If a cup ‘spills’ in an imaginary tea party, the participants will treat the spilled cup as empty, following the normal physics of reality. This occurs because perception involves generative models, using processes we derive from experience to simulate the physical world and predict its behavior. Imagination involves running a generative model ‘alongside’ or ‘on top’ of this internal simulation of reality. Some processes are modified in the imagining, but the ones that are not modified are ‘filled in’ by our default generative model of the real world.
The efficacy of generative models for explaining the imagination is demonstrated by computational models that simulate imagination. Generative models based on artificial neural networks (ANNs) can visualize objects that the network has never seen before, replicating the correctness, coverage, and compositionality of the human imagination (Lee et al, 2008). An ANN can learn the structure of an environment and then simulate or hallucinate it internally, but this process relies on creating an efficient, compressed, thorough, and interpretable model of the world (Ha & Schmidhuber, 2018). For instance, Testolin & Zorzi (2016) show that human perception is analogous to graphical models implemented with generative ANNs, which build high-level representations and extract statistical regularities from the environment in an unsupervised way and use feedback connections to carry-top down expectations. These generative models have psychologically and biologically plausible properties, like unsupervised learning and interactions between feedback and feed-forward activity.
Reichert et al (2013) demonstrates that generative models can explain human internal imagery, showing that the cortical dynamics of spontaneous hallucinations in Charles Bonnett syndrome (CBS) can be simulated and explained by an ANN-based generative model. In CBS, partial blindness results in a deficiency of visual input in early processing stages, resulting in spontaneous activity in the cortex. The authors show that recurrent connections between layers in ANNs are similar to reciprocal synaptic connections between layers in the neural visual processing hierarchy and enable simulating the balance between bottom-up sensory information and top-down internal priors that occurs in the brain. When the trained ANN is given empty or corrupted input, this results in realistic artificial hallucinations that can be strikingly decoupled from the input image. These examples are fascinating demonstrations of the potential of using generative models to facilitate progress in understanding the mechanisms of hallucinations, mental imagery, and perception in the human brain.
Conclusively, the generative model framework offers a fruitful way to understand the imagination. It also suggests the algorithmic components that the imagination involves, which likely correspond to different neural correlates. For instance, the imagination requires a sensory system to collect information about the world and support simulations of it. It also needs a memory system to consolidate these experiences into representations that can be accessed for future imaginings. Then, some sub-system must support compressing a huge number of observations of reality into a generative world model, so that imagination can use this model to create realistic and task-relevant simulations. Finally, there must be some internal workspace that allows the mind to produce, combine, and manipulate imagined objects.
3. The Neural Correlates and Mechanisms of Imagination
Any complete model of imagination must accurately and comprehensively describe how imaginings are produced by complex interactions of neuron assemblies, regions, and networks in the human brain. How do neural circuits create an experienceable representation of an object that is not currently present in the subject’s sensory environment?
3.1 Perception and imagination
Imagination has many parallels with perception. This is unsurprising given our theoretical framework that suggests perception and imagination both involve similar generative models. For instance, research showed that in people with visual disorders, imagination is disabled in the same way as perception – e.g. people with hemispatial neglect cannot imagine things on the neglected side, suggesting that imagery and perception use the same machinery (Koch, 2004, p. 99). Furthermore, more vivid imaginings dilate your pupil more, suggesting that the imagination activates very early perceptual processes (Laeng, 2014). The excitability of the visual cortex also predicts imagery strength (Keogh & Pearson, 2020). Additionally, binocular rivalry experiments conducted by Tartaglia et al (2009) demonstrate that just perceiving something (like an oriented line) can improve your visual sensitivity to that thing, imagining visual content improves your sensitivity to that content. This priming effect indicates that imagination involves processes similar to perception. Finally, the contents of imaginings can be mostly decoded with activity in the early visual cortices like V1 and V2 (Vetter et al, 2014), showing that the representations of imagined objects are partially realized in early sensory areas.
As described in Pearson (2017), top-down imagination functions like a weak version of perception with a “reverse visual hierarchy” (p. 2): imagining begins with an initial conscious choice to create a mental image in the frontal lobe, producing a cascade of activity that runs ‘backwards’ in the brain, retrieving stored info and memories in medial temporal areas, and then finally, sensory and spatial representations of the imagery are created in the parietal and occipital lobes. Additionally, the hippocampus can recruit long-term memories to help give richness and spatial coherence to complex, large imaginings (Buckner, 2010). After all, imagination is closely linked to memory, and more vivid imagery is linked to better performance in visual working memory tasks (Keogh & Pearson, 2014). While perception involves feed-forward information propagating upward from early visual areas, imagination involves a feedback cascade that begins in frontal regions and then recruits memories and visual areas to produce imaginings.
Finally, frontal regions play an executive rule in guiding the imagination, but do not produce the actual imagined content. As the patterns of activity associated with imagination move up from V1 to frontal areas, they become increasingly similar to the neural patterns of perception (Pearson 2017, p. 3). This is likely because the executive control mechanisms and high-level processes involved in triggering imagination are nearly indistinguishable from the ones involved in processing, modeling, and manipulating feed-forward visual information. Attention to perceptual realities and attention to possibilities therefore seem to implicate the same neural mechanisms in frontal-parietal areas.
3.2 Imagination and the Default Mode Network
There is a growing consensus that remembering the past, imagining the future, counterfactual thinking, and simulating possible experiences, all involve similar neural mechanisms in the default mode network (DMN) (Hassabis & Maguire 2017; Mullaly & Maguire, 2014; Pearson, 2019; Addis et al, 2007; Spreng et al, 2009). The DMN is a collection of brain areas often activated during wakeful rest and internal mental activity, and includes the medial prefrontal cortex, the posterior cingulate cortex or precuneus, and the angular gyrus, among other regions (Raichle, 2015). Winlove et al (2018) review 40 neuroimaging studies in a meta-analysis of the correlates of visual imagery, and identified 11 consistently activated regions, finding that the superior parietal lobule was involved in top-down control of imagery, the inferior frontal sulcus semantic processing and working memory, and the frontal eye fields and V1 supported internal visual depictions. Further, Whittingstall et al (2014) show that the posterior cingulate cortex (PCC) is a crucial hub for integrating occipital, parietal, and temporal areas together during visuospatial imagery.
Imagining future events and prospective thinking involved the same generation processes and areas in the right frontopolar cortex and left ventrolateral prefrontal cortex, showing that the episodic memory system is involved in imagining the future and vice versa (Addis & Schacter, 2007). More specifically, future-oriented and counterfactual thinking engages the posterior DMN (pDMN), centered around the posterior cingulate cortex (Xu et al, 2016). Researchers showed this by asking participants in an fMRI scan to make choices about their present situation, and then prospective choices about their future. Their findings demonstrated that people often engage vivid mental imagery in future-oriented thinking, and that this process activates the pDMN while reducing its connectivity with the anterior DMN. This provides a candidate neural process that underlies imaginative generation of possibilities. However, imagination requires not just the DMN, but organized interactions between the DMN, executive control network (ECN), and salience networks to create controlled, meaningful, and actionable imaginings (Gotlieb et al, 2018). The DMN may be essential to generating images, ideas, and possibilities, while other networks allow us to modify, select amongst, and move our attention between them.
3.3 Imagining as binding-by-synchrony
A key cognitive ability that underlies imagination is prefrontal synthesis (PFS), the ability to create novel mental images by combining experienced or remembered objects. The binding-by-synchrony hypothesis claims that this process is performed in the lateral prefrontal cortex (LPFC), which likely acts as an executive controller that synchronizes a network of neuronal ensembles (NEs) that represent familiar objects, synthesizing these objects into a new imaginary experience (Vyshedskiy, 2019). Familiar objects are encoded in the brain by neuronal ensembles, and the sensory component of objects is physically encoded in “the posterior cortical hot zone” (Koch & Tononi, 2016). Remembering or imagining objects requires synchronous resonant activity of the object-encoding neuronal ensembles, and when this synchrony occurs in the posterior cortical hot zone it causes the object to come to consciousness.
Imagining novel things, then, is the processes of synchronizing independent object-NEs through conscious attention. Objects can then be imaginatively modified by desynchronizing parts of an object-NE from the whole (called prefrontal analysis). The LPFC acts as a puppeteer in this process, flexibly synchronizing object-NEs to manufacture an unlimited number of novel mental images (Vyshedskiyp. 99). For example, an ensemble representing Bill Clinton and one representing a lion can be synthesized by synchronizing their firing activity in the same phase, creating a mental image of Clinton holding a lion (Vyshedskiy & Dunn, 2015). Any arbitrary type and number of ensembles can be synchronized in the mental workspace, limited by working memory, experience, and focus. Imagination can be either top-down and intentional, driven by the prefrontal synchronization of lower-level neuronal assemblies, or bottom up and unintentional, when lower-level ensembles synchronize non-volitionally and without a puppeteer, spontaneously producing dreams, hallucinations, or sudden insights and images. Children acquire PFS around 3 to 4 years of age, along with other imaginative abilities like mental rotation, storytelling, and advanced pretend play (Vyshedskiy, 2019, p. 101). While further study is needed, it is plausible that development of PFS is associated with mature modal cognition, advanced creative abilities, and generating more sophisticated imaginings.
Creative thought relies on the ability to manipulate internal representations flexibly in the mental workspace. Schlegel et al (2013) confirm Winlove et al’s finding that 11 regions consistently are activated in imagination, including the occipital cortex, PPC, precuneus, posterior inferior temporal cortex, DLPFC, and frontal eye fields. However, this research also showed that maintenance and manipulation of imagined objects involved separate sub-networks, where maintaining involved a dense network integrated by the MTL and manipulation involved a sparser network with a hub in the precuneus. This supports the hypothesis for separate neural mechanisms for imaginative synthesis (forming and maintaining a mental image or object) and imaginative analysis (applying operations, filters, or decompositions to the imagined objects). Imagination relies on dynamically synchronizing neural assemblies in the mental workspace.
Finally, imagination is an example of type 3 qualia, which is the temporary binding of simple sensory objects (type 1 and 2 qualia) through endogenous attention (Tse, 2017). In daydreaming, mental imagery, and imagination, we can simultaneously experience the contents of the iconic buffer (our current sensory state) in the attentional background, and the contents of the working memory buffer in the foreground (Tse, 2017, p. 17). In contrast, dreams are an example of experiencing type 3 qualia alone, without external inputs or basic sensory. Binding-by-synchrony, and the idea that imaginings are type 3 quales, also explains why imagination only seems to represent an object while your attention is currently focused on it. Unlike perception, in which sensory areas are activated by external inputs, top-down imagination requires the constant, effortful synchronization of neural ensembles to maintain mental objects. When your attention moves, the synchronization collapses, and the imagined object vanishes.
4. Imagination and Free Will Perception
Imagination is fundamental to seeing oneself as a free agent. Here, I do not take a position on the complex and rife debates on free will, compatibilism, and determinism in philosophy. I do not argue that imagination is a literal precursor to free will in any deep metaphysical sense, but rather that it is indispensable to our perception of free will, bracketing away the question of whether this perception is an illusion or not. I support this position with several arguments. The most central argument claims that to represent or see oneself as choosing freely, one must be able to represent alternative possibilities for actions. Representing alternative possibilities requires imagination. Thus, imagination is required for free will perception. Seeing yourself as free requires representing or imagining alternative possibilities for action. Additionally, this implies that the systematic constraints on what possibilities we imagine restricts the choices we can make and limits our sense of free will.
“Free will also includes the creative ability to imagine,” as we can choose to apply our attention to internally generated qualia (Tse, 2013, p. 238). This enables freedom in two ways. First, imagining and mentally ‘playing out’ possible scenarios to form a plan for action is precisely what empowers us to make decisions. By creating an internal virtual reality, it allows us to pre-select and pre-experience actions, an essential part of human decision-making. Second, imagination itself involves choices among internal representations, even if this does not manifest in external actions. The imagination enables freedom in the sense that it supports generating a large number of possibilities with a “high degree of disorder or chance amounting to a kind of ‘freedom’” (Krausz & Bardsley, 2009, p. 133). In this sense, the very manner of representation and the kinds of mental operations involved in imagination are fundamentally intertwined with freedom. While memory and imagination involve similar processes, prospection (simulating the future) is less constrained and subject to ‘reality checks’ than retrospection (Kane et al, 2008, p. 132). Future-looking imagination is vital to volitional action.
4.1 Imagining alternative possibilities and its constraints
Research on modal cognition shows that imagining alternative possibilities is not free and boundless but has important constraints. By default, we only consider a systematically limited subset of the imaginable possibilities. Imagination produces a series of possibilities, and then during decision-making we sample from this distribution of imagined options in an adaptive way, constrained by relevant factors. The set of possibilities we consider is limited systematically by the sampling process (Morris, Phillips, and Cushman 2019). Under the theory of the psychological representation of modality developed by Phillips et al, the set of possibilities we consider is limited by the constraints of probability/normality, physics, and morality (Phillips & Knobe, 2018). For instance, both children and adults under severe time constraints tend to consider immoral options (e.g. stealing or lying) or unlikely and irregular options (e.g. painting polka dots on an airplane) as impossible (Phillips, Morris, & Cushman, 2017). Our perceptions of ourselves as freely acting are systematically limited by the possibilities we can imagine. Although we may be able to choose among possibilities, we do not have complete control over the pool of possibilities that are consciously available to us. Through imagination, we can modify and expand this pool of options, supporting a greater sense of agency and freedom.
We tend to judge agents as free when we can represent alternative possibilities for their action. In a sense, a failure to imagine can preempt free will perceptions, as one cannot choose to act upon a possibility that one does not represent, and one cannot see an option as ‘freely’ chosen if no other possibilities are represented. Indeed, people use judgements of possibility to inform judgements about whether an agent is free (Phillips & Knobe 2018). Generally, if we are able to imagine situations where the action could be different, we judge the agent as free. When participants generated more possibilities, imagining more alternative decisions a ship captain could have made, they were more likely to make the judgement that he was free and not forced (Phillips, Luguri, and Knobe, 2015). Unpublished data from the Dartmouth PhilLab supports this finding, suggesting that as people imaginatively generate more possibilities, these options become less constrained by the norms of probability, normality, morality, and rationality. This may imply that possibilities become more divergent, unconventional, novel, or surprising as the quantity of ideas generated increases. Therefore, imagination is essential to free will judgements, and imagination enhances the sense of freedom by expanding the set of accessible options.
The ability to imaginatively project alternative possibilities may therefore underly individual differences in free will perceptions – if a person imagines many more available options, they see themselves as freer. Simply imagining more possibilities may engender a feeling of more freedom. Developmental research provides strong support for this claim.
Children tend to resist, or fail to generate, impossible and improbable imaginings. Kushnir (2018) also shows that free will beliefs originate in the ability to understand intentional action, inferring when agents are free to do otherwise and when they are constrained. Young children are often unable to imagine alternatives to improbable, irregular, or immoral events, and so tend to see them as impossible. Children’s imaginations are thus surprisingly reality-constrained: children (age 2-8) protest against pretense that contradicts their knowledge of regularity, expecting imaginary things to have ordinary properties (Friedman et al, 2017). Even when pretending, kids expect lions to roar and pigs to oink, and they resist imagining otherwise. Children also protest against pretense that contradicts their knowledge of regularity, expecting imaginary entities to have ordinary properties (Vandervoort and Friedman, 2017).
Furthermore, 82% of the time, children extend fantasy stories with realistic events rather than fantastic events, while adults extend fantasy stories with fantastic events (Weisberg et al, 2013). Young children imagine along ordinary lines even when primed with fantastical contexts, filling in typical and probable causes for fantastical imaginary events (Lane et al, 2016). Children show a strong typicality bias in completing fictional stories, favoring additions to the story that match their regular experiences in reality (Thorburn et al, 2020). This evidence shows that children’s imaginations are limited by typicality, morality, and their understanding of the physical world. This suggests that children have less advanced imaginations than adults—they are using simpler constraints, quick heuristics, and a more basic model of the world to effortlessly generate possibilities.
Most conclusively, an experiment by Flanagan and Kushnir (2019) found that performance on a task that involved generating ideas within an imagined fantasy world was the best predictor of children’s free will judgements: the more fluent the children were in this imagination task, the more likely they were to judge themselves as free. As the authors speculate, “one potential mechanism is a direct pathway from idea generation to judgments of choice and possibility” (p. 5). In my view, the pathway is not completely direct, as existing research indicates that after possibility-generation (imagination) we also evaluate the relevance of possibilities and rank them. However, the initial generation is crucial, and the nature and quantity of generated possibilities has demonstrable impacts on how people think about possibilities, freedom, and choice. Constraints on the imaginative process lead to downstream effects on our choices and our perceptions of freedom.
As children develop, they are able to soften these constraints and imagine more alternatives, and when they do endorse a choice as free rather than forced they often cite imagined alternatives to the scenario as an explanation (Kushnir, 2018). As children develop, the constraints on their imagination relax, leading to less restricted generation of possibilities. Older children are more likely to imagine improbable and physically impossible phenomena (Lane et al, 2016, p. 6). Explicitly prompting children to generate more possibilities leads them to imagine more like older children, producing possibilities less constrained by probability and regularity (Goulding & Friedman, 2020). Cultural contexts mediate this developmental process. For example, American children are more likely than Nepalese and Singaporean children to judge that they are free to act against cultural and moral norms (Chernyak and Kushnir, 2019). This is likely because children in cultures with stronger or more restrictive norms find it harder to generate evaluatively wrong possibilities or see these possibilities as relevant. As free will judgements depend on representing alternative possibilities, these children see themselves as less free to pursue possibilities that violate moral or social norms. When imaginative flexibility increases with age and experience, we can represent a wider range of possibilities for action and cultivate a broader conception of our own free will.
Viewing imagination as a generative model furthers fruitful interpretations of this research. When imagining, young children apply a generative model with the same rules of generation used in perception to produce expectations about reality. This early imagination may use simple constraints and empirical heuristics to allow effortless and rapid generation of possibilities. For instance, if the child regularly encounters an event, they are more likely to imagine this event (Goulding & Friedman, 2020). In later development and adulthood, the imagination generates possibilities in a more deliberative and analytical way. This suggests a dual process model of imagination (Stuart, 2019). Children may use a more uncontrolled, effortless, and unconscious imagination based on simple heuristics and experience-derived rules of generation. In contrast, adults use a more controlled, effortful and conscious imagination that generates possibilities based on more sophisticated and principled rules.
Conclusively, the default representation of imagination results in resistance to imagining possibilities that violate physical laws, irregular or unlikely possibilities, and immoral or evaluatively bad possibilities. Experimental results reveal that the imaginations of young children are limited by precisely these constraints. Adults are able to deliberately generate more and less constrained possibilities. With very limited time or significant cognitive pressure, adult imaginations may resemble the imaginations of young children. However, just as adults can treat immoral possibilities as irrelevant, imaginative resistance shows that the adult imagination is inhibited against immoral possibilities. Finally, individual differences in openness to experience, creativity, and imaginative ability may predict some of the variation in judgements of possibility and freedom. For instance, people who are naturally more imaginative (and thus generate more possibilities) will be more likely to judge agents as free rather than forced.
4.2 Imagination and existential freedom
To imagine enables free consciousness, because it allows you to get beyond the real, developing a broader perspective on the world by imagining beyond it and escaping from it to some degree (Turner, 1968). The imagination is a radical break from the surrounding world, a negation of present circumstances, making-present something that is not there by making-absent what is ‘really’ there (Sartre, 2010). As Husserl writes, free phantasy (imagination) allows one to see more possibilities and attain a wider-ranging knowledge of experience (Husserl 2012, §70). Our sensory lives give us access only to a small selection of possible experiences, and thus we need imagination to explore the immensity of conceivable configurations of experiences, choices, and perceptions. The imagination can supplement our experience, and in turn we can use experiences to pollinate the imagination and enable coherent world-simulations.
This sheds light on an important debate in existentialism. Sartre claimed that human consciousness is able to transcend any given situation by pursuing the possibilities we imagine. He thought that we are radically, infinitely free to choose our possibilities (2015, p. 112). We can define our identity with negation, through the set of possibilities we reject. In contrast, Heidegger had a much limited view of human freedom. He thought that our world, and our set of available possibilities, is defined by social structures that are out of our control. The They (Das Man), or basically our social context, limits the set of possibilities we are capable of considering (Heidegger, 2010, §27). Certain possibilities will never be available to us, not just because we cannot factually achieve them, but because we cannot even conceive them. For him, freedom is the process of personally appropriating one of these socially given options, and authenticity consists in becoming one’s possibilities. Both perspectives are true to some extent: we have immense freedom to generate and choose amongst our imagined possibilities, but these possibilities are also limited by our social context and our cognitive abilities.
Furthermore, imagination is essential to the process of flexible identity-construction: developing a sense of oneself, seeing aspects of one’s identity, and moving towards a hopeful future self (Gotlieb et al, 2018). A sometimes forgotten aspect of free will is that to perceive yourself as acting freely, you must already perceive yourself as an agent. By allowing us to picture ourselves in the future, counterfactually vary features of our identity, and imagine a constant thread of who we are through our lifetimes, imagination supports the cross-temporal identity that is essential to seeing oneself as a free agent. Finally, imagination may enable a kind of existential creativity: an individual’s attitude of exploring life’s possibilities and experimenting with life-plans and versions of herself (Loi & Plas, 2020). Imagination “permits one to take the paths of many varied and opposed ways of thinking,” creating “the excess that gives to the free spirit the dangerous privilege of living for experiments and of being allowed to offer itself to adventure” (Nietzsche, 1996, #4). Imagining alternative future tracks and ways of life, and narratively constructing an identity that persists through these disparate pasts and possibilities, is crucial to a person’s ability to forge a meaningful life.
4.3 Mental disorders and imagination
It may even be the case that certain disorders increase free will perceptions by amplifying imaginative abilities, facilitating unexpected connections and more unpredictable mental pathways. While at its extreme this can lead to psychosis, it also amplifies the exploratory processes essential to generating alternative possibilities for choice. Both bipolar and ADHD are associated with significantly higher openness to experience (Van Dijk et al, 2017; Quilty, 2009). Openness is linked to trait creativity, is even used as a measure of creativity, and is associated with higher volume in brain regions that inhibit control and reduce constraint (Li, 2015). The highly-open personalities of patients with disorders like ADHD and bipolar may facilitate highly associative, fluent, and originative brainstorming of possibilities.
Looser cognitive limitations, weakened top-down control, and more unconstrained thinking may also potentiate imagination and free will perceptions in certain disorders. Creative tasks benefit from a state of hypofrontality, in which reduced PFC activation enables more spontaneous, bottom-up thought patterns (Ramey & Chrysikou, 2014). Bipolar patients exhibit disruptions in the frontoparietal control network which reduce top-down constraints, and mania and involves hypofrontality, a “significant attenuation of task-related activation of right lateral orbitofrontal function” that results in disinhibition and distractibility (Altshuler et al, 2005). Further, individuals with ADHD have impaired executive inhibition, which reduces the person’s ability to suppress creative but unconventional ideas – and ADHD patients exhibit improved performance on tasks like the Unusual Uses Test (White & Shaw, 2006). People with mental disorders associated with impulsivity, including bipolar, Tourette’s, and ADHD, often have more fluent and vivid imaginations, and are biased toward generation over evaluation (Ellamil, 2012). Therefore, the disinhibited imaginations of people with certain disorders may allow them to brainstorm more actionable possibilities. However, this may also explain the pathological aspect of these disorders: it may be harder to select appropriate actions given a far larger pool of possibilities, including irrelevant, unfit, or harmful ones. Limiting the number of projected possibilities is therefore likely adaptive – imaginative constraints are the bonds that set us free.
Finally, aphantasia is a well-documented disorder that involves the absence of a ‘mind’s eye,’ where otherwise normal, healthy individuals report a complete lack of visual experience when they attempt to imagine something (Keogh & Pearson, 2018). There are also degrees of aphantasia – it can involve an impaired imagination with reduced strength, control, or vividness, rather than a complete lack of imaginings. At opposite end of the imaginative spectrum is hyperphantasia, an exceptional strength, control, and vividness of imagination. One of the testable predictions of my theory is that if imagination does indeed play a crucial role in the perception of freedom, then there will be significant differences in the free will perceptions and judgements of aphantasic and hyperphantasic individuals. Specifically, higher scores on tests of imaginative ability will correlate with stronger perceptions of personal free will.
4.4 Imagining to increase agency
Imagination can also be thought of as a trainable ability, which can be practiced to improve self-efficacy, self-control, and agency. For example, individuals who are more skilled at counterfactual thinking are more easily able to self-restrain and delay gratification in the service of later reward (Mischel et al, 2011). Imagination breaks our constant present-orientation and task-focus, moving us into more flexible, open, and future-oriented mode of internal reflection that is crucial for long-term decision-making (Gotlieb et al, 2018). Being able to imagine the future allows us to resist current temptations and focus on long-term goals. Imagination supports the perception of free will, and in turn, an increased belief in free will changes the way persons imagine their futures – promoting a focus on personal agency and interpersonal connection in prospective imaginings (Nagelmann, 2019). Imaginative skill thus promotes a sense of freedom.
Finally, social norms are one of the most powerful sources of constraints on the imagination. Just as a child who is only given certain props and stories will naturally shape their pretend play around these objects and narratives, adults mold their imaginings by their socio-cultural environment. The existing structures of the world can congeal in our minds, ossifying until they seem almost unshakeable, and are not even realized as constraints. So perhaps, for example, “it is easier to imagine the end of the world than it is to imagine the end of capitalism” (Fisher, 2009, p. 8). Overcoming the constraints of collective imagination, the rigid social orthodoxies that tell us what is and is not possible, can have transformative force – liberating entire populations to act more freely because they realize their actions are not as constrained as they thought (Dey & Mason, 2018). Disruptive truth-telling, courageous speech, and utopian imagination in the style of MLK, Ghandi, or Mandela can therefore literally enhance our sense of free will by increasing the number of possibilities for action and boosting their cognitive availability.
Conclusion
The ability to imagine is a core component of consciousness. “Imagination is a specifically human form of conscious activity” (Vygotsky, 1967) which distinguishes us from other organisms, supporting our ability to generate complex mental representations and reconfigure them into innumerable combinations. Imagination dominates consciousness both in duration and degree. The average person spends between 30% and 50% of their waking time daydreaming (McMillan et al, 2013), and even more conscious time is occupied engaged in prefrontal synthesis, dreaming, operating in the mental workspace, or simulating the future. Further, imagination represents a peak of consciousness, where endogenous attention is actively and volitionally applied to synchronize lower-level neural ensembles into complex internal simulations. Research into the neural correlates of consciousness must therefore treat the imagination as a central question.
In this paper, I have reviewed the neural basis of the imagination, from its computational role and diverse functions in supporting creativity, decision-making, and modal cognition, to its algorithmic structure in generative models, to its implementation in the brain through reversed perceptual processes, the default mode network, and binding-by-synchrony. Further, I have argued that imagination is central to the perception of free will. Without imagination, free will is almost unimaginable. Without it, we would not be able to represent alternative possibilities, simulate the consequences of our actions, or construct an identity through time and envisage different ways of living and being. Imagination also explains some of the systematic ‘soft’ limitations on our free will, which prevent us from acting on options that we cannot or do not imagine. By building our imaginative capabilities we can transform our personal and societal futures.
Works Cited
Abraham, A. (Ed.). (2020). The Cambridge handbook of the imagination. Cambridge University Press.
Addis, D. R., Wong, A. T., & Schacter, D. L. (2007). Remembering the past and imagining the future: common and distinct neural substrates during event construction and elaboration. Neuropsychologia, 45(7), 1363-1377.
Altshuler, L. L., Bookheimer, S. Y., Townsend, J., Proenza, M. A., Eisenberger, N., Sabb, F., … & Cohen, M. S. (2005). Blunted activation in orbitofrontal cortex during mania: a functional magnetic resonance imaging study. Biological psychiatry, 58(10), 763-769.
Bechtel, W., & Shagrir, O. (2015). The non‐redundant contributions of Marr’s three levels of analysis for explaining information‐processing mechanisms. Topics in Cognitive Science, 7(2), 312-322.
Berto, F. (2021). Taming the runabout imagination ticket. Synthese, 198(8), 2029-2043.
Buckner, R. L. (2010). The role of the hippocampus in prediction and imagination. Annual review of psychology, 61, 27-48.
Chernyak, N., Kang, C., & Kushnir, T. (2019). The cultural roots of free will beliefs: How Singaporean and US Children judge and explain possibilities for action in interpersonal contexts. Developmental psychology, 55(4), 866.
Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and brain sciences, 36(3), 181-204.
Cook, C., & Sobel, D. M. (2011). Children’s beliefs about the fantasy/reality status of hypothesized machines. Developmental Science, 14(1), 1-8.
Dokic, J., & Arcangeli, M. (2014). The heterogeneity of experiential imagination. Open MIND. Frankfurt am Main: MIND Group.
Eagleman, D. M., & Vaughn, D. A. (2021). The defensive activation theory: REM sleep as a mechanism to prevent takeover of the visual cortex. Frontiers in neuroscience, 15.
Ellamil, M., Dobson, C., Beeman, M., & Christoff, K. (2012). Evaluative and generative modes of thought during the creative process. Neuroimage, 59(2), 1783-1794.
Fisher, M. (2009). Capitalist realism: Is there no alternative?. John Hunt Publishing.
Flanagan, Teresa, and Tamar Kushnir (2019). “Individual differences in fluency with idea generation predict children’s beliefs in their own free will.” In CogSci, pp. 1738-1744.
Gaesser, B. (2013). Constructing memory, imagination, and empathy: a cognitive neuroscience perspective. Frontiers in psychology, 3, 576.
Gotlieb, R. J., Hyde, E., Immordino-Yang, M. H., & Kaufman, S. B. (2018). Imagination is the seed of creativity. The Cambridge Handbook of Creativity. New York, NY: Cambridge University press.
Goulding, B. W., & Friedman, O. (2020). Children’s beliefs about possibility differ across dreams, stories, and reality. Child development, 91(6), 1843-1853.
Ha, D., & Schmidhuber, J. (2018). Recurrent world models facilitate policy evolution. arXiv preprint arXiv:1809.01999.
Hassabis, D., Kumaran, D., & Maguire, E. A. (2007). Using imagination to understand the neural basis of episodic memory. Journal of neuroscience, 27(52), 14365-14374.
Heidegger, M. (2010). Being and time. Suny Press.
Hunt, M., & Fenton, M. (2007). Imagery rescripting versus in vivo exposure in the treatment of snake fear. Journal of Behavior Therapy and Experimental Psychiatry, 38(4), 329-344.
Husserl, E. (2012). Ideas: General introduction to pure phenomenology. Routledge.
Kane, J., McGraw, A. P., & Van Boven, L. (2008). Temporally asymmetric constraints on mental simulation: Retrospection is more constrained than prospection. The handbook of imagination and mental simulation, 131-149.
Keogh, R., & Pearson, J. (2014). The sensory strength of voluntary visual imagery predicts visual working memory capacity. Journal of vision, 14(12), 7-7.
Keogh, R., & Pearson, J. (2018). The blind mind: No sensory visual imagery in aphantasia. Cortex, 105, 53-60.
Keogh, R., Bergmann, J., & Pearson, J. (2020). Cortical excitability controls the strength of mental imagery. elife, 9, e50232.
Kielak, K. (2019). Generative Adversarial Imagination for Sample Efficient Deep Reinforcement Learning. arXiv preprint arXiv:1904.13255.
Koch, C., Massimini, M., Boly, M., & Tononi, G. (2016). Neural correlates of consciousness: progress and problems. Nature Reviews Neuroscience, 17(5), 307-321.
Koch, Christof. (2004). The quest for consciousness: a neurobiological approach. Roberts & Co.
Kushnir, T. (2018). The developmental and cultural psychology of free will. Philosophy Compass, 13(11), e12529.
Laeng, B., & Sulutvedt, U. (2014). The eye pupil adjusts to imaginary light. Psychological science, 25(1), 188-197.
Lane, J. D., Ronfard, S., Francioli, S. P., & Harris, P. L. (2016). Children’s imagination and belief: Prone to flights of fancy or grounded in reality?. Cognition, 152, 127-140.
Lee H, Ekanadham C, Ng Ay (2008). Sparse deep belief net model for visual area V2. Advances in Neural Information Processing Systems, 20.
Leslie, A. M. (1994). Pretending and believing: Issues in the theory of ToMM. Cognition, 50(1-3), 211-238.
Liao, S. Y., Strohminger, N., & Sripada, C. S. (2014). Empirically investigating imaginative resistance. British Journal of Aesthetics, 54(3), 339-355.
Liao, Shen-yi and Tamar Gendler (2011). “Imagination.” The Stanford Encyclopedia of Philosophy.
Loi, M., Viganó, E., & van der Plas, L. (2020). The societal and ethical relevance of computational creativity. arXiv preprint arXiv:2007.11973.
Malinowski, J. E., & Horton, C. L. (2015). Metaphor and hyperassociativity: the imagination mechanisms behind emotion assimilation in sleep and dreaming. Frontiers in Psychology, 6, 1132.
Marques, H. G., & Holland, O. (2009). Architectures for functional imagination. Neurocomputing, 72(4-6), 743-759.
Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. San Francisco: W.H. Freeman.
McMillan, R., Kaufman, S. B., & Singer, J. L. (2013). Ode to positive constructive daydreaming. Frontiers in psychology, 4, 626.
Mischel, W., Ayduk, O., Berman, M. G., Casey, B. J., Gotlib, I. H., Jonides, J., et al. (2011). Willpower over the life span: decomposing self-regulation. Soc. Cogn. Affect. Neurosci. 6, 252–256.
Mullally, S. L., & Maguire, E. A. (2014). Memory, imagination, and predicting the future: a common brain mechanism?. The Neuroscientist, 20(3), 220-234.
Nagelmann, Stina. “How do attitudes on free will relate to students’ imagination of the future?.” Bachelor’s thesis, University of Twente, 2019.
Nietzsche, Friedrich. (1996). Human, all too human: A book for free spirits. Cambridge University Press.
Pearson, J. (2019). The human imagination: the cognitive neuroscience of visual mental imagery. Nature Reviews Neuroscience, 20(10), 624-634.
Phillips, J., & Knobe, J. (2018). The psychological representation of modality. Mind & Language, 33(1), 65-94.
Phillips, J., Morris, A., & Cushman, F. (2019). How we know what not to think. Trends in cognitive sciences, 23(12), 1026-1040.
Phillips, Luguri, and Knobe (2015). “Unifying morality’s influence on non-moral judgments: The relevance of alternative possibilities.” Cognition 145, 30-42.
Quilty, L. C., Sellbom, M., Tackett, J. L., & Bagby, R. M. (2009). Personality trait predictors of bipolar disorder symptoms. Psychiatry Research, 169(2), 159-163.
Raichle, M. E. (2015). The brain’s default mode network. Annual review of neuroscience, 38, 433-447.
Ramey, C. H., & Chrysikou, E. G. (2014). “Not in their right mind”: the relation of psychopathology to the quantity and quality of creative thought. Frontiers in psychology, 5, 835.
Reichert, D. P., Series, P., & Storkey, A. J. (2013). Charles Bonnet syndrome: evidence for a generative model in the cortex?. PLoS computational biology, 9(7), e1003134.
Reichert, D. P., Series, P., & Storkey, A. J. (2013). Charles Bonnet syndrome: evidence for a generative model in the cortex?. PLoS computational biology, 9(7), e1003134.
Sack, A. T., van de Ven, V. G., Etschenberg, S., Schatz, D. & Linden, D. E. J. Enhanced vividness of mental imagery as a trait marker of schizophrenia? Schizophr. Bull. 31, 97–104 (2005).
Sartre, J. P. (2010). The imaginary: A phenomenological psychology of the imagination. Routledge.
Sartre, J. P. (2015). Being and nothingness. Central Works of Philosophy: The twentieth century: Moore to Popper, 4, 155.
Schlegel, A., Kohler, P. J., Fogelson, S. V., Alexander, P., Konuthula, D., & Tse, P. U. (2013). Network structure and dynamics of the mental workspace. Proceedings of the National Academy of Sciences, 110(40), 16277-16282.
Shine, J. M. et al. Imagine that: elevated sensory strength of mental imagery in individuals with Parkinson’s disease and visual hallucinations. Proc. R. Soc. B 282, 20142047 (2014).
Shtulman, A., & Phillips, J. (2018). Differentiating “could” from “should”: Developmental changes in modal cognition. Journal of Experimental Child Psychology, 165, 161-182.
Spreng, R. N., Mar, R. A., & Kim, A. S. (2009). The common neural basis of autobiographical memory, prospection, navigation, theory of mind, and the default mode: a quantitative meta-analysis. Journal of cognitive neuroscience, 21(3), 489-510.
Stuart, M. T. (2021). Towards a dual process epistemology of imagination. Synthese, 198(2), 1329-1350.
Tartaglia, E. M., Bamert, L., Mast, F. W., & Herzog, M. H. (2009). Human perceptual learning by mental imagery. Current Biology, 19(24), 2081-2085.
Testolin, A., & Zorzi, M. (2016). Probabilistic models and generative neural networks: Towards an unified framework for modeling normal and impaired neurocognitive functions. Frontiers in Computational Neuroscience, 10, 73.
Thorburn, R., Bowman-Smith, C. K., & Friedman, O. (2020). Likely stories: Young children favor typical over atypical story events. Cognitive Development, 56, 100950.
Tse, P. (2013). The neural basis of free will: Criterial causation. MIT Press.
Turner, R. (1968). Sartre and Ryle on the Imagination. South African Journal of Philosophy, 20-28.
Van de Vondervoort, J. W., & Friedman, O. (2017). Young children protest and correct pretense that contradicts their general knowledge. Cognitive Development, 43, 182-189.
Van de Vondervoort, Julia W., and Ori Friedman,” Young children protest and correct pretense that contradicts their general knowledge,” Cognitive Development 43 (2017): 182-189.
Van Dijk, F. E., Mostert, J., Glennon, J., Onnink, M., Dammers, J., Vasquez, A. A., … & Buitelaar, J. K. (2017). Five factor model personality traits relate to adult attention-deficit/hyperactivity disorder but not to their distinct neurocognitive profiles. Psychiatry research, 258, 255-261.
Vetter, P., Smith, F. W., & Muckli, L. (2014). Decoding sound and imagery content in early visual cortex. Current Biology, 24(11), 1256-1262.
Vygotsky, L. S. (2016). Play and its role in the mental development of the child. International Research in Early Childhood Education, 7(2), 3-25.
Vyshedskiy, A., & Dunn, R. (2015). Mental synthesis involves the synchronization of independent neuronal ensembles. Research Ideas and Outcomes, 1, e7642.
Vyshedskiy, Andrey. “Neuroscience of imagination and implications for human evolution” (2019). Preprint DOI: 10.31234/osf.io/skxwc.
Walton, K. L. (1990). Mimesis as make-believe: On the foundations of the representational arts. Harvard University Press.
Weisberg, D. S., Sobel, D. M., Goodstein, J., & Bloom, P. (2013). Young children are reality-prone when thinking about stories. Journal of Cognition and Culture, 13(3-4), 383-407.
White, H. A., & Shah, P. (2006). Uninhibited imaginations: creativity in adults with attention-deficit/hyperactivity disorder. Personality and individual differences, 40(6), 1121-1131.
Williams, D. (2021). Imaginative constraints and generative models. Australasian Journal of Philosophy, 99(1), 68-82.
Williams, D. (2021). Imaginative constraints and generative models. Australasian Journal of Philosophy, 99(1), 68-82.
Williamson, T. (2016). Knowing by imagining. Knowledge through imagination, 113-23.
Winlove, C. I., Milton, F., Ranson, J., Fulford, J., MacKisack, M., Macpherson, F., & Zeman, A. (2018). The neural correlates of visual imagery: A co-ordinate-based meta-analysis. Cortex, 105, 4-25.
Xu, Xiaoxiao, Hong Yuan, and Xu Lei, “Activation and connectivity within the default mode network contribute independently to future-oriented thought,” Scientific reports 6 (2016): 21001.