The Juxtaposition Engine: Software with an Ontological Mission

Serendipity and Origins

"When you start juxtaposing things that are not usually brought together, you discover all kinds of things you never expected." (Michael Best, CaSTA, Nov. 14, 2003, UVic).

Michael Best is, of course, referring to that delightful phenomenon of serendipity that makes the joys of research worth the countless hours of solitary effort. The notion of juxtaposition--for Michael Best in the comparison of canonical Shakespearean texts translated into electronic editions--calls to mind a nexus of related ideas originating from the act of juxtaposition: besides serendipity, we have analogy, montage, coincidence, synchronicity, chance, approximation, randomness, and improvisation. All are strategies of artistic production, and counterpoint the meticulous plotting of fictional narratives and prosody, the careful rendering of visual perspective, and the detailed notation of musical transcription into scores. As a community of scholars, we are learning how to use computers to help with our explorations, and generally do so in ways that rely on the machine's predictability, thoroughness, and relentless logic. Michael Best set me to thinking about computers and juxtaposition, chance operations, randomness, and the serendipity of discovery--this was the moment the idea of the juxtaposition-engine was born. Since time is short here, I'll sketch out a few ideas that animate the j-engine as an instrument for generating an aesthetic artifact, or performance, in the tradition of 20th C. humanities. This is by no means a thorough catalogue of effects and influences; rather, a montage of news items related to juxtaposition.

The Resonant Mosaic

In his analysis of the impact of newspapers as a medium, Marshall McLuhan used the term symbolist mosaic to characterize the juxtaposition of elements in the typical newspaper. The symbolist writers--Baudelaire, Verlaine, Mallarmé, and Rimbaud--probed the workings of imagination by, among other techniques, fragmentation and juxtaposition of images. For McLuhan, the juxtaposition of stories in a newspaper, skilfully arranged by the good editor, explained much of the medium's appeal. In many respects, the juxtaposition of story elements in the newspaper syncopates print, animates its usual linear bias, and allows text and image to co-mingle.

In The Gutenberg Galaxy, McLuhan probes the analogy of the mosaic--and its inherent patterns of juxtaposition--and suggests that the form introduces elements of the auditory and tactile into the visual: "[F]or the two-dimensional mosaic or painting is the mode in which there is muting of the visual as such, in order that there may be maximal interplay among all of the senses. Such was the painterly strategy 'since Cezanne,' to paint as if you held, rather than as if you saw, objects" (Essential McLuhan/Gutenberg Galaxy 137).

In the following, McLuhan summarizes one of his main arguments in The Gutenberg Galaxy, fascinated as usual with competing vectors of sensory fragmentation and resonance:

...from the invention of the alphabet there has been a continuous drive in the Western world towards the separation of the senses, of functions, of operations, or states emotional and political, as well as of tasks--a fragmentation which terminated, thought Durkheim, in the anomie of the nineteenth century. The that the two-dimensional mosaic is, in fact, a multidimensional world of interstructural resonance. It is the three-dimensional world of pictorial space that is, indeed, an abstract illusion built on the intense separation of the visual from the other senses. (138)
The juxtaposition of elements in the symbolist mosaic--whether the poem or newspaper--generates, like the j-engine, an "interstructural resonance." Much of this resonance occurs beyond our capacity for cognitive awareness, though it is telling us stories nonetheless.

Cézanne's Passages

In his tactile approach to his visual materials, Cézanne used a technique called passages--the inter-penetration of planes--to suggest the dimensionality and substance of his subjects. The skill of his painterly technique is to use juxtaposition of forms and shapes to convey the reality of his visual impressions even as he moved towards greater abstraction. In doing so, Cezanne is thought by many to be a progenitor of 20th C. abstraction, directly influencing the Cubists, the Dadaists, and the Surrealists in turn. All of these aesthetic movements traded on juxtaposition of visual elements to create an "interstructural resonance."

Duchamp's found objects and ready-mades, as do many of Picasso's best-loved sculptures, take as their deepest meaning the nature of radical juxtaposition. At their best, these objects convey a sense of spontaneity and ad-hocism that delights and instructs.

In like manner, the Dadaists used collage and photomontage to suggest the fracturing of reason and the liberation of the artistic consciousness from the tyranny of convention. For the Dadaists, "[P]hotomontage combined the photograph's proximity to objective reality with a dynamic process of reordering which enacted, at least metaphorically, the revolutionary reordering of society" (Clark 32). The merz-werk of Kurt Schwitters and the constructions of Joseph Cornell succeeded in elevating the found object into an elegant and compelling expression based on the power of juxtaposition. In his semiotic analysis of sitemaps, George Dillon takes as his primary visual model the Dadaist photomontage:

Similar to cubist collage, but far more involved with text, is the practice of Dada photomontage as developed by Hannah Höch, Raoul Hausmann, Kurt Schwitters and others (Georg Grösz, John Heartfield, Max Ernst). The hypertext page has words and images linking to other words and images; Dada photomontage is made up of bits of photos and other images along with words and phrases from the media, not "things" but signifiers. These signifiers are recomposed into a new whole but point always to another "page" from which they were snipped. So the Dada photomontage is like a sitemap--an image of one way all the fragments go together.

At the end of his analysis, Dillon awards "the grand prize for the descendants of Dada" to Mark Napier and his Shredder project to Dadaize web pages.

Convulsive Beauty and the Exquisite Corpse

Surrealists used the power of juxtaposition to plumb the depths of the subconscious. For the Surrealists, the dream was "a source of pure imagination, an expression of the marvellous and the unexpected" (Cunningham). André Bréton's first manifesto identifies the dream as the archetype of the "surrealist image" based on the "unexpected juxtaposition of different realities....which Breton saw most perfectly realised in the nineteenth-century poet Lautréamont's phrase: 'As beautiful as the chance meeting on a dissecting table of a sewing machine and an umbrella.'" David Cunningham comments:

This kind of “convulsive beauty” was sought in the Surrealist writing of Bréton, Aragon, Éluard, Desnos and others, and was characteristically manifested in the combination of images from disparate sources making up a whole which defied any straightforward unification of its elements. Such juxtaposition may be regarded as a radical emphasis upon what Roman Jakobson famously terms the “metaphoric” pole of literary language, one which seeks to engender a certain “shock effect” as the means to an extreme “defamiliarisation” of conventional or logical forms of poetic construction and metaphoric association. Such “defamiliarisation” is exemplified in one of Bréton's most famous poems, “Free Union”, which follows the familiar poetic technique of describing the beauty of a lover in a series of analogies, but which undermines the usual conventions of such verse by the sheer unexpectedness of its comparisons.

Perhaps the most provocative of the Surrealist "love poems" are the images generated by the process of the Cadavre Exquis--the Exquisite Corpse.

Aarseth's Cybertext: Perspectives on Ergodic Literature

Espen Aarseth's exploration of ergodic literature---"open, dynamic texts where the reader must perform specific actions to generate a literary sequence, which may vary for every reading" ( such diverse genres as "hypertext fiction, computer games, computer generated poetry and prose, and collaborative Internet texts such as MUDs." These cybertexts require the active participation of the writerly reader to navigate meshworks, initiate commands and actions, or make other choices, the j-engine requires a different kind of participatiion and interactivity. Depending on its pacing and selection of images, the j-engine involves the co-performer of the "text" in an active process of cognition, both consciously and unconsciously.

We are suggesting here that the j-engine may be a useful vehicle for interrogating the role of the non-rational or uncounscious interactivity with narrative. The j-engine produces ergodic literature of a different order: the participant must perform (un)specific actions to generate an aesthetic text, actions of cognition under the sign of resonance rather than reason.


In his recent Protocol: How Control Exists After Decentralization, Alexander Galloway describes how competing vectors of control and decentralization, or freedom, are managed on the internat by DNS and TCP/IP. Galloway is mapping out a paradigm of coding performance where improvisation and indeterminacy are allowed to occur within certain protocols. For example, Galloway describes how a datagram is "assigned a 'time-to-live'--the maximum number of hops it can take before it is deleted, to ensure that datagrams do not hop around the network indefinitiely" (45). In a telling analogy to illustrate his point about the importance of order for establishing meaning, Galloway writes that a "record is any type of nonrandom information, not simply something that records language or data. Thus, the act of sharpening a raw stone into a tool embodies the stone with the 'information' of its new shape. Arranging randomly scattered leaves into a straight line gives the leaves 'information'" (72).

The j-engine illustrates this tension between control and decentralization that is, ultimately, managed by agreements about binary coding and interoperability. While arranging leaves into a straight line gives the leaves "information," leaving the leaves in a random pile is also information of a different order--though it may be information apprehended quite differently by cognition.

Spatial Montage

Eisenstein, Octobre

In The Language of New Media, Lev Manovich continues his project to assemble the protocols of the emerging digital artifacts by borrowing (again) from the lexicon of film. In the present context, his appropriation of montage is of particular interest. To qualify as a montage, he argues, "a new media object should fulfil two conditions: Juxtaposition of elements should follow a particular system, and these juxtapositions should play a key role in how the work establishes its meaning, and its emotional and aesthetic effects...By establishing a logic that controls the changes and the correlation of values on these dimensions, digital filmmakers can create what I call spatial montage....Borders between different worlds do not have to be erased; different spaces do not have to be matched in perspective, scale, and lighting; individual layers can retain their separate identities rather than being merged into a single space; different worlds can clash semantically rather than form a single universe" (158).

Eisenstein Editing

Manovich makes a further distinction between ontological and stylistic montage that is of interest to a consideration of the juxtaposition engine. An ontological montage establishes "the coexistence of ontologically incompatible elements within the same time and space." He uses the example of Rybczynski's Tango (1982)--which achieved its compositing effects with optical printing--to illustrate how the juxtaposed scenes could never occur in "normal human experience," despite their metaphoric resonance. In contrast, stylistic montage mixes a heterogenous combination of media--live-action shots, stills, 8mm, VHS, DV, advertising, documentary, amateur, professional--into the moving image equivalent of the cubist and dadaist constructions. Digital compositing has accelerated this eclectic montage of styles. "Compositing can be used to hide this diversity--or it can be used to foreground it, creating it artificially if necessary." Manovich uses the example of Forrest Gump to illustrate his point: the "simulation of different film and video artifacts is an important aspect of its narrative system" (159).


Timecode Screen Shot

The basic scheme of organization we have used for the j-engine on the screen has a direct parallel to both the form and method of Mike Figgis' Timecode. Figgis, a musician, composed his film on music sheets, as if scoring a musical composition similar to a quartet. Each of the four interdependent films were shoot in continuous 93-minute takes by four camera operators (including Figgis). The four digital cameras were turned on simultaneously, using a common timecode. The blocking and much of the acting was improvised.

"Figgis has admitted suffering initial qualms over the viability of his split-screen, parallel-plotting device, fretting that Timecode's constant four-way dialogue of sound and visual information would swamp the viewer. He concluded, however, that a diet of channel-surfing and multi-media has made sophisticates of modern-day filmgoers, equipping them to process a bombardment of information thrown at them on various frequencies. Where MTV would sate its audience through frenzied editing, Figgis reasoned, Timecode would do so through simultaneous, real-time narratives. Formally separate, these two modes of communication are spiritual cousins" (Sight and Sound).

Timecode provides the viewers with many choices about what to watch and attend to in the juxtaposed and simultaneous plots, but there is a limit to this freedom. This film is clearly the work of a unifying consciousness to shape, orchestrate and coordinate the diverse materials. As writer, director, producer and composer, Figgis composes his film as Sun Ra directed his Arkestra: a nod, a flourish, a full-blown intervention directed his superb musicians and gave them a vision as much as a direction.

Near the end of the film, one of the characters (Ana, played by Mia Maestro) comes up with an idea for a film that dispenses with the Eisensteinian montage in favour off a new cinematic language. "Montage has created a false reality," she proclaims. "Digital is demanding new expressions." The film she proposes sounds very much like the Timecode we are watching: filmed in real time in one continuous take on four cameras.

The reviewer concludes: "In dispensing with montage, Timecode liberates the medium up to a point, but is still constrained by the simple need (perhaps inherent in all drama) to order its material and tell a story" (Sight and Sound).


In Visual Explanations, Edward Tufte defines a confection as "an assembly of many visual elements, selected...from various Streams of Story, then brought together and juxtaposed on the still flatland of paper. By means of a multiplicity of image-events, confections illustrate an argument, present and enforce visual comparisions, combine the real and imagined, and tell us yet another story" (121).

For Tufte, the assemblages of Joseph Cornell seem to qualify as confections--as opposed to mere collages--because they convey information, an argument, as cryptic as it may be:

Cornell's boxes, miniature theatres of reverie, assemble once-separate materials to crerate magical and cryptic architectures, three-dimensional collages. For art, collage (French, pasting) combines images so as to create pleasing or provoking visual experiences, hardly expressible in words and rarely based on words; on the other hand, confections bring images together to display visual information, often expressible in words and often derived from words. Confection-makers cut, paste, construct, and manage miniature theatres of information--a cognitive art that serves to illustrate an argument, make a point, explain a task, show how something works, list possibilities, narrate a story.(138)
It seems, then, that our juxtaposition engine is a confection in these terms, for it is our intention to "illustrate an argument, make a point, explain a task, show how something works, list possibilities, narrate a story." If there has been some serendipity in this confection for you, the pleasure of discovery has been mutual.

The Virtual Lightbox

The Virtual Lightbox "is a software tool for comparing images online. Comparison, what John Unsworth calls a 'scholarly primitive,' is a basic and probably intuitive operation that is nonetheless not well supported--for images anyway--by conventional Web browser technology; that is, users have no ability to move, juxtapose, or otherwise reposition images beyond the configuration in which they are delivered by a static page layout. As rich image collections continue to come online, it's becoming increasingly apparent that end-users lack the tools to exploit such resources to their full potential." ( Differences between the Lightbox and the JEngine revolve around the user's role: in the Lightbox, the user will consciously place images to search for comparison and meaning; in the JEngine, 4 image subset after subset of will flash onto the screen, allowing the user to seek meaning, and unconsciously reveal similarity and difference.

The Versioning Machine

The Versioning Machine is "a software tool designed by a team of programmers, designers, and literary scholars at Maryland Institute for Technology in the Humanities (MITH) for displaying and comparing multiple versions of texts. The display environment seeks not only to provide for features traditionally found in codex-based critical editions, such as annotation and introductory material, but to take advantage of opportunities of electronic publishing, such as providing a frame to compare diplomatic versions of witnesses side by side, allowing for manipulatable images of the witness to be viewed alongside the diplomatic edition, and providing users with an enhanced typology of notes." (


Aarseth, Espen. Cybertext: Perspectives on Ergodic Literature. Baltimore, Md: Johns Hopkins University Press, 1997. (

Clark, Toby. Art and Propaganda in the 20th Century. New York: Harry Abrams, 1997.

Cunningham, David. "Surrealism, 1919 - 1966." Literary Encyclopedia. n.d. ( 25 May 2004.

Dillon, George. "Dada Photomontage and Sitemaps." PostModern Culture 10:2 (2000).

Galloway, Alexander. Protocol: How Control Exists After Decentralization. Cambridge, MA: MIT Press, 2004.

Haahr, Mads. "Introduction to Randomness and Random Numbers." Random.Org. June 1999. ( Accessed: May 2004.

Jennifer Everywhere.

McLeod, Scott. "I Can't Stop Thinking." Scott McLeod.Com. (

McLuhan, Marshall. Essential McLuhan. Ed. Eric McLuhan and Frank Zingrone. Toronto: Anansi, 1995.

Manovich, Lev. The Language of New Media. Cambridge, MA: The MIT Press, 2001.

Napier, David. Shredder 1.0. 1998.

"Review of Timecode." Sight and Sound. September 2000. ( Accessed: May 2004.

Rubin, William. Dada & Surrealist Art. New York: Harry N. Abrams, 1968.

Soules, Marshall. "Resonant Media." Emerging Digital Technologies. (

Tufte, Edward. Visual Expanations: Images and Quantities, Evidence and Narratives. Cheshire, CT: Graphics Press, 1997.

© 2004 Michael Nixon and Marshall Soules
Fair Dealing Applies