Mangen, A. (2008). Hypertext fiction reading: haptics and immersion

Hypertext fiction reading:haptics and immersionAnne Mangen

National Centre for Reading Education and Research, University of Stavanger

Reading is a multi-sensory activity, entailing perceptual, cognitive and motorinteractions with whatever is being read. With digital technology, reading manifestsitself as being extensively multi-sensory – both in more explicit and more complexways than ever before. In different ways from traditional reading technologies suchas the codex, digital technology illustrates how the act of reading is intimatelyconnected with and intricately dependent on the fact that we are both body and mind– a fact carrying important implications for even such an apparently intellectualactivity as reading, whether recreational, educational or occupational. This articleaddresses some important and hitherto neglected issues concerning digital reading,with special emphasis on the vital role of our bodies, and in particular our fingers andhands, for the immersive fiction reading experience.

Reading with new technologies

The ways we read are constantly being moulded by whatever technological innovations,

devices and platforms come around. The print book is presently being challenged by the

computer and, perhaps in particular, the e-book. Digital technology also pervades our

surroundings – eloquently suggested by expressions such as ubiquitous and invisible

technology, and pervasive computing. Laptops and e-books are beginning to replace print

textbooks in schools. More and more of our daily reading is reading from screens, or from

some version of electronic reading tablets or mobile technology, rather than reading from

print. This raises a number of important research questions concerning digital (screen)

reading compared with print reading – no doubt pedagogically, but also generically: how

does digital technology change the ways we read?

Theorists across disciplinary boundaries largely agree that we read differently when

reading digital texts, compared with when reading print. Moreover, not only is our screen

reading distinctly different from print reading, but our reading modes and habits in

general are changing due to steadily increasing exposure to digital texts (Birkerts, 1994;

Bolter, 2001; Bolter & Grusin, 1999; Hayles, 2003; Jewitt, 2006; Kress, 2003; Levy,

2001; Liu, 2005, 2006; Mackey, 2003, 2007a). There is by now a large number of

empirical and experimental studies on perceptual and cognitive aspects of digital reading,

compared with print readings. However, few pursue to any depth the important questions

pertaining to the impact of the intangibility and volatility of the digital text on the reading

Journal of Research in Reading, ISSN 0141-0423 DOI: 10.1111/j.1467-9817.2008.00380.xVolume 31, Issue 4, 2008, pp 404–419

r United Kingdom Literacy Association 2008. Published by Blackwell Publishing, 9600 Garsington Road,Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA

process and experience. Unlike print texts, digital texts are ontologically intangible and

detached from the physical and mechanical dimension of their material support, namely,

the computer or e-book (or other devices, such as the PDA [personal digital assistant],

the iPod or the mobile phone). Such a detachment has important implications for the

reading experience, and it calls for a more substantial understanding of the impact of the

(im)materiality of the text – and, more precisely, of the relation between the text and its

technological platform – on our reading. Such an understanding in turn mandates an

orientation towards theoretical perspectives and approaches adequately addressing issues

of materiality and, more importantly, how different kinds of materiality impact our

embodied, multi-sensory, reading experience.

Materiality matters

All reading is, as previously mentioned, multi-sensory. Of particular importance, and at

the same time remarkably neglected in theories of reading, is the extent to which reading

is an activity involving and requiring manual dexterity – that is, skilful handling by our

fingers and hands. In her empirical study of young readers reading a variety of (print and

digital) multi-modal texts, Margaret Mackey points to the difficulty of finding ‘a study of

reading processes that takes full account of what the hands are doing as the reader

comprehends the text’ (Mackey, 2007a, p. 112). Haptic perception1 is of vital importance

to reading, and should be duly acknowledged. The reading process and experience of a

digital text are greatly affected by the fact that we click and scroll, in contrast to tactilely

richer experience when flipping through the pages of a print book. When reading digital

texts, our haptic interaction with the text is experienced as taking place at an indeterminate

distance from the actual text, whereas when reading print text we are physically and

phenomenologically (and literally) in touch with the material substrate of the text itself.

This may seem a matter of marginal importance in reading research, but I will claim – and

show in the following – that it is a matter requiring attention as much more than a mantra

for media theorists, or topic of interest mainly to philosophers: materiality matters.

Until quite recently, however, issues of materiality have been largely neglected in

reading research overall. Several studies point to the importance of addressing the multi-

sensory dimension of digital reading (Back, 2003; Bearne, 2003; Kress, 2003; Mackey,

2007a, 2007b; Merchant, 2007; Walsh, 2006; Walsh, Asha & Sprainger, 2007), without

really pursuing the issue any further. Acknowledging the need for a new theory of

meaning that is attentive to the affective affordances of different materialities and their

relation to the physiology of bodily reception of meaning, Kress observes:

Forms of imagination are inseparable from the material characteristics of modes,

from their shaping in a society’s history, and from their consequent interaction

with the sensoriness, the sensuousness, of our bodies. Introducing a concern with

materiality and the senses into representation brings the longstanding separation in

Western thinking of mind and body into severe question, and therefore challenges

the reification and consequent separation of cognition, affect and emotion (Kress,

2003, p. 171).

With the emergence of digital reading devices and technological platforms some

researchers have at least begun addressing questions of materiality from cognitive and

psychoergonomic perspectives – most notably in the field of Human–Computer

HYPERTEXT FICTION READING 405

r United Kingdom Literacy Association 2008

Interaction (HCI). In a comparative study of reading paper and electronic books,

Morineau et al. note: ‘Interaction with the physical support of the e-book during encoding

is very different than with a classical paper book, but we know of no experimental work

on this matter’ (Morineau, Blanche, Tobin & Gueguen, 2005, p. 336). Starting out from

the fact that there is a crucial link between the sensory–motor experience of the materiality

of the support and the cognitive processing of the text content, the study conducted

by Morineau et al. finds that the e-book does not provide the external indicators to

memory in the way that a print book does. In the e-book, the connection between the text

content and the material support is split up, allowing the technological device to display a

multitude of content that can be altered with a click. The book, by contrast, is a physically

and functionally unitary object where the content cannot be distinguished from the

material part. Hence, they conclude that the e-book ‘does not serve as an unambiguous

index to indicate a field of knowledge on the basis of its particular physical form’

(Morineau et al., 2005, p. 346). This is an interesting conclusion in a time when different

versions of the e-book (iRex Technologies’ iLiad, or Amazon’s Kindle, for instance) and

other mobile technologies (such as mobile phone novels in Japan: see Ito, Okabe &

Matsuda, 2005) are again being launched as potentially replacing the print book (both in

and out of schools), after their dismal and quite spectacular failure a decade ago. Once

again, the question begs itself: will we be reading novels on screen – perhaps on our

mobile phones – in the future?

Recreational reading of digital text: the immersive aspect

Recreational reading is done for a number of purposes, one of which is the wish to be

immersed in a fictional world – metaphorically expressed as being lost in the book (cf. for

instance Gerrig, 1993; Nell, 1988; Ryan, 2001a, 2004b). As we have all experienced,

immersion is a matter of degree. Further, and equally important, we can differentiate

between different kinds of immersion. There is, on the one hand, the kind of immersion in

a technologically enhanced environment that we typically experience in different kinds of

virtual reality installations, computer simulations and while playing computer games.

This kind of immersion facilitates a sense of being immersed in a fictional, virtual (in the

literal sense of the word) world which is to a large extent created and sustained by the

technological features and material devices involved in its display (data gloves, head set,

other devices typically providing haptic feedback or also stunning graphics allowing

seamless and fast movement, and other visual features providing a sense of agency)2

rather than by our mental acts of imagination. In contrast, consider the sense of being

immersed in a fictional world which is largely the product of our own mental, cognitive,

abilities to create that fictive, virtual (in the figurative sense of the word) world from the

symbolic representations – the text, whether purely linguistic or multi-modal, digital or

print – displayed by means of any technological platform. This is the kind of immersion

we experience when reading a page-turner novel. In this kind of immersion, the physical

and technical features of the material support – the book – are ideally transparent in order

to facilitate, and not disturb, phenomenological immersion. For heuristic reasons, and

partly following Marie-Laure Ryan’s typology (Ryan, 2001a), I call the first kind

technological immersion, and the second phenomenological immersion. Digital

technology seems the perfect platform for providing the kinds of immersion so obviously

found in computer games of all kinds – from strategic thinking and planning in Sims, to

406 MANGEN


fast and feverish response with mouse or joystick in action-oriented games, or sports

games. In comparison, the kinds of immersion in a fictional world that we typically

experience when reading a thrilling page-turner of a detective story, however, where the

sense of phenomenological presence in a fictional world is created and maintained mainly

by our own mental faculties, such as imagination and fantasy, seems more difficult to

obtain from reading digital texts. Judging from the immense popularity of computer

games, then, compared with the – hitherto – sparse interest in fiction reading on the web,

it seems plausible to say that digital technology is made for technological immersion

whereas it seems less compatible with phenomenological immersion. But why is this?

And is this situation likely to change as new reading devices, such as the (improved)

e-book, electronic reading tablets and advanced mobile technologies are, arguably,

becoming more and more reader-friendly?3

Consider for instance the reading of electronic literature – or hypertext fiction, as it is

also called.4 These works are novels, short stories, poems or any other literary genre5

produced in order to be read on a computer. That is, they take advantage of the

technological features of digital technology, such as hyperlinks, multi-modality and

interactivity, and typically employ these for aesthetic and/or narrative purposes. The

history of hypertext fiction goes back as far as 1987, when one of its early advocates,

Michael Joyce, published Afternoon – A Story, on a magnetic diskette (still available, on

CD-ROM, from eastgate.com). The publication of Afternoon sparked a great interest in

hypertext both aesthetically and theoretically, involving a number of prominent authors

and literary scholars, such as Robert Coover, George P. Landow, Jay David Bolter and

N. Katherine Hayles. After two decades of considerable theorising and creative activity,

its proponents steadily announcing hypertext fiction as the ultimate manifestation of the

future of literature, however, hypertext fiction remains a rather esoteric field of interest.6

Moreover, we also have very few empirical studies of hypertext fiction reading (Douglas,

2000a; Gee, 2001; Mackey, 2007a; Miall, 2004; Miall & Dobson, 2001; Ryan, 2001a;

Walker, 1999), and most commonly they focus on questions concerning cognitive load

and navigation efforts, not on the potential impact of the materiality of the technology.

If we take the main purpose and motivation for our reading to be that of becoming

immersed in a fictional world, then the text will have to provide the necessary setting for

such a phenomenological sense of presence – by way of whatever modality telling the

story. On several occasions, Marie-Laure Ryan (2001a, 2004a, 2004b, 2005) has

investigated the ramifications of digital technology for immersive reading. In Narrative as

Virtual Reality she concludes that ‘the hypertext format could provide the type of

immersivity of the detective novel, as do some computer games, if it were based on a

determinate and fully motivated plot’ (Ryan, 2001a, p. 240). I will argue, however, that

when it comes to the (in)compatibility of digital technology with phenomenological

immersion, plot is not the whole story. In my view, the incompatibility has at least as much,

if not more, to do with the sensory–motor affordances of distinctly different materialities of

technology than with plot. And in order to adequately account for these differences and

their impact on our multi-sensory reading experience, I suggest we turn to phenomenology.

The phenomenology of reading intangible text

The tactility of a mouse click, of touch screen page turning or of a click with the e-book

page turner bar is very different from that of flicking through the print pages of a book.



The feeling of literally being in touch with the text is lost when your actions – clicking

with the mouse, pointing on touch screens or scrolling with keys or on touch pads – take

place at a distance from the digital text, which is, somehow, somewhere inside the

computer, the e-book or the mobile phone. Because of this ontological intangibility of

the digital text, our phenomenological experience – reading – of the digital text will differ

profoundly from that of a print text. The print text is tangible7 – it is physically, tactilely,

graspable, in ways that digital texts are not (until they are printed out and hence no longer

digital). Such a difference is phenomenologically distinct, meaning that it will have

significant – if theoretically overlooked – consequences for our reading of the different texts.

When we pick up a book, look at it closely and take some time to reflect on the

perceptual features of the experience, it becomes evident that the book, as a material

object, consists of more than immediately meets the eye. Even when left front-page up on

the table, it still has a back cover, and numerous pages between the front and back covers,

even though these are not perceptually available for us from our – and the book’s –

position at the time. But the temporarily unavailable facets of an object, such as the back

cover of the book on the table, are nevertheless part of our experience of the book and the

text as an object, so that we would not be surprised, if we were to pick up the book and

turn it around to look on the back cover of it, that it actually has a back side which is as

physically manifest as the front. It may not be visible, or accessible, to our perception at a

particular time, but it is nevertheless an irreducible part of the overall phenomenological

experience. As phenomenologist Merleau-Ponty describes, ‘objects are really there for me,

and their invisible aspects have reality precisely because I can move around so as to bring

them into view and touch them’ (quoted from Moran & Mooney, 2002, p. 425). Here

Merleau-Ponty also points to the close relation between invisibility and tangibility and

how these two dimensions are both intimately connected to our experiencing something as

having material substance. The back cover of the book is not absent as such (i.e., having

no material substance), it is merely invisible for my presently situated, bodily perception.

As embodied phenomenological bodies-in-the-world,8 we grasp the unseen, the

invisible, as real and present, in its own way. Moreover, the invisible is real and present

in a way that significantly impacts our experience of the thing perceived. The invisible,

says Vivian Sobchack, is that which ‘grounds vision and gives the visible within it a

substantial thickness and dimension’ (Sobchack, 1992, p. 290). Such phenomenological

depth, thickness and dimension are – factually, and by definition– absent in whatever

we read on the screen,9 due to its intangibility. The digital text has no material substance;

hence, it has no invisible dimensions. By definition, the digital erases all traces of

tangibility,10 and, hence, invisibility. The constancy, the temporal and spatial permanence,

of a tactile object – say, a print text – has distinctively different sensory–motor

affordances, then, than something intangible. In my view, theorising digital reading –

whether recreational, educational or occupational – mandates acknowledging and

accounting for this important dimension of our reading process and experience, not least

because it might provide vital clues for prospecting the future of recreational reading with

digital technologies.

Haptic capture in hypertext reading

One main effect of the intangibility of the digital text is that of making us read in a

shallower, less focused way. As shown by numerous studies of screen reading (Coiro,

408 MANGEN


2007; DeStefano & LeFevre, 2007; Dyson, 2005; Eshet-Alkali & Amichai-Hamburger,

2004; Eveland & Dunwoody, 2002; Gee, 2001; Levy, 2001; Liu, 2005, 2006; Ohno,

2007), we tend to scan text on screen. Such a reading mode is highly vulnerable to

distractions, particularly when these distractions are as easily available as a click with the

mouse. Psychologists argue that we are psychobiologically inclined to resort to such

means of rekindling our attention when our attention is, so to speak, exhausted – that is,

when there is nothing left in view to maintain our interest. In his economy of attention,

William Thorngate presents the principle of ‘diminishing attentional returns’:

even though we may invest attention singularly, we will not invest attention

exclusively. Instead, we will develop the attentional equivalent of a mixed

portfolio. The diminution of attentional returns is usually experienced as

habituation or boredom, and appears to regulate our susceptibility to new

information and to interruptions (Thorngate, 1988, p. 250).

In other words, and applied to the reading of hypertext fictions, when the stimuli on the

current screen do not contribute sufficiently to holding our attention, we tend to seek

some sources to ‘renew’ it. If no such sources are available, we will, after a while, lose

interest and concentration, and our attention will switch to new stimuli (and we become

susceptible to new information and to interruptions). This phenomenon has been known

in psychology for a long time; William James and Hermann von Helmholtz expressed it

in their early psychological theories of attention:

an equilibrium of the attention, persistent for any length of time, is under no

circumstances attainable. The natural tendency of attention when left to itself is to

wander to ever new things; and so soon as the interest of its object is over, so soon

as nothing new is to be noticed there, it passes, in spite of our will, to something

else. If we wish to keep it upon one and the same object, we must seek constantly to

find something new about the latter, especially if powerful impressions are

attracting us away (quoted from Carroll, 2003c, p. 29).

James echoes Helmholtz in stating that ‘no one can possibly attend continuously to an

object that does not change . . . the conditio sine qua non of sustained attention to a given

topic of thought is that we should roll it over and over incessantly and consider different

aspects and relations of it in turn’ (quoted from Carroll, 2003c, p. 29).

For heuristic reasons, let us compare the reading of digital text with ‘reading’ from TV

and moving images: both television and moving images have several means at their

disposal for rejuvenating our attention to the screen by simply introducing visual change

of different kinds and on different levels. And if the images and sounds on the TV screen

do not themselves provide these means of rekindling our attention, we have another

device handy to keep our minds structured by outside stimuli: the remote control. As we

all have experienced, we can easily become bored and lose concentration even in the

presence of the constant and massive visual stimulation that the television screen offers.

One option we often resort to when such attentional entropy occurs is the well-known

activity of channel surfing. According to Carroll, what we do when we (often quite

apathetically) switch from channel to channel, is auto-stimulate our own attentional

response:



[we do] what filmmakers and video makers do to us by punctuating their spectacles

with a plethora of cinematic events. Channel surfing is a form of home-made

editing that reveals how entrancing the editing on the screen can be from

momentarily suppressing boredom by revivifying attention (Carroll, 2003c, p. 32).

The parallel device when reading hypertext fictions is, of course, the mouse and clicking

on links. A click with the mouse immediately changes the visual input so that our

attentional focus can be maintained. Thus, our urge to click and the consequent impatient

mode of reading can be at least partly explained by reference to psychobiologically

hardwired dispositions of ours. These hardwired dispositions also help explain why the

computer, as a reading device, seems to be poorly suited for the contemplative and deeply

focused reading we associate with the book. When reading a book, the text in the book as

a static and fixed perceptual phenomenon simply does not provide us with options for

attentional switching and for autostimulating our attentional response. What we resort to

when getting bored by reading a book is usually abandoning the activity altogether,

precisely because the activity (reading fixed text) and the technologies involved do not

themselves provide any alternative (external) stimulation. As a psychobiological rule,

then, when we do have options to rekindle our attention easily by outside stimuli, we are

– psychobiologically as well as phenomenologically – inclined to resort to them, rather

than to consciously trying to resist such distractions by attempting to structure

consciousness from within (which is more effortful).

Our urge to click, when reading hypertexts, is a vivid example of cross-modal

attentional capture, but of a different kind from those typically covered by psychological

theories of attention.11 Our urge to click is an example of haptic capture of the visual and

auditory modalities, with the corollary sensory–motor dominance of the haptic and tactile

over the cognitive and perceptual. The links in a hypertext fiction present themselves as

an experiential potential, a latently accessible actualisation of something currently

unavailable, which becomes readily accessible with the click of a mouse. The sensory–

motor affordances of the computer make it very easy to rekindle our attention, getting

access to something beyond our present experience. As such, text or icons that yield (i.e.,

hot spots) afford haptic interaction with the computer. We experience these as links to be

clicked on, and such affordance is necessarily incompatible with phenomenological

immersion.

Consider the first example node from M. D. Coverley’s hypermedia fiction Califia

(Figure 1).

In Califia, the reader is invited to join the three narrators Augusta, Kaye and Calvin in

a search for a lost stash of gold buried somewhere in Southern California. It is a classical

quest story, structured as four journeys (North, South, East and West), each with a

considerable amount of supplementing information and clues, in terms of maps,

background information, archives with historical documents, etc. The first example node

presents the beginning of The Journey North, where Augusta tells about her incitement to

start digging in her own back yard after clues. The text and image in this node, however,

present themselves not primarily as symbols or text to be read (i.e., cognitive operations

and phenomenological immersion) as much as a potential for motor action (i.e., haptic

operations and technological immersion). In such nodes, with the alluring ‘Follow me’

and the cursor turning to a pointing finger, the haptic affordance captures the attention at

the cost of phenomenological immersion. To take another example, the node showing

‘The Blue Blanket’ (Figure 2) displays a vital piece of background information to the

410 MANGEN


quest of the story: a hand-embroidered blanket holding the key to the possible location of

the stash of gold.

More interesting in this context, however, is what sensory–motor interaction the

graphics of the blanket node afford. When moving the cursor across the blanket, you will

realise that some parts of it – such as the spots displaying the stellar constellations –

contain links, whereas others – for instance, the moth holes – do not. As an aesthetic

Figure 1. M.D. Coverley, Califia (2000); opening node ‘Roadhead’. r 2000 M.D. Coverley. Reprinted by

permission of the author.

Figure 2. M.D. Coverley, Califia (2000); node ‘The Blue Blanket’. r 2000 M.D. Coverley. Reprinted by

permission of the author.



strategy, a combination of hidden and visual links is very common in literary hypertext.

As much as it might enhance the thrill of the unknown while reading, however, it might

equally as well cause the reader to impatiently browse the screen in search for ‘words that

yield’.12 The online hypertext fiction Lasting Image (by Carolyn Guyer and Michael

Joyce; http://www.eastgate.com/LastingImage/) is a short story from a Japanese village,

about a group of foreigners and a blind monk walking around in the village, taking some

beautiful pictures. The story is presented as if told from these two viewpoints – that of

the group of foreigners describing their experiences in the village and their appreciation

of the blind monk’s photos, and that of the monk himself. The links appear anywhere on

the page, and they are both visual (the arrows) and hidden. On some of the slides,

resolution is the clue – the clear parts of the page are the ones that yield (see Figure 3).

On other slides, there are no visual signposts (except for the arrows in the bottom right

corner); in such slides, the activated areas must be actively sought for by simply moving

the cursor across the entire image on screen.

Phenomenologically, such scanning and browsing has the effect of making the overall

reading experience one of sensory–motor (and primarily haptic) interaction with the

technological features of the hypertext, rather than a primarily hermeneutic immersion in

the fiction being told. Furthermore, it imbues the narrative with a latent ambiguity that is

a hallmark of digital hypertext; there is always the possibility that the visual display

might change – minimally, or completely, with the click of a mouse. The mere possibility

of the click bringing about some degree and kind of visual change impacts our

phenomenological immersion in a narrative fiction in a way that is simply not possible

when reading print narratives.

The well-known phenomenological experience of such attentional capture is the

impatience we often experience when surfing the web – we are, as Ben-Shaul describes it,

neither here nor there – in an experiential situation bereft of both physical and

phenomenological presence:

[C]ombining passive [e.g., reading ‘static’ and non-yielding text] and behaviorally

active cognitive constructiveness [e.g. clicking on links] demands multitasking that

Figure 3. C. Guyer and M. Joyce, Lasting Image (2000); node ‘sampans2’ (http://www.eastgate.com/

LastingImage/sampans2.html). r 2000 C. Guyer and M. Joyce. Reprinted by permission of the authors.

412 MANGEN


may generate split attention overload . . . More problematic, however, is the split

attention of the viewer/user between what he/she cognitively constructs from

what’s going on in front of him/her, and his/her constant awareness to what may

potentially lie at stake in options made available by behaviorally changing the

course of events. . . . In all of these experiences the behavioral option is restlessly

often activated, resulting in the user/viewer being neither here nor there

(Ben-Shaul, 2004, p. 157).

The experiential position of split attention is by definition irreconcilable with the deep,

immersive state of reading that we experience with, for instance, riveting thrillers and

suspenseful detective stories. In order for phenomenological immersion to be obtained,

our cognitive capacity needs to be more or less fully occupied in a cohering and consistent

way so that we do not experience any perceptual or cognitive surplus of attention available

to other tasks (Douglas, 2000a; Douglas & Hargadon, 2000b; Nell, 1988). When afforded

the possibility to click, however, our attentional allocation is already partly directed

towards the haptic intending of clicking, rather than fully directed towards the contents of

the text itself, and hence the potentially immersive impact of the narrative fiction.

Phenomenologically relating to the computer

Different technologies display different affordances, entailing different reading

experiences. What all technologies have in common, however, is that they to some

extent and in some way transform our experience. We experience and interact with

technology along a continuum of existential relations. Carefully parsing this range,

phenomenologist Don Ihde singles out the following three main human-technology

relations: embodiment relations, hermeneutic relations and alterity relations (Ihde, 1990).

Applied to digital technology, they are all crucial and will each have significant – and

different – impact on the phenomenology of reading.

The embodiment relation can be exemplified by our relating to a hammer in use: when

we are nailing, the hammer withdraws from our perceptual focus, as we focus on the

nailing process and whatever is being nailed. Technologies in embodiment relations must

be ‘technically capable of being seen through’ (Ihde, 1990, p. 73); in other words, they

must be transparent, as the perceptual and experiential terminus in embodiment relations

is not the technology or instrument per se, but that which we experience by means of the

technology. The instrument or technology is experienced as an extension of my body, it is

incorporated into my embodied actions, as is typically our experience of seeing through

eyeglasses or contact lenses, talking on the phone or a blind man’s navigating with a

cane. Embodiment relations display what Ihde calls ‘directly or instrumentally mediated

partial [perceptual] transparency’ (Ihde, 1991, p. 74). There is a perceptual isomorphism

between what is shown and how it is shown through the technology, which has significant

phenomenological implications. In embodiment relations, then, the technology is not

experienced in itself, but is – when it works properly – a means through which we

experience something else. When it is in some way or another malfunctioning, or it is

missing and we have to look for it, its phenomenological transparency is replaced by

opacity; the technology or instrument suddenly and for a brief period of time becomes an

object for our attention, typically experienced as an intrusion or obstruction to what is our

perceptual focus and terminus.



Different from embodiment relations are the relations which Ihde calls hermeneutic.

As was the case with embodiment relations, the technology in hermeneutic relations –

when it works properly – is also primarily experienced as a means through which we

experience something else. However, this ‘something else’ that is now our perceptual and

experiential terminus or focus is not our immediately surrounding lifeworld, but some

kind of ‘text’ (in the broad sense of the term – some representational artefact) or text-like

entity. Hence, in hermeneutic relations, the perceptual isomorphism between what we see

or experience and how this is shown or represented via the instrument or technology that

we have in embodiment relations is gone, and replaced by a fundamentally different kind

of ‘transparency’, requiring a different kind of sensory engagement and yielding very

different experiences from embodiment relations.

As illustrations of the hermeneutic relations, Ihde uses the examples of reading – for

instance a map, a thermometer, or a (literary) text. Such representational technologies, or

displays, require and shape our sensory engagement in a very different way from seeing

through eyeglasses or talking on the phone. The perceptual act directed towards the

technology in a hermeneutic relation is a specialised interpretive act, requiring some form

of reading. And what we read with hermeneutic technologies is some representation of a

world which might be some particular aspect of our lifeworld (as in the map and the

thermometer), or that might be an entirely fictional world (as in a fictional text).

Hermeneutic relations also allow for a kind of transparency, but that transparency is more

appropriately called linguistic-interpretive (or textual) rather than perceptual (Ihde, 1991,

p. 75). Whereas embodiment relations make our immediate lifeworld present for us,

hermeneutic relations make present a represented and referred-to world, and this presence

is therefore ‘a hermeneutic presence’:

Not only does it occur through reading, but it takes its shape in the interpretative

context of my language abilities. . . . [The represented] world is linguistically

mediated, and while the words may elicit all sorts of imaginative and perceptual

phenomena, it is through language that such phenomena occur. And while such

phenomena may be strikingly real, they do not appear as world-like (Ihde, 1990,

p. 84).

Whereas embodiment relations mimic and extend our sensory-perceptual capacities,

hermeneutic relations, then, can be said to mimic and extend our linguistic and interpretive

capacities. Compared with embodiment relations, the technology in hermeneutic relations

is more noticeably present as a mediator (also when it works properly), because it entails

a more perceptually transformational rather than perceptually isomorphic relation to

that which is experienced/read. There is a great difference between the whole-body

experience of sub-zero temperatures and of seeing the numbers on a thermometer from

inside the kitchen and inferring from this display that it is cold outside. As Ihde says, in

hermeneutic relations ‘the world is first transformed into a text, which in turn is read’

(Ihde, 1990, p. 92). But in a similar way as with embodiment relations, when the

technology or instrument in a hermeneutic relation breaks down or somehow fails to

mediate our access to its world, the technology will be experienced as obstructing or

intruding.

The third of Ihde’s human-technology relations is the alterity relation. Whereas the

focal and perceptual terminus of both embodiment and hermeneutic relations is, in

different ways, beyond or ‘through’ the technology, in alterity relations the technology

414 MANGEN


itself appears as the focus and terminus of our perception and experience. Alterity

relations are relations to the technology as an opaque object in our lifeworld. In such

relations, both the perceptual and the hermeneutic transparency are gone, and replaced by

a relation with technology as ‘other’. As outlined above, in both embodiment and

hermeneutic relations, in instances where the technology takes on full objectiveness or

opacity, it is perceived as somehow and to a certain extent obtrusive and intruding upon

our experiential focus. In alterity relations, however, this objectiveness of the technology

is the determining part of the relation. As such, the alterity relation can be said to be

instrument-like or technology-like, whereas embodiment relations are more body-like,

and hermeneutic relations are more text-like or language-like. The technology in alterity

relations takes on a quasi-otherness: rather than being perceived negatively as an

obtruding object, we experience the relation to and with the technology as object as

positive and existential. Ihde’s prime example of an alterity relation is our relation to the

computer when playing a computer game. Observing how both embodiment (hand and

finger control of mouse and keyboard, or joystick) and hermeneutic relations (relating to

the ‘storyworld’ of the game, whether it is a racetrack, a space war or a medieval fantasy

world) are certainly present, Ihde points out how there is another dimension to the

experience which manifests itself during play:

There is the sense of interacting with something other than me, the technological

competitor. In competition there is a kind of dialogue or exchange. It is the quasi-

animation, the quasi-otherness of the technology that fascinates and challenges.

I must beat the machine or it will beat me. In each of these cases, features of

technological alterity have shown themselves (Ihde, 1990, p. 100).

Our relation to the technology finds its focal fulfilment in the interaction with the artefact

– the computer – itself, not through an artefact by embodiment or by the hermeneutics of

interpretive activity, as in hermeneutic relations.

The technological infrastructure and the material platform of the computer, including

the mouse or touch pad, the display, the keyboard and possibly other hardware devices,

potentially configure and embody all these three human-technology relations.

Furthermore, the modes in which they internally intersect and combine, as well as how

they are related to the reader, have significant implications for our reading process and

experience of digital texts. Because of the direct, physical, indeed tangible, relation

between the narrative fiction text in the print book and its technological platform, we

relate to the technology of the book in a way that is supportive of phenomenological

immersion, namely by a primarily hermeneutic relation. While reading a print book, the

technological artefact – the book, the pages – partly withdraws, so that our intentionality

is primarily directed towards the narrative fiction itself, and not to the technological

object as such. Hence, the hermeneutic relation dominates the embodiment relation in our

experiential (phenomenological and perceptual-cognitive) relation to the book. When

reading a hypertext fiction, however, the combination of the intangibility of the text and

the prevalent haptic affordances of the computer make our hermeneutic relation – and

hence phenomenological immersion – highly vulnerable to being captured by the haptic

affordances of the computer and, hence, making us relate to the computer in a primarily

alterity rather than hermeneutic relation. Such a relation is hard to reconcile with

phenomenological immersion, but it is highly compatible with another mode of

engagement with the computer – with playing computer games.



The literature of the future?

Making predictions about the future of literature and of our patterns and modes of

recreational reading in an era of such extreme technological change is certainly a risky

enterprise. The book has been pronounced dead several times, as new gadgets and

reading devices have been developed. The expectations surrounding the next e-book

generation are enhanced by the fact that dominant actors such as Amazon take part,

eagerly launching their Kindle. Also, new media scholars and hypertext theorists still

express a dedicated belief in hypertext fiction advancing from its present state as largely

unknown, to becoming some of, if not the, most exciting forms of literature to come. In a

paper titled Feral hypertext: When hypertext literature escapes control presented at the

ACM Hypertext Conference in 2005, Jill Walker concludes: ‘Perhaps our greatest

challenge . . . lies in recognising literary forms that do not adhere to our conventional

forms of discipline: authors, works and commodities. I suspect that these forms of

literature will be the most interesting in years to come’ (Walker, 2005). Writing about

hypertext fiction in UCLA Today, N.K. Hayles states that ‘[a]s the body of literary

hypertexts grows, I anticipate that it will become an increasingly important part of

literature in the new millennium’ (Hayles, 2004).13 Hayles even emphasises the

increasing importance of hypertext literature for understanding the future of humanity:

As electronic literature matures, it develops rhetorics, grammars, and syntaxes

unique to digital environments. Learning to speak digital, it calls forth from us new

modes of attending – listening, seeing, moving, navigating – that transform what it

means to experience literature (‘read’ is no longer an adequate term). If each era

develops a literature that helps it understand (or create) what it is becoming, a better

comprehension of our posthuman condition requires a full range of literary

expression, print and electronic. The future of electronic literature is our future

(Hayles, 2005).

It may certainly be the case that electronic literature forces us to attend to literature and

fiction in novel and potentially rewarding ways. To me, however, it seems plausible that

the particular sense of being, deeply and for an extended period of time,

phenomenologically immersed that we typically experience when reading a novel, is

related to and at least partly dependent on the very materiality of the print pages of the

book itself. No matter how print-like the quality of the e-book screen, the text as such

remains digital and hence detached from the physical support. More empirical research is

needed in order to bring us closer to fully understanding the impact of different material

platforms and their sensory–motor affordances, and how these are in fact a major part of

our reading experience.

Notes

1. Haptic perception involves both the tactile perception through our skin and the perception of the position

and movement of our joints and muscles (commonly referred to as the kinaesthetic sense modality). For

example, when we click with the computer mouse, we sense the mouse click both through the receptors on

the skin on our fingers, as well as through the position and movement of our hand and fingers.

2. Agency is defined by Janet H. Murray as ‘the satisfying power to take meaningful action and to see the

results of our decisions and choices’ (Murray, 1997, p. 126).

416 MANGEN


3. Much of the criticism of earlier versions of digital reading devices focused on technological performance

issues, such as the computer screen’s inferior reader-friendliness due to flickering screens, or the bulkiness

of the platform itself, making it less comfortable – if at all possible – to hold in our hand (and, hence,

inappropriate for bedtime reading). Both of these features are considerably improved in the most recent

versions of the e-book.

4. For a book-length study from a cognitive-phenomenological point of view, see Mangen (2006).

5. The question of whether or not these texts can rightly be called literature belongs to a different debate, the

pursuit of which is beyond the scope of the present article.

6. As observed by for instance Michel Chaouli: ‘Why are readers with a promiscuous appetite for

contemporary fiction by and large not drawn to the ‘‘interactive’’ fictional texts one finds on the web and in

other electronic form? Shrewd critics have, after all, demonstrated that electronic fiction . . . offers some of

what is the most adventurous, playful, and innovative in contemporary writing, indeed, that the very

structure of the form encodes many of the features that recent theoreticians of literature have most prized’

(Chaouli, 2005).

7. Psychologist James J. Gibson has defined tangibility according to the three following variables –

geometrical (shape, dimensions, proportions, slopes, curves, etc.); surface (texture, roughness, smoothness);

material (heaviness, mass, rigidity, plasticity) (Gibson, 1966).

8. The term is a translation of what Merleau-Ponty calls corps-vecu (Merleau-Ponty, 1962 [1945]).

9. Holograms might be considered an exception here.

10. The obvious tangibility of the computer’s material platform and its hardware is not, strictly speaking, part of

the digital per se, but are features belonging on a different phenomenological ‘level’ – the mechanical,

technological and material platform storing and displaying the digital text.

11. Traditionally, psychological theories of attentional capture have typically focused on the cross-modal

relations between the auditory and the visual senses at the expense of other modalities. Recent research,

however, largely supports the view that cross-modal capture effects can occur between all combinations of

auditory, visual and tactile stimuli (cf. for instance Spence, 2001).

12. In Afternoon – a story, Michael Joyce operates with a mixture of hidden and visual links; hidden links are

termed ‘words that yield’, and the reader is encouraged to follow his or her intuition as to what words these

might be (Joyce, 1996 [1987]).

13. Another evidence of the role and productivity of the hypertext fiction community is the work of the

Electronic Literature Organisation (ELO), whose mission is ‘[t]o facilitate and promote the writing,

publishing, and reading of literature in electronic media’ (www.eliterature.org). ELO also has a Nordic

affiliation, ELINOR (www.elinor.nu). The hypertext fiction community is also regularly represented at

major digital media and technology conferences, such as the annual ACM Hypertext and the biannual DAC

(Digital Arts and Culture).

References

Back, M. (2003). The reading senses. In G. Liest�l, A. Morrison & T. Rasmussen (Eds.), Digital media

revisited: Theoretical and conceptual innovation in digital domains. (pp. 157–182). Cambridge, MA: MIT

Press.

Bearne, E. (2003). Rethinking literacy: Communication, representation and text. Reading: Literacy and

Language, 37(3), 98–103.

Ben-Shaul, N. (2004). Can narrative films go interactive? New Cinemas: Journal of Contemporary Film, 2(3),

149–162.

Birkerts, S. (1994). The Gutenberg elegies: The fate of reading in an electronic age. Boston, MA: Faber and

Faber.

Bolter, J.D. (2001). Writing space: Computers, hypertext, and the remediation of print. (2nd edn). Mahwah, NJ:

Lawrence Erlbaum.

Bolter, J.D. & Grusin, R. (1999). Remediation: Understanding new media. Cambridge, MA: MIT Press.

Carroll, N. (2003c). Film, attention, and communication: A naturalistic account. In N. Carroll (Ed.), Engaging

the moving image. (pp. 10–57). New Haven, CT: Yale University Press.

Chaouli, M. (2005). How interactive can fiction be? Critical Inquiry, 31(3), 599–617.

Coiro, J. (2007). Exploring changes to reading comprehension on the internet: Paradoxes and possibilities for

diverse adolesent readers. Ann Arbor, MI: UMI.



DeStefano, D. & LeFevre, J.-A. (2007). Cognitive load in hypertext reading: A review. Computers in Human

Behavior, 23(3), 1616–1641.

Douglas, J.Y. (2000a). The end of books – or books without end?: Reading interactive narratives. Ann Arbor,

MI: University of Michigan Press.

Douglas, J.Y. & Hargadon, A. (2000b). The pleasure principle: Immersion, engagement, flow. Paper presented at

the ACM Hypertext, San Antonio, TX.

Dyson, M.C. (2005). How do we read text on screen? In H.v. Oostendorp, L. Breure & A. Dillon (Eds.),

Creation, use, and deployment of digital information. (pp. 279–306). Mahwah, NJ: Lawrence Erlbaum

Associates Inc.

Eshet-Alkali, Y. & Amichai-Hamburger, Y. (2004). Experiments in digital literacy. CyberPsychology and

Behavior, 7(4), 421–429.

Eveland, W.P. Jr & Dunwoody, S. (2002). An investigation of elaboration and selective scanning as

mediators of learning from the Web versus print. Journal of Broadcasting and Electronic Media, 46(1),

34–53.

Gee, K. (2001). The ergonomics of hypertext narrative: Usability testing as a tool for evaluation and redesign.

ACM Journal of Computer Documentation, 25(1), 3–16.

Gerrig, R.J. (1993). Experiencing narrative worlds: On the psychological activities of reading. New Haven, CT:

Yale University Press.

Gibson, J.J. (1966). The senses considered as perceptual systems. Boston, MA: Houghton Mifflin Co.

Hayles, N.K. (2003). Translating media: Why we should rethink textuality. The Yale Journal of Criticism, 16(2),

263–290.

Hayles, N.K. (2004). Hypertext narratives, literature come alive [Electronic Version]. UCLA Today. Retrieved

15 November 2005, from www.today.ucla.edu/1997/970411HypertextNarratives.html

Hayles, N.K. (2005). Deeper into the machine: The future of electronic literature [Electronic Version]. Culture

Machine, 5. Retrieved 6 February 2006, from http://culturemachine.tees.ac.uk/Cmach/Backissues/j005/

Articles/Hayles/NHayles.htm

Ihde, D. (1990). Technology and the lifeworld: From garden to earth. Bloomington, IN: Indiana University

Press.

Ihde, D. (1991). Instrumental realism: The interface between philosophy of science and philosophy of

technology. Bloomington, IN: Indiana University Press.

Ito, M., Okabe, D. & Matsuda, M. (2005). Personal, portable, pedestrian: Mobile phones in Japanese life.

Cambridge, MA: MIT Press.

Jewitt, C. (2006). Technology, literacy and learning: A multimodal approach. London: Routledge.

Joyce, M. (1996 [1987]). Afternoon: A story (CD). Watertown, MA: Eastgate Systems Inc.

Kress, G. (2003). Literacy in the new media age. London: Routledge.

Levy, D.M. (2001). Scrolling forward: Making sense of documents in the digital age. (1st edn). New York:

Arcade.

Liu, Z. (2005). Reading behavior in the digital environment: Changes in reading behavior over the past ten

years. Journal of Documentation, 61(6), 700–712.

Liu, Z. (2006). Print vs. electronic resources: A study of user perceptions, preferences, and use. Information

Processing and Management, 42(2), 583–592.

Mackey, M. (2003). Researching new forms of literacy. Reading Research Quarterly, 38(3), 403–407.

Mackey, M. (2007a). Literacies across media: Playing the text. (Vol. 2). London: Routledge.

Mackey, M. (2007b). Mapping recreational literacies: Contemporary adults at play. New York: Peter Lang.

Mangen, A. (2006). New narrative pleasures?: A cognitive-phenomenological study of the experience of reading

digital narrative fictions. Trondheim: Faculty of Arts, Department of Art and Media Studies Norwegian

University of Science and Technology.

Merchant, G. (2007). Writing the future in the digital age. Literacy, 41(3), 118–128.

Merleau-Ponty, M. (1962 [1945]). Phenomenology of perception. London: Routledge.

Miall, D.S. (2004). Reading hypertext [electronic version]. Forum Computerphilologie, 3. Available online:

http://computerphilologie.uni-muenchen.de/jg03/miall.html

Miall, D.S. & Dobson, T. (2001). Reading hypertext and the experience of literature. Journal of Digital

Information, 2(1). Available online: http://journals.tdl.org/jodi/article/view/jodi-36/37

Moran, D. & Mooney, T. (2002). The phenomenology reader. London: Routledge.

Morineau, T., Blanche, C., Tobin, L. & Gueguen, N. (2005). The emergence of the contextual role of the e-book

in cognitive processes through an ecological and functional analysis. International Journal of Human–

Computer Studies, 62(3), 329–348.

418 MANGEN


Murray, J.H. (1997). Hamlet on the holodeck: The future of narrative in cyberspace. New York: Free Press.

Nell, V. (1988). Lost in a book: The psychology of reading for pleasure. New Haven, CT: Yale University Press.

Ohno, T. (2007). EyePrint: Using passive eye trace from reading to enhance document access and

comprehension. International Journal of Human–Computer Interaction, 23(1/2), 71–94.

Ryan, M.-L. (2001a). Narrative as virtual reality: Immersion and interactivity in literature and electronic

media. Baltimore, MD: Johns Hopkins University Press.

Ryan, M.-L. (2004a). Multivariant narratives. In S. Schreibman, R. Siemens & J. Unsworth (Eds.), A companion

to digital humanities. (pp. 415–430). Oxford: Blackwell Publishing.

Ryan, M.-L. (2004b). Narrative across media: The languages of storytelling. Lincoln, NE: University of

Nebraska Press.

Ryan, M.-L. (2005). Narrative and the split condition of digital textuality. Dichtung Digital 1. Available online:

http://www.brown.edu/Research/dichtung-digital/2005/1/Ryan/

Sobchack, V.C. (1992). The address of the eye: A phenomenology of film experience. Princeton, NJ: Princeton

University Press.

Spence, C. (2001). Crossmodal attentional capture: A controversy resolved? In C. Folk & B. Gibson (Eds.),

Attraction, distraction, and action: Multiple perspectives on attentional capture. (pp. 231–262). Amsterdam:

Elsevier Science B.V.

Thorngate, W. (1988). On paying attention. In W. Baker, L. Mos, H. Van Rappard & H. Stam (Eds.), Recent

trends in theoretical psychology. (pp. 247–264). New York: Springer-Verlag.

Walker, J. (1999). Piecing together and tearing apart: Finding the story in Afternoon. Paper presented at the

Proceedings of the tenth ACM Conference on Hypertext and hypermedia: Returning to our diverse roots.

Walker, J. (2005). Feral hypertext: When hypertext literature escapes control. Paper presented at the ACM

Hypertext ’05, Salzburg, Austria.

Walsh, M. (2006). The ‘textual shift’: Examining the reading process with print, visual and multimodal texts.

Australian Journal of Language and Literacy, 29(1), 24–37.

Walsh, M., Asha, J. & Sprainger, N. (2007). Reading digital texts. Australian Journal of Language and Literacy,

30(1), 40–53.

Received 28 February 2008; revised version received 11 August 2008.

Address for correspondence: Anne Mangen, National Centre for Reading Education

and Research, University of Stavanger, NO-4036 Stavanger, Norway.

E-mail: [email protected]



Mangen, A. (2008). Hypertext fiction reading: haptics and immersion

Documents