Multisemiotics Semiotic Landscape

7/21/2019 Multisemiotics Semiotic Landscape

http://slidepdf.com/reader/full/multisemiotics-semiotic-landscape 1/21

Course: Visual and Multimodal

Communication

or Towards Multisemiotics:

Multimodality & Multimediality

Prof. Eija Ventola

Aalto University

[email protected]



Unit 2

Ventola/2 2



Ventola/2 3

I. Introduction - Semiotic orientation – using

Language to move to Multisemiotics

• development of the systemic-functional (SF) model of language associal semiotic

• Lg only one way of construing meaning

•

there are also other semiotic means ofmeaning-making (cf. de Saussure,Morris, Pierce)

• starting point: we assign 'grammars'

to language, but we can also assign toother semiotic systems (cf. Kress & van

Leeuwen)



Ventola/2 4

2. What kind of views on language &

context & social action in SFL -

! Different charts showing thedevelopment of the theory – examples

! 1990’s prominent : Genre (social action)

-Register (Field/Tenor/Mode) -Language

! Language (discourse/lex.gr/phonology &orthography)

! Metafunctional hook-up to context:

ideational, interpersonal and textual



Ventola/2 5

Halliday, Language as Social Semiotic 1978:11, Fig.

1. vs other modes as social semiotic?



Ventola/2 6

Halliday, Language as Social Semiotic

1978, Fig.19.



Ventola/2 7

Butt et al., Using Functional Grammar,

2000 p.7



Ventola/2 8

Ventola, The Structure of Social

Interaction, 1987: 58



Ventola/2 9

Connotative Semiotics: Genre, Register,

Language (see Martin 1997: Eggins andMartin 1997).



Ventola/2 10

LANGUAGE: Discourse semantics, lexicogrammar,phonology/graphology & the metafunctions of

language (Martin 1997: Eggins and Martin 1997).



Ventola/2 11

3. Proceeding to provide a semioticframework for analyses of semiosis

In the 1990’s, analyses of

!

pictures (O'Toole; Kress & van Leeuwen,Iedema, Eggins)

! sculpture and architecture (O'Toole)

! music & film (van Leeuwen)

! museums (Ravelli, Purser, Ventola)

! etc. – a lot of work exists (sometimesframeworks differ, so be aware …)

> As discourse, grammar, various realisations +

developing systematic discriptions of systems and structures involved (cf. grammar – rank ofunits: clause complex (cf, sentence), clause,group, word …)



Ventola/2 12

4. From monomodality > multimodalityFrom monomediality > multimediality- to multisemiotics

!

traditionally the representational and theinterpersonal seen to be organised through thetraditional modes of speaking and writing, ofwhich writing most valued in our society

! but today our world is more visual & the visualrepresentation is getting in some genres (eg. inadvertising) an upper-hand of the writing.

! thus, representation of what the discourse isabout and who the interactants involved are is

more and more expressed multisemiotically –ie. through a range of modes and media



Ventola/2 13

Towards functionally oriented

‘grammar’ of visual design

! ideational (representational) –representing the world inside andaround us

! interpersonal (modal) – enactingsocial relations

! textual (compositional) – howideational and interpersonalcombine into meaningfulcommunicative wholes



Ventola/2 14

O’Toole, M. 1994/2011. The Language ofDisplayed Art + CD – rank, units – cf. lg



Ventola/2 15

5. The semiotic landscape

= the place of visual communicationthat the range of various semioticmodes have achieved and are

available in a given community‘(Kress & van Leeuwen 1996: 33-40



Ventola/2 16

6. Developing literacies:

image-based or text-based?

Receptive literacies: begins with

! Simple images to look at, images +words, image + text, realistic

images

Productive literacies:

!

Images, letters, text (text withoutpictures)



Ventola/2 17

7.Changes in the semiotic landscape –

visualisation of discourses and literacies

Changes in the visualisation of:

! Science textbooks (Kress & v.Leeuwen)

! Foreign language textbooks

education + entertainment= ‘edutainment’

! The press: Frankfurter Allgemeiner, theTimes, the HS

! The Internet Press (see e.g. Ventola, E. (2005. Multimodality,

multimediality and multiliteracies - A new era for the press and using the press. In Lenk,

Hartmut and Andrew Chesterman (eds). Pressesorten im Vergleich – Contrasting Text Types inthe Press. Hildesheim/New York: Georg-Olms-Verlag, 351-387.)



Ventola/2 18

8. Summary and Conclusion

! Linguists - a traditional interest in lg andconsequently the descriptive models havecentered around lg

! K & vL (1966: 40) say: “As modes of

representation are made and remade,they contribute to the making andremaking of human societies and of thesubjectivities of their members.”

! the new semiotic landscape offers a

descriptive challenge which we are aboutto explore



Sindoni article & questions

! Discussion on S’s questions

! Possible topics – group formation –mind-mapping/idea generation

Ventola/2 19



Next time …

! Reading: Kress article; Kress & vanLeeuwen on representation

! Representational function

! Students’ part: bring some initialdata examples – each groupmember brings 2 images that canbe used for initial analysis of

representation - narrative

Ventola/2 20



Thank you!

Ventola/2 21

Multisemiotics Semiotic Landscape

Documents