Visual Semiotics

Maria Giulia Dondero & Kaan Tanyeri

Maria Giulia Dondero & Kaan Tanyeri

(Please click on the above names to access their CV)

Yazıyı Türkçe çevirisinden okumak için lütfen tıklayınız.


Alt­ho­ugh se­mi­otics was con­si­de­red a li­te­rary field due to its first app­li­ca­ti­ons, it has today be­co­me very di­ver­se. You mostly con­cent­ra­te on vi­su­al se­mi­otics. Can you tell us the story be­hind this, of how you got ac­qu­a­in­ted with se­mi­otics? And is there a re­ason you chose to focus on vi­su­al se­mi­otics?

I was a stu­dent at the Uni­ver­sity of Bo­log­na, at the D.A.M.S. (Dis­cip­li­nes of Art, Music and Per­for­ming Arts, Fa­culty of Phi­lo­sophy and Hu­ma­ni­ti­es) and I fol­lo­wed art his­tory clas­ses. In 1996-1997, Paolo Fabb­ri came to teach in Bo­log­na and I was truly cap­ti­va­ted by his vi­su­al se­mi­otics clas­ses. It was then that I re­ali­zed that my in­te­rest lied be­yond art his­tory, that I ne­eded to study the vi­su­al arts from a dif­fe­rent pers­pec­ti­ve, lo­oking for si­mi­la­ri­ti­es and dif­fe­ren­ces be­t­we­en ima­ges from dif­fe­rent time pe­ri­ods. After this en­co­un­ter with Paolo Fabb­ri, I began to study ge­ne­ral se­mi­otics.

Which se­mi­oti­ci­ans have had the gre­atest inf­lu­en­ce on you? Could you tell us about them?

After Paolo Fabb­ri, I met Ja­c­qu­es Fon­ta­nil­le who is the se­mi­oti­ci­an whose se­mi­otic the­ory ins­pi­red me the most with res­pect to image analy­sis and to the ge­ne­ral the­ory of te­x­tu­ality and of bo­dily enun­ci­ati­on. I still think that he is the most in­no­va­ti­ve post-Gre­ima­si­an sc­ho­lar and that he is the most ge­ne­ro­us with his stu­dents. My app­ro­ach to sci­en­ti­fic ima­ges and my book writ­ten with Fon­ta­nil­le, The Se­mi­otic Chal­len­ge of Sci­en­ti­fic Ima­ges. A Test Case for Vi­su­al Me­aning, are also ins­pi­red by Je­an-Fran­ço­is Bord­ron’s the­ory of ico­ni­city and by his pro­po­si­ti­on re­gar­ding the pro­cess of tran­si­ti­on from subs­tan­ce to form on the image’s plane of exp­res­si­on.

Anot­her very im­por­tant se­mi­oti­ci­an in my life is Je­an-Ma­rie Klin­ken­berg. He is so­me­one who did a great job re­gar­ding vi­su­al rhe­to­ric and vi­su­al se­mi­otics and he was pro­bably the first se­mi­oti­ci­an to be con­cer­ned with the ma­te­ri­ality of ima­ges in the 70s. I have often been cri­ti­cal about some parts of his ge­ne­ral se­mi­otics the­ory but everyt­hing he wrote about ima­ges has ins­pi­red me tre­men­do­usly, es­pe­ci­ally his app­ro­ach to ima­ge-te­xt the­ory (what we call “sync­re­tic dis­co­ur­se”).

Anot­her se­mi­oti­ci­an who was and is very im­por­tant as re­gards my se­mi­otic thin­king and met­ho­do­lo­gi­cal app­ro­ach is Pi­er­lu­igi Basso Fos­sa­li, who is so­me­one who is able to con­ci­li­ate st­rong for­ma­li­za­ti­on with very sharp analy­ses of cul­tu­ral ob­jects.

In­de­ed, li­te­rary wri­ting has been and will con­ti­nue to be de­fe­ated by the vi­su­al, if I am cor­rect? The fu­tu­re is built on vi­su­ality. How can we exp­la­in this? Hu­ma­nity has been se­e­ing since the very first human, but our eyes seem to have ac­qu­ired a dif­fe­rent im­por­tan­ce in today’s world.

I think that what has chan­ged in re­cent times is that we un­ders­tand that ima­ges can be stu­di­ed in a more pro­fo­und way than along the lines of Bart­hes’ app­ro­ach or ac­cor­ding to a phi­lo­sop­hi­cal app­ro­ach. Se­mi­otics al­lo­ws pe­op­le to un­ders­tand that an image is a dis­co­ur­se and that it may pro­du­ce ar­gu­ments, conf­lic­ting ar­gu­ments, and me­ta-ref­lec­ti­on. The field of vi­su­al stu­di­es also cont­ri­bu­tes to the un­ders­tan­ding of ima­ges, but the prob­lem with vi­su­al stu­di­es is the ide­ology that gu­ides the re­ading of ima­ges and pre­vents from un­ders­tan­ding the me­aning of an image based on the analy­sis of its spa­ti­al com­po­si­ti­on as such.

I’m also con­vin­ced that the is­su­es of in­te­rest to vi­su­al se­mi­otics are very close to those faced by wri­ting stu­di­es. Both dis­cip­li­nes have the prob­lems of the subst­ra­te and of insc­rip­ti­on in their focus and the cros­sings be­t­we­en the two are be­co­ming im­por­tant today due to a sha­red in­te­rest in di­gi­tal li­te­racy.

I am cur­rently wor­king on li­te­rary se­mi­otics. This is un­do­ub­tedly due to my com­mit­ment to li­te­ra­tu­re. How do vi­su­al semioticians view li­te­rary se­mi­oticians? Are we out of date?

I think that ever­yo­ne has a lot of res­pect for li­te­rary se­mi­otics! Vi­su­al se­mi­otics took ins­pi­ra­ti­on from ver­bal and li­te­rary se­mi­otics – one need only look at the use of enun­ci­ati­on the­ory in image analy­sis – but I think that li­te­rary se­mi­otics can also learn a lot from vi­su­al se­mi­otics. Not only can the dis­tinc­ti­on be­t­we­en fi­gu­ra­ti­ve and plas­tic analy­sis be use­ful to li­te­rary analy­sis for stud­ying the nar­ra­ti­on of events and the sc­he­ma­tic re­la­ti­ons be­t­we­en them, but also the idea of a text as a ma­te­ri­al body (no­vels or nar­ra­ti­ons are also ma­te­ri­al ob­jects) and as a me­di­atic ob­ject can carry cur­rent li­te­rary analy­sis to­wards gre­ater comp­le­xity, which is es­pe­ci­ally re­le­vant today as li­te­ra­tu­re is being cir­cu­la­ted over new me­di­ums. Cur­rent vi­su­al se­mi­otics is also stud­ying the image as a ma­te­ri­ality and is cont­ri­bu­ting to the ma­te­ri­al turn that began in vi­su­al anth­ro­po­logy (see Pho­tog­raphs Ob­jects His­to­ri­es: On the Ma­te­ri­ality of Ima­ges, Eli­za­beth Ed­wards and Ja­ni­ce Hart, eds., 2004). I think that li­te­rary the­ory needs to take ad­van­ta­ge of what se­mi­otics has been doing to study vi­su­ality and, more ge­ne­rally, media.

So how does vi­su­al se­mi­otics study the image? Th­ro­ugh which pro­ces­ses do you con­duct your analy­sis?

To analy­ze ima­ges, I use some basic con­cepts from Gre­imas [“Fi­gu­ra­ti­ve Se­mi­otics and the Se­mi­otics of the Plas­tic Arts.” New Li­te­rary His­tory, 20, no. 3 (1989):627-649] that have been fi­ne-tu­ned by se­mi­oti­ci­ans such as Ja­c­qu­es Fon­ta­nil­le th­ro­ugh the no­ti­ons of subst­ra­te (sup­port matériel et sup­port for­mel) and of comp­le­ment, as well as some ot­hers by Pi­er­lu­igi Basso (it is pos­sib­le, in his view, to per­form both a plas­tic and a fi­gu­ra­ti­ve re­ading of the plas­tic enun­ci­ati­on). But the most im­por­tant is to as­so­ci­ate the plas­tic and fi­gu­ra­ti­ve analy­sis with the study of ut­te­red enun­ci­ati­on (énon­ci­ati­on énoncée), be­ca­use the lat­ter al­lo­ws you to un­ders­tand the re­la­ti­ons­hip be­t­we­en the image’s com­po­si­ti­on (types of pers­pec­ti­ve, in­de­xi­cal add­res­ses, conf­licts be­t­we­en po­ints of view, and so on) and the po­si­ti­on (spa­ti­al but also cog­ni­ti­ve and emo­ti­onal) that the image has set for the ob­ser­ver.

What is the main pur­po­se of these analy­ti­cal pro­ces­ses? Get­ting the hid­den me­aning be­hind what is vi­sib­le? If so, does every image ne­ces­sa­rily have a la­tent me­aning? Be­ca­use I don’t think that an or­di­nary every­day pho­tog­raph pos­ted on so­ci­al media will have much of a hid­den me­aning. In such case, would your analy­sis chan­ge?

I’m not very in­te­res­ted in the hid­den me­aning of ima­ges! I’m more in­te­res­ted in stud­ying their com­po­si­ti­on, their genre, and their sta­tus. The sta­tus is what de­ter­mi­nes the type of in­terp­re­ta­ti­on app­li­ed to the image and the con­di­ti­ons of its cir­cu­la­ti­on. You speak about ima­ges on di­gi­tal so­ci­al media. They have a dif­fe­rent sta­tus: they cir­cu­la­te th­ro­ugh dif­fe­rent eco­lo­gi­es of va­lu­es than do ar­tis­tic and sci­en­ti­fic ima­ges. The analy­sis has to begin by ta­king ac­co­unt of the sta­tu­ses of ima­ges, of their gen­res (port­ra­it, still life, and so on), and of their ge­ne­alogy. The met­ho­do­lo­gi­cal inst­ru­ments to use are dif­fe­rent whet­her we must analy­ze an ar­tis­tic image or a sci­en­ti­fic one. In the first case, it is the analy­sis of the plas­tic and ma­te­ri­al di­men­si­ons (ges­tu­res of pro­duc­ti­on) that are cri­ti­cal, whe­re­as sci­en­ti­fic ima­ges have to be stu­di­ed from wit­hin a se­ri­es of ima­ges in re­la­ti­on to the pa­ra­me­ters that have been ins­ti­tu­ti­ona­li­zed to pro­du­ce them. Every sci­en­ti­fic image has to be rep­ro­du­cib­le by other sci­en­tists, in order to be sci­en­ti­fi­cally valid. Or­di­nary pho­tog­raphs have an ama­te­ur sta­tus and must be stu­di­ed in the way that fa­mily and per­so­nal pho­tog­raphs have been in the past, with at­ten­ti­on to im­per­fec­ti­ons, er­rors, fil­ters, and ut­te­red enun­ci­ati­on in re­la­ti­on with the com­mu­nity to which the pho­tog­raph is add­res­sed (fa­mily, fri­ends, large com­mu­ni­ti­es, etc.).

Spe­aking of so­ci­al media, we can also talk about big vi­su­al data. Could you ple­ase tell us about big vi­su­al data? How does se­mi­otics re­la­te to big vi­su­al data?

Not so many sc­ho­lars in se­mi­otics are de­aling with this issue. One of the first has been Dario Com­pag­no, who wor­ked on the analy­sis of big data re­la­ting to po­li­ti­cal and jo­ur­na­lis­tic in­for­ma­ti­on. For my part, I am tr­ying to un­ders­tand how the analy­sis of big vi­su­al data can help the se­mi­otic analy­sis of cor­po­ra and to de­ve­lop an analy­sis of the ge­ne­alogy of forms that was not pos­sib­le in War­burg’s and Fo­cil­lon’s time (see: https://​ce­serh.​hy­pot­he­ses.​org/​997). I stu­di­ed the vi­su­ali­za­ti­on of big vi­su­al data as a me­ta-ima­ge in my last book The Lan­gu­age of Ima­ges. The Forms and the For­ces and I tried to study it in the tra­di­ti­on of “ima­ges wit­hin ima­ges” in art his­tory and in the do­ma­in of lan­gu­age sci­en­ces. I think that the study of big vi­su­al data can be very use­ful to se­mi­oti­ci­ans, who have al­ways been in­te­res­ted in fin­ding “com­mon di­ag­rams” – as Paolo Fabb­ri said in La svol­ta se­mi­oti­ca – be­t­we­en dif­fe­rent kinds of dis­co­ur­ses. The analy­sis of vi­su­al forms th­ro­ugh com­pu­ta­ti­onal analy­sis can also offer the pos­si­bi­lity of de­fi­ning analy­ti­cal pa­ra­me­ters in a way that our per­cep­ti­on does not allow. It’s ab­so­lu­tely cri­ti­cal, for se­mi­oti­ci­ans, to un­ders­tand how mac­hi­ne le­ar­ning works and how we can un­ders­tand the al­go­rith­mic pro­duc­ti­on of va­lu­es.

In this con­te­xt, can we say that se­mi­otics will be­co­me an even more va­lu­ab­le met­hod in the fu­tu­re? I think our need for it will inc­re­ase day by day to bet­ter un­ders­tand the world.

It’s pos­sib­le. But it is ne­ces­sary for young se­mi­oti­ci­ans to be spe­ci­alists in new tech­no­logy and new com­mu­ni­ca­ti­on for­mats. I think that in this res­pect, se­mi­oti­ci­ans who work in Italy are more dis­po­sed to learn and study the new fron­ti­ers of com­mu­ni­ca­ti­on tech­no­logy.

What is the state of the stu­di­es on vi­su­al se­mi­otics aro­und the world? Which aca­de­mi­ci­ans can we talk about in this re­gard? Is there an en­vi­ron­ment cons­ti­tu­ted by vi­su­al se­mi­olo­gists or is there a jo­ur­nal they are gat­he­red in?

The Jo­ur­nal Visio does not exist any­mo­re, but the In­ter­na­ti­onal As­so­ci­ati­on for Vi­su­al Se­mi­otics (AISV/IAVS) is wor­king on the di­gi­ta­li­za­ti­on of past Visio is­su­es to make them pub­lic. I think that the best vi­su­al se­mi­oti­ci­ans are the ones I al­re­ady men­ti­oned in my first an­s­wer. Some very im­por­tant analy­ses have also been pro­du­ced by Anne Be­ya­ert-Ges­lin, Ma­ri­on Co­las-Bla­ise, Odile Le Guern, Mas­si­mo Leone and Anne Hénault.

You know I often dis­turb you with my qu­es­ti­ons. What would you sug­gest to re­se­arc­hers like me and ot­hers who are in­te­res­ted in se­mi­otics? What sho­uld we do, what sho­uld we read?

I would give you only one piece of ad­vi­ce: it is ne­ces­sary to be able to read a lot of Ro­man­ce lan­gu­ages to build skills and a cul­tu­re in se­mi­otics. Even if many im­por­tant books are trans­la­ted into Eng­lish (al­most the en­ti­re oe­uv­re of Gre­imas is trans­la­ted, for ins­tan­ce), the Euro­pe­an se­mi­otic tra­di­ti­on (which is also pre­sent and being re­ne­wed in La­ti­no-Ame­ri­ca) and very im­por­tant pa­pers are mostly writ­ten in French, Ita­li­an, Por­tu­gu­ese, and Spa­nish. It’s very im­por­tant not to limit one­self to Eng­lish-spe­aking li­te­ra­tu­re and cul­tu­re.

Fi­nally, I would like to hear your wis­hes for the Turkey Se­mi­otics Circ­le. Do you have a mes­sa­ge for us?

I’m very glad to know that the Turkish Se­mi­otics Circ­le is be­co­ming st­rong. Turkish sc­ho­lars and in­tel­lec­tu­als are mul­ti­lin­gu­al spe­akers; this is an im­por­tant qu­ality and I hope for yo­un­ger ge­ne­ra­ti­ons to ac­hi­eve pre­ci­sely that: being cul­ti­va­ted pe­op­le who can read a lot of lan­gu­ages…

Thank you very much for everyt­hing…


Copy Protected by Chetan's WP-Copyprotect.