Pictorial semiotics

Although pictures are mentioned, and compared to verbal language, already by such precursors of semiotics as Lessing and Degérando, and in spite of the fact that Saussure, and even more Peirce, refer to pictorial signs repeatedly, pictorial semiotics must be considered a recent discipline indeed: the Russian formalists have little to say about pictures, and the Prague school merely uses them to illustrate general principles of semiosis. Only with the advent of French structuralism did a body of knowledge particularly geared to the elucidation of general principles underlying the organisation of the picture sign start to emerge.

Pictorial semiotics is, of course, concerned with the study of pictures as particular vehicles of signification. To many of its pioneers, however, this speciality has merely been a practical way of mapping an individual picture onto a verbal description, while retaining a minimum of confidence in the objectivity of the procedure. It is only recently that the practitioners of pictorial semiotics have come to realise the importance of determining the nature of their study, which is why the differing ways in which its tasks has been fixed and its limits circumscribed may be more clearly seen once we have considered the history of the field and the issues falling within its scope.

First and foremost among the pioneers of pictorial semiotics must be mentioned Roland Barthes, whose article "La rhétorique de l’image", stands at the origin of two diverging developments inside semiotics, the semiotics of publicity, and the semiotics of visual art, represented by, among others, Louis Marin, Hubert Damisch and Jean-Louis Schefer. Not only did Barthes and his followers try to reduce all meaning to the linguistic kind, employing a model inspired in structuralist linguistics, but in so doing, they unfortunately misunderstood the import of most linguistic terms. This also applies to Damisch’s (1979) refutation of the linguistic model, identified with semiotics tout court, which, moreover, testifies to a much more serious confusion in comparing the merely intuitive, pre-theoretical notion of the picture with the concept of language as reconstructed in linguistic theory (just as Metz did in the case of the notion of film; cf. Sonesson 1989a,I.1.2.). What is confused in Barthes’ work tends to become even more so in that of his followers, who, moreover, inherit his exclusive attention to the content side of the pictorial sign, or more exactly, to the extra-signic referent and its ideological implications in the real world, even to the point of ignoring the way in which the latter are modulated in the sign.

The second most influential figure in early pictorial semiotics was no doubt Umberto Eco, who defined two of the basis issues of the domain, and whose resolution of these issues was hardly contested until recently. Probably because only conventional signs, according to Saussure, were of interest to semiotics, Eco set out to show that pictures are as conventional as linguistic signs. Pursuing even further the analogy with linguistic signs, Eco went on to suggest that pictures could be analysed into elementary signs, which, in turn, could be dissolved into features having no meaning of their own. Although Eco himself was to quality this latter idea ever more through the years, one or other version of his conception continues to be accepted by many scholars in the field.

Less influential than Barthes and Eco, but certainly as important for the development of pictorial semiotics, René Lindekens in his two early books (1971; 1976) discusses questions pertaining to the basic structure of the pictorial sign (e.g., conventionality and double articulation), using photography as a privileged example. His theoretical baggage is complex: Hjelmslevian semiotics, of which he has a much more solid knowledge than Barthes, combined with an inkling of the Greimas school approach; phenomenology, which, however, affected him in the subjectivist reinterpretation of the existentialists; and the experimental psychology of perception, mainly derived from the Gestalt school. Yet, the different theoretical strands remain badly integrated, and much knowledge present in these perspectives is insufficiently exploited (cf. Sonesson 1989)

In order to demonstrate the conventionality of pictures, and to show how they are structured into binary features, Lindekens (1971) suggests, on the basis of experimental facts and common sense, the existence of a primary photographic opposition between the shaded-off and the contrasted; at the same time, he also turns to experiments involving geometric drawings which have the function of brand marks, in order to discover the different plastic meanings (which Lindekens calls "intra-iconic") of elementary shapes. In fact, Lindekens would seem to argue for the same conventionalist and structuralist thesis as the early Eco (1968), but while the latter tends to ignore the photograph as the most embarrassing counter-example, Lindekens attacks its frontally from the beginning.

In the late seventies and in the eighties, pictorial semiotics made something of a new start, or, rather, produced three fairly different, new beginnings: one, which is associated with the Greimas school, and whose main representatives are Jean-Marie Floch and Felix Thürlemann; another, which comes out of the "general rhetoric" defended even earlier by the Liège group known as "Groupe µ"; and, finally, a development centred around Fernande Saint-Martin and her disciples in Montréal and Québec. To this could be added an even more recent strand rooted in the "social semiotics" of M.A.K. Halliday.

Jean-Marie Floch, Felix Thürlemann, and their followers accept the basic tenets of the Greimas school, and make use of its abundant paraphernalia, albeit with unusual restraint. Thus, like all contributions from the Greimasian camp, their articles employ an array of terms taken over from the linguistic theories of Saussure, Hjelmslev, Chomsky, and others, but given quite different meanings. The real problem resulting from this approach, therefore, it not, as it is often claimed, that it deforms pictures and other types of non-linguistic meanings by treating them as being on a par with language, but that, in attributing quite different significations to terms having their origin in linguistic theory, it renders any serious comparison between linguistic and non-linguistic meanings impossible. Moreover, Floch and Thürlemann agree with other Greimasians in taking all knowledge about the object of study to be irrelevant to semiotics, so that they must refrain from using the knowledge base of, for instance, perceptual psychology.

The interest of this approach resides not only in the fact that it involves the application of a model having fairly well-defined terms, which, at least to some extent, recur in a number of text analyses, but also is due to the capacity of this model to account for at least some of the peculiarities of pictorial discourse. Thus, for example, Floch and Thürlemann have noted the presence of a double layer of signification in the picture, termed the iconic and plastic levels. On the iconic level the picture is supposed to stand for some object recognisable from the ordinary perceptual Lifeworld (which is of course a much more restricted notion of iconicity than that found in the Peirce tradition); while concurrently, on the plastic level, simple qualities of the pictorial expression serve to convey abstract concepts. Floch, it is true, has tried to generalise these notions to other domains, most notably to literature, but they seem much better adapted to pictorial discourse.

A second, more controversial aspect of, in particular, the work of Floch, is the idea that pictorial meaning is organised into contrasts, i.e. binary terms, one member of which is an abstract property and the other its opposite ("continuity" vs "discontinuity", "dark colours" vs "light colours", etc.), both of which are present in different parts of the same picture. Indeed, each analysis starts out from an intuitive division of the picture into two parts, which may then be repeated inside one or both the division blocks. The remaining task of the analyst is thereafter to justify this segmentation, setting up long series of oppositional pairs, the members of which are located in the different division blocks resulting from the segmentation. Although Floch shows considerable ingenuity for discovering a binary division in all pictures studied, one may wonder whether such an analysis is equally adequate in all cases, and whether it remains on the same level of abstraction.

Equally of seminal importance to pictorial semiotics, the Groupe µ, or Liege school has consisted of different members through the years, the most constant of which are the linguists Jean-Marie Klinkenberg and Jacques Dubois, the chemist Francis Edeline and the aesthetician Philippe Minguet. Starting in the late sixties, this Belgian group of scholars produced a book of "general" rhetoric, in which they analysed in a novel way the "figures" appearing in the elaborate taxonomies of classical rhetoric, using linguistic feature analysis inspired in the work of Hjelmslev, as well as the mathematical theory of amounts. As in classical rhetoric, a figure is taken to exist only to the extent that there is a deviation from a norm. The latter is understood as redundancy, and thus identified with the Greimasian concept of isotopy, which henceforth becomes one of the essential building-blocks of the theory. At this stage, Groupe µ seems heavily dependent on a set of Hjelmslevian concepts (which they may not interpret quite correctly; cf. Sonesson 1988,II.1.3.7., and 1989a,II.3-4.), as well as on the notion of isotopy as conceived by Greimas (which in itself may be incoherent, cf. Sonesson 1988,II.1.3.5).

In spite of being general in import, the theory to begin with was mostly concerned with figures of rhetoric as they appear in verbal language. In a short study of a coffee pot disguised as a cat, Groupe µ (1976) tries to implement the theory also in the pictorial domain. Over the years, the theory has been continuously remodelled, so as to account better for the peculiarities of pictorial meaning. Recently, Groupe µ rhetoric appears to leave behind at least part of the linguistic strait-jacket inherited from Hjelmslev, in order to incorporate "a certain amount of cognitivism" (Klinkenberg, personal communication). Yet, the theory still seems far from integrating the perceptual and sociocultural conditions that constitute the foundations of all rhetorical modulations.

Like the Greimas school, Groupe µ recognises the difference between the iconic and plastic layers of the picture sign (again using a notion of iconicity which is much more restricted than that of Peirce). In this conception, iconic figures can be interpreted because of the redundancy of the iconic layer, and plastic figures acquires their sense thanks to a corresponding redundancy of the plastic layer (thus, for instance, we recognise the bottles substituted for the eyes of Captain Haddock as a figure, because of the context of his body; and we identify the geometrical shape substituted for the circle in one of Vasarely’s works, because of the environment of repeated circles). More recently, Groupe µ (1992) also recognises iconico-plastic figures, which are produced in the plastic layers, while the redundancy occurs in the iconic one, or vice-versa (a comic strip personage which is like a human being but has blue skin would be of this kind, the bodily shape permitting recognition while the blue colour creates the deviation). Norms may be either general, valid for all pictures, or local, if they are created in a particular picture in order to be overturned: thus, the repetition of identical geometrical shapes in Vasarely’s works is the backdrop on which another geometrical shape stands out as a deviation.

The third conception of importance in the domain of pictorial semiotics is the one propounded by Fernande Saint-Martin and her collaborators, sometimes termed the Quebec school. In a number of publications (1985, 1987a), Saint-Martin has been elaborating a theory of visual semiotics which is based on the conviction that a picture, before being anything else, is an object offered to the sense of visual perception. Visual meaning, according to this conception, is analysable into six variables, equivalent to a set of dimensions on which every surface point must evince a value: colour/tonality, texture, dimension/quantity, implantation into the plane, orientation/vectorality, and frontiers/contours generating shapes. The surface points, specified for all these values, combine with each other, according to certain principles, notably those of topology, and those of Gestalt theory (cf. Saint-Martin 1980 and 1990). The principle merit of this approach is to have systematised a series of analytical conceptions familiar from earlier art history and Gestalt psychology.

Much of the importance of the Quebec schools resides in its explicit criticism of the Greimasian approach, most clearly spelled out by Marie Carani, who is also the author of important studies concerned with pictorial abstraction and perspective, respectively (Carani 1987; 1988). As compared to the binary opposition, which is the regulatory principle of the Greimas school approach, as well as to the norm and its deviations, which determines the conceptual economy of Groupe µ rhetoric, the Quebec school offers a much richer tool-kit of conceptual paraphernalia, more obviously adapted to the analysis of visual phenomena. Yet this very richness also appears to constitute the basic defect of the theory: it is not clear whether it furnishes any restrictions on what may be taken as relevant in the picture sign, which means that no analytical direction have been presented.

The constraints imposed by the grid taken oven from the linguistic theory of M.A.K. Halliday by, notably, Michael O’Toole (1994), are, in this respect, much more enlightening. According to this conception, every work realises some alternative from among the ideational, interpersonal and textual "macro-functions", renamed by O’Toole the representational, modal and compositional functions. The first function is involved with the relationships between the participants and processes in the real world, the second concerns the way in which this world is presented by the creator of the sign, and the third has to do with rules of internal patterning applying to the work as such. It seems unfortunate, however, that in trying to specify the different options available for the realisation of the different functions, O´Toole often employs traditional art-historical terms, without giving them any new definition.

In his extensive, critical review of pictorial semiotics devoted to an analysis of the linguistic heritage preserved by this science, as it appears in the conceptions of, most notably, Barthes, Floch, Thürlemann, and Groupe µ, Sonesson (1989) emphasises the basically perceptual nature of the picture sign, and expounds some of the consequences of this observation, invoking the testimony of contemporary perceptual psychology, and of philosophical and phenomenological theories of perception. Contrary to, most notably the Greimas school, he thus shuns the autonomy postulate of semiotics, admitting that pictorial semiotics has a lot to learn from psychology and other sciences, while claiming that their results must be inserted into a specifically semiotic problematics evolved from the history of this science. Critically reviewing the use of many linguistic and otherwise semiotic concepts, such as sign, feature, connotation, iconicity, and so on, he argues that these are useful only to the extent that their import are clearly spelled out, so that the specificity of pictorial meaning could emerge.

In the work of the pioneers, pictorial semiotics, even when it concerned itself with advertisement pictures, tended to make its own the traditional conception of art history and literary history alike, according to which the object to be studied was the individual, purportedly unique, work of art. Although some scholars developed models of analysis which embodied hypotheses about wide-ranging regularities found in pictorial semiosis, there has been little awareness, until recently, that pictorial semiotics, if it is to be a part of general semiotics, must be concerned with all kinds of pictures, and formulate principles applicable to all empirically occurring picture kinds, and even to all objects potentially recognisable as pictures. Such a conception, although extended to the wider domain of visual semiosis, is implied by Saint-Martin’s (1987) recent work (but is only applied to artistic pictures). Arguments to the effect that pictorial semiotics should be a general science of depiction, or of visual images, are only presented in the recent books by Groupe µ (1992: 11ff), Sonesson (1989:9ff), and O´Toole (1994:169ff).

To elucidate the meaning of pictorial semiosis must mean, among other things, to find out in what respects pictures are like other signs, and how they differ from them, most notably perhaps how they are differentiated from other signs of such sign categories to which they undoubtedly belong: the category of visual signs, and the category of iconic signs. Such as task, and even the very specificity of pictorial semiotics, obviously dissolves itself if we accept the idea of the Greimas school, according to which all meaning is of a kind, or is identical in nature as far at it is pertinent to semiotic theory (cf. Floch 1986b; Thürlemann 1990, on the other hand, conceives of pictorial semiotics as an ancillary of art history).

Curiously, Floch (1984: 11, 1986a: 12f) who defends this theory, also argues, on the other extreme, that semiotics should not concern itself with middle-range categories like "photography" and "painting", described as "socio-cultural", but should instead attend to the minute details of an individual picture. Groupe µ (1992:12) follows suit in denying the pertinence of these same categories, which the group conceive of as being "sociological" or "institutionalised". Whatever the sociological status of photography and painting, however, it seems that they are also, and primarily, particular varieties of the picture sign, embodying a particular principle of pertinence which serves to rely expression and content, and as such they should be of interest to semiotic theory.

Pictorial kinds could be differentiated from the point of view of their rules of construction, that is, the rules specifying which traits of the expression plane are relevant for conveying the content, and vice-versa. From this point of view, a photograph differs from a painting and a cut-out; and a linear drawing, to use traditional art historical terms, is different from a painterly one.

Then we may also distinguish categories of pictures according to the effects which they are intended to produce (not the actual effects, which may vary, and which cannot really be known). Thus, in our society, publicity pictures are expected (among other things) to sell commodities, pornographic pictures are thought to stimulate sexual imagination, and caricature supposedly hold the depicted person up to ridicule. Very much less well-defined is the intended effect of fine art.

In the third place, pictorial categories may be differentiated on the basis of the channels through which pictures circulate. The picture post card, for instance, follows another trajectory to reach the receiver than a publicity poster, a wall painting, a television picture, or the illustration of a weekly review.

There is an additional, very different way of distinguishing picture categories, which depends on the nature of the configuration occupying the expression plane of the picture. Ordinary language does not possess any terms for differentiating pictures in this way, but the existence of such a classification, however tacit, is suggested by the fact that some text analytical models which are very productive when applied to some pictures fail to yield any result when transposed to other pictorial "texts". To some extent, this appears to be the kind of categories which are addressed in the "rhetorical" analysis offered by the Groupe µ.

Pictorial semiotics, then, could well be conceived as that particular branch of semiotics which is concerned to determined in which way the picture sign is similar and different from other signs and meanings, in particular as far as its relationship to other iconic and/or visual meanings are concerned; and which is also called upon to analyse the systematic ways in which signs which are pictures may yet differ form each other, thus, for instance, as to construction, socially intended effects, channels of circulation, and configurational kinds.

Göran Sonesson

Bibliography:

See also:

Advertising, Icon, Iconicity, Image/picture, Isotopy, Picture (perception of), "Rhétorique de l’image", Visual semiotics