Case Study: ‘Poetics of Search’
Digital Search Processes as a Critical Methodology in Photocuratorial Practice
Exhibition, research and text: Nadine Isabelle Henrich
Starting points
The ‘Poetics of Search’ project positions search processes and database structures as new sub-areas of photography. The exhibition introduces search as a narrational entity and digital co-curator by activating the semantics of digital databases as a contemporary infrastructure of curatorial practice at the level of content and making this semantics visible in the exhibition display. The focus here is on the question of what possibilities arise for the orders of photographic visibility through the testing of new search methodologies.
The project, which was originally devised as an exhibition concept for Fotozentrum Schweiz in spring 2022 under the title ‘Poetics of Search’, is intended as the sketch of a method, a case study and an experimental form of dialogue. It is meant to act as a stimulus, prompting tests of new approaches to interacting with the digital infrastructures of public museums and making them fruitful for the curatorial concept. The idea emerged from the linking of different fields that raise new challenges and questions in current curatorial work. First and foremost, how can an exhibition display be developed that clearly manifests a collaborative process, makes itself vulnerable and reveals the dynamic quality of institutional structures and museum practices? In addition, the project focuses on the question of what new methodological approaches can emerge from current technological developments. For example, how approaches to perceiving and exploring historical material can be reshaped through cooperation with non-human co-actors, such as search algorithms, artificial intelligence (AI), Stable Diffusion, or Generative Adversarial Networks (GAN) technologies. ‘Poetics of Search’ thus constituted a sketch in the form of an exhibition that tested the process of reflecting on and critically engaging with digital search processes and their potential for novel ways of interacting with and approaching the collections and archives of Fotomuseum Winterthur and beyond.
Visibility as searchability
The architecture of digital databases has radically changed the orders and structures of visibility in the field of photography. In a present where unmanageable digital information platforms and image and data collections are navigated, digital search is an instance of mediation that is upstream of the viewing process. Instead of visiting real spaces, rummaging through boxes and searching through folders, contemporary research in photographic collections and archives is mainly done via databases and their search functions – and is thus text-based and remote. Here, these new digital infrastructures and tools not only organise digital photographic data but are prioritised over contact with physical objects. Digital search consequently also moderates access to museum collections. Today, the orders of photographic visibility are essentially structured by their searchability. However, this transformation has not yet been represented to any great extent in curatorial theory and practice.
How can we reflect on the transformation of the way visibility is mediated by search engines and tools and test it in interactions with collections to discover new patterns at a remove from familiar paths, while countering encoded biases with shifts in perspective? Machine-supported digital search is both a tool and a stimulus that inscribes itself in all processes of idea generation, research and selection for exhibition projects, as well as in research-based museum work in general. Nevertheless, the search process itself is understood as a kind of standardised, logical path within the limited framework of the search screen and is executed in a purely functional way. It is precisely this (mis)understanding of search as a standardised procedure that is addressed and examined in ‘Poetics of Search’. The search process is tested as a critical method and specifically activated by a ‘voice’ and made visible as a co-actor in the cultural present. The exhibition Poetics of Search sees and treats Fotomuseum Winterthur and Fotozentrum Schweiz as a space of search processes, offering a case study to involve visitors with both a professional and a general interest in the topic and raise awareness among this public of the ‘epistemology of search’ [1] as an area of the photographic. The main concern is to put searchability and its semantics at the heart of the exhibition’s genesis, while at the same time using digital search as an instrument of counter-narrative. By making the focal points of the collection and its blind spots part of the narrative and by incorporating into the display metadata that is normally only used internally, we are able to weave the logic of the database and its wealth of information into the physical presentation.
The exhibition offers different reading directions, multiple levels of content and possibilities of perception by feeding the terms of two selected poems by under-represented voices into the search functions of the museum database of Fotomuseum Winterthur; from the search, a multi-perspectival dialogue emerges as an open constellation of image and text. The lyrical text of the poems allows us to engage with those voices whose perspectives are absent from the collection holdings. The search results yielded by the searchable keywords create an image-based translation of the poetic texts that is, however, fundamentally different from their point of view: Photographs that tell stories of industry, economics, prosperity, artistic freedom, the military and the documentation of ‘the other’ by white photographers enter into dialogue with a poetic counter-narrative of resistance to racism, exploitation, discrimination and systematic marginalisation. The vocabulary thus becomes concrete in its meaning from different perspectives. The metadata on the number of search results also provides insight into those motifs and thematic complexes that are particularly well represented in the collection and those that remain absent. The display thus weaves together different narratives that can be interpreted individually by the viewer. There is no intention for there to be a clear overarching narrative. Rather, an approach is presented as to how we can approach archives differently and how new modes of access and unforeseen connections can be tested employing experimental usages of search tools. In its intention to challenge linear narratives, methodological fetishisms and chronological orders of photographic archives and collections, the project ties into the recent iteration of the anarchive discourse heralded by Hal Foster in his essay ‘An Archival Impulse’, which looks at contemporary artistic practices. [2]
Dialogues
‘Poetics of Search’ takes two poetic texts as the starting point of dialogue with the collections and archives. The voice of a poet from a community whose perspective is currently under-represented in the collections – for example, Americans like Audre Lorde who identified as Black and lesbian, and people like Quique Avilés who left their homeland of El Salvador and often struggle with living and working in the US as part of the second largest group with ‘illegal’ status. The search text thus brings the photographs into contact with perspectives that are not included in the collections. What is the significance of the worlds in which the authors’ voices are located for their texts and the perspective they reveal relating to the objects in the collection? For example, when Lorde writes ‘tree’, this supposedly generic word is connoted with an experiential, historical and political dimension that is fundamentally different from the range of meanings that would apply to the use of ‘tree’ in a descriptive image title in everyday museum settings. ‘Tree’ in Lorde’s work evokes the history of hatred, violence, racism, nationalism, radicalisation and crime that spreads through the word ‘tree’ in the United States like a toxic root system, giving it a fundamentally different gravity than it implies in other people’s usage. Lorde’s ‘tree of anger’ is not the same ‘tree’ that photographer X shoots or chooses for the title. Even though Lorde writes ‘march’, the meaning is historically specific and presumably refers to the critique of white US American feminism in the 1960s and 1970s, which struggled for equality with white men within the very limited perspective of white, privileged women, but was still far from a holistic, intersectional perspective of cross-class, anti-racist equality. The marcha política that Graciela Iturbide invokes in the title of her photograph, and which is displayed as a corresponding search result to ‘march’, enters into dialogue with Lorde’s perspective insofar as it is directed at the struggle of indigenous communities of Mexico against their oppression, disenfranchisement and exploitation. Although the geographical context differs, Iturbide’s photographic perspective rues the colonial-historical and intersectional context of different protest movements and struggles against oppression, to which Lorde’s poetry is also dedicated.
But beyond these specific close readings of such combinations of a poetic search word and photographic search results, the effect of destabilising and pluralising institutional semantics is central. This is because the transformation of the semantic stability of the collection vocabulary exposes gaps and reveals those other readings and disparate experiences that differ from the selected positions in the collection and archive holdings. Consequently, the activation of the search texts through the voice of the absenter creates a destabilisation and reordering of existing contexts, allowing new readings to open up in supposedly familiar photographs and the messages they convey. The poetic search text here reveals an untrodden (search) path through the collection.
Point of Reference
In the hyperspace of the Internet, where ‘there are no longer isolated entities in themselves, that is, no “subjects”’, Byung-Chul Han locates a ‘hypertextual mode of experience’. [3] Referring back to Literary Machines, Theodor Holm Nelson’s standard work on hypertext published in 1981, he develops the concept of ‘windowing’ in this context: ‘All mirror each other or let others shine through in them.’ [4] What is relevant here is the dissolution of solitary subjects who pluralise themselves online in favour of an essential mode of ‘multiplication’ of the self, of connectivity and polyphonic cooperation. Furthermore, the role of the reader is transformed: they can move in different directions and make new connections in a newly open information structure, as Han points out: ‘The reader is no longer thrown into a predetermined, quasi-monochrome structure of meaning and order. Rather, they move actively, laying paths independently through the multicolored space of hypertext.’ [5] How can these observations be related to the attempt made here to create a different kind of exhibition and display in the work with photographic archives?
Nelson’s concept of ‘active reading’ is relevant to ‘Poetics of Search’ in that it conceives of reading as an active and formative practice and presents it as a critical and self-determined method. Hypertext allows connections to be made between disparate texts and narratives from different contexts, thereby linking different sources and allowing one to create one’s own connections not provided for in the material. It consequently allows one’s own narratives to be woven into existing texts, allowing them to be given a new reading. This shift from a supposedly standardised practice to a malleable method corresponds to the way in which digital search takes on a critical agency in this project and acquires a curatorial-conceptual dimension.
The recognition of the fundamental mode of interpenetration and mutual mirroring that Nelson and Han postulate for our interaction with information online can be linked to the recently coined term ‘networked co-curation’ with regard to curatorial practice. [6] Annet Dekker and Gaia Tedone introduced this in 2019 to accentuate the mode of cooperation that conditions curating in the digital space: ‘Networked co-curation can be understood as both a theoretical concept and an operational strategy for forging a critically reflexive mode of online curation. It opens the curatorial activity to participation from human and non-human agents and largely relies on digital tools and network infrastructures.’ [7] Dekker and Tedone show that digital curating is constitutively embedded in networks, economic causal structures, automated information flows and social dynamics. However, this sphere cannot be sharply delineated from putatively ‘analogue’ or ‘physical’ museum practice. Rather, digital curation and museum practice are in exchange with each other and are steadily becoming more closely linked through increasingly digital data management, online research, and database-driven collection management. The ‘use’ of online search engines and search functions in dealing with databases in museum practice must be understood and reflected upon beyond digital curating, analogous to an assemblage of non-human and human actors, since the digital user’s search tool interface moderates our dealings with physical collections and archives and essentially structures their accessibility and visibility, albeit unnoticed.
Automated search, as an indispensable non-human collaborator, allows us to navigate unmanageable amounts of data online and to pick out those records that are relevant to us from the sea of information. The usability of large data collections is fundamentally linked to effective search functions and the search operates, following Nelson’s hypertext, as an upstream co-actor actively suggesting connections. Our perception of patterns, the making of new connections and the formatting of data are at the heart of contemporary ideation. The reflections of ‘Poetics of Search’ therefore tie in with the ideas of US American art historian David Joselit, who addressed them in 2011 with the term ‘epistemology of search’. [8] In his text ‘What to Do with Pictures’, he put forward the important thesis that in contemporary cultural production there is a shift from the production of new content to the formatting of already existing content. Today, new knowledge is created by finding meaningful patterns, which only become recognisable when navigating large databases with the help of search engines. Thus, search, as the overlooked non-human protagonist of the present, fundamentally shapes our current cultural production. With an awareness of this framework, conclusions must be drawn and new methodological approaches to curatorial practice developed.
‘Windowing’: The vulnerable display
The digitisation of the collections of public museums and the novel possibility of arranging them according to specific search criteria, such as gender, the nationality of the artists or the geographical origin of the collection objects, mean that collections can be subjected to a new kind of statistical analysis. They make institutions ‘transparent’ by revealing their patterns and their mechanisms of inclusion and exclusion manifested in numerical form. What conclusions can be drawn for curatorial work from the possibility of such digital transparency becoming visible? Expanding on Han’s description of the digital constitution as the state of allowing others to shine through in oneself, we can view ‘windowing’ as a vulnerable display within the terms of the exhibition concept. This allows institutional mechanisms, internal ways of working and the resulting biases to shine through. The internal information of the search results in the museum database ties the exhibited photographs into the fabric of the collection, linking them to the images that are not exhibited and, in the absence of search results, to those that have not been collected. The gaps in the display and the metadata from the search process are visibly marked, making the internal conditions that gave rise to the selection of works comprehensible at least to a degree. To understand the exclusion mechanisms and incompleteness in their social and political dimension, the information that is offered constitutes a tool for raising awareness among visitors and bringing them into contact with hidden fields of decision-making that are often unknown or remain invisible at least for the most part.
In the context of the ‘Poetics of Search’ project, ‘windowing’ is thus intended to denote the attempt to understand the exhibition display as a window into institutional structures and internal processes. The opening of the closed form of the display allows visitors to see the exhibition as a momentum in an ongoing process of historiography, the exploration of value systems and processes of inclusion and exclusion. To actively break open the appearance of the exhibition as a closed outcome – generated by the professional processes of an influential institution whose products suggest a certain sovereign power of interpretion – making the processes of creation and of search and selection transparent is an approach to making oneself vulnerable as an institution and opening up to visitors playing a critical and active role. Which objects are thematically important for the exhibition but could not be exhibited owing to conservation concerns? What searches were undertaken but without success? What other arguments were weighed but discarded in the process? And further, what photographs from other perspectives would have been equally relevant to the depiction of the historical event but are not in the collection? Why were they not purchased?
The ‘Poetics of Search’ project makes the search the starting point for addressing some of these questions and making the relationship between the exhibited objects and the collection transparent holdings by indicating the internal search results in the work information. If around ten results were found for the term ‘triumph’, with two of the hits included in the display, while the search for ‘confusion’ remained fruitless, this becomes immediately comprehensible in the display. In this way, a parallel narrative about collection holdings, the selection processes that led up to their acquisition and the collection focuses that have developed over time is woven into the exhibition, affording an insight into the policies of the institution.
Through the text-image combination in the exhibition display and the search and museum database metadata in the work labels, the chosen display ventures to intervene somewhat in the autonomy of the work. The display ties the individual photograph and the individual photo book into an informational structure that engages with the processes and politics of selecting and collecting in a subliminally self-reflexive way. The institution’s long-invoked gesture of neutrality – its aesthetic withdrawal and attempt to make itself transparent in the exhibition space, by being invisible, especially in the field of contemporary art – is countered here by a contrary vision of transparency: the transparent institution is not invisible but rather visible and viewable in its internal processes and policies. As a result, the collections it has amassed over time are understood not so much as exemplary canons but rather as the result of human decisions in a specific context and a framework of interests. The design and the information provided in an exhibition are closely linked to the question of whether one wants to think about and question the conditions under which institutions emerge and are produced in the context of knowledge formation and historiography and to examine this as an exhibition object.
Case study Winterthur
Using the specific example of the collections and ephemera of Fotomuseum Winterthur and the publications in Fotozentrum’s photo library, this means reflecting, among other things, on the fact that the collections have a focus on American photography of the 1970s, 1980s and 1990s. For example, they have prominent holdings by Robert Frank, Lewis Baltz, Joel Sternfeld, Nan Goldin and Larry Clark. These collection holdings tell stories about the United States of that time: we can see in them, for example, the crack epidemic that gripped the United States in the mid-1980s and the human losses caused by the AIDS crisis. They tell stories from the perspective of white America. What is missing, on the other hand, are the lives and struggles of people with a migratory background in the US, the experience of racist violence against African American, Latinx people and people of color in American society. Oppressed and exploited people are often ‘documented’ as marginalised or as objects of social grievance and unrest. In Sternfeld’s photograph Domestic Workers Waiting for the Bus, Atlanta, Georgia, April 1983, from the series American Prospects, a small group of workers stands many steps away from the photographer in the middle of a wide leafy street that is home to their white employers.
Another shot by Sternfeld shows a black activist protesting against the newly created Civil Rights Museum, which documents and musealises violence against the African American civil rights movement. The image condenses both a white documentary photographer’s view of the protest of ‘others’, as well as oppressed people’s discomfort and mistrust of museums that are about them but fail to involve their perspectives or invite their participation.
How can an exhibition be created that addresses these missing perspectives and gaps in collections without hastily filling in blind spots through loans or new acquisitions, thus obscuring the imperfections that are a constituent feature of historically developed collections? Isn’t it desirable for viewers to be aware that every collection can be understood as the result of historical selection processes that manifest (past) value systems? Collections tell the story, contained in their inventories, of how the institution sees itself, of art historical methodologies, the value concepts of circles of patrons and friends, the dynamics of the art market, economic structures and cultural-political interests. The contextual setting and the conditions governing an institution cannot be separated from its collections and should not be veiled retrospectively but rather made visible and conscious.
Making absences visible pursues the goal of transforming the perception of the collection into a conscious encounter with its hierarchies and systems of order, and of understanding the institution not as morally superior and omniscient but rather as a register of the transformations of social power relations and the value systems that result from them. The decision to put one’s own collection under scrutiny and to ask who is not represented, what stories are not told by their protagonists but mediated by others, who did not profit from the institutional acquisition of the works and whose works and archives were not actively preserved is consciously revealed with ‘Poetics of Search’ in the constellation of text and search results. For absence is not to be misunderstood as non-existence. In the words of Dominick LaCapra, ‘Paradise absent is different from paradise lost…’ [9] Couldn’t the act of empowering the visitors to understand the inclusions and exclusions of a collection as a discursive, dynamic practice that needs to be explored again and again be the impetus for making exhibition concepts and displays vulnerable and open to renegotiation on the basis of critical search processes?
African American perspectives
The photographers collected by museums were often those found through human networks, especially before the Internet’s new search capabilities became available: galleries and publications drew attention to artists who were able to establish contacts with curators and museums. The often prominent representation of American photographers in European and Swiss collections is rooted in the early institutionalisation of the medium in the United States in the mid-20th century, as well as the representation of female photographers and artists working in photography by influential agencies and galleries. This early embrace of photography, however, was selective in the US, to the exclusion of African American women photographers. Beginning in the 1980s, research and exhibitions by photo historians and curators like Deborah Willis specifically sought to fill these fundamental gaps. Early on, Willis showed great commitment to addressing the voids in the photographic and art historical canon. Even before her first university degree in 1975, she wrote a proposal to begin her own freelance research project on African American photographers who were still alive and not part of the white establishment’s mainstream narrative. At the same time as Lorde’s Who Said It Was Simple was being written, she was working on the basics of making biographical information on African American photographers extensively available for research for the first time. At the time it was Gordon Parks who was the first photographer to respond to her research request and who became a lifelong friend and dialogue partner. In 1985 and 1989, two directories of biographies of African American photographers were published, in which Willis published her research results as foundational works. [10] In an interview with Leica, Willis later named a number of photographers whose work she considered particularly relevant: Gordon Parks, Bruce Davidson, Hank Willis Thomas, Carrie Mae Weems, Lorna Simpson, Deana Lawson, Sheila Pree Bright, Adama Delphine Fawundu, Allison Janae Hamilton, Tyler Mitchell, Moneta Sleet, Jr, Jamal Shabazz, Marc Ferrez, Miriam Romais, Susan kae Grant, Eudora Welty, Jeanne Moutoussamy-Ashe, Ruddy Roye, Jack Franklin, Ray K. Metzker, Roy DeCarava, Bayete Ross Smith and Ernest Withers.
When I entered the above names in the search mask of the online collection of Fotomuseum Winterthur on 27 December 2022, I did not find any works by Gordon Parks in the collection, for example, although he is at least mentioned in two very readable articles by the American photo historian David Campany in the discursive blog Still Searching… A search of all the names yields the following results: Gordon Parks (0 works, 2 mentions in blog essays), Bruce Davidson (0), Hank Willis Thomas (0), Lorna Simpson (1 series of 10 works), Deana Lawson (0 works, 1 product in the bookshop), Sheila Pree Bright (0), Adama Delphine Fawundu (0), Allison Janae Hamilton (0), Tyler Mitchell (0), Moneta Sleet (0), Jr., Jamal Shabazz (0), Marc Ferrez (0), Miriam Romais (0), Susan kae Grant (0), Eudora Welty (0), Jeanne Moutoussamy-Ashe (0), Ruddy Roye (0), Jack Franklin (0), Ray K. Metzker (0), Roy DeCarava (0), Bayete Ross Smith (0), Ernest Withers (0); of 23 African American photographers with medium to high levels of familiarity, work by only one photographer is found in the collection, making the hit rate in the collection database online 4.34 per cent.
Let us return to the Poetics of Search display and the attempt to use it as a window into the collection database and institutional processes and policies. Making the conditions under which knowledge and history are produced in museum institutions visible and thereby also vulnerable is tested here through metadata on search results in internal databases. If a term in the search text does not yield a result, this is just as immediately comprehensible as the decision not to show works due to restoration concerns. The setting of the collection is put into perspective by two under-represented voices, for in the contrasting connection of poetic textual level and the photographs collected together using the search tool, the difference in perspective becomes evident, as does their linkage through historical, economic and political contexts. The vocabulary Lorde uses to evoke her struggle for equality and empowerment meets vernacular photographs from Swiss industrial history from the 1960s–1980s. On the photographic level, as a supposed counterpoint to the text, industry and the military often form central thematic areas that can be read as linked to Lorde’s and Avilés’s families and migration stories.
Collection revision
To satisfy the need for collection and exhibition parameters to be revised and constantly adjusted, digital search is a key tool – a collaborative partner in the analysis of biases, voids and one-dimensionalities. In addition to the necessity of dedicating exhibitions to under-represented artists and image producers, ‘Poetics of Search’ explores the question of how collections and archives can be approached in a way that does not accord with the classical paths of everyday museum life and thereby implicitly perpetuate the structures, hierarchies and orders of the institution. The revision of collections from a postcolonial, antiracist, intersectional and empowering perspective implies a dynamisation and destabilisation of the traditional orders and categories. This necessary shaking affects how museum collections are perceived, especially by the people who interact with collections and choose what is made visible to the public and preserved in collections for the future. Each new exhibition project is a readjustment of how we look at the collection and understand its narratives.
If one imagines the collection as an island, its formation results from the different forces and actors that model the ocean’s boundary and thus determine the island’s shape. Below the water, however, it is connected to and in a process of exchange with extensive formations that have produced its form in the first place. However, this fabric and the invisible areas that are not part of the collection’s holdings or its visible presentations are no less relevant. Rather, the collection can be understood as much from what is absent – from what has not been collected, not stored, not exhibited, not cataloged, not named, not made searchable – as from its visible, present holdings.
Dark Archives vs. Negative Archives
In databases and online archives, we speak of ‘dark archives’ when data collections remain invisible as well as unsearchable and thus inaccessible and inoperative for users. This state resembles a shadow existence, in which information is managed as existing but inaccessible. Let us transfer this scenario to the physical holdings of public museums: with regard to the museum landscape in Switzerland, for example, experts estimate that up to 90 per cent of the works in museum storage are rarely or never exhibited; it is likely that the situation is much the same in Germany. This bulk of collection objects, which remain stored out of sight in archives and collection repositories, can be regarded, by analogy with databases, as physical ‘dark archives’, unused, dormant resources in our museum culture. If they are accessible at all, these holdings are made searchable and digitally visible, with the help of online databases, only as digital copies. Mediated by the experimental search of ‘Poetics of Search’, several photographs came to light that had never been exhibited before.
Lorde’s search text thus also led to those areas of the collection that exist in the shadows, whose photography holdings lie dormant and remain unseen away from well-trodden thematic or scholarly paths. The digital search in the museum database is democratic: vernacular photographs, ephemera and artistic photographs of diverse contexts and disparate economic values emerge as search results with identical probability. However, whether a digital image has been entered into the dataset often reflects the relevance attached to the object internally.
The discourses of postcolonial theory and feminist institutional critique have fundamentally challenged the canon of ‘Western’ institutions in its claim to validity and put its parameters, values and power structures up for discussion. Against the background of this claim to a redefinition of the self-image of public museums, every museum collection with a long history is to be interrogated about those positions that were not collected. Through this shift in perspective, the collection becomes tangible as the result of processes of selection and, therefore, exclusion. The ‘negative archive’ is therefore meant here to denote what is absent, excluded, not found, etc., and to function as a negative form and invisible counterpart to the photographs that are present in the collection. This absent, negative archive is to be evoked through the activation of search as an exhibition object and the strategy described above of ‘windowing’ failed searches that indicate absences without results.
Text-image (re)configurations
Starting from the link between searchability and visibility, the exhibition Poetics of Search focuses on the transformation of the photographic medium through its novel relationship with (search) text. To make this paradigm shift tangible and visible, it not only introduces the search text as a content narrative to engage with collections and their biases but also explores artificial intelligence (AI) image generators (GAN), such as DALL-E, Midjourney and Stable Diffusion. To this end, poetic text was versioned as a prompt and thus translated into GAN photographs. These poetry-based synthetic photographs are placed in dialogue with the search results from the collection to point to the perspectives and voices that have been systematically rendered invisible across the holdings.
In the process, the exhibition explores the emergence of these AI-based new imaging technologies in terms both of their potential as artistic and activist tools for visualising alternative historical narratives and counter-narratives and of their problematic nature as a synthetic perpetuation of historical biases. These result from the sexist and/or racist patterns inscribed as biases in the archives used to train machine-learning technologies. In this broad field, which is being transformed on a daily basis by the learning processes of artificial intelligence, Poetics of Search focuses primarily on three questions: How is the new role of text-based digital search changing the relationship between photographic data and text? How can this transformation of the configuration of text and image be reflected and exhibited? Can new AI-based image technologies be used to highlight missing perspectives and visualise counter-narratives?
From text to image and back
Artificial intelligence (AI) – or so-called synthetic cognitions (SC) – is increasingly developing the ability to recognise and categorise image information by training its algorithms on large datasets from image databases. We all contribute to the training of these image-recognition algorithms when, for example, we are asked by Google to prove our human identity by having to select images of traffic lights, for example, from nine motifs.
Based on these developments, it is now possible to produce new images that resemble photographs based on information in text form. For example, GAN can be used to synthesise new images from millions of bits of image data. GAN platforms like Midjourney or Dreamstudio, which allow users to create images by entering text and thereby transform their own ideas and fantasies into pictures, are tools that redefine the relationship between image and language.
In the second part of the exhibition Poetics of Search, poetry texts by Audre Lorde and Quique Avilés were translated into synthetic images using GANs such as Midjourney and Dreamstudio. The photographic quality of the images of the poems were created in dialogue with the GANs in an iterative process. The entire poem was inserted into the text prompt. The adaptation of the prompt that caused Midjourney to create the image was reworded several times to comply with the platform’s guidelines. This prohibits, among other things, terms related to violence and sex. Therefore, the author, in collaboration with Juan Diaz Bohorquez, replaced the term ‘sex’ with ‘touch’ when using Midjourney on Lorde’s poem. This involved an interpretation or versioning of the original text.
From this modified prompt, the platform ended up producing four draft images, from which one was selected. This was then further developed again by GAN and resulted in another four versions, from which one motif was chosen for further work. This process was repeated about twenty times and the image results were further developed step by step in dialogue between GAN and the user. The process is documented by screenshots so that it can be reconstructed moment by moment in the exhibition.
94 words in a photograph
GAN and Stable Diffusion platforms like Midjourney and Dreamstudio, which were used to translate the poetic texts into photographic images, are continuously trained on image databases. These contain millions of image records that are accessible online and come from very different contexts. They might originate, for example, from freely available digitised copies taken from a large number of historical archives. Owing to the varying progress of digitisation processes in historical collections worldwide and the technology companies whose market leaders are often based in the US, numerous GANs were initially trained primarily on US American datasets.
Upon closer examination, the image versions of Lorde’s poem Who Said It Was Simple generated with Midjourney reveal this focus on US image archives and their racist and violent dimensions. A faceless crowd has gathered under a mighty tree. The broad trunk tapers downward, and its bark opens there to a burnt-looking surface. The figures are shadowy, their faces unrecognisable, partly covered by masks made of black cloth. Around the tree, the pictorial space brightens, perhaps hinting at a fire. The figures stand close together but are not engaged in concrete action. It seems to be a gathering to witness something, to observe. One group of three figures wear cowl-like garments reminiscent of anthropological photographs of indigenous communities in African countries around 1900. In the process of viewing, the texture of their clothing becomes tree bark, which is superimposed on the garment like a projection. It is these shifts in the process of perception that destabilise the image and characterise it as a network of unequal pictorial information.
Above this group of figures, thick ropes hang down from a branch. The scene triggers unease and carries a violent subtext, as it recalls the ‘lynching murders’ of African Americans: a public form of murder practised in the US between the 1890s and the 1950s and extensively documented in photographs and distributed as picture postcards until the 1920s. The racist image regime served as an element of social control and oppression both under colonialism and in US society.
These problematic photographic archives feed into contemporary image technologies, perpetuating the patterns of historical power structures and their violent image regimes in the present. Thus, while the synthetic image remains ambivalent in its pictorial plot, upon close inspection it nevertheless seems to combine patterns of different types of images from colonial contexts as well as the history of slavery and racial violence in the United States. On the part of the technology companies, there is only limited transparency with regard to the image databases used to train the image technologies. For example, the LAION-5B training dataset, a library of 5.85 billion images used for Stable Diffusion and Google Images, could not be examined for this project to assess observed patterns. Thus, the interpretation outlined here remains a hypothesis to be explored in further research.
Tags
Let us return to the collection database in Winterthur. In museum databases that are not described in more detail by tags and linked on the basis of content and motif, the meaning of artists’ names and original picture titles and descriptive titles takes on new relevance. Today, access to unknown artists and vernacular photography is often dependent on terms and locations found in their image titles. This ineffective searchability of digitised collections, with only rudimentary work information fed into the database, stands in opposition to the potential ‘democracy’ of search. This is because, in theory, digital search promises to break down the hierarchies that structure the visibility of museum collections according to the principles of familiarity, economic value, etc. Practically, however, very few museum collections are extensively tagged and entered into databases with a range of information.
If we acknowledge the role of searchability in working with photographic data and digitised collections and archives, then the relevance of the breadth and diversity of information that museums link to photographic data becomes evident. Let us, therefore, imagine the collection of Fotomuseum Winterthur as a potentiality of photographic search activities, since these are currently only searchable to a limited extent: by name, title, series title, year, location and technique. Let us now contrast this state with a future networked breadth of information, in which images from the most diverse contexts can be linked by tags that are also based on implicit connections. A notional example: historical photographs of Swiss textile factories in Winterthur would be related by tags such as #textile industry or #cotton to photographs of cotton plantations in the southern states of the US based on the cost of the unpaid labor of enslaved people. Switzerland’s industry also played a role in the transatlantic slave trade triangle that connected West Africa, the Americas and Europe. Thus, even the connection to the island nation of Grenada, home to Audre Lorde’s family and for a time the most important French colonial naval base in the Caribbean, would be a distant reference point in the digital cartography of an interconnective Swiss industrial and photographic history.
How can such a pluralisation and diversification of ways of dealing with databases be achieved? A revision of the collection database that understands tagging as part of a reflective and critical practice that influences the visibility and contexts of the collection would be conceivable. For example, it would be interesting to invite people interested in photography from different social contexts to participate in tagging the collection to assign tags from diverse perspectives and make connections that may be outside the perceptual modes of the institution itself.
Conclusion
Recognising a fundamental link between the photographic dispositif and the searchability of digital and digitised images both in museum databases and online using search algorithms and search engines, ‘Poetics of Search’ opens up the testing of new search avenues as a tool of curatorial methodology. Activating the search text as a voice was implemented through the use of poetic texts that brought under-represented perspectives into dialogue with the collection. The project is intended as a case study to test the extent to which digital search can be understood as a reflective practice of curatorial work, allowing new methodologies to be developed. Keep Searching!
References
[1] David Joselit, ‘What to Do with Pictures’, OKTOBER 138 (2011), 81–94.
[2] Hal Foster, ‘Ein archivarischer Impuls’, OKTOBER 110 (2004), 3–22.
[3] Byung-Chul Han, ‘Windowing und Monaden’, in Hyperkulturalität: Kultur und Globalisierung (Berlin: Merve Verlag, 2005), 48–50, here: 49.
[4] Ibid.
[5] Ibid., 48.
[6] Annet Dekker and Gaia Tedone, ‘Networked Co-curation: An Exploration of the Socio-Technical Specificities of Online Curation’, Arts 8, no. 3, 86 (2019), 1–14.
[7] Ibid., 11.
[8] Joselit, ‘What to Do with Pictures’.
[9] Dominik LaCapra, Writing History, Writing Trauma (Baltimore, MD: Johns Hopkins University Press, 2001), 57.
[10] Deborah Willis-Thomas, Black Photographers, 1840–1940: An Illustrated Bio-Bibliography (New York: Garland, 1985) and Deborah Willis-Thomas, An Illustrated Bio-Bibliography of Black Photographers, 1940–1988 (New York: Garland, 1989).