CAS

Content Aware Studies

Started in 2018, ongoing to this date

Marble, polyamide, machine learning algorithms, custom software, original dataset, multi-channel video installation;

Machine learning technical assistance:

Artem Konevskikh

Content Aware Studies (CAS) Series is comprised of ideas, AI experiments, objects, moving image works, films and essays. It inquires how the use of machine learning in historical analysis and reproduction as a scientific tool brings to the forefront ethical questions of bias contamination within data and automation of its analysis. Inspired by the examples of confusing para-scientific interventions such as AI-based Voynich Manuscript decryptions CAS series examines the various sides of this inquiry. It also speculates about material objects as synthetic documents of machine-rendered histories.

The objects from the CAS series came through methodologies developed with data scientists and based on training artificial neural networks aiming to replenish lost fragments of sculptures, friezes and other objects of classical antiquity as well as to generate never before existing, yet “algorithmically genuine” objects of that era. The research examined outputs of advanced AI models trained on datasets consisting of thousands of 3D scans of classical sculptures from renowned international museum collections. The models generated by the algorithm were then 3D-printed in various synthetic materials, filling the voids in the eroded and damaged marble sculptures. Some of these algorithmic outputs were turned into new entire marble sculptures carved by machines. Uncanny in their algorithmic integrity, they posed questions about whether they can be considered objects of classical antiquity. They render the work of a synthetic agency that lends a faithful authenticity to the forms, while also producing bizarre errors and algorithmic normalizations of forms previously standardized and regulated by the canon of Hellenistic and Roman art. In its second iteration, CAS challenged previously established AI methodologies, against data from prehistoric and geologic time archives including first stone tools, writing systems, paleontological archives of fossilised plants, organisms and other biogenic data.

It operated on the datasets primarily comprised of the findings archived and documented in repositories of contemporary natural history museum collections, through which the objects of synthetic histories came about as a result.

These speculative forms of restoration, museology, and historiography provide a case study for the critical examination of misleading trajectories in knowledge production and epistemic focal biases that occur at the level of computational software operations. Preoccupied with ontologies derived from biases, misleading guises, seeming authenticities and mixed-up materialities it seeks to highlight and warn about epistemological issues of computationally accelerated studies. The series focuses on questions: what are the ethical, philosophical, and historical challenges we’re facing when using computationally automated means of knowledge production and investigation? What epistemics do such methodologies hold by uncovering deeper and sharply unsuspected new knowledge or instead masking unacknowledged biases?

A series of essays address these experiments via the new materialist frameworks in a non-anthropocentric way, while seeking to locate the subjects of investigations as encounters between non-organic bodies. In the optics of a non-human agency of the AI investigator, what of our historical knowledge and interpretation encoded into the datasets will survive this digital digestion? How are historical narratives, documents, their meaning, and function perverted when their analysis has been outsourced to machine vision and cognition? In other words, what happens to historical knowledge and documentation in the age of the information-production epidemic and computational reality engineering?

Content Aware Studies film

Computational documentary film-essay, machinima; apr 30 min. The film is currently in a work-in-progress state.

The film intends to open up speculations and thought experiments into history, matter, agency and computation. History in this context is seen as data; while data is seen as a crude material and critical resource for content-form-knowledge production through which production and investigation? questions of origin and genuine-ness are posed and aesthetic implications can be studied. How are historical narratives, documents, and their meaning and function perverted when they collide with ubiquitous machine vision and translation? In other words what happen to historical knowledge in the age of the information epidemic aforementioned and computational reality engineering? These questions are asked about synthetic forms of knowledge production as a result of outputs of machine-learning (ML)-technologies operating on historical archives. They inquire about the capacities and consequences of such machine-learning technologies as a means of automated historical investigation and question whether these findings still hold historical qualities.

One of the main questions about technology and culture, posed here is: what are the ethical, philosophical, and historical challenges we’re facing when using such automated means of production and investigation? Can applications of such technology allows us to uncover deeper and sharply unsuspected new knowledge or do they mask unacknowledged biases?

As part of this investigation, we look into the project Content Aware Studies (CAS), which through artistic practice seeks to establish investigative methods of these machine-learning-capacities. This research examines how various advanced AI, or more specifically General Adversarial - Networks (GANs), which are particularly known for their recent advancements in computer vision, cognition, and hyper-realistic image rendering operate when trained on datasets consisting of thousands of 3D scans from renowned international museum collections. Specifically trained neural network models are directed to replenish lost fragments of friezes and sculptures and thus generate previously never existing objects of classical antiquity. The algorithm generates results convertible into 3D models, which are then 3D-printed in synthetic materials and used to fill the voids of the original sculptures, or turned into entirely new machine-fabricated marble objects; Faithfully restoring original forms, while also producing bizarre errors and algorithmic interpretations of previously familiar to us Hellenistic and Roman art, which are then embodied in machine carved stone blocks. Uncanny in their algorithmic integrity they render the work of a synthetic agency that lends a faithful authenticity to the forms, while also producing bizarre errors and algorithmic normalisation of forms previously standardised and regulated by the canon of Hellenistic art.

Content Aware Studies | Alexander Levy, Berlin, DEU, 2019

Human (un)limited | Ars Electronica Export, Moscow, RUS, 2019

NTAA’22, New Technological Art Award | Zebrastraat Ghent, BEL, 2022

You and AI: Through the Algorithmic Lens | Onassis Foundation, Athens, GR, 2021

Thoughts on materiality

Materiality has reappeared as a highly contested topic in recent art. Modernist criticism tended to privilege form over matter — considering material as the essentialized basis of medium specificity — and technically based approaches in art history reinforced connoisseur- ship through the science of artistic materials. But in order to engage critically with materiality in the post-digital era, the time of big data and automation, we may need a very different set of methodological tools.

We may need to address digital infrastructures as entirely physical and to reexamine the notion of “dematerialization”, by addressing materialist critiques of artistic production, surveying relationships between

matter and bodies, exploring the vitality of substances; and looking closely at the concepts of inter-materiality and trans-materiality emerging in the hybrid zones of digital experimentation. The image below is a result of the interpretation of an antique portrait by a general adversarial neural network based on the analysis of nearly 10,000 3D scans. The custom-created dataset included 3d scans of sculptures from the collections of the Metropolitan Museum, Hermitage, British Museum, National Museum of Rome and other world-renowned collections of antiquity;

CAS essayS ON SYNTHETIC HISTORIES

A series of essays on Synthetic Histories, Hylomorphism and Materiality, and Predispositions by Design in which the work Content Aware Studies is at the centre as a case study for the proposed critique of AI-driven methodology in historiography.

Links to downloadable PDFs:

Essay On Content Aware and Other Case-Studies: Historical Investigations at Blazing Ultra Resolution

Back to main page

Content Aware and Other Case Studies: Museum of Synthetic History