Window

Installation transforms digital data into physical form

How can one gain an insight into other people's personal, subjective aesthetic experiences in a gallery context?

Window is an installation designed for the National Gallery of Scotland to help gallery visitors gain an insight into other people’s personal, subjective aesthetic experiences in a gallery context. Acknowledging the complexity of these internal experiences, Window takes a pragmatic approach by focusing purely on the externally discernable elements of visual aesthetic experience – i.e., how someone is looking at an artpiece. Window analyzes the position of bodies relative to paintings to infer where a person is looking, which is then translated through a projector onto an elastic canvas replica of the painting that is stretched and distorted based on what parts of the original painting are being looked at. Where people are looking is not only represented by physical distortions in the fabric but also by the opacity of the projection. Initially, the entire canvas is blank, but the painting is revealed over time, providing a glimpse into the cumulative aesthetic experience the person looking at the artpiece may be having.

How it works

Window consists of separate data collection and data visualization components, keeping our solution unintrusive within the main gallery space, while being visually engaging outwith. A discreet Microsoft Kinect camera is installed in the gallery space, collecting data on the position of gallery visitors relative to a particular painting. This is visualized on an elastic canvas, by a projection, and physically manipulated by a three-dimensional movement table we created underneath the canvas. This ‘xyz table’ is powered by five Arduino microcontrollers. The projection visuals are coded in Processing.

The Making of Window

 

Context of Enquiry

There is an interesting dichotomy that becomes immediately evident upon stepping into an art gallery. Going to galleries is often a social activity, but the actual viewing of the art and the aesthetic experience is a highly individual and personal affair. Galleries are filled with people, but they are silent. Each painting is gazed upon thousands of times, but these experiences remain personal unless the social norms of the silent gallery are transgressed.

The phenomenology of aesthetic experience is obscure. Even if one were allowed to talk in a gallery, how would one possibly describe the experience? The entire aesthetic experience is a perception in the mind. Philosophers today still argue about what constitutes aesthetic experience, there is no clear cut answer. It was our hope, however, that this experience could be interpreted at least partially by being quantified and translated.

Research, Methodology and Process

Embarking with the double diamond design process, our research initiated with ‘discovery’ – we delved into the project, attempting to abstain from preconceptions of what potential issues to address.

In the ‘define’ phase we played with ideas such as an Augmented Reality scavenger hunt to find particular objects in paintings, or a kaleidoscopic camera to allow artworks to be seen in new and strange ways, and then through social media. It became apparent that many of our ideas revolved around the experience of the gallery on a personal and interpersonal level.

Our breakthrough occurred upon the realization that subjective aesthetic experience is both extremely concrete, but also impossibly intangible by its nature of being subjective and personal. This tied in with the dual concepts of materiality and immateriality, and provided interesting challenges from both data collection and streaming, and design standpoints. This thusly led to the narrowing of the context to focus on subjective aesthetic experience, and how could be shared and communicated with others.

Design Guidelines

The perception in the mind is psychologically manifested, and hence difficult to discern externally. Early ideas involved interventions like asking visitors to wear eye trackers or attaching electrodes to gallery their heads and reading their brainwaves, but were quickly discarded due to their invasive nature. After some thought, we drew up a number of guidelines for our solution:

  1. It should be unitrusive
  2. It should add a new dimension to the gallery-going experience without hampering it
  3. It should not force visitors to break established social conventions of gallery etiquette

Operating within these guidelines, we settled upon a two­part solution; a discreet data gathering element within the gallery space, and a data visualization component elsewhere, perhaps in the foyer or outside. Dividing these two elements would allow for data to be collected within the gallery unobtrusively, but then presented separately without compromising the experience inside the gallery.

 

Data Collection

Given that aesthetic phenomenology is an internal experience, data collection necessitates either directly measuring brain activity, or using some proxy or external measure for internal states. Because measuring brain activity is impractical, we turned our efforts towards external/proxy measures. Aesthetic experience of visual art is primarily sensed through sight, so vision became the focus of our investigation. Our idea was that where people look on a painting could provide insights into how the painting is being subjectively experienced. For example, a 1993 study (Nodine, Locher & Krupinski) comparing how people trained and untrained in art and art history found that there are significant differences in how individuals with from these differing backgrounds take in a painting. One does not merely take in an entire art piece at once, but rather focuses on individual compositional elements sequentially to form a mental “schema” of the image as a whole, as the human eye has a sharp field of view of only 1­2 degree (Hochberg, 1968). Hochberg proposed the idea that momentary foci of attention and saccades work recursively with “schema” to influence how a person’s gaze moves around an artwork.

An eye­tracking apparatus would have been ideal for data collection (Nodine, Kundel, Toto, Krupinski, 1992), but it became apparent that currently available technology is either too invasive, e.g. requiring headgear, or prohibitively expensive. In light of this, we hypothesized that for sufficiently large and detailed paintings, a person’s position relative to the painting could approximate where in the painting they were looking. The efficacy of this approach, however, is highly dependent on the type of painting in question. It ought to be large enough to warrant substantial movement, such as a large landscape format painting. It also ought to be detailed enough that a person may move in closer for a better look or take a step back to take in the entire painting. The painting we selected was “Francesca da Rimini” by William Dyce, as it was suitably large. It also used little perspective, and so could be viewed from any position with little visual distortion (Pirenne, 1970, Hagen, Jones, 1978).

Amongst the options that were considered for measuring a person’s location relative to a painting were using arrays of microphones or pressure sensors, however, these were abandoned due to impracticality, expense, and probably imprecision. A compromise between subtlety, accuracy, and cost was eventually found, with a Microsoft Kinect camera. This could be used to track individuals in the camera’s range, with x,y, and z coordinates for their movement.

Dyce, William; Francesca da Rimini; National Galleries of Scotland; http://www.artuk.org/artworks/francesca-da-rimini-210011
Data Presentation 

Crucially, we wanted to provide a unique insight into subjective experience, but we were also wary of drawing conclusions with insufficient evidence or misrepresenting a person’s experience. To this end, we determined that the most effective way to present the data would be to simultaneously visualize live and cumulative time lapse data of where people are looking on a painting (and by proxy aesthetic experience) in real­time.

The solution we present is an elastic canvas with a dynamically self­painting artwork matching a real painting in the gallery (created in processing) projected onto it from above. Underneath this, a machine stretches and distorts the canvas at the point of focus of a viewer on the original painting. This point distortion, which represents where a person is looking as they move about, mirrors the 1­2 degree point of focus of the human eye (Hochberg et al., 2007) as well as the distortions that occur through looking at a two­dimensional painting from different vantage points (Gombrich et al., 1972, Shimamura, 2013, Hagen, 1976). Initially the projection is black, but the art piece fades into view based as people stop and look at the original painting.

Conclusion

We approached the difficult challenge of gaining insight into other people’s personal, subjective, aesthetic experiences in a gallery context, and managed to design and build an artifact that largely achieves this goal. By separating the data collection and visualization components, we managed to keep our solution unintrusive within the main gallery space, while being visually engaging outwith. In this way, we believe Window adds a new dimension to the gallery, without detracting from the original experience, whilst still fitting in aesthetically given its canvas design. It can stand on its own, (if not somewhat separated) and attract visitors. Despite limitations with the fidelity of visitor data, the arduino components, we believe Window effectively provides a ‘window into the subjective’.

References

Gombrich, E., Hochberg, J. and Black, M. (1972). Art, perception and reality. Baltimore: Johns Hopkins University Press, p.155.

Hagen, M. (1976). Influence of picture surface and station point on the ability to compensate for oblique view in pictorial perception. Developmental Psychology, 12(1), pp.57­63.

Hagen, M. and Jones, R. (1978). Differential patterns of preference for modified linear perspective in children and adults. Journal of Experimental Child Psychology, 26(2), pp.205­215.

Hochberg, J., Peterson, M., Gillam, B. and Sedgwick, H. (2007). In the mind’s eye. Oxford: Oxford University Press.

Pirenne, M. (1970). Optics, painting & photography. London: Cambridge University Press, p.96.

Shimamura, A. (2013). Experiencing art. Oxford: Oxford University Press, pp.37­38.