If it’s creative, it doesn’t feel like evaluation: implications for practice from the Artcasting project
Jen Ross, PhD
The Museum Review, Volume 2, Number 1, 2017
Abstract The Artcasting research project, undertaken in 2015-16 in partnership with the ARTIST ROOMS program in the UK, developed a new digital and mobile form of evaluation of arts-based engagement. The project was designed to understand how visitors would respond to creative questions and methods for connecting their experience of artworks to places and times beyond the gallery. It also sought to understand how working with a platform like Artcasting could constitute meaningful evaluation practice. This paper describes some key implications for practice from the project: how inventive methods can help to shed new light on challenging issues; how digital engagement might shift meanings and approaches to co-production; and what can be gained from creative research collaborations between academic and cultural heritage partners.
Keywords Digital and mobile engagement; evaluation; co-production; inventive method; partnership working
About the Author Jen Ross is a senior lecturer and co-director of the Centre for Research in Digital Education at the University of Edinburgh. She has been working on digital learning and engagement projects with the museum and gallery sector since 2007, when she was part of the research team for the National Museums Online Learning Project. She is Deputy Director (KE) of Research and Knowledge Exchange in the Moray House School of Education. Her research interests include online distance learning, Massive Open Online Courses (MOOCs), digital futures, reflective practices, and cultural and educational institutions online.
I think when it’s something that feels more creative and kind of participatory it doesn’t feel like evaluation (Artcasting interviewee)
Artcasting was a research project funded by the Arts and Humanities Research Council in the United Kingdom in 2015-16. Working closely with colleagues from the National Galleries of Scotland and Tate, in the context of the ARTIST ROOMS collection and exhibition, the researchers developed a new digital and mobile form of evaluation of arts-based engagement- Artcasting – and piloted this with two ARTIST ROOMS exhibitions. ARTIST ROOMS is a collection of more than 1,600 works of international contemporary art, jointly owned and managed by Tate and National Galleries of Scotland. ARTIST ROOMS shares the collection in a series of monographic exhibitions throughout the U.K. in a program of exhibitions organized in collaboration with local associate galleries.
The project team sought to understand how visitors would respond to a creative, digital way to articulate their engagement with artworks; and how the Artcasting approach, which asked visitors to make connections between art and place, could constitute meaningful evaluation practice.
This article provides some background and context for the project, and outlines its key implications for practice around approaching evaluation inventively; re-imagining co-production; and undertaking creative research collaborations.
Background
The Artcasting project explicitly sought to bring together a number of ideas – around engagement, evaluation, and digital and mobile methods – and the richness of each of these areas has given us some very fruitful findings on which to build. The project was explicitly geared towards imaginative methods and a creative approach to the thorny issue of evaluating visitor experience, engagement, and learning, and this inventive approach opened up many interesting pathways and possibilities as the project progressed.
The UK’s Arts and Humanities Research Council recently evaluated their Cultural Value program, and a number of their recommendations echo the pre-occupations of the Artcasting project. The program was geared towards “the value of art and culture: to be precise, the value associated with people’s engaging with and participating in art and culture,”[1] and in reviewing the program’s projects and activities, the report authors argue that:
thinking about cultural value needs to give far more attention to the way people experience their engagement with arts and culture, to be grounded in what it means to produce or consume them or, increasingly as digital technologies advance as part of people’s lives, to do both at the same time.[2]
The Artcasting project sat precisely in this space – it developed a digital and mobile method for producing responses to art, and explored how such a method could be generative for richer approaches to evaluation.
Artcasting was an interdisciplinary project involving researchers in digital education and design informatics, and as a result it was able to take a highly creative approach to conceptualizing, designing for, and empirically analyzing the questions it sought to explore. Through a series of interviews with ARTIST ROOMS stakeholders, workshops with young people, and interdisciplinary design conversations and experiments, the project established a solid foundation of insights for developing its main output – a mobile application (app) called Artcasting.
The Artcasting app invited visitors to select an artwork from one of the two exhibitions involved in the pilot – Roy Lichtenstein at the Scottish National Gallery of Modern Art in Edinburgh, Scotland, and Robert Mapplethorpe: The Magic in the Muse at the Bowes Museum in County Durham, England. Then, the visitors were asked to choose a location and time period to which their chosen artwork would be digitally sent (or ‘cast’). This could be any location in the world, and any time in the past, present, or future. The reasons for the choice of location and time were captured in textual form in the app.
The artcasts sent by visitors were visible on a world map, with lines of trajectory showing the artworks’ starting and ending locations, and indicating the timescales of their journeys. The purpose was to visualize individuals’ connections with art in a way that emphasized movement and imaginative journeys. Furthermore, the app would create a “geofence” for each artcast so that people with the app installed on their own mobile device could re-encounter artcasts if they happened to be in a physical location to which an artcast had been sent. Throughout the project, researchers sought to understand if such a focus on mobility would be generative for visitors in terms of articulating their engagement with artworks; and simultaneously, if it could be a productive way for gallery staff to evaluate and represent engagement.
By the end of the project, approximately 170 artcasts had been sent by a wide range of visitors to the two galleries, including young people, students, families, and older visitors. The artcasts, taken collectively, expressed a very diverse set of responses to and perspectives on the artworks selected and cast. Here are some examples of the range and types of casts visitors sent.
In the sections that follow, I will set out what the research team saw as the main implications for practice for the cultural heritage sector that emerged from the Artcasting project.
Inventive evaluation
The Artcasting app was the expression of a methodological approach, a way of testing ideas through engagement with visitors about visitor interpretations of artworks and exhibitions. The primary idea we wanted to grapple with was the possibility of approaching evaluation more imaginatively. We drew on a range of research, including key work done by Belfiore and Bennett on humanities-based approaches to evaluation, which focus on understanding the complexity of the aesthetic experience, and on “open inquiry,”[3] rather than on more instrumental approaches to evaluation that are aimed at meeting highly prescribed goals and securing ongoing support and funding by evidencing impact. We sought to show that evaluation and engagement could be closely integrated, and that visitors could engage creatively with exhibitions in ways that could also provide rich insights for reflection and practice. We were inspired in this by our analysis of interviews with nine museum and gallery colleagues associated with ARTIST ROOMS, who simultaneously understood evaluation as being constrained, routine, and instrumental; and talked with energy and enthusiasm about their experiences of “open inquiry” through dialogues with peers, funders, and others that sparked their curiosity about how visitors engaged with exhibitions and with institutions.[4]
Artcasting is an example of “inventive method” that helps shape the problem that it was diagnosed to address.
Artcasting was an example of “inventive method” that helped to shape the problem it was designed to address – it provoked the researchers and partners to consider the values around evaluation they held; as well as providing new ideas about how to categorize and visualize engagement (for example by looking to see which artworks had the most intensity of engagement around them; which travelled the furthest; and so on – discussing and exploring what movement and intensity might have to say about engagement and impact). By showing that such imaginative approaches are possible, and that there is potentially much more space for thinking differently about evaluation than had previously been thought, the project made an impact that went beyond the piloting of an app, and towards a richer context of evaluation for ARTIST ROOMS and its partners.[5]
Digital co-production
Ultimately, design and platform decisions, and unanticipated effects, were highly consequential for the project, as is often the case when technologies, practices, and ideas are brought together. Mobile and digital technologies, brought into cultural heritage spaces and approached creatively, can serve to unsettle assumptions and illuminate issues. One issue that we came to understand quite differently as a result of the Artcasting project was that of co-production: activity in which representatives of an institution and representatives of one or more of that institution’s publics are engaged in reciprocal forms of participation, interpretation, co-operation, or exchange. Co-production is increasingly highly valued as a way of involving communities and individuals as stakeholders in cultural heritage settings, and of proving impact and relevance.[6] It is also increasingly the focus of critical attention as researchers and professionals in the field have questioned exactly how collaborative, empowering, or participatory co-production can be.[7]
One reason for this are the more or less stable, but sometimes unacknowledged, power dynamics at play when co-production is initiated and managed by institutional representatives. When institutions set the terms and draw the boundaries around public engagement and participation in co-production initiatives, there is perhaps inevitably a sense in which co-production “reflects the agendas of the institution where the processes, such as the final right to edit content, are tightly controlled by the museum.”[8]
The process of artcasting, and most importantly the potential for artcasts to be re-encountered by other visitors both via the app interface and in physical locations to which artcasts could be sent, provides a new perspective on co-production. Artcasting certainly did encourage visitors to engage in specific ways with the collections, and to do so on terms set out by the project and its partners. The artcasting data was collated and visualized through a dashboard interface to which the galleries had access, and was a novel, but not especially challenging, mode of user response. It could be kept within the bounds of a “safe” visitor-gallery power dynamic.
Artcasting allowed individual, sometimes quote personal and idiosyncratic, interpretations of artworks to be publicly encountered in spaces at times well beyond the gallery.
At the same time, though, Artcasting allowed individual, sometimes quite personal and idiosyncratic, interpretations of artworks to be publicly encountered in spaces and times well beyond the gallery, and indeed beyond the timeframe of the exhibition and the project. Though the app is no longer available to download, those who already have it might still be encountering artcasts. The gallery space was shifted, or perhaps multiplied, by the movement of artcasts, and the geofences that the app created meant that any place could become an exhibition space.[9]
As a result, the form of co-production enacted by Artcasting was one of distributed interpretation – an unstable and shifting dynamic that does not undo, but certainly changes, what it means for a gallery to “host” an exhibition, or for a visitor to be a “guest” at one. Bell suggests that mobile technologies produce not hosts and guests but “more or less stable or fragile places and/or times when hosting-guesting occurs.”[10] He calls these “host-spots.” Artcasting produced such host-spots through its invitations to users to interpret and to share their interpretations of where artworks belonged.
Artcasting is not unique in destabilizing relationships of hospitality or co-production – it may be that many digital and mobile cultural heritage projects could usefully be seen in this way.[11] What is important is that those tasked with engaging publics in these types of activities and projects are open to how this might happen, and what it could mean. What can we do with host-spots, and how might they help us grapple with issues of authority and control when they bump uncomfortably against our ambitions for working alongside publics?
Creative research collaborations
The final area in which I would like to suggest some implications for practice from the Artcasting project is around what it means to undertake creative research collaborations between academic and cultural heritage organizations. This is important because there often seems to be a degree of mutual scepticism about how successfully colleagues can work across their respective interests and obligations to arrive at satisfying research questions, approaches, and outcomes. To be able to undertake such a speculative project, where the methods and theoretical approaches were outside the norm for exploring topics around evaluation, meant building trust and solid partnerships between the research team and our gallery colleagues. Even from the earliest stages of the project, gallery colleagues had to take a risk to support a vision that was initially quite difficult to explain to others, and whose purpose was to test ideas rather than to generate a product. The proposed value of the project was in opening up new possibilities for bringing engagement and evaluation together, for analyzing how people articulated their engagement with art, and for exploring how less instrumental modes of evaluation could generate useful insights for the cultural heritage sector. These benefits were felt to be worth taking risks, and the support and engagement from partners at Tate, NGS, and the Bowes Museum was at the heart of the success of the project.
We built on this support by working to keep partners closely involved in the project at all stages, including through membership on the project advisory board, during the development of the Artcasting platform, and planning of the pilots. In return, our key partners took the lead on keeping their colleagues and other stakeholders informed, sharing emerging findings and ideas, and providing invaluable help in developing connections with key communities with links to their institutions.
Strong partnerships proved especially important in the context of the development and launch of the pilot app. Flexibility and creative approaches meant that we were able to test at a number of stages, adapt our plans when timings changed or when technical obstacles (both in the app development and in the galleries) emerged, and overall to engage with a range of visitors who generated extremely interesting artcasting data.
With those partnerships in place, the sorts of inventive approaches described in the earlier sections of this article became possible, through a shared willingness to take risks and explore the territory of evaluation in new and unexpected ways.
Conclusions
The Artcasting project was not intended to resolve all the thorny questions around engagement, interpretation, and evaluation that it grappled with. It was instead about opening up space for different kinds of approaches to those questions. In a cultural heritage evaluation landscape “dominated by the language of targets, outcomes, outputs, and delivery,” this in itself is a contribution worth making.[12] Beyond that, the platform itself, the Artcasting method, and the project’s particular approach to bringing mobilities theory, digital engagement, and evaluation together have generated some valuable possibilities, and the research team is developing plans with a range of small and large cultural heritage and arts organizations to explore these further.
Acknowledgements
Many thanks to the Artcasting research team: Jeremy Knox, Claire Sowton, and Chris Speed. The ‘Artcasting’ research described in this paper was supported by the Art and Humanities Research Council [grant number AH/M008177/1].
Notes
[1] Crossick and Kaszynska, 2016, p. 13.
[2] Ibid, p. 7.
[3] Belfiore and Bennett, 2010, p. 139.
[7] See: Lynch, 2009; Graham, 2012; Morse, Macpherson and Robinson, 2013; Ashley, 2014.
[8] Morse, Macpherson and Robinson, 2013, p. 92.
[11] See: Hogsden and Poulter, 2012.
[12] Simons and McCormack, 2007, p. 295.
References
<!–[if supportFields]> ADDIN ZOTERO_BIBL {"custom":[]} CSL_BIBLIOGRAPHY <![endif]–>Ashley, S. L. T. (2014) ‘“Engage the World”: examining conflicts of engagement in public museums’, International Journal of Cultural Policy, 20(3), pp. 261–280. doi: 10.1080/10286632.2013.808630.
Belfiore, E. and Bennett, O. (2010) ‘Beyond the “Toolkit Approach”: arts impact evaluation research and the realities of cultural policy-making’, Journal for cultural research, 14(2), pp. 121–142.
Bell, D. (2012) ‘Moments of Hospitality’, in Molz, D. J. G. and Gibson, D. S. (eds) Mobilizing Hospitality: The Ethics of Social Relations in a Mobile World. Ashgate Publishing, Ltd.
Crossick, G. and Kaszynska, P. (2016) Understanding the value of arts & culture: The AHRC Cultural Value Project. Swindon: Arts and Humanities Research Council.
Graham, H. (2012) ‘Scaling Governmentality: Museums, co-production and re-calibrations of the “logic of culture”’, Cultural Studies, 26(4), pp. 565–592. doi: 10.1080/09502386.2012.679285.
Hogsden, C. and Poulter, E. K. (2012) ‘The real other? Museum objects in digital contact networks’, Journal of Material Culture, 17(3), pp. 265–286. doi: 10.1177/1359183512453809.
Kidd, J. (2014) Museums in the New Mediascape: Transmedia, Participation, Ethics. Ashgate Publishing, Ltd.
Knox, J. and Ross, J. (2016) ‘“Where does this work belong?” New digital approaches to evaluating engagement with art’, in. MW2016: Museums and the Web 2016, Los Angeles. Available at: http://mw2016.museumsandtheweb.com/paper/where-does-this-work-belong-new-digital-approaches-to-evaluating-engagement-with-art/.
Lynch, B. (2009) Whose Cake is it Anyway?: A collaborative investigation into engagement and participation in twelve museums and galleries in the UK. Paul Hamlyn Foundation. Available at: http://www.collectionstrust.org.uk/home/whose-cake-is-it-anyway-a-collaborative-investigation-into-engagement-and-participation-in-twelve-museums-and-galleries-in-the-uk (Accessed: 13 July 2015).
Morse, N., Macpherson, M. and Robinson, S. (2013) ‘Developing dialogue in co-produced exhibitions: between rhetoric, intentions and realities’, Museum Management and Curatorship, 28(1), pp. 91–106. doi: 10.1080/09647775.2012.754632.
Ross, J. et al. (in press) ‘Artcasting, mobilities, and inventiveness: engaging with new approaches to arts evaluation’, in Ciolfi, L. et al. (eds) Cultural Heritage Communities: Technologies and Challenges. London: Routledge.
Simons, H. and McCormack, B. (2007) ‘Integrating Arts-Based Inquiry in Evaluation Methodology: Opportunities and Challenges’, Qualitative Inquiry, 13(2), pp. 292–311.
Sowton, C. (under submission) ‘Cultural sector perspectives of evaluation: Collage, dialogue, and the holy grail’, under submission.
The Museum Review 