Presentation: Platform Supported Anticipation: Multimodal data from design ideation as platform level metadata for distributed ideation.

Theme: How does the future get made? Part of, New Ideas Session: Deep time, design, survivalism, robots, embodied interaction, and almanacs.

Anticipation 2017 Conference
Senate House, School of Advanced Study, London.


Individuals are viewed more and more as data points for various platform level digital products and services. This is a by-product of the combination of carrying sophisticated sensor arrays in the form of smartphones and advances in technologies such as computer vision deployed in wearable tech and embedded within the ‘smart’ city. This situation has opened up new possibilities for novel types of metadata to be gathered and aggregated at the platform level for various purposes and will be key to anticipatory practices. The media effects of this environment are not well understood, from fake news and filter bubbles to new concepts of normalcy we now perform presence in important ways by contributing to larger datasets to be mined later for value by emerging systems of machine learning. In this context anticipatory practices such as design will be augmented by systems built on the meshing together of these data sets and researchers in many domains need rapid methods to study the effects of such augmentation. This is, in part how the future will get made.

My research explores this situation as it touches design ideation process and asks questions about what a multimodal dataset drawn from design ideation might look like and how it may generate metadata for distributed ideation platforms.

Our culture has become increasingly indexed online. As design and ideation processes have shifted online, there has been a growing repository of resources and implicit connections that can be indexed, analysed and used in augmenting design ideation processes. The idea that knowledge can be obtained by mining web relationships is not new and many researchers, for example in the field of HCI, have proposed systems that do just this. What my research proposes as a distinct area of study is the effect that a system such as this may have on the experience of ideation in design practice, and asks if this can give rise to a new data type that is amenable to iterative development, forking or branching outside the temporal bounds of physical ideation situations. My research also proposes an ontological exploration of design as a collaborative endeavour, given the associated wide-scale collaborative effects of such systems.

My research uses Paul Dourish’s idea of Embodied Interaction and specifically his idea of context (Dourish, 2004) in conjunction with the use of Multimodal methods as these methods have been developed to specifically investigate complex social and expert use of technology and to gauge their impact on meaning making in this context. The thesis here is that this approach may allow for new taxonomies of multimodal data to be developed that could both; a. explore how design ideation is supported with elements of ubiquitous computing and, b. speculatively explore how design ideation may be encoded enabling iterative, asynchronous development within networked information platforms.

The ideas above outline what could be thought of as some of the components of anticipatory work in the design field and looks to establish how to investigate such phenomena using a combination of research through design, multimodal analysis and embodied interaction.

Key references:
Dourish, P. (2004) ‘What we talk about when we talk about context’, Personal and Ubiquitous Computing. Springer-Verlag, 8(1), pp. 19–30. doi: 10.1007/s00779-003-0253-8.

Norris, S., & Maier, C. D. (2014). Interactions, images and texts: a reader in multimodality.

Price, S., Jewitt, C., & Brown, B. (2013). The SAGE Handbook of Digital Technology Research. 1 Oliver’s Yard, 55 City Road, London EC1Y 1SP United Kingdom : SAGE Publications Ltd.

Zimmerman, J., Forlizzi, J. and Evenson, S. (2007) ‘Research Through Design as a Method for Interaction Design Research in HCI’.