by Eoin Kilfeather
Cultural meaning does not reside in individual objects but in the patterns of knowledge and events, belief and thought that link them to each other and to the observer. This is why narrative is so important to the communication of, and meaningful understanding, of culture.
Digital heritage and semantic web technologies have held out the promise of nearly unlimited access to cultural knowledge. However, the vision is as beguiling – and currently as unobtainable – as the invocation in John Donne’s Song (see Figure 1). The problem is that cultural meaning does not reside in individual objects (however beautiful and marvellous in themselves) but in the patterns of knowledge and events, belief and thought that link them to each other and to the observer. It is well understood that cultural narratives are not static, but change in time and in relation to the viewpoints from which they are both created and explored. Curators have developed ways of presenting their collections that guide visitors and make sense of the experience. Although we have new ways of accessing cultural objects, the narrative structures and arguments that could be found in a handcrafted presentation are often lost. There is a tension between providing user control over what is shown and the coherence of the overall experience.
Figure 1: A Donne' poem.
Figure 2: A mock-up of a visualization.
Although there is a vast and growing store of digital objects, most of them do not provide the information for explicit interpretation. On the other hand many cultural websites are extensive and elegant, but their narratives are ‘burnt-in’ and fixed by their authors: they can never be complete, and there is a good chance that the searcher’s interests will not be fully met. Also, current systems are tied to the technology platforms for which they are authored. Web-based approaches offer a two dimensional sensory experience that is limited in comparison with the real world; immersive and haptic systems still require expensive special-purpose hardware, whose use is restricted to small numbers of people at the heritage sites themselves.
The DECIPHER project proposes new solutions to the whole range of narrative construction, knowledge visualization and display problems. It will combine much richer, event-based metadata with causal reasoning models. This will result in a reasoning engine, virtual environment and interfaces that can present digital heritage objects as part of a coherent narrative, directly related to individual searches and user contexts. This will allow the user to interactively assemble, visualize and explore, not just collections of objects, but the knowledge structures that connect and give them meaning.
The partners bring together the skills and experience in the technical fields required by the project, with the authorial and curatorial authority of national institutions, the innovative impetus of a technology-based SME, and the drive to market of a large company that combines heritage and media interests. They are: 1) Two national heritage institutions, the National Gallery of Ireland and the Irish Museum of Modern Art, with major collections that span the range of physical and born digital art works. 2) A large company, Alinari 24 Ore (IT), which is both the world’s oldest photographic archive and a part of a large, modern media group. 3) A research-based SME System Simulation Ltd. (UK), specialising in software for professional media and content management. 4) Three leading academic research centres, Dublin Institute of Technology (IE), the Open University’s Knowledge Media Institute (UK), and Brno University of Technology (CZ) expert in knowledge management, narrative construction, data mining, semantic web technologies, language technologies, multimodal interfaces and digital heritage applications.
Background work carried out by KMi has succeeded in uncovering interesting patterns from data by using a combination of machine reasoning, knowledge visualization and user tagging. DECIPHER will produce an advance in the process by combining much richer, event-based metadata with causal and dynamic reasoning models. We will do this by associating an object with a set of events, making it possible to apply machine reasoning in a much more interesting way, not only to identify clusters of events or objects but also to construct narratives.
To do this we need to generate the event-based metadata. In this project, colleagues in Brno University of Technology are producing the event-based semantic metadata automatically from networked sources and the traces of social interaction. Artificial intelligence techniques, using Kripke structures (a type of nondeterministic finite state machine proposed by Saul Kripke in 1963 to represent the behaviour of a system )and Kripke semantics (a formal semantics for non-classical logic systems, also known as relational semantics or frame semantics) then make it possible to identify and formally represent regions and tipping points in sets of content and related events. By maintaining and accruing the metadata associated with the objects and transitions, the machine representation becomes richer and more powerful. This will make it possible to see if there are relevant transitions or regions in other people’s narratives, thereby enabling social interpretations and recommendations. It will also support counterfactual or ‘what if’ reasoning by showing the effect of moving or removing a transition.
Clearly, it is not enough to develop a narrative: we must present it in a way that allows the visitor to explore the knowledge space and objects in the richest way possible, by taking full advantage of whatever technology platform is available (fixed or mobile, 2D or 3D, window or immersive). The visual formalisms, interfaces and interaction techniques draw on DIT’s previous work in mobile interaction and virtual museum spaces. Customised rich interfaces support digital curation in other ways. For example, some installation art consists of a digital file and the curatorial instructions for “how to show it.” Such a system that links media resources, curatorial metadata and multi-modal interfaces would allow a much deeper exploration of such works.
The DECIPHER project (FP7-270001-Decipher) began in January 2011 and runs for three years. It is supported by the EU’s 7th Framework Programme.
Link:
http://decipher-research.eu/
Please contact:
Eoin Kilfeather
Digital Media Centre, Dublin, Ireland
E-mail:
