IConCMT 2023
The 5th International Conference on Creative/Media Technologies (IConCMT) took place at the St. Pölten University of Applied Sciences (UAS) from 28 to 30 November 2023.
At IConCMT, our primary objective is to foster interdisciplinary dialogue and collaboration within the broad spectrum of digital media. By exploring the intersection of technological possibilities, creative design, and socio-cultural frameworks, the conference creates an environment for professionals from diverse backgrounds to interact and create novel synergies.
The conference focused on participatory culture, cultural heritage, open data, and hybrid public spaces, inviting submissions that explore digitalization and hybridization of culture, along with technological, ethical, and regulatory considerations. Additionally, the intersection of visualization, AI, media production, and storytelling was a key area, unraveling possibilities and implications in these integrated fields. Climate change communication patterns and immersive education paradigms were also explored, providing a platform for innovative approaches and strategies.
We extend our heartfelt thanks to all participants for making IConCMT 2023 an enlightening and enriching event, filled with engaging presentations, interactive workshops, and thought-provoking discussions. We look forward to welcoming you again and exploring exciting developments in these multidisciplinary domains.
The proceedings will be available in spring 2024.
Programme | 28 to 30 November 2023
The Work of Art in the Age of Synthetic Knowledge Production
Egor Kraft - nomadic, investigative & interdisciplinary artist working on the intersection of arts, media, technology and research
The lecture's focus lays at the intersection of culture & automation. More specifically, it aims to unpack on the current state of art & image production in the context of Ai, while looking at technologies of automation and peer-to-peer computation networks within the studies of derivative aesthetics, epistemic challenges posed by synthetic forms of knowledge production and, of course, socio-humanitarian discourses.
It is hard to underestimate the impactful effect of the current state of computational advancements across many areas, including artistic production. An underlying premise to the course is that we acknowledge that recent computational advancements in Ai have radically changed how we understand the methodology of production of image & knowledge. Not only this suggests new ontology of synthetic images, but also the need to figure out epistemologies of their provenance and future function. Such new terms as promptography or synthography suggest the popularisation and quick adoption of aforementioned developments. While more complex notions, i.e. latent space or data cannibalisation require a little more profound understanding of the technical backend of how Ai is used in content production.
Attendants will be invited to look at AI through the lens of media theory following critical and speculative thinking in order to learn how to work with AI within their individual practices. Approaching Ai, conceptually, may involve recognition of an agential relationship between synthetic and “natural” forms of intelligence; such a framework may help to understand and navigate across the approaches of Ai as a tool vs Ai as a form of intelligence. The key question at the core of this session is: what conceptual & technical frameworks are available to us for working with Ai, via Ai, about Ai and maybe even for Ai in critically reflexive ways?
Edith Blaschitz: Connecting Place, People, Art, and Digital Technologies: Criteria of Success … and Failure – an Analysis of the remembrance project “Making Traces Readable in the NS Forced Labour Camp Roggendorf/Pulkau”
After being largely neglected for a long time, NS forced labour camps and camp-like facilities used for the internment of prisoners of war and civil forced labourers are increasingly drawing interest from the realms of research and historic preservation and from the general public. So how can we create an awareness for these “loaded” but forgotten locations that are largely spaces in our everyday lives? This question is framed by fundamental challenges of the current culture of memory that arise from the growing temporal distance to the time of the Shoah and NS terror, and especially from the foreseeable end of direct historical testimony. The search for new anchor points for reference and confrontation, and associated new formats of dissemination and involvement is characteristic of this present stage of “post-memory”. The historic location has come into focus as a new anchor point that is attributed “authenticity” and truthfulness.
Against the background of the question of how to deal with forgotten camp sites in spaces of everyday life and what digital technologies can do, the project “Making Traces Readable in the NS Forced Labour Camp Roggendorf/Pulkau” was developed and implemented. The objective was to visualise – in other words, explore, represent, and make accessible – the history of forced labour in the quarry close to the town Pulkau. The quarry experienced an eventful history and evolved from an early industrial location into a leisure paradise and event venue – and during the NS regime, prisoners of war and civil persons were forced to do extremely hard physical labour there.
For the project, a concept was developed that combines the physical location with artistic and specialist approaches, participatory remembrance work, and digital technologies. This connection between the place, different actors (artists, scientific experts, digital / creative media technologies experts, and citizen scientists), and digital technologies can be referred to as an “assemblage” in which heterogeneous components form a social fabric in a collaborative process and in relation to one another. Within the framework of the implementation, the main focus was on the question of how the various components can be entangled with each other in the most meaningful way possible in order to render the historical events and experiences visible. Another question addressed in the project was how a productive way of dealing with these gaps and fragmentary knowledge can be found.
The presentation will focus on success criteria and possible problems in such a highly complex project, which deals with "burdened" cultural heritage.
Ingo Zechner and Vrääth Öhner: Film as Witness? Holocaust Education with Digital Means
Not least in view of the current turning point in Holocaust education – the fact that there will soon no longer be any eyewitnesses left to report on the events – educational issues are increasingly focusing on places, but also on images and the possibilities of the digital.
The international research project Visual History of the Holocaust has taken on the task of digitizing stocks of so-called "liberation images," i.e., pictures taken during or shortly after the liberation of concentration and extermination camps, from a wide variety of archives and making them available online in bundled form, accompanied by an incredible wealth of contextualizing information: Where were these films and photos taken, by whom, with what intention, and by what technological means? How were they used – and how do they resonate in a wide variety of contexts?
In the course of the presentation, this very comprehensive project will be briefly introduced with a focus on its possible applications, which will then also be discussed with the help of examples. In addition, the role as well as the handling of this kind of (moving) image material will be critically examined: in museums, memorials and other contexts of mediation and education, on site and with the means of the digital.
(Chair: Georg Vogt)
Nuno Cintra Torres: Magi, the Magical Word: Gamified Online Collaborative Funding in the Arts
Let's Put the Sequeira in Its Rightful Place was the slogan of the successful online and gamified collaborative funding campaign for the purchase by the National Museum of Ancient Art (MNAA) (Lisbon, 2015-2016) of the painting The Adoration of the Magi (1828). Sequeira referred to painter Domingos Sequeira. The rightful place was the MNAA. Failure to collect €600 thousand would entail the painting being sold abroad, a loss to national heritage. The strategic objective was the construction of social capital for giving based on formal and informal hierarchical and weak networks. Adverse exogenous economic and political macro forces spurred the museum's entrepreneurial initiative. The Magi became pixel art during the campaign’s six months. The collection of donations was obtained via a gamified “shopping” process. “Which part of The Magi Adoration will you sponsor”? was asked.
Research questions: Were the gamification of the giving process and the marketing campaign the sole levers of the necessary social capital for giving, or were there other unrecognised, subjacent, and implicit factors not expressed but unwittingly unleashed by the campaign that may have surreptitiously contributed to the funding outcome? Did habitus, deep social and cultural constructs associated to the Magi myth exert a resonating and confirmative function? Did timeless visual mnemonics, Renaissance pictorial compositional principles, and a «Turneresque» style play a persuasive role?
Methodologies. Semi-structured interviews were conducted with the then director of the MNAA, and with the main actors of the core group of organisations and institutions that conceived, launched, and managed the campaign at the newspaper Público, advertising agency FUEL, and the MNAA. Research was undertaken on what remained in the public domain of the ten-year-old campaign. Documentary data from news, interviews, opinion articles, broadcast and cable TV videos, and analysis of still-existing interactive pages of the campaign’s website in Público. Interviews with donors were a source of giving motivations. Mediated Discourse Analysis was adopted as a methodological approach providing an action-oriented unit of analysis: material histories of use and access, nexus of practice, focal point, site of engagement, mediational discourse, mediated action, and mediational means.
Results. The campaign rejuvenated the museum's brand. The basic elements of the social capital theoretical model were fulfilled: trust, transparency, and accountability. WoM and eWoM proved essential to participatory activism. The Advertising Value Equivalency (AVE) of the media exposure was significant and would have been financially unattainable for the museum as paid advertising. It is argued that efficacity resided in a plurality of engineered factors, foremost the pixelization of the painting and gamification of the giving process producing collective mass entertainment based on the “purchase” of pixels, and in non-orchestrated factors arising from the Biblical Magi myth so popular in Europe and elsewhere. Deep social constructs, cultural meanings, and artistic discourse practices emanating from the depicted myth contributed to the desired outcome. Success resulted from the conjunction of everyday technologies, the power of the myth, timeless visual mnemonics, and the syncretic application by Sequeira of Renaissance and Turner-influenced compositional and depiction techniques.
Michele Varini: Mirror on the Wall: What is Fashion and What is not? New Dynamics and New Inequalities within Digital Fashion
The new paradigms of consumption and production made possible by digital technologies have been affecting the fashion industry for several years, with a further acceleration in response to (and as a consequence of) the pandemic situation. Within this complex, variegated and stratified current, a phenomenon has recently started to manifest itself and grow: several fashion brands have experimented with incursions, of various kinds, into the world of gaming, a subculture peculiar for its imagery, rules and languages. The growth of the gaming sector, and of the economic profits related to it, is a well-known reality, but for various fashion brands, especially prestigious fashion houses with a reputation of a certain type, mixing with this digital world, strongly characterised by its rules and aesthetics, can often bring with it a lot of difficulties. There are profiles where these digital 'outfits' are re-shared, re-edited, processed, especially on Instagram, the platform of choice for fashion communication for a long time now.
Another sign of 'cross-fertilisation' can be found in fashion shows hosted in digital environments: several fashion houses have created digital garments that can be used and purchased directly on the chosen media platform. Both users and brands develop skills and practices by reworking the tools of the gaming platform, and the result is a re-mediation of productions, in a circle of image creation that becomes a co-production. The paradigms of consumption, production and creativity itself behind fashion objects seem to break out of the traditionally followed trajectories. One of the aims of the study is to explore this new reality: how is this phenomenon shaping fashion imaginaries and stereotypes, breaking traditional top-down dynamics? What dynamics are emerging in a world where the economic, cultural and social capital is not that of the offline world and where the possibilities of expression are unbound by traditional materiality? To address a field such as this, which moves between on and offline, making even these hermeneutically less relevant distinctions, a netnographic and visual methodology of investigation was chosen. The material thus sampled was then used as a photostimulus in semi-structured interviews with Italian gamers, with the aim of investigating what the users themselves perceive and think about the phenomenon in question. The aim of this work is to reconstruct a visual imaginary of this co-production and consumption of fashion in a media context, a hypothetical 'metaverse', and to explore the new dynamics of inequality generated there, on the increasingly opaque boundary between the usual materiality, to which fashion has always been accustomed, and this new 'non-constitutionality', proper to the digital.
Patryk Wasiak: Bedroom Programmers, the Software Industry in the 1980s, and the Emergence of Information Capitalism
In my presentation, I discuss how the figure of the ‘bedroom programmer’ that dominated the imagery of the emerging consumer-oriented software industry of the 1980s was embedded in the emerging structures of information capitalism. Here, I intend to answer the question, of how the ‘bedroom programmer’ as a model of selfhood represented the broader sociotechnical imaginaire (Jasanoff and Kim 2015) of the information capitalism of the 1980s which was structured with the neo-liberal Reaganomics/Thatcherism ideology.
‘Bedroom programmers’ made software, primarily computer games, single-handily designing the technical structure of a program and also designing a product that will be attractive to consumers. The culture of bedroom programming offered social training in the commodification of software (Appadurai 1986) since it promised adolescents an opportunity to move from writing games to thriving in the software industry as a ‘programmer entrepreneur.’ One of the dominant symbols of this culture was a flashy sports car that can be purchased by a teenage programmer from royalties of a successful game. This was a promise of a personal reward for designing a successful product in a competitive software market.
I argue that the ‘bedroom programmer’ can be interpreted as a figure that merges the imagery of information society, knowledge labour, and, referring to Thomas Streeter (2011), the neoliberal high-technology fantasy of Reaganomics and Thatcherism. While developing my argument I highlight the continuities and interconnections of the culture of the software industry of the era and digital capitalism as we know it today.
There is a substantial body of works that discuss contemporary digital capitalism. However, aside from Streeter’s work (2011), there is a scarcity of studies on the process of formation and evolution of digital capitalism from a historical perspective. Only Streeter offers an investigation into the imaginaire related to digital data as a property that can be patented and sold in the context of the neo-liberal ideology of the 1980s. Similarly, the scholarship on the history of computing lacks studies that discuss the conditions of labour and largely dismiss how the development of the software industry was situated on broader economic and social developments of the 1980s. Moreover, aside from a study on contemporary digital patriarchs (Little and Winch 2021), there are no works that deconstruct how personhood can be embedded in the imagery of digital capitalism.
I contribute to filling this knowledge gap by offering a theoretically informed historical case study that is grounded in a framework that connects the history of computing, and more broadly STS (Streeter 2011, Abbate 2018, Jasanoff and Kim 2015), and studies on digital capitalism.
Patricia Derbyshire: Counter Colonial Narratives. Kinship Computer interaction
This paper explores Siksikaitsitapii/Blackfoot teachings of Saaponsstaa (--magical, mysterious experiencing) and Kitastoko (-- land intimacy) for their impact on planning inclusive and just Immersive Experiencing (IE) within game-worlding. It highlights the absence of Indigenous scholarship in Human-Computing Interaction (HCI) and proposes adopting Kinship-Computer Interactions (KCI) to counter colonial narratives and reduce user dissonance. The paper applies the Currere methodology and land-based research epistemologies to examine Indigitalized IE, incorporating Saaponsstaa and Kitastoko to design KCI prototypes. Expected results include descriptive narratives, generative strategies, and recommendations to counter colonial memory and promote epistemic expansion and design justice. The study emphasizes KCI's significance in centring genocide-informed development and programming, thereby reducing harms to Indigenous users and their communities.
(Chair: Victor Schetinger)
The media prize of the St. Pölten University of Applied Sciences.
Scalable Interactive Visual Analysis Through Storytelling
Eduard Gröller - computer scientist and professor for visual computing and human-centred technology
Scalable Interactive Visual Analysis combines computer-supported, interactive, visual representations of (abstract) data with automatic techniques to amplify cognition and facilitate modeling. In recent years data complexity has grown considerably, and data-driven storytelling has gained increased interest. To exemplify data-driven storytelling, we present molecumentaries for producing documentary-style content using real-time scientific visualization. Molecular documentaries feature structural models from molecular biology, created through adaptable methods instead of the rigid traditional production pipeline.
Our work is motivated by the rapid evolution of scientific visualization and its potential in science dissemination. Without some form of explanation or guidance, however, novices and lay-persons often find it difficult to gain insights from the visualization itself. We integrate such knowledge using the verbal channel and provide it along an engaging visual presentation. We turn texts written by domain experts into verbal representations using text-to-speech functionality and provide them as a commentary. Afterwards storytelling as promising communication medium will be put into a social context. Possible future directions will be briefly touched in areas like immersive/situated analytics, conversions between text and visuals, and industry 4.0 to 6.0.
Elvin Luku: Recreating News on TikTok: Challenges and Opportunities for Mainstream Media to Reach New Audiences
The audiences of traditional media are getting older, while the young generations have chosen social media as the main source of information and entertainment. The study of audience behavior from the Department of Journalism & Communication in the University of Tirana have found that Generation Z (under 25s) “spent more than 4 hours using a smartphone, but none of them to consume news during this time”. Meanwhile the latest survey of MediaLook Center (2020) has identified that “Instagram and TikTok are the main source of information for more than 70% of teenagers”. These trends comply with the latest international report of Reuters Institute (2022) which underlines that "the so-called GenZ have embraced Instagram and TikTok for news, entertainment and distraction, to express their political rage – but also to tell their own stories in their own way”.
Taking in consideration these data and the challenge of the traditional newsrooms to reach the young audiences this paper raises several research questions: Can traditional media produce reports, storytelling and criticism through Tik Tok’s specific language and visual rhetoric in the platform to gain the attention of GenZ? May these new forms of video production be considered as a journalism genre in the social media 3.0? Can traditional newsrooms have a different approach of various social, political, economic and cultural phenomena, to produce infotainment Tik Tok videos?
This study employs a participatory research methodology. The author of this paper has been part of the board of ethics of the first Tik Tok newsroom in Albania called TikTok Corruption. During a seven-month period from May 1 to November 30, 2020 the team of 5 Tik Tokers have published 366 productions on the platform, based on the social, political and economic developments in Albania. This productions have been analyzed using a quantitative and qualitative method, aimed at highlighting the limitations and achievements of this new form of journalism on Tik Tok. Moreover, five interviews have been conducted with young Tik Tokers. The preliminary findings indicate that people with a journalistic background have found difficulties to adapt to the Tik Tok culture of storytelling, where the journalist conveys information through his own performance. Secondly, Tik Tokers face difficulties to express through visual symbolism and therefore they complement visual rhetoric with long explanatory texts.
Sohail Dahdal: Enhancing Youth Engagement in Climate Action Messages through Virtual Reality Immersive Gameplay: A Case Study
Effective communication of critical messages, such as the urgency of addressing climate change, to the youth demographic poses a formidable challenge. Often, the call to action is subtly embedded within the message, limiting the degree of engagement required to catalyze meaningful responses (Bandura, Cherry, 2020). This paper investigates this issue by presenting a case study centered on a virtual reality (VR) immersive series game meticulously crafted to convey the intricacies of climate change. Unlike conventional approaches, this VR game merges narrative-driven gameplay with a embedded call to action, thereby enabling participants to comprehend the issue, explore potential solutions, and pledge proactive measures (Boncu, Candel, Popa, 2022).
Virtual Reality, recognized for its capacity to induce a heightened sense of presence and immersion (Sanchez-Vives, Slater, 2005), offers a unique platform for amplifying engagement. By enveloping participants within the narrative, VR holds the potential to elevate the youth's involvement in serious matters. The integration of gameplay elements further enhances this potential by fostering a deeper connection with the subject matter (Laamarti, Saddik, 2014). This augmented engagement not only provides a more effective means of conveying complex concepts but also contributes to increased motivation for real-world action beyond the virtual realm (Hügel, Davies, 2022).
Through the lens of the case study, this paper demonstrate the effectiveness of the VR immersive serious games in bridging the gap between message delivery and tangible action (Martini, Resceanu, Tilea, 2023). It explores how the combination of immersive storytelling and interactive gameplay influences youth perception, knowledge retention, and willingness to commit to climate-conscious behaviors. The findings suggest that the synergy between VR immersion and gameplay mechanics is conducive to fostering a sense of personal responsibility and agency, leading to a higher likelihood of meaningful actions being taken outside the confines of the game environment.
In conclusion, this paper underscores the potential of VR immersive gameplay as a transformative tool for conveying critical messages to the youth. By facilitating a profound understanding of intricate issues like climate change and nurturing a stronger connection to these matters, VR has the capacity to propel the younger generation towards more active and impactful participation in addressing global challenges. This research contributes valuable insights into the realm of innovative persuasive communication strategies, where technology converges with meaningful engagement to inspire genuine change.
Aline Roes Dalmolin, Ada Cristina Machado Silveira and Camila Hartmann: Parallel Brazil: Analysis of the Bolsonarist Media Sphere
After the second round of the Brazilian presidential elections in 2022, a large part of the population denied basic principles of the democratic process. Supporters of former President Jair Bolsonaro contested the result of the polls, refusing to accept the victory of Luis Inácio Lula da Silva. Reflecting on the communicational and media processes that allow these episodes to occur, is essential to understand the consequences of Bolsonarism and the many ills it creates or exposes. It is a refusal of communicational barbarism, seeking elements to combat it based on the understanding of the circuits and circulatory processes typical of the far-right media anatomy.
Brazil has an environment favourable to the dissemination of disinformation and with freedom of expression problems, as well as lack of public trust, encourage the consecration of autocratic societies, or the confrontation between visions of exercise of political power (Stier, 2015). This is accompanied by a journalism institutional crisis scenario, which reflects on the constant decrease in their circulation (IVCB, online) provoked by decreased renown print vehicles in Brazil business model. In addition, the productive structures are still very much linked to industrial logic, there is frequent contestation of the credibility of the national mainstream media. Readers migrate to other information sources taken as more credible and with innovative publication formats, such as WhatsApp and Telegram groups’ contents.
Our proposal is to investigate the Bolsonarist media sphere (Rocha, 2023). That media sphere aims to produce fake news and polarizing narratives (good against evil, left against right), creating a parallel reality and uniting political militancy. In this context, communication is restricted to the exercise of the phatic function within polarized bubbles. The extreme right polarization structure exhausts the communication process, promoting discirculation. The intense circulation in an environment of communicational barbarism, paradoxically, configures itself in discirculation and the informational moral panic (Carlson, 2018) is correlated with the emergence of that cognitive dissonance process.
This text analyzes Bolsonarist media sphere exemples, more specifically two Bolsonarist channels on Telegram between the 1st and the 8th of January 2023, in which an intense amount of posts and disinformation content circulated daily. The chosen clip covers the period of one week between the inauguration ceremony of Luís Inácio Lula da Silva as president of Brazil, which took place on January 1st, and the January 8th riots, with the invasion and depredation of the National Congress, the Federal Supreme Court (STF) and the Planalto Palace by radical Bolsonarists in brazilian capital. The Bolsonarist Telegram groups were observed from the methodological perspective of mediatized interactional circuits (Braga, 2017). It is expected to observe an escalation of radicalized mobilization, in which the members of the group were summoned to participate in anti-democratic acts. On the other hand, the mobilization of these groups are being investigated by Brazilian Judiciary and police agencies, in order to determine the responsibilities for anti-democratic acts and seek to punish the participants and those responsible.
(Chair: Yulia Belinskaya)
Video Presentation and remote talk
Thomas Weibel: Mont - An Open Data Mountain Peak Recognition App in Vanilla JS
Gerhard Sprung and Martin Schweighofer: Project DesART: Utilising an AR Sand Table to Understand Complex Interconnections
Julian Rubisch and Tobias Leibetseder: Vibrate resonAlte - Prompt the Artist
Robert Halwaß, Steffen Prowe, Evelyne Becker, Joachim Villwock, Martina Mauch, Linnea Pehl, Clara Simon, and Lena Ziesmann: Immersive STEM -Laboratories to Improve Teaching and Learning (Demo)
Alexander Rind, Wolfgang Aigner, Magdalena Boucher, Florian Grassinger, Stefanie Größbacher, Lydia Popp, and Johannes Pflegerl: Data-Driven Infographics for Counselling Interviews with Family Caregivers in Dementia Care
Dementia is a cognitive disorder that progressively impairs activities of daily living. Thus, a high level of care is needed, which is very often provided by family members or other informal caregivers. This puts family caregivers under a lot of physical and emotional strain. The Angehörigendialog is a social-diagnostic tool that was developed to identify and alleviate the needs of family caregivers in the Austrian region Burgenland. It provides a questionnaire that mobile nurses use as guide for counseling interviews with family caregivers. Then, a visual report of the individual answers is used to reflect the caregiver’s situation reflected together so that coping strategies and available support can be discussed.
This talk presents a follow-up project, in which we designed and implemented a web-based questionnaire and bespoke data-driven infographics for the report. The tool was created in a human-centered design process in cooperation between the visualization research group of and institute of social inclusion research. Target users were involved in focus group interviews in the beginning and finalization of the project. The resulting infographics designs are, on the one hand, robust regarding the possible inputs that care givers can provide in the questionnaire. On the other hand, they apply metaphors to be easily understandable and motivate a resource-oriented reflection. The software and the designs are available as free and open-source software at https://github.com/fhstp/angehoerigendialog and can be tested at https://angehoerigendialog.fhstp.ac.at/. The Angehörigendialog is a case study for a visualization scenario that involves data at a tiny, personal scale that is visualized for personal reflection and as support mutual communication in the counseling interviews. Thus, it marks the intersection of the fields of collaborative data visualization with personal visualization.
Julia Boeck, Alexander Rind and Florian Zahorka: Personnel Fluctuation in Austrian Emergency Medical Services: A Data Visualisation Approach
Ensuring steady medical care in Austria's Emergency Medical Services depends heavily on paramedics. Experts in this field stress the importance of understanding paramedic turnover in a systematic way to maintain consistent medical care. Various factors contribute to paramedics leaving their stations, making it challenging for EMS stations to prevent dropouts. To address this, an annual meeting between paramedics and team managers aims to tackle exit reasons, but some factors may go unnoticed. To assist commanders and team managers, a visualization method was developed to track paramedic data. This includes identifying paramedics not meeting shift requirements and showcasing their preferred shift partners and types. Developed through a design study methodology and interviews with two ambulance station experts, the visualizations—mosaic-like, lollipop chart, slope graph, and highlighted table—proved beneficial. They enable commanders to grasp turnover patterns, helping to find solutions. In essence, these visualizations are valuable according to rural ambulance station domain experts.
(Chair: Kajetan Enge)
„Raise the Treasure!“ - Cultural Data and its potential for visualization & storytelling
Kasra Seirafi - Co-founder of Fluxguide
The lecture highlights current possibilities, trends and technologies for cultural mediation. Mobile computing, AR/VR/XR, data analytics, visualization methodologies, and more, offer new opportunities for outreach, learning and digital innovation to museums. We will connect an EU funded research project („InTaVia“) with museum/cultural practice (Web Application for the „Stiftung Preussische Schlösser und Gärten Berlin-Brandenburg“), and discuss concepts, technologies and lessons learned. We will show approaches how to raise the cultural treasure of exhibitions, collections, and cultural data and how to make it available for experts, the broad public and education.
Bruno Caldas Vianna: Visual Artificial Intelligence's Matter of Taste
This paper delves into the concept of taste as a classifier and its role in perpetuating cultural capital. Pierre Bourdieu's study on class and taste highlights how those with higher cultural capital dictate societal notions of good taste, influencing dominated classes. In the era of machine learning visual generative methods, the perpetuation of cultural capital ownership occurs through statistical processes.
The evaluation of visual neural networks' aesthetic quality involves two crucial steps: filtering out low-quality images and creating test synthetic images for evaluation during training. However, these evaluations are biased, reflecting the preferences of a select group of individuals. Aesthetic evaluation data is obtained through public rating systems, but most users belong to a specific demographic, leading to further homogeneity in taste. Consequently, current rating systems reinforce a limited cultural capital rooted in access to technology and computational creativity.
To foster diversity in neural generative systems, the paper proposes new scoring systems, incorporating ratings from individuals in the Global South, those unfamiliar with generative systems or computers, and marginalized cultures. This endeavor aims to build a more inclusive aesthetic guidance and address the dominance of specific cultural capital in the field of visual culture.
Alaz Okudan: Operational Mishap and Unexpected Outcomes: Failed Imagery and Accidental Aesthetics of Artificial Intelligence
This proposal offers an investigation into the user reception of accidental aesthetics encountered in AI-generated images produced by text-to-image technology. Although it is a technology that draws growing interest, AI’s technical and social norms, as well as its dialogue with its users, are still in the developmental stages. Furthermore, due to the discrepancy and incongruity between language and the visual world, AI-generated image platforms carry a strong capacity to create results that may transcend user expectations. With the hypothesis that the appeal of AI-generated images partly stems from their ability to produce unexpected and accidental outcomes that can challenge human expectations, this research aims to comprehend how accidents and failures are perceived in AI images by their users and how their interaction with such results affect their artistic expression and critical thinking.
Isabel Pina: Media Arts Suspensions and Deviations: Paz Dos Reis between Cinema and Photography
Media have non-linear evolutions. This means that new media failures or retractions can highlight overlooked features of old media. The very early history of Portuguese Cinema is one of those moments: the new and promising cinema technologies were first experimented and then discarded by Aurélio da Paz dos Reis, the pioneer of Portuguese cinema. Instead, he intensely explored in photography a solution for covering current affairs.
The creation of the ‘Portuguese Stereoscope’, a commercial stereoscopic card series that depicted political events such as the agitated republic rallies, the social life of the bourgeoisie and the daily life in the streets of Porto, was an international innovation: it consisted of the very first photojournalism in 3D.
This research project will investigate this historical fact by checking in the press the events covered by Paz dos Reis and by creating an online media art installation that will allow the matching of these materials with his stereos to understand how these 3D news materials relate to current VR 360º new videos.
This digital curation will be developed as an artistic research method which will also allow to experiment with film materials: Aurelio’s original footage, a new short film shot with his original camera kept by the Portuguese Cinematheque (ANIM) and a documentary film focused on some of his most symbolic photographs. The short film we propose to be shot with his original camera is part of an experimental media archaeology method that will be a core part of this research. It is expected that this research project leads to a new approach towards the work of Paz dos Reis.
(Chair: Markus Wintersberger)
Bärbel Bissinger, Christian Märtin and Christian Herdin: Applied Emotion AI in Video Conferences
Artificial Emotional Intelligence, Affective Computing or Emotion AI deals with the ability of machines to recognize human emotions. Our physical signals can be analyzed and categorized which makes it possible to train machines to recognize emotions and respond to it. This changes how we interact with technology, and it could also change how we interact with each other. There are more and more research activities in this field as well as products on the market that apply Emotion AI. According to a recent forecast, emotion detection and recognition is a rapidly growing market which will be worth more than 42 billion USD by 2027. As technology becomes ubiquitous in interpersonal interactions and activities, Emotion AI could make our tool-based interactions more human-like.
Emotions play a central role in our communication and decision-making and should therefore get more attention, even in business environments. Since Covid-19, more and more meetings are being held virtually which has advantages but also many disadvantages. For example, the transmission of non-verbal signals becomes more difficult and changes our interaction behavior. People also report exhaustion caused by the huge number of video conferences, the so-called Zoom-Fatigue phenomenon. One issue is the constant self-view. Seeing yourself all the time is not natural and can have a positive effect on self-awareness, but, e.g., a negative one on enjoyment and perceived productivity. Therefore, alternatives to activated cameras that nevertheless transmit emotional reactions automatically could be useful.
In our previous research, we used Facial Expression Recognition (FER) in video conferences to detect emotional states of participants. In these small-scale user studies, we analyzed the facial expressions of participants to detect and visualize emotions with a FER-tool and human observers. We discovered situations in which facial expressions alone were not sufficient to correctly identify emotions outside the lab in our small-scale study. Circumstances such as changing video quality or participant movement made automatic emotion recognition via faces challenging. In these scenarios, we could only analyze emotions when participants shared their videos during the meeting.
In this paper, we would like to present a different approach, where we integrate and analyze a set of different physical signals in addition to facial expressions, and where people do not share their video in the conference, but an interface which changes depending on the emotional state of the participant. This makes it possible to share emotions without sharing the own video. For this purpose, we use our SitAdapt system.
SitAdapt is an integrated software system for enabling situation-aware real-time adaptations for web and mobile applications. SitAdapt uses the different APIs of the devices such as eye-tracker, wristband, facial expression, and EEG signal recognition software, as well as metadata from the application to collect data about the user. The included rule editor allows the definition and modification of situation rules, e.g., for specifying the different user states and the resulting actions. The rule editor can use all input data types and attribute values as well as their temporal changes for formulating rule conditions. At the runtime of the application, the rules are triggered by the adaptation component for adapting the user interface, if the conditions of one or more rules apply.
Benjamin Roszipal, Sebastian Egger-Lampl and Markus Karlseder: Enhancing Higher Education through XR Technology: An Innovative Approach to Knowledge Acquisition, Skill Development, and Examining
Knowledge acquisition and skill development are fundamental components of effective learning in higher education. Traditional methodologies involving pattern recognition, knowledge reproduction, and hands-on practice have long been utilized to build competencies in various fields. However, certain disciplines, such as healthcare and large industries, often face limitations in hands-on training due to safety concerns or resource-intensive simulations.
This contribution explores the potential of experiential learning in digital simulations supported by Extended Reality (XR) technology as a novel and efficient approach to address these challenges. XR presents an advanced platform for creating immersive and engaging experiential learning experiences. Leveraging XR, we have developed a didactic concept that enables learners to acquire knowledge, develop skills, and achieve successful test results in a meaningful and effective manner.
The implementation of XR supported experiential learning (XREL) offers several advantages over traditional methods. It provides learners with a safe and risk-free environment to practice complex tasks, promoting higher retention of knowledge and skill transfer. XREL can be tailored to diverse educational needs, catering to both beginners seeking foundational knowledge and seasoned experts looking to enhance their proficiency.
Our research investigates the effectiveness of XREL in learning outcomes across various disciplines. We present a comprehensive analysis of our XREL-based didactic approach, demonstrating its applicability in upskilling employees, students, and experts alike. The study showcases how our XREL solutions facilitate experiential learning, empowering learners to acquire knowledge and skills through immersive experiences.
Through this conference contribution, we aim to shed light on the potential of XREL technology in transforming higher education, bridging the gap between theoretical understanding and practical application. We provide empirical evidence of the positive impact of XR simulations on knowledge acquisition, skill development, and proficiency assessment. Ultimately, this work paves the way for a more efficient and accessible approach to learning and training in various fields, fostering continuous growth and advancement in education and professional development.
XRCONSOLE is an R&D-driven SME focused on incorporating cutting-edge research into interactive educational XR technologies. Engaging in various national and international research projects, we develop XR prototypes, integrate behavioral probes and analytics to create educational and training KPIs. Our interdisciplinary team with scientific, development, and educational backgrounds ensures evidence-based practices, fostering meaningful collaboration with academia, industry, and educators. With a vision for immersive learning experiences, we strive to revolutionize education through XR technology, empowering learners of all backgrounds to acquire knowledge and skills in unprecedented ways.
Dennis Rosenberg: Predictors of the Perceived Health-Related Usefulness of Mobile Devices in Later Life: Results of the Health Information National Trends Survey
Objectives and Research Question: The goal of this study was to assess the impact of the older adults' background characteristics on their perceptions of the health-related mobile device usefulness. The third-level digital divide approach was employed as the theoretical framework for the study. According to this approach, (older) people tend to benefit differently from using technology even when they have equal access to it and are equally competent or skillful in its use. The third-level digital divide approach maintains that the differences in these benefits or their perceptions are attributed to the categories (older) people belong. The study attempted to answer the following question: which socio-demographic and health-related characteristics of older adults are associated with their perception of mobile device usefulness in various health domains? Methods: The data were obtained from the Health Information National Trends Survey (Wave 5, Cycle 4) conducted in February-June 2020. The sample included 1373 U.S. older mobile device owners, i.e. people who reported owning either smartphone, tablet, or both. The lower limit for the respondent age, in accordance with the definitions of the United Nations and World Health Organization, was set at 60 years. The items served as dependent variables asked whether tablet or smartphone helped respondents in the following domains: tracking progress on a health-related goal (achieving health-related goal), making a decision about how to treat chronic illness or condition (medical decision making), and discussions with healthcare provider (patient-provider communication). Since each item was dichotomously introduced, the data were analyzed using logistic regression models. Results: Being 75 years old or older and self-definition as White significantly interacted with respect to perceived mobile device usefulness in achieving health-related goal and patient-provider communication. Age and White race variables were negatively and independently associated with perceived mobile device usefulness for medical decision making. Being married, female gender, better self-rated health, and having a hypertension diagnosis corresponded to a greater likelihood of perceiving own mobile device as useful in achieving health-related goal. Mental health disorder diagnosis positively related to perceived usefulness for medical decision making and patient-provider communication. Education and income levels also demonstrated some associations, mainly with respect to perceived usefulness in achieving health-related goal. Discussion: The findings provide support for the employed theoretical framework by showing that some categories of older adults are more likely than others to perceive their mobile devices useful in health domains. Health background seems to be less dominant than socio-demographic background in explanation of the mobile device health-related usefulness, providing additional support for the third-level digital divide approach. The findings may serve for preparation of programs aimed at encouraging older adults to use their mobile devices for health purposes.
Robert Halwaß, Steffen Prowe, Evelyne Becker, Joachim Villwock, Martina Mauch, Linnea Pehl, Clara Simon and Lena Ziesmann: Immersive STEM-Laboratories to improve teaching and learning
The project “Interactive Teaching in Virtual STEM labs | MINT-VR-Labs” (MINT is the German acronym for STEM) takes up the didactic potential of virtualizing laboratories in order to anchor innovative interaction formats in virtual space and thus implement new blended learning and virtual teaching/learning formats in practical modules for the Berlin University of Applied Sciences (BHT). In these virtual labs, students can, for example, explore biotechnology labs, look behind the scenes of a theater stage, learn basic programming skills in a pizza factory, or experience complex mathematical functions. Virtual laboratory exercises are intended to supplement classroom teaching, reduce the heterogeneity of students' prior knowledge, and increase student success. For this reason, the development and testing of these virtual laboratories will be didactically accompanied and the effectiveness of the use of these digital media in university teaching will be evaluated.
(Chair: Georg Vogt)
Gallery
Fotocredits: Biberovic, Muthenthaler, Steinwendtner
Team
-
Researcher
Media Business Research Group
Institute of Creative\Media/Technologies - Department of Media and Digital Technologies
- Course Leader Digital Photography and New Visual media (MA)
- Associate Lecturer
- Department of Media and Digital Technologies
-
Senior Researcher
Media Creation Research Group
Institute of Creative\Media/Technologies - Department of Media and Digital Technologies
- Lecturer
- Department of Media and Digital Technologies
Cooperation
