DHQ: Digital Humanities Quarterly

2021 15.1

AudioVisual Data in DH

Editors: Taylor Arnold, Jasmijn van Gorp, Stefania Scagliola, and Lauren Tilton

Front Matter

[en] Introduction: Special Issue on AudioVisual Data in DH

Taylor Arnold, University of Richmond; Stefania Scagliola, Université du Luxembourg; Lauren Tilton, University of Richmond; Jasmijn Van Gorp, Utrecht University

Abstract [en]

[en] Founding the Special Interest Group Audio-Visual in Digital Humanities: An Interview with Franciska de Jong, Martijn Kleppe, and Max Kemman

Stefania Scagliola, Université du Luxembourg

Abstract [en]

Section 1: Annotation of AV Material as Method and Theory

[en] Exploring Film Language with a Digital Analysis Tool: the Case of Kinolab

Allison Cooper, Bowdoin College; Fernando Nascimento, Bowdoin College; David Francis, Bowdoin College

Abstract [en]

[en] Audiovisualities out of Annotation: Three Case Studies in Teaching Digital Annotation with Mediate

Joel Burges, University of Rochester; Solvegia Armoskaite, University of Rochester; Tiamat Fox, University of Rochester; Darren Mueller, University of Rochester; Joshua Romphf, University of Rochester; Emily Sherwood, University of Rochester; Madeline Ullrich, University of Rochester

Abstract [en]

[en] The Media Ecology Project: Collaborative DH Synergies to Produce New Research in Visual Culture History

Mark Williams, Dartmouth College; John Bell, Dartmouth College

Abstract [en]

[en] Audiated Annotation from the Middle Ages to the Open Web

Tanya E. Clement, University of Texas at Austin; Liz Fischer, University of Texas at Austin

Abstract [en]

Section 2: Analyzing (Meta)Data

[en] Healing the Gap: Digital Humanities Methods for the Virtual Reunification of Split Media and Paper Collections

Stephanie Sapienza, Maryland Institute for Technology in the Humanities; Eric Hoyt, University of Wisconsin-Madison; Matt St. John, University of Wisconsin-Madison; Ed Summers, Maryland Institute for Technology in the Humanities; JJ Bersch, University of Wisconsin-Madison

Abstract [en]

[en] PodcastRE Analytics: Using RSS to Study the Cultures and Norms of Podcasting

Eric Hoyt, University of Wisconsin-Madison; J.J. Bersch, University of Wisconsin-Madison; Susan Noh, University of Wisconsin-Madison; Samuel Hansen, University of Michigan and University of Wisconsin-Madison; Jacob Mertens, University of Wisconsin-Madison; Jeremy Wade Morris, University of Wisconsin-Madison

Abstract [en]

[en] Transdisciplinary Analysis of a Corpus of French Newsreels: The ANTRACT Project

Jean Carrive, Institut National de l'Audiovisuel; Abdelkrim Beloued, Institut National de l'Audiovisuel; Pascale Goetschel, Centre d'Histoire Sociale des Mondes Contemporains; Serge Heiden, ENS Lyon; Antoine Laurent, Laboratoire d'Informatique de l'Université du Mans; Pasquale Lisena, EURECOM; Franck Mazuet, Centre d'Histoire Sociale des Mondes Contemporains; Sylvain Meignier, Laboratoire d'Informatique de l'Université du Mans; Bénédicte Pincemin, ENS Lyon; Géraldine Poels, Institut National de l'Audiovisuel; Raphaël Troncy, EURECOM

Abstract [en]

[en] Topological properties of music collaboration networks: The case of Jazz and Hip Hop

Lukas Gienapp, Leipzig University; Clara Kruckenberg, Leipzig University; Manuel Burghardt, Leipzig University

Abstract [en]

Section 3: Creative and Liberatory Ways to Remix AV Data

[en] Afrofuturist Intellectual Mixtapes: A Classroom Case Study

Tyechia L. Thompson, Virginia Tech; Dashiel Carrera, Virginia Tech

Abstract [en] This article is a classroom case study of the Intellectual Mixtape Project, an AudioVisual digital humanities module. The intellectual mixtape uses jazz and hip hop as a framework to create an audio compilation and “conversation” that samples literary-audio texts (such as SunRa speeches, Octavia Butler interviews, Tracy K. Smith’s poetry readings, etc.). Each track of the intellectual mixtape has three audios: 1) the literary-audio texts from the syllabus, 2) the students’ voice in their own words, and 3) an audio of the students’ choice. As a companion to each track, students write 500 words of liner notes that must include the title of their track and their curation and mixing decisions.Students then publish their entire intellectual mixtape (three or more tracks) with original or “remixed” cover art on an online platform. The first part of the study will discuss the structure of the intellectual mixtape assignments. In these assignments, students are provided with literary-audio texts, required to complete and submit audio homework assignments, and taught the basics of audio editing. This method of teaching and analyzing literature shifts the practice of literary analysis from top down approaches that privileges the authority of the text and instead encourages the student to “converse” with the text to create new knowledge. This methods also reflects the artistic practice of Afrofuturist artists and theorist who improvise, remix, and sample to create their work. The second part of the study will discuss a performance and midterm adaptation of the Intellectual Mixtape Project entitled Sound of Space: An Interactive Afrofuturist Experience. “Sound of Space” was an immersive performance with four-sensory stations that featured Afrofuturist themes. The midterm adaptation was showcased in the Cube, a four-story high, state-of-the-art multimedia black box theater at Virginia Tech. In preparation for the performance, students merged sound engineering, 360 degree-video-projection, improvisational performance, and light design. “Sound of Space” introduced students and audiences’ to an immersive Afrofuturist-audio experience and pushed the boundaries of literary analysis. The third part of the study will address challenges with the Intellectual Mixtape Project. Challenges include finding relevant literary-audio texts and dealing with the many limitations imposed by U.S. copyright law. Some ways to address the challenges imposed by U.S. copyright law might be to 1) reclassify sampling audio as a form of quotation, 2) use databases of copyright-free music, 3) find culturally significant works from lesser-known artists who will license their tracks, and/or 4) pay royalties.

[en] Annotating our Environs with the Sound and Sight of Numbers: The DataScapes Project

John Bonnett, Brock University; Joe Bolton, Business Insight 3; William Ralph, Brock University; Amy Legault, Billyard Insurance Group; Erin MacAfee, University of Ottawa; Michael Winter, Brock University; Chris Jaques, Badal.io; Mark Anderson, Independent Consultant

Abstract [en]

[en] What Does A Photograph Sound Like? Digital Image Sonification As Synesthetic AudioVisual Digital Humanities

Michael J. Kramer, SUNY Brockport

Abstract [en] Computers have the capacity to transpose the pixels, shapes, and other features of visual material into sound. This act of data correlation between the visual and the audial produces a new artifact, a sonic composition created from the visual source. The new artifact, however, correlates precisely to data in the original, thus allowing for fresh ways of perceiving its form, content, and context. Seeming to distort the visual object into an aural one paradoxically allows an observer to observe the visual evidence anew, with more accuracy. A kind of generative, synesthetic criticism becomes possible by cutting across typical boundaries between the visual and the audio, the optic and the aural. Listening to as well as looking at visual artifacts by way of digital transpositions of data enables better close readings, more compelling interpretations, and deeper contextual understandings. Building on my earlier scholarship into image glitching, remixing, and sonification, this essay investigates a photograph of Joan Baez performing at the Greek Amphitheater in Berkeley, California, during the early 1960s. The image comes from my project on the Berkeley Folk Music Festival and the history of the folk music revival on the West Coast of the United States. Here, the use of digital image sonification becomes particularly intriguing. While we cannot magically recover the music being made in the photograph, we can more closely attend to the ghosts of sound within the silent snapshot. Digital image sonification does not recover the music itself, but it does help to amplify issues of gender, power, embodiment, spectacle, performance, hierarchy, and performance in my perceptions of Baez making music in the photograph. Using the ear as well as the eye to scan the image for its multiple levels of meaning leads to unsuspected perceptions, which then support more revealing analysis. In digital image sonification, a cyborgian dance of data, signal, image, sound, history, and human perception emerges, activating visual materials for renewed scrutiny. In doing so, this mode of AudioVisual DH activates the scholarly imagination in promising new ways.

[en] From close listening to distant listening: Developing tools for Speech-Music discrimination of Danish music radio

Iben Have, Aarhus University; Kenneth Enevoldsen, Aarhus University

Abstract [en]

[en] Hearing Change in the Chocolate City: Computational Methods for Listening to Gentrification

Alison Martin, Dartmouth College

Abstract [en]

Section 4: Reconfiguring Computational Methods for AV

[en] Advances in Digital Music Iconography: Benchmarking the detection of musical instruments in unrestricted, non-photorealistic images from the artistic domain

Matthia Sabatelli, Montefiore Institute; Nikolay Banar, University of Antwerp; Marie Cocriamont, Royal Museums of Art and History, Brussels; Eva Coudyzer, Royal Institute for Cultural Heritage; Karine Lasaracina, Royal Museums of Fine Arts of Belgium, Brussels; Walter Daelemans, University of Antwerp; Pierre Geurts, University of Liège; Mike Kestemont, University of Antwerp

Abstract [en]

[en] Music Theory, the Missing Link Between Music-Related Big Data and Artificial Intelligence

Jeffrey A. T. Lupker, The University of Western Ontario; William J. Turkel, The University of Western Ontario

Abstract [en]

[en] Comparative K-Pop Choreography Analysis through Deep-Learning Pose Estimation across a Large Video Corpus

Peter Broadwell, Stanford University; Timothy R. Tangherlini, University of Calfornia, Berkeley

Abstract [en] The recent advent of deep learning-based pose detection methods that can reliably detect human body/limb positions from video frames, together with the online availability of massive digital video corpora, gives digital humanities researchers the ability to conduct "distant viewing" analyses of movement and particularly full-body choreography at much larger scales than previously feasible. These developments make possible innovative, revelatory digital cultural analytics work across many sources, from historical footage to contemporary images. They are also ideally suited to provide novel insight to the study of K-pop choreography. As a specifically non-textual modality, K-pop dance performances, particularly those of corporate and government-sponsored "idol" groups, are a key component of K-pop’s core mission of projecting "soft power" into the international sphere. A related consequence of this strategy is the ready availability in online video repositories of many K-pop music videos, starting from the milieu's origins in the 1990s, including an ever-growing collection of official "dance practice" videos and fan-contributed dance cover videos and supercuts from live performances⁠. These latter videos are a direct consequence of the online propagation of the "Korean wave" by generations of tech-savvy fans on social media platforms. In this paper, we describe the considerations and choices made in the process of applying deep learning-based posed detection to a large corpus of K-pop music videos, and present the analytical methods we developed while focusing on a smaller subset of dance practice videos. A guiding principle for these efforts was to adopt techniques for characterizing, categorizing and comparing poses within and between videos, and for analyzing various qualities of motion as time-series data, that would be applicable to many kinds of movement choreography, rather than specific to K-pop dance. We conclude with case studies demonstrating how our methods contribute to the development of a typography of K-pop poses and sequences of poses ("moves") that can facilitate a data-driven study of the constitutive interdependence of K-pop and other cultural genres. We also show how this work advances methods for "distant" analyses of dance performances and larger corpora, considering such criteria as repetitiveness and degree of synchronization, as well as more idiosyncratic measures such as the "tightness" of a group performance.

[en] Moving Cinematic History: Filmic Analysis through Performative Research

Jenny Oyallon-Koloski, University of Illinois at Urbana–Champaign; Dora Valkanova, University of Illinois at Urbana–Champaign; Michael J. Junokas, University of Illinois at Urbana–Champaign; Kayt MacMaster, University of Illinois at Urbana–Champaign; Sarah Marks Mininsohn, University of Illinois at Urbana–Champaign

Abstract [en]

[en] Towards a User-Friendly Tool for Automated Sign Annotation: Identification and Annotation of Time Slots, Number of Hands, and Handshape

Manolis Fragkiadakis, Leiden University; Victoria Nyst, Leiden University; Peter van der Putten, Leiden University

Abstract [en]

Section 5: Forms of AV Scholarship

[en] Books Aren't Dead: Resurrecting Audio Technology and Feminist Digital Humanities Approaches to Publication and Authorship

Emily Edwards, Bowling Green State University; Robin Hershkowitz, Bowling Green State University

Abstract [en]

[en] Another Type of Human Narrative: Visualizing Movement Histories Through Motion Capture Data and Virtual Reality

Eugenia S. Kim, The Hong Kong Academy for Performing Arts

Abstract [en]

[en] Deformin' in the Rain: How (and Why) to Break a Classic Film

Jason Mittell, Middlebury College

Abstract [en]

Reviews

[en] Book Review: Digital Sound Studies (2018)

Tracey El Hajj, University of Victoria

Abstract [en]

Göttingen Dialogues in Digital Humanities

Editor: Marco Büchler

Articles

[en] Introduction to Göttingen Dialogues 2016

Marco Büchler, Institut für Angewandte Informatik (InfAI)

Abstract [en]

[en] Hierarchical or Non-hierarchical? A Philosophical Approach to a Debate in Text Encoding

Alois Pichler, University of Bergen

Abstract [en]

[en] Annotating ritual in ancient Greek tragedy: a bottom-up approach in action

Gloria Mugelli, Laboratorio di Antropologia del Mondo Antico, Università di Pisa; Federico Boschetti, CoPhiLab, ILC, CNR Pisa; Andrea Bellandi, CoPhiLab, ILC, CNR Pisa; Riccardo Del Gratta, CoPhiLab, ILC, CNR Pisa; Anas Fahad Khan, CoPhiLab, ILC, CNR Pisa; Andrea Taddei, Laboratorio di Antropologia del Mondo Antico, Università di Pisa

Abstract [en]

[en] Can an author style be unveiled through word distribution?

Giulia Benotto, Extra Group

Abstract [en]

[en] Using an Advanced Text Index Structure for Corpus Exploration in Digital Humanities

Tobias Englmeier, CIS, Ludwig-Maximilians University, Munich, Germany; Marco Büchler, Institute of Computer Science, University of Göttingen, Göttingen, Germany; Stefan Gerdjikov, FMI, University of Sofia "St. Kliment Ohridski", Sofia, Bulgaria; Klaus U. Schulz, CIS, Ludwig-Maximilians University, Munich, Germany

Abstract [en]

[en] Computer Vision and the Creation of a Database of Printers’ Ornaments

Hazel Wilkinson, University of Birmingham, UK; James Briggs, Graphcore; Dirk Gorissen, Machine Learning Ltd.

Abstract [en]

[en] Inferring standard name form, gender and nobility from historical texts using stable model semantics

Davor Lauc, Chair of Logic, Department of Philosophy, Faculty of Humanities and Social Sciences, University of Zagreb; Darko Vitek, Department of History, Centre of Croatian Studies, University of Zagreb

Abstract [en]

[en] German Narratives in International Television Format Adaptations: Comparing Du und Ich (ZDF 2002) with Un Gars, Une Fille (Quebec 1997-2002)

Edward Larkey, University of Maryland, Baltimore County

Abstract [en] This article cross-culturally compares the German remake of the Quebec sketch comedy/sitcom series Un Gars, Une Fille ("A Guy and a Girl") the the original version by correlating quantitative data derived from the Adobe Premiere Pro annotation function on the duration of narrative segments and incorporating these data into a an interpretation of family conflict management strategies, gender roles and conflicts between the mother-in-law and the the young 30-something couple as protagonists. The article examines a scene in which the daughter confronts her mother's trauma-inducing behavior on her as a young girl, and the boyfriend confronts the mother-in-law's animosity toward him. The article delves into the background for the transactional, belligerent, and obligational thinking behind the family relationship of the German couple compared to the more affectionate and conciliatory relationship in all other versions. The investigation postulates that the family relationships in the German context must be seen within the context of the Inability to Mourn, a major psychological study by Margarete and Alexander Mitscherlich in the 1960s of politically salient post World War 2 trauma among a large variety of social groups in West (and East) Germany. The emotional repression as a result of various forms of guilt, which never explicitly surfaced in the confrontations, was passed down from generation to generation, while this same, or similar social psychological contexts was seemingly not a factor in other countries, many of which had also experienced repressive dictatorships during World War 2 and afterwards, in which adaptations of these series were produced. Further collaborative investigations would be required to uncover the reasons for this discrepancy.

Articles

[en] From the Presupposition of Doom to the Manifestation of Code: Using Emulated Citation in the Study of Games and Cultural Software

Eric Kaltman, Department of Computer Science, California State University Channel Islands; Joseph Osborn, Department of Computer Science, Pomona College; Noah Wardrip-Fruin, Department of Computational Media, University of California, Santa Cruz

Abstract [en]

[en] Fostering Community Engagement through Datathon Events: The Archives Unleashed Experience

Samantha Fritz, Department of History, University of Waterloo; Ian Milligan, Department of History, University of Waterloo; Nick Ruest, Digital Scholarship Infrastructure Department, York Univeristy; Jimmy Lin, David R. Cheriton School of Computer Science, University of Waterloo

Abstract [en]

[en] Leonardo, Morelli, and the Computational Mirror

Alison Langmead, University of Pittsburgh; Christopher J. Nygren, University of Pittsburgh; Paul Rodriguez, San Diego Supercomputer Center; Alan Craig, Independent Consultant and Researcher

Abstract [en]

Reviews

[en] Networks, Maps, and Time: Visualizing Historical Networks Using Palladio

Melanie Conroy, The University of Memphis

Abstract [en]

[en] A Review of Twitter and Tear Gas

Nanditha Narayanamoorthy, York University

Abstract [en]

[en] A Review of Intergenerational Connections in Digital Families

Sucharita Sarkar, D.T.S.S. College of Commerce, Mumbai, India

Abstract [en]

Author Biographies

URL: http://www.digitalhumanities.org/dhq/vol/15/1/index.html
Comments:
Published by: and
Affiliated with: Digital Scholarship in the Humanities
DHQ has been made possible in part by the National Endowment for the Humanities.
Copyright © 2005 -

Unless otherwise noted, the DHQ web site and all DHQ published content are published under a Creative Commons Attribution-NoDerivatives 4.0 International License. Individual articles may carry a more permissive license, as described in the footer for the individual article, and in the article’s metadata.

Announcements

2021 15.1

AudioVisual Data in DH

Editors: Taylor Arnold, Jasmijn van Gorp, Stefania Scagliola, and Lauren Tilton

Front Matter

Section 1: Annotation of AV Material as Method and Theory

Section 2: Analyzing (Meta)Data

Section 3: Creative and Liberatory Ways to Remix AV Data

Section 4: Reconfiguring Computational Methods for AV

Section 5: Forms of AV Scholarship

Reviews

Göttingen Dialogues in Digital Humanities

Editor: Marco Büchler

Articles

Articles

Reviews

Author Biographies