Categories
Blog

PhDs for Polymaths (2011)

Over the past couple years I’ve taught a few graduate classes that incorporate ideas from the Digital Humanities and emphasize “multimodal scholarship,” and I’ve been conducting research on praxis-based PhD programs. It’s for these reasons, I assume, that the planning committee for our graduate students’ Critical Themes in Media Studies conference asked me earlier this year to organize an opening-night panel on multimodal doctoral work and praxis-based PhDs. So, for the past couple months I’ve contacted graduate directors and colleagues at various local institutions to ask if any of their students are completing non-traditional (i.e., multimedia, performance-based, practice-based, etc.) dissertations on media studies-related topics. Their recommendations have helped me to pull together an impressive panel of three inspired young artist-scholars. Next Friday evening, April 15, at 5:30pm, before Clay Shirky’s opening keynote, we’ll be gathering in the Teresa Lang Center, on the 2nd floor at 55 W 13th Street, to talk about “The Multimodal Dissertation.” Come join us.

Multimodal scholarship, writes USC’s Tara McPherson (2009), deploys “new experiential, emotional, and even tactile aspects of argument and expression” in order to “open up fresh avenues of inquiry and research.” How might we in Media Studies transform the media technologies that have traditionally been our research subjects, into researchtools, and thereby “open up fresh avenues” of creative scholarship? This panel examines how these new modes of scholarly practice are informing doctoral education. Our three panelists discuss how they’re infusing media-making into their dissertations, and how they’re navigating the still largely uncharted terrain of multimodal scholarship.

The Sound of America: Sound, Sensation, Sentiment, and Knowledge in American West Tourism

Jennifer Heuson 

Links between the American West and American identity, memory, and history are well documented. America constructs its uniqueness through the land and people west of the Mississippi. American West tourism is a crucial form of this construction.

Traveling west has become a ritual of citizenship, a pilgrimage to the birthplace of a mythical America. This is the America of cowboys and Indians, of gold mines and train robberies, of wild horses and still wilder people. It is an America of the past, performed in the present, informing the future. While scholars have devoted much energy to unpacking the significance of Wild West mythologies, two important areas remain underdeveloped: tourism and sound. My work engages both as key to the production and circulation of the “Wild” American West and its meanings. Tourist experiences of the American West play a pivotal role in knowledge of American history and identity. Yet, such experiences are neither natural, nor benign. They are mediated, historical, and political. They are actual and imagined. They are also sensual. It is the power of the sensual, living tourist encounter I hope to uncover by engaging its sonic contours. The sound of the American West, as a national soundscape, reveals much about how America is known, remembered, and imagined. It also hints at the future forms of American politics, at home and abroad.

Marquee Survivals: Racialized Urbanism in Cinema’s Recycled Spaces

Veronica Paredes

Marquee Survivals is an interactive, digital dissertation that explores contemporary conceptions of the repurposed movie theater. Across the United States, twentieth-century movie theaters have been converted into a variety of different establishments, including churches, swap meets, clothing and electronics stores. This project unravels how discussions surrounding these former movie houses racialize the spatial and historical perceptions of American popular media. In unpacking nostalgia’s place in touring the extant structures of film exhibition, Marquee Survivals highlights the roles race, ethnicity, and nation play in constructing the cultural narrative of cinema’s decline in the American downtown.

Incorporating methodologies from diverse academic disciplines, Marquee Survivals is also a networked digital dissertation that complicates dominant understandings of cinema’s early exhibition spaces by connecting them to present-day media consumption. Working toward an alternative media historiography of the repurposed movie theater, Marquee Survivals marries film theory and history, cultural studies, and digital media production. This presentation will feature documentation of Marquee Survival’s design processes and struggles. What are the challenges of building a distributed dissertation project that has equal investment in achieving rigorous scholarship and an affective user experience?

Hitting Walls (v.XVII): Some Strategies, Several Projections

Carlin Wing

Hitting Walls uses the sport of squash to address colonial histories, globalization and the potential for serious play within overdetermined structures. The project exists as a series of iterations made in a variety of media including large format photography, appropriated webgrabs, video, sculpture, performance, participatory activities and academic lectures. The most recent completed iteration took the form of a lecture and workshop on ball-making methods at Machine Project in Los Angeles this past January.

I expect my dissertation to exist as one more iterative element of this larger project. My broad goal is to use my dissertation as an opportunity to experiment with and make a claim for hybrid formats of intellectual work. As this is my first year in a doctoral program, it does not seem particularly helpful to pretend that I already know what form my dissertation will take. I cannot even, at this stage, claim with absolute confidence that it will make sense to me four years from now to consider the project to be part of Hitting Walls. I do expect a large amount of the work to be written but I also intend for there to be play within that writing, as well as essential elements, visual, aural or otherwise, which will work with the written components.

I would like to take the opportunity of this panel to briefly share a few of the Hitting Walls projects and to discuss various ways to experiment with academic, as well as other, forms. I would then like to open up a conversation that I am just beginning to have within my own department about how a dissertation is, can and should be defined. Right now it seems like a matter of shaping some good questions, setting them loose, and seeing how they ricochet.

Categories
Blog

The Chimerical Praxis PhD

Josef Albers, Homage to the Square, 1949
Josef Albers, Homage to the Square, 1949

For the past five or six years I’ve been on various iterations of the same committee — one charged with rethinking the PhD. Our more specific charge has been to investigate what doctoral-level training should mean at an institution that regards design, artistic production, and media-making as intellectual pursuits — research-based, informed by theory, and potentially theoretically generative. While we were having our discussions, lots of other professional organizations and institutions were having similar conversations about “praxis-based” PhDs, and some schools (like Carnegie Mellon) even put such programs into place. Of course Australia and the UK (and schools in a few other places) have been doing this sort of thing for years.

Last year, the Provost charged me with filtering those years’ worth of experience and dialogue (including, most recently, with the wonderful Lisa Grocott, Clive Dilnot, Simone Douglas, and Joseph Heathcott) into the creation of an official formatted-in-accordance-with-state-requirements proposal for a new PhD program. I worked for a couple months with an external research organization to flesh out our committees’ “environmental scans” of the various design-, arts-, and praxis-oriented doctoral programs around the world, then set to work on the proposal. There are quite a few parts of that document — including all the stuff about budgets and faculty workload, etc. — that I’m not qualified to complete. I’ve done what I can, so now I wait for the “operationalizers” to help me workshop these ideas and address the nuts and bolts. I’m not sure when, or if, that’ll happen. But because I dedicated a good part of the past several years of my life to this project, I kind of want some of these ideas to see the light of day. So, here are a few excerpts from the proposal:

Design/Media Praxis PhD

What defines and distinguishes this program are: 1) its orientations toward praxis, that is, the integration of theory and practice in the active creation of new artifacts, processes, and understandings; and 2) its embrace of a wide variety of research methods, creative approaches, and means of dissemination.

  • At its core is the conviction that various forms of practice – including but not limited to designing, media-making, curating, art, performance, and writing – can generate new knowledge and understanding of the world, and therefore can constitute fully legitimate pathways of scholarship.
  • Forms of acting, making, and practice outside the traditional limits of doctoral research require inquiry as much as do the classic academic disciplines. This is acutely the case in the arena of complex problem-solving/possibility-seeking – of central importance in The New School’s mission – where creative, transdisciplinary practices are well suited to exploring solutions through the production of new artifacts, processes, and understandings.
  • The advanced reflection made possible through this mode of doctoral study allows the exploration of knowledge through creative practice, without forsaking scholarly rigor.

Students enrolled in the program would propose and pursue a project germane to any of the fields of study represented at the graduate level at The New School, subject to approval by their advisors and programs/departments.

  • All projects would be hybrid, incorporating both writing and some form of practice or action-research. The form of such projects could range from a designed object or system to a documentary film to a written dissertation with supporting media, depending upon the student’s objectives and the project’s demands.
  • The form of the dissertation in turn informs, and is informed by, the choice of methods, which might range from traditional qualitative methods to iterative and performative approaches to experimental design research.
  • The program emphasizes the careful selection of multimodal methods and means of dissemination, and the applicability of students’ experiences to fields and professions that may extend beyond, but are in meaningful partnership with, the academy.
  • The dissertation project can be designed to prepare the student to practice not only as an academic, but also professionally as an informed high-level actor in the increasingly complex spheres of action that will emerge over the next decades.

The relation of the Praxis PhD to the traditional PhD

  • Students wishing to complete traditional dissertations in the social sciences or in urban policy may pursue a PhD through one of our existing programs in the New School for Social Research and the Milano School of International Affairs, Management, and Urban Policy. The Praxis program will serve students interested in transdisciplinary projects that employ methods, and make use of modes of presentation and dissemination, that are not typically employed in traditional doctoral programs.
  • The Praxis PhD program will emphasize the process of scholarship. As part of working collaboratively, faculty will address methodologies of collaboration and team-management. Participants will consider socialization within and between various fields and professions, as well as the range of knowledge/s and practices desirable in a collaborative community. Students will of course cultivate particular areas of expertise and skill, but will also develop proficiency in various methods of inquiry, forms of production, and modes of dissemination.
  • The Praxis PhD will emulate the commitment of traditional doctoral training to doing work of breadth and depth with a small group of advisors and peers. However, unlike the traditional doctorate, which tends to frame the dissertation as its primary end-goal, the Praxis PhD program will regard the dissertation also as an exploration of the intellectual skills and competencies we want student to develop through their education. The dissertation, and the research process, will allow students to demonstrate both the expertise and understanding gained in a particular field, and the knowledge gained about what it means to be a scholar-practitioner working in a collaborative, interdisciplinary environment…..

109_lambert1-600x781

Curriculum

The process of scholarship will be an emphasis of the program. As part of working collaboratively, we’ll address processes of group-building and methodologies of collaboration and team-management. We’ll consider how we socialize within and between various fields and professions. We will also consider the range of knowledge/s and practices we’d want a collaborative community to have. Students will of course cultivate particular areas of expertise and skill — they will enter the program with Masters degrees, which certify that they’ve cultivated a solid foundation in a particular area of specialization, and they’ll further develop that knowledge through elective cognate courses — but will also develop literacy in various research methods and modes of publicizing or disseminating their work.

There has been much debate over the length of time and the cost that U.S. students invest in completing PhDs. We aim to decrease time-to-degree without sacrificing rigor, by offering more flexible scheduling options, including minimal residency requirements and instructional formats that aren’t constrained by the traditional 15-week course.

The Praxis PhD program will consist of 30 credits, distributed across three years of coursework and supervision. Students are required to be registered full-time, and in residence in New York City, for the first year of study. Years two and three can be completed remotely, but students must return to New York twice each year, for two two-week workshop/conference/exhibitions. The distribution of courses is as follows:

Year One / Semester One

  • Core Studio I (required 3-credit praxis studio): A team of faculty will guide students through a case study requiring interdisciplinary investigation and engagement. In this semester, students will focus on problem definition, design research, and methodology. In the process, students will learn about collaboration strategies, including how to build international partnerships; project and team management, how to “do interdisciplinarity,” etc.
  • Methodologies and Epistemologies (required 3-credit seminar/workshop): This seminar examines how epistemologies and methodologies are mutually informed; offers an overview of methods represented within the humanities, social sciences, design, and fine arts field; and helps students develop strategies for choosing among methods – and particularly choosing from among the methods courses available at The New School – as appropriate for their own dissertation projects, and in other applications and contexts. We will also examine how the myriad media and design modalities – from documentary and data visualization to field recordings and various approaches to prototyping – can function as research tools and techniques. In addition, during this course students will begin to develop their Literature Reviews / Environmental Scans, and their Portfolios/Process Journals (about which more below), which they will maintain throughout their enrollment in the PhD program, and which will be an integral part of the submitted dissertation project.
  • Praxis Symposium (required 3-credit symposium): Each week students will meet with two faculty from across The New School, paired to highlight their common concerns and disparate approaches, or with representatives from various faculty research clusters, labs, or centers, who model the type of interdisciplinary, collaborative work that Praxis students are preparing to do. Faculty will discuss practical and ethical issues germane to their practice, lead students through close readings of texts that are central to their work, and lead workshops on methods that are likewise integral to their practice.
  • Elective: In consultation with an advisor, students can choose from among all graduate courses at The New School. Elective credits might be used to develop a cognate area or to cultivate new design or production skills.

Year One / Semester Two

  • Core Studio II (required 3-credit praxis studio/workshop): A continuation of Core Studio I. In this semester, students and faculty will employ the methods they’ve chosen in Semester I, evaluate the results, and iterate. They’ll also examine the variety of means through which students can disseminate their work – e.g., traditional or experimental publication, video documentaries, interactive platforms, curated exhibitions, designed artifacts and systems. In the process, students also explore how they can continue to uphold rigorous academic standards, including citation and peer review, in these diverse modalities.
  • Three Electives: In consultation with an advisor, students can choose from among all graduate courses at The New School. Elective credits might be used to develop a cognate area or to cultivate new design or production skills.

Years Two/Three

  • Students must enroll in six credits (at no cost?) of Internship, Applied Fieldwork, and/or Research Methods (about which more below). If the Internship/Applied Fieldwork involves intensive research that allows for the development of a strong set of research methods, students need not enroll in separate Research Methods courses. These determinations are to be made on a case-by-case basis in consultation with the student’s advisors.
  • Students are welcome to audit additional Elective courses to further develop their cognates or cultivate their design and production skills. If they remain in New York, they can choose from among all graduate courses offered at The New School or from any local consortium institution. If they are working remotely, students can take a graduate course at a local institution. Or they can design an independent study with a New School advisor.
  • Each semester students must participate in online Professionalization Workshops (about which more below; no credit)

Non-Course Requirements

  • Students entering the program with limited design and/or production experience must participate in a self-directed summer tutorial(?). (How will we level the playing field for students who enter the program with different skill sets? Should this be a concern — or should we simply allow students to learn from one another in the first-year courses, and to fill in their individual gaps via Elective coursework?)
  • During the second semester of their second year, students must submit an approved Literature Review / Mediagraphy / Environmental Scan.
  • During the second semester of their third year(?), students enroll in Dissertation Proposal Supervision (no credit) with their dissertation supervisors. They defend this proposal publicly during the summer Praxis PhD Conference/Exhibition.
  • Each semester beyond successful defense of the dissertation proposal, students must enroll in Dissertation Supervision (no credit) with their dissertation supervisor. The faculty member and student determine how frequently they will meet, either in-person or virtually. The student is also responsible for maintaining his/her Process Journal, about which more below.
  • Each Fall, students must participate in the annual two-week Praxis PhD Workshop (should this have credits attached to it?), on-site in New York.
  • Each Spring/Summer, students must participate in the annual two-week Praxis PhD Conference/Exhibition (should this have credits attached to it?): Here students demonstrate their annual progress and receive feedback from their student colleagues, all Praxis PhD faculty, and a group of external critics, designers, scholars, and other professionals.

Internships / Applied Fieldwork

Students are required to complete at least three credits’, but no more than six credits’, worth of internship(s) or applied fieldwork. The program faculty and External Partnerships Administrator (EPA) will cultivate relationships with various New York-based design and media production companies, cultural institutions, not-for-profits, development organizations, etc. — all potentially eventual employers of graduates of the PhD program — and identify opportunities where our students’ particular areas of expertise and skill sets could be put to good use in addressing real-world problems. Students are also welcome to cultivate their own external partnerships and propose their own internships.

Students must submit, and have approved by both their advisor and the EPA, an extensive proposal, developed in consultation with the external organization, that offers (1) a description of the specific project(s) the students will undertake in partnership with the external organization; (2) a list of the student’s responsibilities and expected time commitment; and (3) a discussion of the cognate knowledge, research methods, design and/or production skills, and modes of dissemination the student will likely cultivate through completion of the internship. Each student’s advisor and the program’s EPA will ensure that the nature of the work is appropriate for doctoral-level students, and that it will support the student’s course of study. If the proposal is approved, external organizations must sign a contract agreeing to the nature and extent of work the intern will undertake.

Students may also propose to undertake self-directed (but faculty advisor-supervised) Applied Fieldwork, potentially involving partnerships with multiple organizations. For instance, a student might choose to investigate the design of urban interfaces for the new Hudson Yards development project in Manhattan; his or her work might involve liaising between various government agencies, design and technology firms, and the city’s data managers. Again, students are required to submit an extensive Applied Fieldwork proposal, which must be approved by the student’s advisor and the program’s EPA.

Research Methods

Students will be able to choose from a variety of variable-credit methodology courses represented within the humanities, social sciences, design, and fine arts fields – ranging from qualitative methods to iterative and performative approaches to experimental design research – based on the nature of their dissertation projects. The New School already offers a suite of such one-, two- and three-credit graduate methods courses, some dedicated to a particular method – interviewing or content analysis, for example – and others dedicated to the integration of a variety of methods for a particular application – e.g., participatory design research, or data gathering and analysis. Praxis PhD students are welcome to take these courses, provided they are offered for graduate credit. The PhD faculty will also develop new courses based on student interest and need; many will be offered online and/or during intensive summer or intersession periods. In addition, a core requirement for the Praxis PhD will be a methodology seminar, to be taken in students’ first semester, that discusses how methodologies and epistemologies are mutually informed, offers an overview of the various methods options available, and addresses strategies for choosing and mixing methods as appropriate for students’ individual projects.

Professionalization Workshops (need a better name!)

While students are working remotely throughout years two and three, the program will host monthly online workshops on various aspects of professionalization – e.g., team-building, grant-seeking, event planning, software for project management, job-seeking. The workshops will allow for asynchronous discussion and occasional synchronous presentations with Q&A.

Dissertation

There is, of course, much in traditional doctoral education that is worth retaining – particularly, the commitment to doing work of breadth and depth with a small group of advisors and peers. However, unlike the traditional doctoral program, which tends to frame the dissertation as its primary end-goal, our Praxis PhD program will regard the dissertation also as an exploration of the intellectual skills and competencies we want student to develop through their doctoral education. The dissertation, and the process leading up to it, will allow students to demonstrate both the expertise and understanding they’ve gained in a particular field or practice, as well as the knowledge they’ve gained about what it means to be a scholar-practitioner, particularly one working in a collaborative, interdisciplinary environment.

To this end, the dissertation must be accompanied by an online Process Journal in which students reflect on and analyze their work process – their choice of methods and modes of representation, their integration of lenses from myriad disciplinary fields and practices, how collaboration and professional work have informed their project – throughout the dissertation development process, and after the dissertation has been successfully defended. Students are encouraged to explore various forms of writing and modes of presentation in this journal.

All dissertations will be hybrid, incorporating both writing and some form of practice or action-research. This work represents, in the words of Clive Dilnot, a “critical reflection on, and analytical translation of, enacted practice into knowledge.” The form of such projects could range from a designed object or system to a documentary film to a written dissertation with supporting media, depending upon the student’s objectives and the project’s demands. The dissertation project can be designed to prepare the student to practice not only as an academic, but also professionally as an informed high-level actor in the increasingly complex spheres of action that will emerge over the next decades. Thus, students are encouraged to partner wit external organizations – design firms, cultural organizations, etc. – in the design and evaluation of their dissertation projects.

Dissertations are intended to address significant real-world or practice-based problems, rather than focus on the students’ own self-defined design and/or creative work. As education journalist Andrew McGettigan writes in regard to the art practice PhD, “a frame has to be constructed that places the practice in relation to a significant problem and thereby escapes the solipsism that might result by beginning from one’s own work…. Research that advances knowledge goes beyond a personal exploration and requires a clear sense of how what is being pursued will be of significance to a broader community of academics and practitioners.”

During the annual Praxis PhD Workshop, in the Fall semester, students in the second year will participate in a Dissertation Proposal Lab. They must submit a dissertation proposal by the end of the following semester, and that proposal will be defended – in front of the dissertation committee and two or three external reviewers – within the two-week period of the annual Praxis PhD Conference/Exhibition. For the third – and, if necessary, fourth – year, students will enroll in Dissertation every fall, during the annual Praxis PhD Workshop and Conference/Exhibition. Students will also present their work-in-progress each spring at the Praxis PhD Conference/Exhibition, where they will receive feedback from their committee and a panel of invited guests.

Categories
Blog

Evaluating Multimodal Work Republished in Journal of Digital Humanities

ScantronTest

My post on strategies and rubrics for evaluating multimodal student projects has been adapted and republished in the Journal of Digital Humanities. This probably isn’t the most riveting thing I’ve ever written, but I hope it’s at least somewhat useful. It has been for me and (I hope!) my students.

This new issue of JDH contains several great articles on evaluating/assessing Digital Humanities work. The editors explain:

Some scholarly societies, universities and colleges, and departments have called for a redefinition — or at least an expansion — of what is considered creditable scholarship. There have been scattered initial attempts to understand how digital scholarship might be better assessed, but the editors of JDH felt, and many of our readers agreed, that there was not a single place to go for a comprehensive overview of proposals, guidelines, and experiences. We attempt to provide a single location here, with an issue and living bibliography that will grow as additional examples are published across the web.

The various posts address the support needed for, and collaboration involved in, DH projects, as well as the need to give credit to a wider range of collaborators and contributors; and several focus on processes of peer review and standards for tenure and promotion, including guidelines put forward by several professional organizations. A few, mine included, talk about evaluating student work.

Categories
Presentations

THATCamp Theory Presentation

This past weekend I led a workshop at THATCamp Theory, at Rutgers, on evaluating and critiquing multimodal projects.I must admit, my talk was kind-of a mash-up of two older projects: my CUNY DHI talk from last October (video here) and this post. Above are my slides, and below are my notes.

I unfortunately was able to attend only Day One of the two-day conference (Rory said Day Two was quite the brain-bender). Yet I thoroughly enjoyed the sessions I was able to take part in!

EVALUATION / CRITIQUE OF DH PROJECTS

This workshop will focus on developing a critical vocabulary for responding to DH and systems for providing meaningful evaluative feedback, including 1) developing critical evaluative criteria for various formats of multimodal work and 2) identifying theoretical frameworks that inform those criteria. We’ll consider both professional and student projects and spend some time considering how to make project evaluation an integral part of the DH classroom. Depending on the interests of the group, our case studies might include data visualizations, map-based projects, crowdsourced archival projects, and other interactive publications.

  • Recognize that there’s a history of considering “multimodal evaluation” in composition

[SLIDE 2] I’m not fully ensconced in the DH community – sympathetic to their interest in different forms, practices, praxes, of scholarship.

  • Craft as a useful model for considering how similar intellectual values and practices span domains – reading, writing, making in various modalities
  • But not all making is scholarship

[SLIDES 3-4] McPherson article: Multimodal Humanist – this term, still a mouthful, resonated more with me

[SLIDE 5] Scrivener on when production is research

[SLIDE 6] Question about Feedback & Evaluation — not simply so I could assign a grade, but so we could provide meaningful feedback

  • Work – particularly technical skills – were sometimes outside my area of expertise
  • How to balance weighting of form and content – “rigor” in concept or execution?
  • Individual vs. Group Accountability

[SLIDE 7] Revisited the list of criteria two years later

[SLIDES 8-10] Fall 2010 / 2011 / 2012 : Urban Media Archaeology

  • [SLIDE 11] Semester Schedule – discuss theories representing each unit
  • [SLIDE 12] PROJECT PROPOSALS – not different from trendy “contracts”
    • Justify choice of “genre” and format – use of media tools as method
  • [SLIDES 13-14] Student Proposed Projects
    • Carrier pigeons, electrification of lower Manhattan, video game arcades, newspaper company headquarters, “media actors” in Atlantic Yards using actor-network theory, etc.
    • I provide individual feedback; students post to blogs and classmates comment
    • This semester’s class hasn’t yet posted their proposals online
  • [SLIDE 15] Learn Data Modeling (interface now looks a bit different)
  • [SLIDE 16] User Scenarios
  • [SLIDE 17] Look inside Black Box – Software Development
  • [SLIDE 18] Pecha Kucha
    • DH projects inherently collaborative – need experts from multiple fields
  • [SLIDE 19] All the while, we’re collectively developing criteria for evaluation:
    • [SLIDE 20] By working in small groups and as a class to evaluate other “multimodal projects” + Hypercities
    • [SLIDE 21] Through individual map critiques
    • Thru Peer Review of one another’s projects
  • [SLIDE 22] Process Blogs – Self-Evaluation
    • Make public their process
      • [SLIDE 23] Discuss work w/ other public/cultural institutions – e.g., archives
    • [SLIDES 24-26] Practice “critical self-consciousness” – about their work processes, choice of methods, media formats, etc.
    • Hold themselves accountable for their choices
  • [SLIDE 27] Peer Evaluation: Paper Prototypes
  • Final Presentation: [SLIDE 28] My Feedback + [SLIDE 29] Students’ Peer Reviews

[SLIDE 30] Where was theory throughout?

  • Underlying the entire project, informing their understanding of the way cities work, informing their understanding of how maps work as media, informing how they design their data models, which are in shaped by how they want their projects to look for users – thus, theories about the visualization of data mix in with their theories about how databases work
  • And in order for students to know how we were going to evaluate success, these theories had to be made an integral part of our development process

[SLIDE 31] Through critique, we’ll reverse-engineer student and professional projects and find the theory that informed it

  • [SLIDE 32] From my list of evaluative criteria – Concept + Content; Concept-/Content-Driven Design + Technique; Documentation and Transparent, Collaborative Development; Academic Integrity and Openness; Review and Critique – are backed by theories: theories central to the project’s content, theories of design, theories of knowledge production, theories of labor, etc.
  • [CLICK] But we’ll focus on the few dimensions that are overtly theoretical, and that we can potentially discern in a quick review, in the short time we have here
  • [SLIDE 33] Break up into groups and assess the Concept + Content and Concept-Content-Driven Design + Technique of a few sample DH project and reverse-engineer that theories that might’ve informed their creation

[SLIDE 34] Case Studies:

  • These are the cases we choose from in my UMA class.
  • Solicit ideas for classes of projects to critique (e.g., data visualizations, map-based projects, crowd-sourced archival project, interactive publications)
  • Solicit ideas for specific projects groups can collaborative assess

Examples:

 

 

 

Categories
Publications

Evaluating Multimodal Work, Revisited

Evaluating Multimodal Work, Revisited,” Journal of Digital Humanities 1:4 (Fall 2012).

Categories
Blog

Evaluating Multimodal Work, Revisited

Cat Sidh on Flickr 

Two years ago I was preparing for a semester in which all of my classes involved “multimodal” student work — that is, theoretically-informed, research-based work that resulted in something other than a traditional paper. For years I’d been giving students in my classes the option of submitting, for at least one of their semester assignments, a media production or creative project (accompanied by a support paper in which they addressed how their work functioned as “scholarship”) — but given that that this cross-platform work would now become the norm, I thought I should take some time to think about how to fairly and helpfully evaluate these projects. How do we know what’s good?

So I wrote up a little pseudo-literature review, “Evaluating Multimodal Student Work,” which, I was glad to see, quite a few folks seemed to find helpful. Since I’ve been explicitly discussing the process and politics of evaluation with my students for a couple years now — and because I was invited to lead a workshop on Evaluation & Critique of DH Projects at the upcoming THATCamp Theory at Rutgers in October — I thought I should revisit the issue.

I’ll try to clean up that post from two years ago and add some insights I’ve gleaned from other sources since then, including the collection of essays on “Evaluating Digital Scholarship” that came out in the MLA’s Profession late last year. In recent years the MLA and other professional organizations have made statements and produced guides regarding how “digital scholarship” should be assessed in faculty (re)appointment and review — and these statements are indeed valuable resources — but I’m more interested in here in how to evaluate student work.

[UPDATE 11/12: A few months after posting this, Cheryl Ball wrote to let me know me that she’s written two fabulous, and highly relevant, articles about multimodal assessment: “Assessing Scholarly Multimedia: A Rhetorical Genre Studies ApproachTechnical Communication Quarterly, 21:1 (2012): 1-17 and “Adapting Editorial Peer Review for Classroom UseWriting & Pedagogy (Forthcoming 2013). These aren’t journals I’d typically read, so I’m grateful to Cheryl for bringing these articles to my attention!]

Alright, then….

*   *   *   *   *

A different take on multimedia evaluation. Television-set testing at Underwriters Labs. via

In most of my classes we spend a good deal of time examining projects similar to those we’re creating — other online exhibitions, data visualizations, mapping projects, etc., both those created by fellow students and “aspirational” professional projects that we could never hope to achieve over the course of a semester – and assessing their strengths and weaknesses. Exposing students to a variety of “multimedia genres” helps them to see that virtually any mode of production can be scholarly if produced via a scholarly process (we could certainly debate what that means), and can be subjected to critical evaluation.

Steve Anderson’s “Regeneration: Multimedia Genres and Emerging Scholarship” acknowledges the various genres — and “voices” and “registers” and “modes” of presentation – that can be made into multimedia scholarship. Particularly helpful, I think, is his acknowledgment that narrative — and, I would add, personal expression — can have a place in scholarship. Some students, I imagine, might have a hard time seeing how the same technologies they use to watch entertainment media, the same crowd-sourced maps they use to rate their favorite vegan bakeries or upload hazy Instagrams from their urban derives – the same platforms they’re frequently told to use to “express themselves” – can be used as platforms for research and theorization. Personal expression and storytelling can still pay a role in these multimodal research projects, but one in service of a larger goal; as Anderson says, “narrative may productively serve as an element of a scholarly multimedia project but should not serve as an end in itself.”

The class as a whole, with the instructor’s guidance, can evaluate a selection of existing multimodal scholarly projects and generate a list of critical criteria before students attempt their own critiques – perhaps first in small groups, then individually. Asking the students to write and/or present formal “reader’s reports” – or, in my classes, exhibition or map critiques – and equipping them with a vocabulary tends to push their evaluation beyond the “I like it” / “I don’t like it” /“There’s too much going on” / “I didn’t get it” territory. The fact that users’ evaluations frequently reside within this superficial “I (don’t) like it” domain is not necessarily due to any lack of serious engagement or interest on their part, but may be attributable to the fact that they (faculty included!) don’t always know what criteria should be informing their judgment, or what language is typically used in or is appropriate for such a review.

Once students have applied a set of evaluative criteria to a wide selection of existing projects, they can eventually apply those same criteria to their own work, and to their peers’. (Cheryl Ball has designed a great “peer review” exercise for her undergraduate “Multimodal Composition” class.)

EVALUATIVE CRITERIA

After reviewing a great deal of existing literature and assessment models – all of which, despite significant overlap, have their own distinctive vocabularies – I thought it best to consolidate all those models and test them against our on-the-ground experience in the classroom over the past several years, to develop a single, manageable list of evaluative criteria.

Steve Anderson and Tara McPherson remind us of the importance of exercising flexibility in applying these criteria in our evaluation of “multimedia scholarship.” What follows should not be regarded as a checklist. Not all these criteria are appropriate for all projects, and there are good reasons some projects might choose to go against the grain. Referring to the MLA’s suggestion that projects be judged based on how they “link to other projects,” for instance, Anderson and McPherson note that linking may be a central goal for some projects, but, “linking itself should not be an inflexible standard for how multimedia scholarship gets evaluated.” Nor should the use of “open standards,” like open-source platforms – which, while generally desirable, isn’t always possible (142).

The following is a mash-up up these sources, with some of my own insight mixed in: Steve Anderson & Tara McPherson, “Engaging Digital Scholarship: Thoughts on Evaluating Multimedia Scholarship” Profession (20111): 136-151; Fred Gibbs, “Critical Discourse in the Digital HumanitiesFredGibbss.net (4 November 2011); Institute for Multimedia Literacy, “IML Project Parameters,” USC School of Cinematic Arts: IML (29 June 2009); Virginia Kuhn, “The Components of Scholarly MultimediaKairos 12:3 (Summer 2008); MLA, “Short Guide to Evaluation of Digital Work, “ wiki.mla.org (last updated 6 July 2010).

CONCEPT & CONTENT

  • Is there a strong thesis or argument at the core of this project? Does the project clearly articulate, or some way make “experiential,” this conceptual “core”? Is this conceptual core effectively developed through the various segments or dimensions of the project?
  • “Does the project display evidence of substantial research and thoughtful engagement with its subject?” (IML) Does it effectively “triangulate” a variety of sources and make use of a variety of media formats?
  • Is the platform merely a substrate for a “cool data set” or a set of media objects – or are individual “pieces” of content – data and media in various formats, etc. – contextualized”? Are they linked together into a compelling argument?
  • Is the data sufficiently “enriched”? (MLA) Is it annotated, linked, cited, supplemented with support media, etc., where appropriate?
  • Does the project exploit the “repurpose-ability” of data? Does it pull in, and effectively re-contextualize, data from other projects? (Students should also recognize that their own data can, and should, be similarly repurposed.) This recognition that individual records – a photo or video a student uploads, or a data-set they import, etc. – can serve different purposes in different projects offers students great insight into research methodology, into the politics of research, into questions regarding who gets to make knowledge, etc. As Fred Gibbs acknowledges, discussing how a project uses data also “encourages conversations about ownership [and] copyright.”

CONCEPT/CONTENT-DRIVEN DESIGN & TECHNIQUE

  • Does the project’s form suit its concept and content? “Do structural and formal elements of the project reinforce the conceptual core in a productive way?” (IML)
  • Is the delivery system robust? Do the chosen platforms or modes of delivery “fit” and “do justice to” the subject matter? Need this have been a multimedia project, or could it just as easily have been executed on paper?
  • Does the project “exhibit an understanding of the affordances of the tools used,” and does it exploit those affordances as best possible – and perhaps acknowledge and creatively “work around” known limitations?
  • Is there a “graceful balance of familiar scholarly gestures and multimedia expression which mobilizes the scholarship in new ways?” (IML) A balance of the old and familiar, to help users feel that they can rely on their tried-and-true codes of consumption; and the new, to encourage engagement and promote reconsideration of our tradition ways of knowing?
  • At the same time, do the project creators seem to exercise control over their technology? Or does technology seem to be used gratuitously or haphazardly? “Are design decisions deliberate and controlled?… Does the project “demonstrate[e] authorial intention by providing the user with a carefully planned structure, often made manifest through a navigation scheme?” (IML)
  • Do the project creators seem to understand their potential users, and have they designed the project so it accommodates those various audiences and uses?
  • How does the interface function “rhetorically” in the project? Does it inform user experience in a way that supports the project’s conceptual core and argument? Does it effectively organize the “real estate” of the screen to acknowledge and put into logical relationships the key components – subject content, technical tools, etc. – of the project?
  • Has the project been tested? Are their plans for continual testing and iterative development? Is the project adaptable?
Perhaps in some utopian future, when cognitive science is integrated into *all* disciplines, we can use brain scans as a form of assessment. Just kidding! Photo: Simon Fraser/Science Photo Library via Guardian.uk

DOCUMENTATION AND TRANSPARENT, COLLABORATIVE DEVELOPMENT

  • Do project creators practice self-reflexivity? Do they “accoun[]
    for the authorial understanding of the production choices made in constructing the project?” (IML)
  • Do project creators document their research and creative processes, and describe how those processes contributed to their projects’ “formal structure and thematic concerns?” (IML) McPherson and Anderson (2011) also emphasize the importance of “finely grained accounts of the processes involved in the production of multimedia scholarship in order to evaluate properly the labor required in such research” (142).
  • Do project developers document and/or otherwise communicate their process – perhaps through a “ticket” system like Trac– and make it transparent and understandable to students?
    • As Rory Solomon pointed out in a comment on my earlier blog post, the adoption of open standards also contributes to longevity: “…the open source movement provides means to help minimize these concerns in that open source projects provide many ways to evaluate a given software tool / format / platform. Any serious project will have an open, public web presence, including developer and user mailing lists, documentation, and etc. It is fairly easy then to evaluate the depth and breadth of the developer and user communities. It is useful to check, via wikipedia and other open source project websites, whether there are competing initiatives, whether the project is getting support from one of the larger foundations (eg, FSF, Apache, etc), and if there is competition then what trends there are in terms of which tools seem to be “winning out”. Once a critical mass is reached and/or once a certain level of standardization has been achieved (through things like IETF, ISO, RFC’s, etc), one can be fairly confident that a tool will be around for a very long time (eg, no one questions the particular voltage and amp levels coming out of our wall sockets) and even if a tool does become obsolete, there will be many users and developers also contending with this issue, and many well-defined and well-publicized “migration paths” to ensure continued functioning, accessibility, etc.”
  • Are students involved in the platform’s development? Does this dialogue present an opportunity for students to learn about the process of technological development, to see “inside the black box” of their technical tools, to develop a skill set and critical vocabulary that will aid them not only in their own projects, but in the collaborative process?
    • Students should be asked for feedback on technical design; this conversation needs to happen as part of a structured dialogue, so it’s made clear to students what would be required to implement their requests — and whether or not such implementation is even feasible. Students should also be encouraged to translate their technical snafus — bugs, error messages, etc. — into opportunities to learn about how technology functions, about its limits, and about how to fix it when it’s not cooperating. Ideally, students should have a sense of ownership over not only their own projects, but also the platform on which they’re built.
    • I wrote about some of these frustrations-turned-into-positive learning-experiences in regard to my Fall 2011 Urban Media Archaeology class. Besides, these hiccups — and yes, on occasion, outright disasters — are an inevitable part of any technological development process. The error-laden development process defines every project out in the “real world”; why should a technological development project taking place within the context of an academic class be artificially “smoothed out” for students, artificially error-free?

ACADEMIC INTEGRITY & OPENNNESS

  • Does the project evidence sound scholarship, which upholds all the traditional codes of academic integrity?
  • Does it credit sources where appropriate, and, if possible, link out to those sources? Does it acknowledge precedents and sources of conceptual or technical inspiration?
    • For my classes, I’ve made special arrangements with several institutions for copyright clearances and waiver of reproduction fees. In other cases, students will have to negotiate (with the libraries’ and my assistance) copyright clearances; this is a good experience for them!
  • Does the project include credits for all collaborators, including even those performing roles that might not traditionally be credited?
  • “Is it accessible to the community of study?” (MLA) Is the final “product” available and functional for all its intended users – and open enough to accommodate even unexpected audiences? Is the process sufficiently well documented to make the intention behind and creation of the project accessible and intelligible to its publics?
    • Telling students that their work will be publicly accessible, and that it could have potential resonance in the greater world, can be a great motivator. Of course some students might feel vulnerable about trying out new ideas and skills in public view — and teachers should consider whether certain development stages should take place in a secure, off-line area.
  • “Do others link to it? Does it link out well? (MLA) Does the project make clear its web of influence and associations, and demonstrate generosity in giving credit where it’s due?
    • Emphasizing proper citations – of data, archival work, even human resources that have contributed to the project – reinforces the fact that academic integrity matters even within the context of a nontraditional research project, and it allows both the students and the collaborating institutions and institutions to benefit from their affiliation – e.g., the archives can show that researchers are using their material, and the students can take pride in being associated with these external organizations.

REVIEW & CRITIQUE

  • “Have there been any expert consultations? Has this been shown to others for expert opinion?” (MLA) Given the myriad domains of expertise that most multimodal projects draw upon, those “experts” might be of many varieties: experts in the subject matter, experts in graphic design, experts in motion graphics, experts in user experience, experts in database design, etc.
  • “Has the work been reviewed? Can it be submitted for peer review?… Has the work been presented at conferences?… Have papers or reports about the project been published?” (MLA)  Writing up the work for publication or presentation at conferences elicits feedback. Grant-seeking also gives one an opportunity to subject the project to critique. There are also a few publications focusing on multimodal work — e.g., Vectors, Kairos, Sensate— that are developing their own evaluative criteria.
    • Individual students in my classes have presented their own projects at conferences, submitted them to multimodal journals, or written about their multimodal work for more traditional journals. More informal, though no less helpful, forms of “peer review” can take place in the classroom — through design critiques with external “experts,” student peer-review, etc.
Categories
Blog

IYLSSIF 5: What Media Studies Makes: Forms of Scholarship

The fifth in an epic, six-part series of lectures from my intro to graduate studies lecture course, which I’m posting online in the hope that others will find them useful. [Part 1 Here, Part 2 Here, Part 3 Here, Part 4 Here; the lectures are unedited — hence, you might be a bit confused by a few inexplicable notes and slides about administrative issues]. We started off by describing the premise of the class; then discussed how students could find their own position within the program and the field; then helped students map that field, appreciate its breadth and the various intellectual and create traditions it draws from; then talked about practical methods for maintaining one’s orientation within the field and within one’s own work. Now we talk about the various forms one’s scholarship can take, ranging from traditional academic writing to more experimental writing forms, to “multimodal” scholarship and theoretically informed, research-based media production.

UMS5_Oct3

[SLIDE 1] LOCATION OF SIGN-IN SHEETS!

[SLIDE 2]

  • Academic Sources?
  • Next Week: begin w/ visit from GPIA re: IFP – Deadline: 10/15

[SLIDE 3] WHAT MEDIA STUDIES MAKES: FORMS OF SCHOLARSHIP

In our field, media studies, we take a cross-platform, comparative approach to studying various modes of communication. This comparative approach characterizes not only our subjects of study, but also our methods and our means of presenting the outcomes of our work.

[SLIDE 4] Just as, last week, we talked about different platforms and software for taking notes, organizing projects, etc., we also have to think about what technologies can serve us as research tools – as methods – and what can help us present our work in the most effective way possible. That’s in part what multimodal scholarship is about: thinking about how different media might allow you ask new research questions, engage your subject in new ways, and share your in-progress or finished work in ways that “do justice” to your subject and your argument, that give appropriate form to your content.

As your reading for this week suggested, media studies makes scholarship in traditional written forms and in “multimodal” forms. Film, field recordings, databases, etc. can all function as research tools and as platforms for presenting our research-based, theoretically-informed work. Or course there’s still room for using these media as creative forms – as means of pure artistic expression – but today we’re going to focus on how these technologies might shape the forms of our research and theorization.

[SLIDE 5] Guest: Amir Husak

[SLIDE 6] DIGITAL HUMANITIES

McPherson

  • Researchers are more than “content providers” – they “fully engage with the platforms and tools of the digital era” (120)
    • Computing humanist
    • Blogging humanist
  • “Who better to reimagine the relationship of scholarly form to content than those who have devoted their careers to studying narrative structure, representation and meaning, or the aesthetics of visuality (and aurality)?” (120)
  • [SLIDE 7] Remember Henry Jenkins from our 2nd Lecture, on History of the Field: “New media literacies include the traditional literacy that evolved with print culture as well as the newer forms of literacy within mass and digital media…. [We] must expand [our] required competencies, not push aside old skills to make room for the new.

Beyond core literacy, students need research skills…. Students also need to develop technical skills…. Yet, to reduce the new media literacies to technical skills would be a mistake on the order of confusing penmanship with composition….

  • [SLIDE 8] “The multimodal humanist…brings together databases, scholarly tools, networked writing, and peer-to-peer commentary while also leveraging the potential of visual and aural (and interactive) media that so dominate contemporary life… She aims to produce work that reconfigures the relationships among author, reader, and technology while investigating the computer simultaneously as a platform, a medium, and a visualization device. She thinks carefully about the relationship of form to content, expression to idea” (120)
  • [SLIDE 9]  CommentPress
  • [SLIDE 10] “The multimodal scholar explores new forms of literacy that include authoring and analyzing visual, aural, dynamic, and interactive media….[and imagines] what it would be like to immerse yourself in a scholarly argumentas you might immerse yourself in a movie or a video game. She investigates what happens when scholarship looks and feels differently, requiring new modes of engagement from the reader/user” (120)
    • “’How do you ‘experience’ or feel’ an argument in a more immersive and sensory-rich space?’ ‘Can scholarship show as well as tell?’ ‘Will representing data differently change the ways we understand, collect, or interpret it?’ ‘What happens to argument in a nonlinear environment?’” (121) – AFFECTIVE dimension

     

  • [SLIDE 11] “…hands-on engagement with digital forms reorients the scholarly imagination, not because the tools are cool or new (even if they are) or because the audience for our work might be expanded (even if it is), but because scholars come to realize that they understand their arguments and their objects of study differently, even better, when they approach them through multiple modalities and emergent and interconnected forms of literacy. The ability to deploy new experiential, emotional and even tactile aspects of argument and expression can open up fresh avenues of inquiry and research” (121)
  • [SLIDE 12]Book vs. Database:
    • Book calls for linear organization
    • Database allows for tangents – allows us to “present multiple lines of thought in relation to the materials at hand and to invite others to join us in this process in extended collaboration and convention. Working with databases allows us both to present our arguments differently and to understand our materials differently. Thus, the database might itself be understood das an interpretive platformthat can support and extend the core methodologies of the interpretive humanities…” (121)
    • [SLIDE 13] Wunderkammer: even though it’s organized like a book, it allows for tangents, links etc.: http://www.technorhetoric.net/13.2/topoi/delagrange
    • [SLIDE 14] Thinking/Making
  • [SLIDE 15: Korsakow] New forms of argumentation: “multiple, associative, digressive, even contradictory” (122)
  • “navigating new pathways through scholarly materials that can transform the questions scholarship might ask” (122)
    • [SLIDE 16] John Snow’s 1854 Cholera Map of London

These claims are not unique to the database!

  • [SLIDE 17: Audio/Video] “…imagine very different scholarly ‘outputs’at the surface of the screen – we might create powerful simulations, visualize space and time in compelling ways, or structure data that the user can then play like a video game, richly annotate on the fly, or capture and represent in interesting new ways” (122)
    • [SLIDE 18] Mark Kann’s “Deliberative Democracy and Difference” on Vectors: http://www.vectorsjournal.org/projects/index.php?project=81
  • STILL NEED A METHOD – Topic for next week!

Digital Humanities Manifesto 2.0

  • [SLIDE 19] Process (research AND design!) over product! Collaboration, Interdisciplinarity, distributed networks of knowledge production
  • McPherson: “imperative that we be involved in the design and construction of the emerging networked platforms and practices” – design our own tools (123)
    • [SLIDE 20] Manifesto: “…not only to seek to understand and interrogate the cultural and social impact of new technologies, but to be engaged in driving the creation of new technologies, methodologies, and information systems, as well as in their détournement (turnabout, derailment), reinvention, repurposing, via research questions grounded in the Arts and Humanities: questions of meaning, interpretation, history, subjectivity, and culture” (6)
    • [SLIDE 21]: URT + Mapping the Social Life of Zines
  • “determining and designing the interface to information, data, and knowledge becomes just as central as the crafts of writing, curating, and coordinating” (7)
    • [SLIDE 22: Dr. Strangelove] Faden on Media Stylos: “In 1998, on the 50th anniversary of French critic and filmmaker Alexandre Astruc’s inspiring essay “The Birth of a New Avant-Garde: Le Caméra Stylo,” I began making short films and videos in lieu of academic conference papers. Astruc’s essay called for a new film practice that moved beyond both avant-garde abstraction and narrative story telling and embraced a full range of intellectual practicesfrom filming philosophy to emulating the 17th century literary essay.”
      • Would not advocate misrepresenting your presentation in a conference or workshop proposal – or aiming to alienate your fellow panelists
      • Still, Faden has taken cues from Astruc in thinking about film- or video-making as a means of thinking through theory, or creating media that critiques itself or its own process of construction
      • Mix of media formats and rhetorical modes
      • Allows for consideration of form in relation to content– format of argumentation parallels format of its subject
        • [SLIDE 23] Issues of FAIR USE
        • [SLIDE 24] SCMS Fair Use
  • [SLIDE 25] Evaluating Multimodal Work
  • [SLIDE 26] CUNY DHI
  • [SLIDE 27] Communities of Digital Humanities-inspired graduate students: HASTAC: http://www.hastac.org/scholars

WRITING

IMPORTANCE OF WRITING – EVEN IN PRODUCTION WORK

  • Production faculty claim that the biggest problems with students production projects are (1) problematic conceptualization, which is related to weak writing and (2) sound design
  • [SLIDE 28: Book of Hours, 1460] Grad students tend to write in a way that constitutes what they think is “academic” writing
    • Gerald Graff, education historian, author of “Scholars and Sound Bites” in PLMA: “When students write ponderously and evasively, it is often not because they could not do otherwise, but because they are convinced that such writing is what their professors want” (1041)
    • Becker, sociologist well known for his work on “art worlds,” also addresses the compulsion grad students feel to “sound academic,” to put on a particular “writerly persona” – advocates for simplicity

[SLIDE 29] Toor on Orwell:

  • “A scrupulous writer, in every sentence that he writes, will ask himself at least four questions, thus: What am I trying to say? What words will express it? What image or idiom will make it clearer? Is this image fresh enough to have an effect? And he will probably ask himself two more: Could I put it more shortly? Have I said anything that is avoidably ugly?”
  • Toor: 83 comments!

[SLIDE 30-31] Graff’s Tips:

  • “Be dialogical. Begin your text by directly identifying the prior conversation or debate that you are entering” (1050)
  • “Make a claim, the sooner the better, and flag it for the reader”
  • “Remind readers of your claim periodically, especially the more you complicate it”
  • “Summarize the objections that you anticipate can be made (or have been made) against your claim.” (1050)
  • “Say explicitly – or at least imply – why your ideas are important, what difference it makes to the world if you are right or wrong, and so forth” – “So what?” (1051)
  • “Generate a metatext that stands apart from your main text and puts it in perspective” – “I do not mean to suggest that…” “Here you will probably object that…” (1051)
  • “you are probably so eager to prove that you’ve left no thought unconsidered that you find it hard to resist the temptation to say everything at once, and consequence you say nothing that is understood while producing horribly overloaded paragraphs and sentences” (1051)
  • “Be bilingual. It is not necessary to avoid academese – you sometimes need the stuff… [Bu]
    try to say it in the vernacular as well” (1051)
  • “If you could not explain it to your [friends], the chances are you don’t understand it yourself” (1051)

[SLIDE 32] Most “tips” focus on the style, and take for granted the “how” – how to start a paper when you’re not given a particular assignment, a paper that might be longer than those you’ve written in the past, a paper that serves a different purpose than those you’ve written as an undergrad or for work…

DIFFERENT WRITERS HAVE DIFFERENT TIPS FOR GRAD-LEVEL WRITING

[SLIDE 33] Moxley, Publish, Don’t Perish:

  • Audience Analysis
  • Purpose Analysis: reporting, critiquing, objecting, investigating, persuading?
  • Voice Analysis
    • Becker on Persona & Authority: speaking in imperatives, passive voice, etc
  • Process Analysis
    • “What are your writing rituals? What is the best time of day for you to write? Where do you like to write?” (Moxley 39)
    • Becker: “you have already made many choices when you sit down to write, but probably don’t know what they were” (17)

[SLIDE 34] DRAFTING MODELS

  • “…some academics believe that they are violating the rules when they write without an outline. Or, more sadly, when they cannot come up with an outline, some academicians fear that their idea is weak and insignificant, that they lack the critical thinking skills necessary to write well. In fact, recommending that one always outline before writing is based on the foolish assumption that thinking and writing are not related, that first one thinks and then one writes” (Moxley 27)
    • Becker: You needn’t work out everything before starting to write; “Writing can…shape your research design” (Moxley 18)
  • [CLICK]Freewrite Drafts: write without hesitation – “try to ignore critical thoughts and focus on generating ideas” (29)
    • Helps you “(1) develop ideas that you otherwise would not develop, (2) overcome the tightness and frustration associated with beginning new writing projects, and (3) create a flow that helps establish a voice in your prose” (29)
    • “…when you let your thoughts about the research flow, they often gain a forcefulness, a sense of directness and insight, that they otherwise might lack” (29)
    • “When reviewing your freewrites, identify the details that seem most significant. Put brackets around the sections that you believe are worth keeping.” (29)
    • Zinsser on Style
  • [CLICK]Dictate Drafts
    • “sometimes dictated drafts have a strong, natural voice” (31)
    • “can speak faster than you can write” (31)
  • [CLICK] Draw a Cluster Diagram
    • “Rather than trying to force your ideas into a formal outline, you can pictorially represent them on the page and then draw lines between ideas that seem somewhat related” (31)
    • Remember discussion on CONCEPT MAPPING
  • [CLICK] Draw a Pie Diagram
    • “…allows you to estimate visually how much tie you should spend addressing each aspect of your / subject” (31, 33)
  • [CLICK] Make a Formal Outline

[SLIDE 35] FORMATS

  • Intro / Methods & Materials / Results / Discussion / Conclusion
  • Intro / Subheaded Sections / Conclusion – with Transitions!

Gregory Colon Semenza, Graduate Study for the 21st Century:

[SLIDE 36] TYPES OF PAPERS

  1. The Controversy Paper: “claim that purports to end a controversy or debate” (93)
  2. The Textual Crux Paper: “for years readers have pondered the meaning of an ambiguous, unclear, or even a missing part of a given text…Your research leads you to a strong conclusion about the meaning of the problematic text or term..” (93)
  3. The Gap in Scholarship Paper: “in reading the scholarship about a particular subject, you are struck that no one has said anything about a related and seemingly important matter. You decide to widen the scope of the conversation” (93)
  4. The Historical Contextualization: “clarify the meaning of a particular work or explain its provenance, immediate reception, of influence on other contemporary texts” (94)
  5. The Pragmatic Proposal: “more interested in praxes than theory for its own sake” (94)
  6. The Theoretical Application
  • The “so what?” question + Situating your argument

MORE INFORMAL DH-INSPIRED FORMS OF ACADEMIC WRITING – Work through some ideas before formal publication

[SLIDE 39] Revision

  • Zinsser’s revised m.s.: pp. 10-11
  • Becker on necessity for writing multiple drafts – “writing need not be a one-shot, all-or-nothing venture. It could have stages, each with its own criteria of excellence” (14)
  • [SLIDE 40] Use of Writing Center

[SLIDE 41] Giving Credit Where It’s Due

  • Plagiarism – Don’t do it; honor the collective thought and creation that inspired your own thoughts – just as you would have others do to your own work
  • [SLIDE 42] Crediting Photos
  • Make sure you’re familiar with what constitutes academic honesty and dishonesty
    • “Piece-mealing” an argument is plagiarism
  • Style Guides

[SLIDE 43] MULTIPLE WAYS TO MAKE SCHOLARSHIP IN MEDIA STUDIES

  • Each has affordances and limitations
  • Choice should be guided by what “tool” is right for the job (must cultivate “material consciousness”) – We’ll talk more next week about methods.
  • In this week’s discussion section, you’ll look at various platforms for multimodal scholarship.
Categories
Presentations

Beyond the Seminar Paper

What follows is the short presentation I delivered tonight as part of the CUNY Digital Humanities Initiative’s fall series. Mark Sample and I talked about “DH in the Classroom.” [Edit: 10/19] Mark has posted the slides and text for his fantastic talk, in which he explored the relationships between “building” and “thinking” — and made me question how I’d translate much of what I do in the classroom into an undergraduate context. The Q&A that followed our talks was among the most spirited and thoughtful of any I’ve had the pleasure of participating in. Thanks to Mark and Matt and Charlie for making the evening possible!

Mattern_CUNYDHI

[SLIDE 2]History of practice-based teaching: e.g., Practices of Placemaking at Penn

[SLIDE 3] Came to TNS in 2004 – program that combined theory and practice

  • [SLIDE 4] Theory and Practice often bifurcated – separated into two separate halves of the curriculum
  • Students’ instrumentalist conception of this integration: knowing a little theory will make you a better practitioner, knowing how machines work will make you a better theorist
  • I started to allow students to complete theoretically-informed, research based creative projects in lieu of a mid-term or final paper in my seminar classes
  • Foray into project-based classes
    • [SLIDE 5] 2005: Sound & Space
    • [SLIDE 6] 2005: MSPS
    • [SLIDE 7] Larger 2005-6 MSPS project
    • [SLIDES 8-9] 2006: Media Exhibition Design
    • [SLIDE 10] 2006: Immediacy

[SLIDE 11] Praxis-based Courses: Degree of investment rarely witnessed in traditional seminar courses

  • Students had to not only grapple with, but internalize the course content – the theory – because they’d be held accountable for putting it into action.
  • Did a lot of thinking – especially for the intro-to-grad-studies lecture course I teach – about theories of praxis and theories of craft
    • Questions of STANDARDS

About this time I became aware of HASTAC & MacArthur & DH – not sure I completely identified, for reasons explained in my “DH: The Name That Does No Favors” Post – but I’m sympathetic to a number of values that seem central to the community.

[SLIDES 12-13] McPherson article: Multimodal Humanist – this term, still a mouthful, resonated more with me
[SLIDE 14]
Scrivener on when production is research
[SLIDE 15]
Question about Feedback & Evaluation — not simply so I could assign a grade, but so we could provide meaningful feedback

  • Work – particularly technical skills – were sometimes outside my area of expertise
  • How to balance weighting of form and content – “rigor” in concept or execution?
  • Individual vs. Group Accountability

FORGED AHEAD AND TAUGHT SEVEARL PRAXIS-BASED COURSES:

[SLIDE 16] Fall 2010: Media & Materiality
[SLIDE 17]
Semester Schedule
[SLIDE 18]
Student Projects – Can look during conversation period

[SLIDE 19] Spring 2011: Libraries, Archives & Databases – touches on many DH themes

[SLIDES 20-21] Fall 2010 / 2011: Urban Media Archaeology

  • [SLIDE 22] Semester Schedule
  • [SLIDE 23] PROJECT PROPOSALS – not different from trendy “contracts”
    • Justify choice of “genre” and format – use of media tools as method
  • [SLIDES 24-25] Student Proposed Projects
    • I provide individual feedback; students post to blogs and classmates comment
  • [SLIDE 26] Learn Data Modeling
  • [SLIDE 27] User Scenarios
  • [SLIDE 28] Look inside Black Box – Software Development
  • [SLIDE 29] Pecha Kucha
    • DH projects inherently collaborative – need experts from multiple fields
  • [SLIDE 30] All the while, we’re collectively developing criteria for evaluation:
    • [SLIDE 31] By working in small groups and as a class to evaluate other “multimodal projects” + Hypercities
    • [SLIDE 32] Through individual map critiques
    • Thru Peer Review of one another’s projects
  • [SLIDE 33] Process Blogs – Self-Evaluation
    • Make public their process
      • [SLIDE 34] Discuss work w/ other public/cultural institutions – e.g., archives.
      • This semester, students are working w/ youth media centers, independent bookstores, etc.
  • [SLIDES 35-37] Practice “critical self-consciousness” – about their work processes, choice of methods, media formats, etc.
  • Hold themselves accountable for their choices
  • [SLIDE 38] Peer Evaluation: Paper Prototypes
  • Final Presentation: [SLIDE 39] My Feedback + [SLIDE 40] Students’ Peer Reviews
  • [SLIDE 41] Possible Topic for Q&A: Committee work on implications for the Dissertation

 

Categories
Presentations

Urban Research Mobile Media

Here’s my presentation for the Urban Research & Mobile Media panel at Mobility Shifts:

Mattern_MobilityShifts_URT

Categories
Blog

DH: The Name That Does No Favors

via ToastyTreat87 @ Flickr: http://bit.ly/f6HVOp

I just returned from a workshop at NYU called “Why Digital Humanities?” I went primarily because Kathleen Fitzpatrick, who’s always fantastic, and Diana Taylor were on the panel — but also, I must admit, because, even though I work just a few blocks away, I often miss my alma mater and its fancy facilities and nice catered lunches.

After the four panelists presented, roughly a half-hour remained for questions — and it became clear after the third or fourth person spoke that most of their questions centered on the issue of definition. “If I show a YouTube clip in my writing class, am I practicing the digital humanities?” “Will history departments ultimately split into traditional and digital camps?” I’ve been working with digital archives for 15 years. I was a digital humanist before there even was such a term!”

Last summer I participated in a think tank organized by our Provost’s office at The New School. The challenge I set for myself was to do an “environmental scan” of “alternative modes of scholarly practice” — focusing in particular on DH, a “field” I’d been curious, but also skeptical, about for a while — and to figure out how to translate some of those findings into my classes in the fall. After several months of research, some skepticism lingered — but I also managed to find a new, more fruitful, less techno-fetishist way to conceive of the really valuable things that DH has to offer.

I came to the conclusion that the “Digital Humanities” name doesn’t do anybody any favors. It prioritizes the digital, implying that the insertion of new media into any endeavor inevitably makes it better — and, conversely, that print and old media are inherently retrograde. This is not the message I need to send to my students, many of whom already assume that the world was reborn — and humanity reached its apotheosis — with the rise of the Internet. What’s more, it singles out the humanities, suggesting that in this evolving educational universe, they can go it alone — with the help of a few technical gadgets. DH (or [insert better name here]) is necessarily interdisciplinary and collaborative.

Anyway, here’s the summary report I wrote. It’s no masterpiece, since I wrote this while simultaneously finishing the syllabi for two new grad classes. Still, I think it gets at some of these “image” and “self-definition” problems that seem to get some people stuck.

*     *     *     *     *

via LaughingQuid @ Flickr: http://bit.ly/e575FN

In the work I completed in my preliminary literature review[1], and in research I’ve conducted since then, I’ve discovered that a sizable portion of the Digital Humanities literature is dedicated to addressing when the Digital Humanities began, what the Digital Humanities (DH) are, and what counts as a DH project.[2] These identity-negotiation discussions are perhaps to be expected of a “diverse and still emerging” field[3]. Yet I find that the prevalence of these debates, and their focus on self-justification, limit the attention directed toward meaningful applications. In addition, the efforts to define the Digital Humanities as a discipline often mean that a great deal of (liquid and digital) ink is spilt in establishing the particular nature of DH’s relationship to “the humanities” and “the digital.” Patrik Svensson, in his recent article on “the landscape of digital humanities” – the second in a three-part series in Digital Humanities Quarterly – writes:

there is [even] a question of whether “the digital” needs to be specified at all, and it is not uncommon to encounter the argument that technology and the digital are part or will be part of any academic area, and hence the denotation “digital” is not required.[4]

The continued insistence on (and seeming fetishization of) the digital, however, seems to privilege these media at the expense of other, non-digital, yet equally appropriate and effective, media forms. Rather than fetishizing the database, as some “humanities computing” (what some call “Digital Humanities 1.0”) scholars seemed to do, however, I’d prefer that we consider other modes of “processing” a research project – that we apply the valuable lessons that DH has to offer to a broader scope of scholarly modes. I’d prefer that we consider how particular questions or problems might lend themselves to investigation or representation through aural, visual, or interactive media; through maps, audio archives, documentaries, video games – even architecture, designed products, clothing. In some cases, we should remember, a print document – designed so that its material form reinforces its argument – might be the most appropriate means of giving form to an argument.

I have found media scholar Tara McPherson’s approach most similar to my own vision, and her voice most refreshing. McPherson, who is affiliated with USC’s Institute for Multimedia Literacy, calls such a cross-platform oriented scholar “multimodal”: the multimodal scholar “thinks carefully about the relationship of form to content, expression to idea.” She examines “what happens when scholarship looks and feels differently, requiring new modes of engagement from the reader/user.”[5] She wonders: “How do you ‘experience’ or ‘feel’ an argument in a more immersive and sensory-rich space?” “Can scholarship show as well as tell?” “Will representing data differently change the ways we understand, collect, or interpret it?”

I prefer to use McPherson’s terminology – multimodal scholarship (although I think that there’s still something inelegant in this construction) – so, for the rest of this document I’ll be referring to “MS” instead of “DH.” However, I do think that there are a great many intellectual and ethical parallels between DH and MS, and in some cases we might even be able to use the terms interchangeably. So, although I’ll be using the term (or, rather, acronym) MS in what follows, I’ll be drawing from the literature on both MS and DH.

Rather than get caught up in the debates over labels and territory and disciplinary status, I have found it much more beneficial to focus, like McPherson, on those intellectual practices and values that are central to the new scholarly practices defining MS – values that seem consistent with the new pedagogies and university structures called for by a host of recognizable figures and entitites, including Henry Jenkins, Cathy Davidson, David Theo Goldberg, and the MacArthur Foundation.[6] Those values, which I highlighted in my preliminary literature review, include (1) opening up, laying bare, and critically reflecting on the process of scholarship; (2) collaboration; and (3) a deep concern with epistemological questions (e.g., how is knowledge “made,” who gets to make it, what’s done with it, etc.) I’ll say a few words about each:

First, the practice of chronicling one’s research process, Johanna Drucker says, benefits the researcher him- or herself in that it opens up “occasions for critical self-consciousness.”[7] The practice also benefits academia’s publics – both the limited ones it has now, and the potentially wider and more numerous ones it could have in the future; explaining what we, as researchers/critical-practitioners/critical-educators/etc., do can “illuminate the shadowy process of critical thinking, encouraging readers not only to digest finished works, but also to learn from and evaluate the mechanisms of their creation.”[8] Second, collaboration allows participants to “leverage the increasingly distributed nature of expertise and knowledge and transform this reality into occasions for scholarly innovation, disciplinary cross-fertilization, and the democratization of knowledge.”[9] Collaborators on DH/MS projects increasingly come from outside the university: libraries, museums, archives, historical societies, arts organizations, advocacy groups, non-profits, schools, and local communities all have the potential to participate (as I’ll explain below, I’ve attempted to integrate several outside participants in my fall application). Third, as Stephanie Barish and Elizabeth Daley, also affiliated with USC’s IML, argue, “To be literate today, one must understand how strategically chosen and juxtaposed combinations of media enable the construction and dissemination of meaning in ways that bypass of enrich traditional text and the spoken word. Indeed, one must not only be able to read such media, but also to author it.”[10] Such knowledge work calls into question the distinction between theory and practice. And, through its continual reflection on process, this work also has the potential to raise critical questions about what constitutes knowledge, “who gets to create [i]
,… how it gets legitimated and authorized, and how it is made accessible to a significantly broader (and potentially global) audience.”[11]

My late-summer research has focused primarily on how these values can be integrated into the classroom. The final section of my preliminary literature review addressed the challenges and opportunities of introducing faculty to DH- or MS-inspired pedagogies, and incentivizing them to make the extra effort to incorporate these new modes of teaching into their courses (and to take some risk in doing so). Much of our online ATT discussion throughout the summer has focused on these issues, too. But my literature review closed by bringing these issues back to bear on the students: I focused on how the collaborative, process-focused, multi-disciplinary, “multiple literacies” approach is central to USC’s IML. While the program is immensely inspiring on paper – and it has no doubt achieved tremendous success during its few years of existence – we heard from Holly Willis recently about the challenges even it, with its generous funding and active fellowship programs, has faced. These types of issues require structural changes and widespread institutional commitment to change – efforts that, as we discussed, are beyond individual faculty members’ purview but within the realm of responsibility of the Provost’s office.

A more small-scale, but no less significant, issue that I then turned my attention to was the issue of assessment. I wrote at the end of my preliminary literature review: “As the rampant DH boosterism and invariably positive commentary on [particular high-profile] projects…reveals, the Digital Humanities community has yet to build a tradition of critique.”[12] But how does one critique a research-based interactive map? Or a theoretically informed performance-installation? The standard processes and rubrics of grading, or of peer review, fail in these cases. So, I spent the final few weeks of my summer investigating models for assessing multimodal student projects. My blog post on this topic, I learned (much to my surprise), was tweeted around a bit. And given the specific multimodal form of my students’ projects – an interactive dabatase-driven map – I’ve begun an effort to integrate criticism sensitive to the medium-specificity of the map, with these multimodal evaluative rubrics. I will continue to work through these issues with my students as the fall progresses. And throughout the semester we will be blogging our design and deliberation and evaluation processes, for the benefit of those who might learn from our experience…..

X


X

[1] See here, here, and here.

[2] Sometimes it seems as if it would be easier to explain what doesn’t count, given the expansive nature of some DH definitions. Todd Presner, for instance, defines DH as “humanistic practice anchored in creation, curation, collaboration, experimentation, and the multi-purposing or multi-channeling of humanistic knowledge.” (“Digital Humanities 2.0: A Report on Knowledge,” May 13, 2010, Module m34246, Connexions.)

[3] Digital Humanities Quarterly, “About DHQ”: http://digitalhumanities.org/dhq/about/about.html

[4] Patrick Svensson, “The Landscape of the Digital HumanitiesDigital Humanities Quarterly 4:1 (Summer 2010): ¶ 51.

[5] Tara McPherson, “Introduction: Media Studies and the Digital Humanities,” Cinema Journal 48, no. 2 (Winter 2009): 120-1.

[6] Henry Jenkins, “Confronting the Challenges of Participatory Culture: Media Education for the 21st Century” [white paper] (Cambridge, MA: MIT Press, 2006); Cathy N. Davidson & David Theo Goldberg, “The Future of Thinking: Learning Institutions in a Digital Age” (Cambridge, MA: MIT Press, 2010).

[7] Johanna Drucker and Bethany Nowviskie, “Speculative Computing: Aesthetic Provocations in Humanities Computing,” in A Companion to Digital Humanities, ed. Ray Siemens, John Unsworth, and Susan Schreibman, Hardcover., Blackwell Companions to Literature and Culture (Oxford: Blackwell, 2004).

[8] Avi Santo and Christopher Lucas, “Engaging Academic and Nonacademic Communities through Online Scholarly Work,” Cinema Journal 48, no. 2 (Winter 2009): 133-134.

[9] Jeffrey Schnapp, Todd Presner, et. al., “The Digital Humanities Manifesto 2.0,” n.d.: 5.

[10] Stephanie Barish and Elizabeth Daley, Multimedia Scholarship for the 21st Century, Educause Forum for the Future of Higher Education (Educause, 2005): 39.

[11] Presner, “Digital Humanities 2.0: A Report on Knowledge.”

[12] See Jennifer Howard, “Hot Type: No Reviews of Digital Scholarship = No RespectChronicle of Higher Education (May 23, 2010).