About
I’m a researcher in artificial intelligence, human-computer interaction, and creativity. I primarily design and develop AI-based creativity support tools to assist human designers and artists, particularly in the domains of storytelling, poetry, and game design. As an interdisciplinary scholar with a background in technical games research, I’ve also worked on a wide variety of topics related to interactive narrative and digital games.
Currently, I’m a research scientist at Midjourney, where I lead the Storytelling Lab. Before that, I was an assistant professor of computer science and engineering at Santa Clara University. I hold a Ph.D. in Computational Media from the University of California, Santa Cruz, where I was a member of the Expressive Intelligence Studio. I’ve also been a resident at Stochastic Labs; a consultant at various game industry and tech companies; and a researcher in several labs, most notably the Mobile & Environmental Media Lab, at the University of Southern California.
Here’s a complete list of my papers (current count: 66; awards and nominations: 10). You can also check out my full CV for links to many of my talks and other output.
Selected projects and papers
Toyteller is a novel AI-based interface for playful storytelling: users steer the generation of story text by physically acting out a scene with abstract shapes, as if playing with toys. We’ve found that unusual new interaction techniques – like shaping textual stories through motion – can serve as a complement to traditional linguistic specifications of expressive intent, scaffolding user development of more complex and multifaceted scenes.
Patchwork is a collaborative AI-supported worldbuilding canvas. Users work together to craft a coherent fictional setting and tell stories in that setting, with a multimodal AI system helping them gradually expand and refine their shared vision for the storyworld. We’re also using Patchwork to prototype a variety of novel control and steering techniques for generative AI, including several that have become spinoff projects (e.g., Patchview).
📝 Papers: UIST 2024, HAI-GEN @ IUI 2024
🗣️ Talks: Autodesk Research, South Park Commons
🗞️ Press: VentureBeat, The Decoder
🤝 With: John Chung, Melissa Roemmele, Uchan Sun
🗣️ Talks: Autodesk Research, South Park Commons
🗞️ Press: VentureBeat, The Decoder
🤝 With: John Chung, Melissa Roemmele, Uchan Sun
Homogenization analysis is an approach to the evaluation of creativity support tools (CSTs). By quantifying semantic similarity within and between sets of artifacts created by users of different CSTs, we can diagnose homogenization effects that CSTs (particularly generative AI-based CSTs) exert on their users’ creative outputs and begin to characterize how these effects arise.
📝 Papers: C&C 2024 (best paper), CHI 2024, In2Writing @ CHI 2024
🗞️ Press: Digital Frontier
🤝 With: Barrett Anderson, Jash Shah
🗞️ Press: Digital Frontier
🤝 With: Barrett Anderson, Jash Shah
Blabrecs is a rules modification for the wordgame Scrabble that forbids players from using actual English dictionary words. Instead, players may only use nonsense words that sound like English to an AI trained on the dictionary.
📝 Papers: NeurIPS 2023, AIIDE 2021
🗣️ Talks: !!Con 2024, GDC 2024 (skip to 1:51:00)
🗞️ Press: New Scientist, The Next Web, Dicebreaker
🤝 With: Isaac Karth
🗣️ Talks: !!Con 2024, GDC 2024 (skip to 1:51:00)
🗞️ Press: New Scientist, The Next Web, Dicebreaker
🤝 With: Isaac Karth
Fuzzy linkography is a jointly visual and quantitative approach to the analysis of creative activity traces. Fuzzy linkographs serve as interpretable graphical summaries of design sessions and can be generated at very large scales, allowing us to identify recurring temporal patterns of creative activity; characterize the creative “give and take” between humans and machines; cluster activity traces to discover recurring designer archetypes; and more.
📝 Papers: arXiv preprint
🤝 With: Amy Smith, Barrett Anderson, Jasmine Otto, Isaac Karth, Uchan Sun, John Chung, Melissa Roemmele
🤝 With: Amy Smith, Barrett Anderson, Jasmine Otto, Isaac Karth, Uchan Sun, John Chung, Melissa Roemmele
Loose Ends is an AI-based narrative instrument that can be played to produce narrative, much like a musical instrument can be played to produce music. It aims to help players interleave multiple parallel plot threads into an effective high-level plot structure that satisfies player-specified storytelling goals.
Expressive range coverage analysis is an evaluation method for creativity support tools (CSTs). By analyzing artifacts created by CST users in relation to the tool’s underlying expressive range, we can capture striking visual portraits of interaction dynamics; gauge similarities and differences between users’ exploration patterns in a shared design space of possible creations; and generally illuminate tool impacts on creative outcomes.
Story sifters are computational recognizers of potentially compelling story structures. I’ve worked to frame story sifting as logic programming (Felt); developed AI-based tools that make logical sifters easier to build (Synthesifter); and enabled sifters to reason about possible future events (Winnow) and prioritize unexpected stories (StU).
📝 Papers: ICIDS 2019, INT @ AIIDE 2020, AIIDE 2021, ICIDS 2022, chapter
🗣️ Talks: Concordia University
🗞️ Press: The Verge
🤝 With: Melanie Dickinson
🗣️ Talks: Concordia University
🗞️ Press: The Verge
🤝 With: Melanie Dickinson
Academical is an interactive narrative game that uses roleplaying to train new researchers in responsible conduct of research. Our user studies have found that Academical is better than existing RCR training materials at engaging users, teaching some RCR topics, and promoting moral reasoning, without falling behind in other areas.
📝 Papers: FDG 2020 (exceptional paper), CHI 2021, CHI 2020, chapter, Frontiers in VR
🤝 With: Eddie Melcer, Katy Grasse, Nic Junius, many others
🤝 With: Eddie Melcer, Katy Grasse, Nic Junius, many others
Why Are We Like This? is an AI-augmented collaborative storytelling game. Two players write a story in a pastiche of the cozy mystery genre, with support from a simulation-based AI system that suggests what might happen next based on both character and authorial goals.
StoryAssembler is an interactive narrative system that assembles choice-based stories on the fly from a library of author-provided story fragments, using planning to ensure that every story it produces meets a set of author-specified requirements.
Redactionist is an AI-based tool that helps you create poetry by erasing words from a source text. It builds on my earlier work in this direction: a browser bookmarklet that can turn any arbitrary webpage into blackout poetry.
📝 Papers: HTTF 2024, HAI-GEN @ IUI 2022, ICCC 2021, PCG @ FDG 2019
🗣️ Talks: !!Con West 2019, Computational Poetry Workshop 2019
🗣️ Talks: !!Con West 2019, Computational Poetry Workshop 2019
GBS is a genre-specific game description language for Game Boy RPGs, extracted from the GB Studio game creation tool. Using GBS as a target language for game generation enabled us to explore the generation of larger and more expressively sophisticated games than would otherwise be tractable; we believe that GBS and similar GSGDLs might serve as especially good foundations for future AI-based game design tools.
Retellings studies are a series of investigations into the stories that players tell about their experiences in games, especially simulation-driven emergent narrative games. This work helps us better understand how to provide computational support for human storytelling practices.
📝 Papers: ICIDS 2021a (best short paper nominee), ICIDS 2021b (best short paper nominee), AIIDE 2019, UX of AI @ FDG 2019
Epitaph is a procedural narrative idlegame about existential risks and the death of civilizations. Bennett Foddy (the creator of QWOP) considers it “one of the great web games”. It was the precursor to my academic work on gardening games, which use procedural content generation to support gameplay experiences of caretaking rather than extraction.
📝 Papers: PCG @ FDG 2018
Social VR environments are immersive virtual social spaces. I’ve been involved in several attempts to make social VR environments better for creative collaboration – particularly by endowing users with social superpowers, or communicative augmentations that couldn’t exist in meatspace.
📝 Papers: CHI 2022, DIS 2023, SocialVR @ CHI 2021, DIS 2018, HCII 2018
🤝 With: Josh McVeigh-Schultz, Katherine Isbister, many others
🤝 With: Josh McVeigh-Schultz, Katherine Isbister, many others