HyeonsuKang
Hyeonsu is a CS Ph.D. candidate at Carnegie Mellon University (CMU), advised by Niki Kittur and affiliated with the Human-Computer Interaction Institute.
His work combines cutting-edge AI tools with a deep knowledge of human cognition to build systems that demonstrably augment human creativity and innovation. His overarching research interest involves developing new AI and interaction techniques that allow users to mine the millions (and eventually billions) of design inspirations on the web and in scientific papers to find creative solutions to their problems outside their usual domains.
His systems have been deployed in real-world settings including allocating over $2M in prize money at Conservation X Labs (a conservation-focused non-profit) and at Semantic Scholar (by Allen Institute for Artificial Intelligence, also known as "AI2"), where they have shown statistically significant increases in the novelty and usefulness of ideas found and generated for scientists and designers, and in engagement with them. This work has resulted in several first-authored papers at premiere conferences and journals such as ACM SIGCHI, UIST, TOCHI, and NAACL.
Hyeonsu's work garnered him recognition as a Google Cloud Research Innovator (2021). He was previously supported by the South Korean National Scholarship for Science and Engineering. He received his BS in Computer Science and Engineering at Seoul National University. He also worked at MIT, AI2, UC San Diego, and Tableau Software.
Publications
Efficiently reviewing scholarly literature and synthesizing prior art are crucial for scientific progress.
Yet, the growing scale of publications and the burden of knowledge make synthesis of research threads more challenging than ever.
While significant research has been devoted to helping scholars interact with individual papers, building research threads scattered across multiple papers remains a challenge.
Most top-down synthesis (and LLMs) make it difficult to personalize and iterate on the output, while bottom-up synthesis is costly in time and effort.
Here, we explore a new design space of mixed-initiative workflows.
In doing so we develop a novel computational pipeline, Synergi, that ties together user input of relevant seed threads with citation graphs and LLMs, to expand and structure them, respectively.
Synergi allows scholars to start with an entire threads-and-subthreads structure generated from papers relevant to their interests, and to iterate and customize on it as they wish.
In our evaluation, we find that Synergi helps scholars efficiently make sense of relevant threads, broaden their perspectives, and increases their curiosity.
We discuss future design implications for thread-based, mixed-initiative scholarly synthesis support tools.
In order to help scholars understand and follow a research topic, significant research has been devoted to creating systems that help scholars discover relevant papers and authors. Recent approaches have shown the usefulness of highlighting relevant authors while scholars engage in paper discovery. However, these systems do not capture and utilize users’ evolving knowledge of authors. We reflect on the design space and introduce ComLittee, a literature discovery system that supports author-centric exploration. In contrast to paper-centric interaction in prior systems, ComLittee’s author-centric interaction supports curation of research threads from individual authors, finding new authors and papers with combined signals from a paper recommender and the curated authors’ authorship graphs, and understanding them in the context of those signals. In a within-subjects experiment that compares to an author-highlighting approach, we demonstrate how ComLittee leads to a higher efficiency, quality, and novelty in author discovery that also improves paper discovery.
Analogies have been central to creative problem-solving throughout the history of science and technology. As the number of scientific papers continues to increase exponentially, there is a growing opportunity for finding diverse solutions to existing problems. However, realizing this potential requires the development of a means for searching through a large corpus that goes beyond surface matches and simple keywords. Here we contribute the first end-to-end system for analogical search on scientific papers and evaluate its effectiveness with scientists' own problems. Using a human-in-the-loop AI system as a probe we find that our system facilitates creative ideation, and that ideation success is mediated by an intermediate level of matching on the problem abstraction (i.e., high versus low). We also demonstrate a fully automated AI search engine that achieves a similar accuracy with the human-in-the-loop system. We conclude with design implications for enabling automated analogical inspiration engines to accelerate scientific innovation.
Reviewing the literature to understand relevant threads of past work is a critical part of research and vehicle for learning. However, as the scientific literature grows the challenges for users to find and make sense of the many different threads of research grow as well. Previous work has helped scholars to find and group papers with citation information or textual similarity using standalone tools or overview visualizations. Instead, in this work we explore a tool integrated into users' reading process that helps them with leveraging authors' existing summarization of threads, typically in introduction or related work sections, in order to situate their own work's contributions. To explore this we developed a prototype that supports efficient extraction and organization of threads along with supporting evidence as scientists read research articles. The system then recommends further relevant articles based on user-created threads. We evaluate the system in a lab study and find that it helps scientists to follow and curate research threads without breaking out of their flow of reading, collect relevant papers and clips, and discover interesting new articles to further grow threads.
Finding and engaging with the relevant scientific knowledge is foundational for intellectual progress in a society. Yet, with an exponential growth in publication rates, this becomes a challenging task. While personalized recommendations can help, they still may lack explanations of how certain papers are relevant and thus should be prioritized or attended to. To combat this, we developed a citation-based and two kinds of social relation-based approaches to boost user engagement with scholarly paper recommendations. For users who opted in, these approaches augmented paper recommendations included in email alerts with textual relevance descriptions underneath the recommendations. We evaluated our approaches in a randomized field experiment that ran for over two months and with 7,000+ users, and also in a controlled lab study (N=14) for deeper qualitative insights. We report on our findings and implications for the design of future approaches that aim to augment scholarly recommendations.
We explore a novel representation for automatically breaking up product ideas described in natural language into fine-grained functional aspects. This representation can capture the core purposes and mechanisms in ideas, and support the backbone interactions (e.g., functional search of ideas, mapping and exploration of the design space around a focal problem) for augmenting human intelligence and accelerating the rate of innovation.
“A picture is worth a thousand words.” We developed Paragon, a system that supports crowdworkers and peers during feedback exchange by enabling search of design examples that supplement the written feedback. In two lab studies, we found that i) feedback providers select poster examples that complement their feedback and align with a provided rubric and that ii) feedback providers give significantly more specific, actionable, and novel input when using an example-centric approach, as opposed to text alone.
We developed Omnicode, a programming environment with an always-on run-time visualization that helps novice programmers directly see how the variables and their relations change in real-time, in response to the changes they make in the program code. In our lab study, we found Omnicode to be useful for debugging, forming proper mental models, explaining their code to others, and discovering moments of serendipity that would not have been likely within an ordinary IDE.
Scholarly publications are key to the transfer of knowledge from scholars to others. However, research papers are information-dense, and as the volume of the scientific literature grows, the need for new technology to support the reading process grows. In contrast to the process of finding papers, which has been transformed by Internet technology, the experience of reading research papers has changed little in decades. The PDF format for sharing research papers is widely used due to its portability, but it has significant downsides including: static content, poor accessibility for low-vision readers, and difficulty reading on mobile devices. This paper explores the question "Can recent advances in AI and HCI power intelligent, interactive, and accessible reading interfaces-even for legacy PDFs?" We describe the Semantic Reader Project, a collaborative effort across multiple institutions to explore automatic creation of dynamic reading interfaces for research papers. Through this project, we've developed ten research prototype interfaces and conducted usability studies with more than 300 participants and real-world users showing improved reading experiences for scholars. We've also released a production reading interface for research papers that will incorporate the best features as they mature. We structure this paper around challenges scholars and the public face when reading research papers-Discovery, Efficiency, Comprehension, Synthesis, and Accessibility-and present an overview of our progress and remaining open challenges.
Exposure to ideas in domains outside a scientist's own may benefit her in reformulating existing research problems in novel ways and discovering new application domains for existing solution ideas. While improved performance in scholarly search engines can help scientists efficiently identify relevant advances in domains they may already be familiar with, it may fall short of helping them explore diverse ideas outside such domains. In this paper we explore the design of systems aimed at augmenting the end-user ability in cross-domain exploration with flexible query specification. To this end, we develop an exploratory search system in which end-users can select a portion of text core to their interest from a paper abstract and retrieve papers that have a high similarity to the user-selected core aspect but differ in terms of domains. Furthermore, end-users can 'zoom in' to specific domain clusters to retrieve more papers from them and understand nuanced differences within the clusters. Our case studies with scientists uncover opportunities and design implications for systems aimed at facilitating cross-domain exploration and inspiration.
Matching Open Innovation Projects for Analogical Feedback Exchange
Collective Intelligence 2019
We developed an algorithm for matching teams in open innovation contests that tackle related conservataion challenges using diverse approaches, thereby encouraging the transfer of analogical inspirations between teams. To this end, our algorithm used pre-trained language models to encode the natural language text descriptions of team challenges and their solution approaches into a vector similarity space, then computed semantic similarity between them to systematically find teams tackling similar problems using diverse approaches, shown as a conducive mechanism for the transfer.
Custom Blocks in StarLogo Nova: A Template-Based Approach to Abstraction for Improved Ease of Use and Expressive Power
SIGPLAN 2018
We developed a general extension to the StarLogo Nova language to support end-user programming in various disciplines such as evolutionary biology, physics, and ecosystem sciences. This extension allowed end-users to select blocks that correspond to low-level programming constructs such as looping and variable assignment statements, and group them to create abstraction blocks that hide the low-level implementation details that oft-times distract learners from disciplinary learning objectives and system-level conceptual understanding. Using such abstraction blocks can also reduce the complexity of the programming language itself and lower the barrier to entry for novice learners.