Scholarly publications are key to the transfer of knowledge from scholars to others. However, research papers are information-dense, and as the volume of the scientific literature grows, the greater the need for new technology to support scholars. In contrast to the process of finding papers, which has been transformed by Internet technology, the experience of reading research papers has changed little in decades. For instance, the PDF format for sharing papers remains widely used due to its portability but has significant downsides, inter alia, static content and poor accessibility for low-vision readers. This paper explores the question "Can recent advances in AI and HCI power intelligent, interactive, and accessible reading interfaces - even for legacy PDFs?" We describe the Semantic Reader Project, a collaborative effort across multiple institutions to explore automatic creation of dynamic reading interfaces for research papers. Through this project, we've developed a collection of novel reading interfaces and evaluated them with study participants and real-world users to show improved reading experiences for scholars. We've also released a production research paper reading interface that will continuously incorporate novel features from our research as they mature. We structure this paper around five key opportunities for AI assistance in scholarly reading - discovery, efficiency, comprehension, synthesis, and accessibility - and present an overview of our progress and discuss remaining open challenges. Augmenting scholarly documents through AI-powered interactive reading interfaces.

We explore a novel representation for automatically breaking up product ideas described in natural language into fine-grained functional aspects. This representation can capture the core purposes and mechanisms in ideas, and support the backbone interactions (e.g., functional search of ideas, mapping and exploration of the design space around a focal problem) for augmenting human intelligence and accelerating the rate of innovation.

“A picture is worth a thousand words.” We developed Paragon, a system that supports crowdworkers and peers during feedback exchange by enabling search of design examples that supplement the written feedback. In two lab studies, we found that i) feedback providers select poster examples that complement their feedback and align with a provided rubric and that ii) feedback providers give significantly more specific, actionable, and novel input when using an example-centric approach, as opposed to text alone.

We developed Omnicode, a programming environment with an always-on run-time visualization that helps novice programmers directly see how the variables and their relations change in real-time, in response to the changes they make in the program code. In our lab study, we found Omnicode to be useful for debugging, forming proper mental models, explaining their code to others, and discovering moments of serendipity that would not have been likely within an ordinary IDE.

Nature is often used to inspire solutions for complex engineering problems, but achieving its full potential is challenging due to difficulties in discovering relevant analogies and synthesizing from them. Here, we present an end-to-end system, BioSpark, that generates biological-analogical mechanisms and provides an interactive interface to comprehend and synthesize from them. BioSpark pipeline starts with a small seed set of mechanisms and expands it using an iteratively constructed taxonomic hierarchies, overcoming data sparsity in manual expert curation and limited conceptual diversity in automated analogy generation via LLMs. The interface helps designers with recognizing and understanding relevant analogs to design problems using four main interaction features. We evaluate the biological-analogical mechanism generation pipeline and showcase the value of BioSpark through case studies. We end with discussion and implications for future work in this area.

Exposure to ideas in domains outside a scientist's own may benefit her in reformulating existing research problems in novel ways and discovering new application domains for existing solution ideas. While improved performance in scholarly search engines can help scientists efficiently identify relevant advances in domains they may already be familiar with, it may fall short of helping them explore diverse ideas outside such domains. In this paper we explore the design of systems aimed at augmenting the end-user ability in cross-domain exploration with flexible query specification. To this end, we develop an exploratory search system in which end-users can select a portion of text core to their interest from a paper abstract and retrieve papers that have a high similarity to the user-selected core aspect but differ in terms of domains. Furthermore, end-users can 'zoom in' to specific domain clusters to retrieve more papers from them and understand nuanced differences within the clusters. Our case studies with scientists uncover opportunities and design implications for systems aimed at facilitating cross-domain exploration and inspiration.

We developed an algorithm for matching teams in open innovation contests that tackle related conservataion challenges using diverse approaches, thereby encouraging the transfer of analogical inspirations between teams. To this end, our algorithm used pre-trained language models to encode the natural language text descriptions of team challenges and their solution approaches into a vector similarity space, then computed semantic similarity between them to systematically find teams tackling similar problems using diverse approaches, shown as a conducive mechanism for the transfer.

We developed a general extension to the StarLogo Nova language to support end-user programming in various disciplines such as evolutionary biology, physics, and ecosystem sciences. This extension allowed end-users to select blocks that correspond to low-level programming constructs such as looping and variable assignment statements, and group them to create abstraction blocks that hide the low-level implementation details that oft-times distract learners from disciplinary learning objectives and system-level conceptual understanding. Using such abstraction blocks can also reduce the complexity of the programming language itself and lower the barrier to entry for novice learners.