Liberty Library Cadence

report were chosen for their special competences and with regard for appropriate balance. This project was supported by the Alfred P. Sloan Foundation under grant number 2011-10-28. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organization that provided support for the project.

International Standard Book Number 13:  978-0-309-29848-3 International Standard Book Number 10:  0-309-29848-2 Additional copies of this report are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.

Suggested citation: National Research Council. 2014. Developing a 21st Century Global Library for Mathematics Research. Washington, D.C.: The National Acad­ emies Press.

Copyright 2014 by the National Academy of Sciences. All rights reserved.

Printed In The United States Of America

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare.

Upon the authority of the charter granted to it by the Congress in 1863, the Acad­ emy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding ­engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineer­ ing programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. C. D. Mote, Jr., is presi­ dent of the National Academy of Engineering.

Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Insti­ tute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V.

The National Research Council was organized by the National Academy of Sci­ ences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government.

Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Dr. Ralph J. Cicerone and Dr. C. D. Mote, Jr., are chair and vice chair, respectively, of the National Research Council.

Of The Mathematical Sciences

CLIFFORD A. LYNCH, Coalition for Networked Information, Co-Chair

Vii

This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge.

The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. The committee wishes to thank the following individuals for their review of this report: Thierry Bouche, Cellule MathDoc and Institut Fourier, Université de

Heinz Weinheimer, Springer

Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before

Viii

its release. The review of this report was overseen by C. David Lever­ ­Research Council, he was responsible for making certain that an indepen­ dent examination of this report was carried out in accordance with institu­ tional procedures and that all review comments were carefully considered.

­Responsibility for the final content of this report rests entirely with the authoring committee and the institution. The committee also acknowledges the valuable contribution of the following individuals, who provided input at the meetings on which this

Report Is Based Or By Other Means:

Thierry Bouche, Cellule MathDoc and Institut Fourier, Université de

Grenoble

Jim Crowley, Society for Industrial and Applied Mathematics

Wayne Graves, Association For Computing Machinery

David Lipman, National Center for Biotechnology Information

Overview, 8

Study Definition and Scope and the Committee’s Approach, 8

Previous Digital Mathematics Library Efforts, 11

The Universe of Published Mathematical Information, 14

Brary

What Is Missing from the Mathematical Information Landscape?, 28 What Gaps Would the Digital Mathematics Library Fill?, 29

Fundamental Principles, 72

Constitution of the Digital Mathematics Library Organization, 80

A Meeting Agendas And Other Inputs To The Study

Biographical Sketches of Committee Members and Staff C The Landscape of Digital Information Resources in

Athematics And Selected Other Fields

Like most areas of scholarship, mathematics is a cumulative discipline: new research is reliant on well-organized and well-curated literature. Be­ cause of the precise definitions and structures within mathematics, today’s information technologies and machine learning tools provide an opportu­ nity to further organize and enhance discoverability of the mathematics literature in new ways, with the potential to significantly facilitate math­ ematics research and learning. Opportunities exist to enhance discoverabil­ ity directly via new technologies and also by using technology to capture important interactions between mathematicians and the literature for later sharing and reuse.

In most scientific disciplines, including mathematics, Web-based access to digital resources representing the disciplinary literature is now mature and quite effective. Through a mixture of open and proprietary tools, mathematicians are able to search the enormous and very rapidly grow­ ing literature using attributes such as subjects, titles, authors, dates, and keywords; they can follow chains of citations among works backward and forward in time. While much information is contained in individual items in the mathematical literature, a greater amount of information is represented by the way they are linked. This is not just via references but through the interrelation of concepts, insights, and techniques as they are developed, refined, and spread from one mathematical discipline to another. For ex­ ample, if mathematicians were able to search the literature for instances where a specific equation was used or solved, it would allow them to con­ sider alternative approaches toward solving their own research questions.

This search capability could be facilitated through the use of a database

Eveloping A 21St Century Mathematics Library

of machine-generated and human-cultivated information about the math­ ematical literature and allow for a variety of other capabilities to be built. This report discusses how information about what the mathematical literature contains can be formalized and made easier to express, encode, and explore. Many of the tools necessary to make this information system a reality will require much more than indexing and will instead depend on community input paired with machine learning, where mathematicians’ expertise can fill the gaps of automatization. The Committee on Planning a Global Library of the Mathematical Sciences proposes the establishment of an organization; the development of a set of platforms, tools, and ser­ vices; the deployment of an ongoing applied research program to comple­ ment the development work; and the mobilization and coordination of the mathematical community to take the first steps toward these capabilities.

Mathematics today has the opportunity to expand and redefine the way in which mathematical knowledge is represented and used, the character of the mathematical literature and how it evolves, and the way that math­ ematicians interact with this collection of knowledge. This new relationship with the literature and the mathematical knowledge corpus goes beyond new forms of access and analytical tools; it must also include the tools and services to accommodate the creation, sharing, and curation of new kinds of knowledge structures.

To be clear, what the committee proposes builds on the extensive work done by many dedicated individuals under the rubric of the World Digi­ tal Mathematical Library,1 as well as many other community initiatives.2 Comparing desired capabilities going forward with what has been achieved by these efforts to date, the committee concludes that there is little value in new large-scale retrospective digitization efforts or further aggregations of mathematical science publications (both traditional journal articles and newer preprint, blog, video, and similar resources) beyond the federation of distributed repositories already achieved through existing search services.

Nor is another bibliographically based secondary indexing service needed at this time. Necessary incremental improvements will likely continue to occur in these areas, but they do not require an initiative on the scale of what is being called for in this report.

The real opportunity is in offering mathematicians new and more direct ways to discover and interact with mathematical objects and mathematical knowledge through the Web. The committee’s consensus is that by some 1  The World Digital Mathematics Library rubric has been used by a variety of organizations for many distinct projects. A history of many of these efforts and the current state-of-the-art can be found on the wiki page from the International Mathematics Union’s Digital ­Mathematics Workshop in June 2012, http://ada00.math.uni-bielefeld.de/mediawiki-1.18.1/index.php/.

2  Examples include the Encyclopedia of Integer Sequences, the NIST Digital Library of Mathematical Functions, and the Guide to Available Mathematical Software.

Summary

combination of machine learning methods and community-based editorial effort, a significantly greater portion of the information and knowledge in the global mathematical corpus could be made available to researchers as linked open data3 through a central organizational entity—referred to in this report as the Digital Mathematics Library (DML).

The DML would aggregate and make available collections of ontolo­ gies, links, and other information created and maintained by human con­ tributors, curators, and specialized machine agents, with significant editorial input from the mathematical community. The DML would enable function­ alities and services over the aggregated mathematical information that go well beyond simply making publications available, to include capabilities for annotating, searching, browsing, navigating, linking, computing, and visualizing both copyrighted and openly licensed content. While the DML would store modest amounts of new knowledge structures and indices, it would not generally replicate mathematical literature stored elsewhere.

Instead, it would strive to represent the mathematical knowledge presented within a publication and illustrate how it is connected with other resources. While the committee believes that the DML could begin development soon, it notes that this work would need to be complemented by an ongoing research program to fill in gaps, improve quality and performance, increase the robustness of available technologies, and increase the automation of processes that still rely heavily on human intervention.

The DML would facilitate discovery of and interaction with math­ ematical information from diverse sources with varying levels of copyright. The committee envisions the DML as a growing corpus of public-domain and openly licensed mathematical information, Web services, and software agents, which would coexist with present mathematical publishing and indexing services for the foreseeable future.

A key early issue for the DML organization is how to establish con­ structive and effective partnerships with existing publishers, Web services, and other resources, both those specific to mathematics and those serving the much broader scholarly community. Some of these partnerships might be challenging because of copyright concerns. However, establishing fruit­ ful partnerships is essential to the success of the DML. While the DML would sometimes provide services and functional features that overlap with existing services and tools provided by both commercial and not-for-profit 3  Broadly defined, linked open data are structured data that are published in such a way that makes it easy to interlink them with other data, therefore making it possible to connect them with information from multiple sources. These connected data can provide a user with a more meaningful query of a subject by consolidating relevant information from a variety of places—e.g., in different research papers—and pulling out specific components that the user might be particularly interested in.

Eveloping A 21St Century Mathematics Library

entities, the committee suggests partnering with current service providers whenever possible rather than replicating capabilities of existing resources. For example in MathOverflow,4 a question-and-answer website for research mathematicians, research articles and papers are often referenced in answers given. While the DML would not want to replicate the inter­ face and social networking features of MathOverflow, it would be wholly appropriate for the DML to instigate and participate in a multi-party col­ laboration with MathOverflow and publishers of research ­mathematics to automatically capture citations entered in MathOverflow answers and republish them as linked open data annotations. In this scenario, the DML could help broker standard practices for interoperability and help main­ tain the software agents and annotation repositories that would allow publishers to make mathematicians coming to their websites aware of MathOverflow discussions potentially relevant to the papers they are view­ ing. The converse could also be supported. Posts on MathOverflow could be automatically annotated when errata or other commentary is added to the publisher’s website for an article mentioned in the MathOverflow post.

This illustrates the potential for chains of annotations as a new mode of scholarly discourse (Sukovic, 2008). To visualize how an annotation chain might come about, begin by assuming that a post in MathOverflow refer­ encing a particular article is automatically added as an annotation to this article on the publisher’s website. A subsequent reply to this annotation made by a reader of the publisher website is then automatically added to the thread on MathOverflow. A new reply subsequently added to the thread on MathOverflow is then automatically added as a further annotation on the publisher’s website, and so on. This would allow users of two disparate services—i.e., one scholar using MathOverflow and the other using only the publisher’s website—to nonetheless carry on a substantive discourse about published mathematics research in spite of the fact that each is using a dif­ ferent utility to access the publication being discussed.

Similarly, MathSciNet and Zentralblatt Math (zbMath) already clas­ sify research papers according to the Mathematics Subject Classification (MSC)5 schedule. The DML would not want to replicate this indexing.

However, it might be beneficial for the DML to provide complementary indexing on other dimensions—e.g., by the occurrence in articles of well- known special functions (hierarchies of which are maintained by the Na­ 4  MathOverflow, http://mathoverflow.net/, accessed January 16, 2014.

5  American Mathematical Society, 2010 Mathematics Subject Classification, http://www. 6  NIST, Digital Library of Mathematical Functions, Version 1.0.6, release date May 6, 2013, http://dlmf.nist.gov/.

Summary

Research7). Used in concert, one could then envision a collaboratively built interface that allows refinement of an initial MSC search via attributes such as which special functions are used in the articles that appear in the results from the MSC search.

Such partnerships and collaborations are essential. It is vital that users see a well-integrated interface that incorporates both the DML services and commercial services for those affiliated with institutions that have access to the commercial services. The committee envisions the resources, services, and tools offered by the DML as coexisting with, and often enhancing, the offerings from existing players in the mathematical information landscape.

The committee hopes that relevant organizations will contribute to the work of the DML in various ways, such as by providing financial support, allowing appropriate access to their content and services, or by participat­ ing in the collaborative development, with the shared goal of enhancing the value of the mathematics literature. Building these partnerships would likely require significant negotiations and collaborations, and the DML organization would have to allocate much time and effort to their planning and execution.

The biggest challenge, however, will be in establishing the technical, organizational, and community-coordinating capabilities to deliver on the construction of the resources, services, and tools described earlier in this summary and then planning and implementing the development and deploy­ the requisite tools and services do not exist today or are not sufficiently mature. The committee sees the DML as having a minimal direct research role; rather, the committee believes that the establishment of the DML needs to be complemented by a long-term (5 to 10 years) commitment to a focused and applied research program that would encompass both needed technology, tools, and services and (to a lesser extent) independent research to understand how the DML is being used and how well it is working. Ide­ ally, the commitment to fund this program could come in parallel with the commitment for the initial funding for the DML itself (whether from one or multiple sources). These research programs need to be well connected to the work of the DML. This could be achieved either by ensuring that the DML is deeply involved in the development of the calls for proposals and the subsequent proposal evaluation or by actually placing the DML in the role of a re-granting organization (although the committee sees some potential bureaucratic complications with the latter option).

7  Wolfram Research, Inc., The Wolfram Functions Site, http://functions.wolfram.com/, ac­ cessed January 16, 2014.

Organization And Resources Needed

The committee’s vision of an incremental development of the DML starts with the creation of a small nonprofit organization, referred to here as the DML organization. The DML organization will need a small and dedi­ cated paid staff, including a well-respected mathematician in a senior role, to ensure its development and growth. Other staffing needs may become necessary as the needs and status of the DML evolve, although much of the software development and operations could be contracted out. Ideally, the DML would be attached to and draw support from some host institution (a sharing of services and to reduce overhead. The DML organization could be governed ultimately by the mathematical sciences community through organizations such as the International Mathematical Union and, thence, through their member organizations.

The first and foremost challenge that the DML will face is finding a set of primary funding sources that could support its initial development and early operations (a period of between 5 and 10 years). It is the committee’s hope that the DML would become a self-sustaining entity once some of its key capabilities are established and a potential sustainable business model

Is Chosen From Among Options.8

For the first few years, perhaps the best approach would be to split operational governance from high-level, longer-term policy governance, be­ cause these two tasks will be quite distinct. Both in the short and the ­longer term, appropriate connections are needed between funding and revenue sources and governance, and these connections may well need to shift over time. Particularly in the early days, a light and agile governance mechanism is crucial. Upon launching the DML effort, there would likely be a coalition of partners with a commitment to the DML concept.

Onclusion

Like other scientific disciplines, mathematics is now completing a com­ plex multi-decade transition from print to a digital system that closely emulates print for authors and readers. The mathematics community is thus at an inflection point where it has the opportunity to think about how its collective knowledge base is going to be constructed, used, structured, man­ aged, curated, and contributed to in the digital world and how that knowl­ edge base will be related to the existing literature corpus, to authoring 8  There are many lessons on sustainability to draw upon, including experiences with digital libraries (such as arXiv) and open or community source software as well as work on research data curation.

Summary

and learning mathematics. Colleagues in other disciplines—­astronomy, molecular biology, genomics, chemistry—are in many cases well advanced in formulating their own disciplinary-specific answers that take into ac­ count disciplinary practices (such as the mix of experimental, observational, theoretical, and computational approaches) and the conceptual models that underlie disciplinary thinking.

Mathematics is unusual in many ways; it maintains a healthy and con­ structive relationship with its past, as documented in the literature of the field going back hundreds of years, and some of its literature has a long “shelf life.” The committee believes that investments in refreshing and restructuring the corpus of mathematical literature and abstracting it into a knowledge base for future centuries is a valid and sound investment in the future of mathematical scholarship. The DML proposed in this report provides a platform and a context to achieve this and also offers a criti­ cal point of focus for the mathematical community in a genuinely digital environment to engage in discussions about the creation, curation, and management of mathematical knowledge.

Reference

Sukovic, S. 2008. Convergent flows: Humanities scholars and their interactions with electronic

Overview

Mathematics is facing a pivotal junction where it can either continue to utilize digital mathematics literature in ways similar to traditional printed literature, or it can take advantage of new and developing technology to enable new ways of advancing knowledge. This report details how infor­ mation contained in individual items within the literature could be readily extracted and linked to create a comprehensive digital mathematics infor­ mation resource that is more than the sum of its contributing publications.

That resource can serve as a platform and focal point for further develop­ ment of the mathematical knowledge base. This new system, referred to throughout the report as the Digital Math­ ematics Library (DML), could support a wide variety of new functionalities and services over aggregated mathematical information, including dramati­ cally improved capabilities for searching, browsing, navigating, linking, computing, visualizing, and analyzing the literature.

The Committee’S Approach

The Alfred P. Sloan Foundation commissioned this study and charged

•

Evaluate the potential value of a virtual global library of math­

•

Assuming that a stable context for sharing copyrighted information has been achieved, assess the remaining issues to be addressed in

•

Identify a range of desired capabilities of such a library; and

•

Characterize resource needs. While a traditional library is perhaps the oldest formal information resource available, the manifestation of libraries has evolved dramatically over the past few decades. In many cases within mathematics, as for other fields of scholarship, buildings housing paper publications have given way to online collections of downloadable documents. While this increased ­access is not perfect—not all material is readily available to all ­researchers, and search tools vary from site to site—widespread digitization has made it easier for many to access the mathematical literature. Overall, a much greater proportion of the mathematical literature is available to more ­people than at any time before. The research libraries, scholarly societies, and other players that curate and steward this material continue to grapple with issues, such as long-term preservation of digital materials, but it is fair to say there exists a fairly comprehensive, distributed “digital library” for mathematics offering a much improved but not fundamentally different version of what existed in the time of printed books and journals.

The committee has thus taken the term library in its charge to mean a system that accumulates and shares knowledge, rather than the more traditional library that houses documents, either digital or physical. The committee’s focus has been on functionality that can meet the needs of mathematicians facing a rapidly expanding and diversifying knowledge base. The committee has largely ignored traditional issues of assembling and stewardship of those collections, which are being handled well, for the most part, by the existing distributed digital library.

The committee envisions its target digital library users to be work­ ing research mathematicians and advanced graduate students beginning their research careers throughout the world (hence the word global). The library discussed does not specifically target students below the advanced graduate student level or researchers outside of mathematics, although both sets would likely constitute some of the library’s user base. Having a clear understanding of the target user base directly impacts the types of content the library targets and the types of services it provides. The com­ mittee also believes that the disciplinary scope of the mathematics that this library could provide is best left undefined for now. Mathematics and the mathematical sciences have diffuse boundaries, and this committee takes no stance on where appropriate content lies. However, this is an issue that will have to be addressed by either a future management organization or the community of users.

Eveloping A 21St Century Mathematics Library

The committee believes that there is much room for innovation and progress in the mainstream mathematical information services. To deter­ mine which potential areas for innovation are of the most interest to the mathematics community, the committee held three meetings where it heard from outside presenters on issues relevant to mathematics (November 27- 28, 2012; February 19-20, 2013; and May 30-31, 2013—agendas for these meetings can be found in Appendix A) and two public data-gathering ses­ sion forums (MathOverflow1 and Math 2.02), and wrote a guest entry on Professor Terry Tao’s mathematics blog.3 The committee also referred to the information shared at the World Digital Mathematics Library workshop held by the International Mathematical Union (IMU) on June 1-3, 2012.4 The committee made an assessment of what computers can do today, what computers can help mathematicians to do, and how rapidly these capabilities are likely to grow, if provided with some ongoing focused re­ search funding. The committee’s consensus is that by some combination of machine learning methods and community-based editorial effort, a signifi­ cant portion of the information and knowledge in the global mathematical corpus could be made available to researchers as linked open data. Broadly defined, linked open data are structured data that are published in such a way that makes it easy to interlink them with other data, thereby making it possible to connect them with information from multiple sources. This connected data can provide a user with a more meaningful query of a sub­ ject by consolidating relevant information from a variety of places (e.g., in different research papers) and pulling out specific components that the user might be particularly interested in. The committee envisions that much of the existing mathematical information can be provided as linked open data through a central organizational entity—referred to in this report as the DML. It should be noted that linked open data are not the only way that this can be accomplished, but they are essentially today’s standard for ontologies and other important representations. The committee believes that the DML should make use of current best practices rather than trying to develop some other alternative, whenever possible.

1  I. Daubechies, “Math Annotate Platform?,” MathOverflow (question and answer site), February 18, 2013, http://mathoverflow.net/questions/122125/math-annotate-platform. 2  I. Daubechies, “Math Annotate Platform?,” Math2.0 (discussion forum), February 18, 2013, http://publishing.mathforge.org/discussion/163/.

3  I. Daubechies, “Planning for the World Digital Mathematical Library,” What’s New (blog by Terence Tao), daily archive for May 8, 2013, http://terrytao.wordpress.com/2013/05/08/. 4  Many of the materials presented at the International Mathematics Union’s DML work­ shop can be found at http://ada00.math.uni-bielefeld.de/mediawiki-1.18.1/index.php/, updated April 23, 2013.

Structure Of The Report

This report consists of five main chapters and several appendices. The rest of this chapter discusses previous digital mathematics library efforts, the universe of mathematical information, relevant conceptual tools, and current mathematical resources. Chapter 2 discusses what is missing from the mathematical information landscape and what gaps the DML would fill, and elaborates on the desired DML capabilities from a user’s perspec­ tive. This includes a discussion of what types of features would make the mathematical literature and current resource capability more meaningful to a mathe­matical researcher. Chapter 3 discusses some of the broad issues that the DML would face during development, including developing partner­ ships, managing large data sets, navigating open access, and planning for system and data maintenance. Chapter 4 provides a strategic plan for the development of the DML, including a discussion of fundamental principles, the constitution of a governing organization, steps toward initial develop­ ment, and resources that would be needed. Chapter 5 discusses some details of entity collections and technical considerations for the DML that will be needed to make the features and capabilities discussed in Chapter 2 a reality.

In preparing this report, the committee reviewed many existing digital resources for mathematics, as well as relevant initiatives in some other sci­ ences. A brief discussion of these tools is given in Appendix C.

Previous Digital Mathematics Library Efforts

The idea of a comprehensive digital mathematics library has been around for decades, and there have been several incarnations of the idea with different foci. The first step in this vision was retrospective digitization of the older parts of the literature that did not already exist in digital form, and this has largely been achieved (though the quality, and hence utility, of these converted materials varies widely, ranging from simple page scans to carefully proofread markups).

was funded by the National Science Foundation from 2003 to 2004 as a step “toward the establishment of a comprehensive, international, dis­ tributed collection of digital information and published knowledge in mathematics.”5 Its vision statement reads as follows: In light of mathematicians’ reliance on their discipline’s rich published heritage and the key role of mathematics in enabling other scientific disci­ gator, R.K. Dennis and J. Poland, co-principal investigators, http://www.library.cornell.edu/ dmlib/, last updated December 2, 2004.

Eveloping A 21St Century Mathematics Library

plines, the Digital Mathematics Library strives to make the entirety of past mathematics scholarship available online, at reasonable cost, in the form of an authoritative and enduring digital collection, developed and curated by a network of institutions.

A follow-up report from the International Mathematical Union (IMU, 2006) shared this vision of a distributed collection of past mathematical scholarship that served the needs of all science, and it encouraged math­ ematicians and publishers of mathematics to join together in implementing this vision. However, it was clear within a few years that this vision was not going to become a reality soon. As David Ruddy of Project Euclid wrote

(Ruddy, 2009):

The grand vision of a Digital Mathematics Library, coordinated by a group of institutions that establish policies and practices regarding digitization, management, access, and preservation, has not come to pass. The project encountered two related problems: it was overly ambitious, and the ap­ proach to realizing it confused local and community responsibilities. While the vision called for a network of distributed, interoperable repositories, the committee approached and planned the project with the goal of build­ ing a single, unified library.

At the time of this study, there has been some progress in this vision of a single, unified library in the form of the European Digital ­Mathematics ­Library (EuDML) project.6 The EuDML project, funded from 2010-2013 by the European Commission, created a network of 12 European repositories acquiring selected mathematical content for preservation and access and made progress in establishing a single distributed ­library with a collection of about 225,000 unique items, spanning 2.6 million pages. The EuDML succeeded in creating a unified metadata framework7—which includes items about a document such as the title, ­authors, abstract, comments, report number, category, journal reference, direct object identifier, Mathematics Subject Classification (MSC), and Asso­ciation for Computing Machinery (ACM) computing classification—that is shared by these repositories and providing a single point of access to publications in these repositories, albeit with limited rights to search the full text from some sources. Impressive as the EuDML is, when compared to the full size and scope of the universe of published mathematics (described in the next section), and given the 6  T. Bouche, Université de Grenoble, “From EuDML to WDML: Next Steps,” Presentation to the committee on November 27, 2012.

7  European Digital Mathematics Library, “Appendix, EuDML Metadata Schema (Final)/ Tagging Best Practices,” in EuDML Metadata Schema Specification (v2.0-final), https://­project. eudml.org/sites/default/files/d36-appendix_uncropped.pdf, accessed January 16, 2014.

Ntroduction

essen­tial requirement to integrate with copyrighted materials and the clear desirability and cost-effectiveness of leveraging existing repositories and services, the EuDML experience only emphasizes the difficulties inherent in aiming for a single, centrally managed and truly comprehensive collection of digitized mathematics as the cornerstone for a comprehensive DML. With the advent of recent advances in technology and the advantage of experience gained on EuDML and other projects, the study committee concluded that a more effective approach going forward would be to partner with exist­ ing content providers and focus instead on the innovations and elements of shared infrastructure and knowledge management that are not being adequately addressed by other entities (i.e., rather than on central harvest­ ing and aggregation of primary content). The committee believes that this vision is consistent with the original vision of the EuDML, although it was not realized by that project.

Another example of an online resource that helps users connect with knowledge is the National Science Digital Library (NSDL).8 NSDL is an on­ line educational resource for teaching and learning, with current emphasis on the sciences, technology, engineering, and mathematics. NSDL does not hold content directly—instead, it provides structured metadata about Web- based educational resources held on other sites by providers who contribute this metadata to NSDL for organized search and open access to educational resources via NSDL.org and its services.

A discussion of many other efforts and current digital resources can be found in Appendix C. The Alfred P. Sloan Foundation supported a World Digital Mathe­ matics Library workshop in June 2012,9 which was planned by the IMU’s Committee on Electronic Information and Communication. This workshop provided a wealth of information to the committee on the current state of the art and research efforts aimed at making the World Digital Mathe­matics Library a reality.

Much of the straightforward work of assembling digital mathematics libraries has been done (e.g., digitizing material, aggregating it into small to medium-sized collections). The difficulties that the EuDML faced in creat­ ing a single large aggregation of mathematics literature and the difficulty of other World Digital Mathematics Library efforts in gaining community support indicates that these challenges are unlikely to be overcome soon.

The committee notes that there has been sizable ongoing investment from publishers (both commercial and noncommercial) to retrospectively digi­ 8  National Science Digital Library, http://nsdl.org/, accessed January 16, 2014.

9  International Mathematics Union, “The Future World Heritage Digital Mathematics Library: Plans and Prospects,” updated April 23, 2013, http://ada00.math.uni-bielefeld.de/ mediawiki-1.18.1/index.php/Main_Page.

Eveloping A 21St Century Mathematics Library

tize historical runs of their copyrighted journals and also, in many cases, even earlier historical materials that are now out of copyright, in order to capture comprehensive representations of their journals. However, broad services such as Google Scholar now provide much of the functionality that many of these specialized efforts had hoped to achieve in building compre­ hensive and coherent collections of the mathematical literature. Such ser­ vices achieve this functionality by searching across a range of repositories, rather than trying to collect all of the material in one (or a very few) reposi­ tories. In the committee’s view, efforts to build centralized comprehensive resources are reaching a point of diminishing returns.

Finding: The construction of mathematical libraries through centralized aggregation of resources has reached a point of diminishing returns, particularly given that much of this construction has been coupled with retrospective digitization efforts.

While there is still a substantial amount of historical (mostly out of copyright) mathematical literature that would benefit from retrospective digitization, or higher quality digitization than has currently been done, the committee does not believe that there is justification for a major new program and investment in this area. In particular, although there is value in modest, sustained investment in existing efforts, these will make only incremental contributions. While the fundamental importance of the heri­ tage literature remains, its size, as a fraction of the overall mathematics literature, is diminishing steadily. No amount of additional retrospective digitization will result in a fundamental change in the way that the math­ ematical literature can be used in new ways or evolved to meet new research needs. Moreover, while the historical (e.g., out of copyright) segments of the mathematical literature are valuable, any genuinely meaningful large- scale change in accessing the mathematical literature and knowledge base must encompass not only heritage but also current literature. Thus, the committee believes that a very different set of investments (as described in this report) is where the transformative opportunities await.

The next section provides some more detailed information on the exist­ ing landscape of mathematical literature and how much has been digitized.

The Universe Of Published Mathematical Information

Mathematics shares more with the arts than the sciences, in that its primary data are human creations, perhaps representations of ideas in a platonic realm, rather than data derived by observation or measurement of the physical universe. Mathematical information is primarily mined from its own literature or derived by computation. This section describes the state of

Ntroduction

mathematical publishing and the world of mathematical objects that exist within the publications.

Igital Mathematical Publications

Most of the mathematics literature of the 20th century is now available digitally. Through the Jahrbuch Electronic Research Archive for Mathemat­ ics10 project and the independent efforts of publishers and others, much of the most important mathematical research of the last half of the 19th century also has been digitized. Appendix C provides an overview of the many sources for digitized mathematical source material, including reposi­ tories and many other types of sources, whether freely accessible or behind paywalls (and thus only accessible to subscribers). A large part of the math­ ematics literature in electronic form consists of papers written in the past 20 years. This portion of the literature is searchable and navigable by any user of a library with access to the main subscription services controlled by libraries and publishers.

In addition, a considerable body of the heritage literature in mathe­ matics has been digitized over the past 15 years. The most comprehensive listing of the retro-digitized mathematics literature is Ulf Rehmann’s list of Retro­digitized Mathematics Journals and Monographs,11 which is a list of titles of serials and books that have been digitized without meta­ data.12 Much of this metadata has found its way into indexes maintained by Google, ­MathSciNet, and Zentralblatt (zbMATH).13 The digital corpus of mathematics literature is extensive. The ­MathSciNet14 database includes approximately 2.9 million publica­ tions from 1940 to the present, with direct links to 1.7 million of them.

­MathSciNet currently indexes more than 2,000 journal/serial titles and contains about 100,000 books (post 1960). Of the items currently avail­ able on ­MathSciNet, 2.6 million of them are from the 1970s or later, and 1.7 million are from 1990 onward. The American Mathematical ­Society has kept track of new journal titles in the field since 1997, and there has been an average growth of about 40 new journal titles per year in mathematics.

10  The Jahrbuch Project, Electronic Research Archive for Mathematics, last modified Octo­ ber 31, 2006, http://www.emis.de/projects/JFM/. 11  DML: Digital Mathematics Library, http://www.mathematik.uni-bielefeld.de/~rehmann/ DML/dml_links.html, accessed January 16, 2014.

12  Metadata are broadly defined as data about data. In the case of a typical mathematics journal digital publication, metadata may include information such as author, journal name and volume, date of publication, time of file creation, size of file.

13  zbMATH, http://zbmath.org/, accessed January 16, 2014. January 16, 2014.

Eveloping A 21St Century Mathematics Library

zbMATH (1931-­present) contains more than 3 million publications and currently ­indexes approximately 3,500 journals. The annual production of mathe­matics papers is more difficult to quantify. There has been a steady increase in the number of math papers added to arXiv15 over the past 5 years (shown in Table 1-1), although it is not clear from these data if this shows an increase in mathematics publications or an increase in mathemati­ cians’ willingness to post their papers. Annual entries on ­MathSciNet and the number of mathematics papers listed in Web of ­Science16 have both remained relatively constant around 90,000 and 20,000, respectively (see Tables 1-2 and 1-3).

Components of the digitized corpus of mathematics are increasingly included in a variety of stable, well-curated repositories, although access to much of this corpus remains limited by copyright or other intellectual rights restrictions. For example, in terms of retrospectively digitized works cataloged under the subject heading (or subheading) of “mathematics,” the HathiTrust Digital Library17 includes approximately 40,000 biblio­ graphically distinct resources.18 Of these, only 6,800 were digitized from public-domain works; the rest were digitized from copyrighted originals.

These numbers are a mix of monograph titles and serial titles (a serial title in HathiTrust typically encompasses a complete run of a journal, edited series, or conference publication series). Each serial run could be expected to include tens or even hundreds of issues, with each issue containing at least several articles or papers. In terms of pages, using the HathiTrust repository-wide ratio of pages per bibliographic resource to estimate, this translates to a rough estimate of 25.5 million pages of retrospectively digi­ tized mathematics in HathiTrust with approximately 17 percent (6,800 out of 40,000) digitized from public-domain sources.

The basic trends seem clear: more and more of the corpus of math­ ematical literature will be in digital form, including some with high-quality markup, specifically those items that are “born” digital or retro-digitized to be in a machine readable format and that use typesetting such as LaTeX or MathML (as opposed to page images of publications). As mentioned before, the fraction of the overall corpus that is pre-1970 is rapidly dimin­ ishing due to the relative explosion in the annual rates of publication in recent decades (however, this should in no way be seen as diminishing the fundamental importance of heritage literature).

15  arXiv, http://arxiv.org/, accessed January 16, 2014. 16  Thomson Reuters, “Web of Science Core Collection,” http://thomsonreuters.com/web-of- science/, accessed January 16, 2014.

17  HathiTrust Digital Library, http://www.hathitrust.org/, accessed January 16, 2014. 18  Current as of September 2013.

Ntroduction

TABLE 1-3  Mathematics Papers Listed in Web of Science Annually

,760

SOURCE: Thomson Reuters, “Web of Science Core Collection,” http://thomsonreuters.com/ web-of-science/, accessed January 16, 2014. TABLE 1-2  Number of Articles in Research Journals in MathSciNet

,191

NOTE: A steady growth of about 3 percent per year is seen. accessed January 16, 2014. TABLE 1-1  Number of Mathematics Papers Added to arXiv Annually

,176

SOURCE: arXiv, http://arxiv.org/, accessed January 16, 2014.

Objects In The Mathematical Literature

Information found in the mathematical literature is diverse but largely

Bibliographic Information, Such As

a. Documents (e.g., articles, books, proceedings, talks, diagrams,

Homepages, Blogs, Videos);

b. People (e.g., authors, editors, referees, reviewers); c. Events (e.g., discoveries, publications, conferences, talks, births,

Deaths, Degrees, Awards);

d. Organizations (e.g., universities, publishers, journals, libraries,

Service Providers);

e. Subjects (e.g., major branches of mathematics—algebra, ­geometry, analysis, topology, probability, statistics—as well as their intersections and interactions and their various sub- branches, down to even finer topics and including ubiquitous

Mathematical Terms Like “Number,” “Set”)

2. Mathematical concepts (e.g., axioms, definitions, theorems, proofs, formulas, equations, numbers, sets, functions) and objects (e.g., groups, rings).

Collecting and aggregating mathematical bibliographic information has been the path many digital libraries and digital resources have taken in the past (Chapter 2 and Appendix C discuss many of these efforts to date). While there are many challenges in collecting this information, the even more difficult work lies in collecting mathematical concepts, which lack the standardization that most bibliographic information has acquired.

However, an ability to explore these mathematical objects within the litera­ ture offers the potential to uncover currently under-explored connections in mathematics.

The recent National Research Council report The Mathematical Sci­ ences in 2025 (NRC, 2013) discusses the importance of mathematical struc­ tures, which are part of the larger mathematical concepts described above: A mathematical structure is a mental construct that satisfies a collection of explicit formal rules on which mathematical reasoning can be car­ ried out. . What is remarkable is how many interesting mathematical structures there are, how diverse are their characteristics, and how many of them turn out to be important in understanding the real world, often in unanticipated ways. Indeed, one of the reasons for the limitless pos­ sibilities of the mathematical sciences is the vast realm of possibilities for mathematical structures. A striking feature of mathematical structures is their hierarchical nature—it is possible to use existing mathematical

Ntroduction

structures as a foundation on which to build new mathematical structures . . Mathematical structures provide a unifying thread weaving through and uniting the mathematical sciences. (pp. 29-30) Given the size, diversity, and inherent nature of mathematics informa­ tion in categories 1 and 2 above, it is clearly not sufficient to simply pro­ vide undifferentiated access to the universe of mathematics monographs, journal articles, and conference papers. Instead, the online research litera­ ture of mathematics must be organized into a well-structured network of resources linked together based on a variety of attributes—bibliographic and topical, of course, but also linked in a highly granular fashion on com­ monalities of mathematical structures and the shared use of mathematical objects, reasoning, and methodologies. The committee believes that the greatest potential for the DML lies in providing mathematicians access to a well-structured network of information and building services that both enhance and utilize this data. In the context of today’s Web environment, a well-structured network implies adherence to the Semantic Web19 and linked open data principles and to community-endorsed standards and best practices. While the foundation for such a well-structured network of digi­ tal research mathematics exists in established repositories and component digital libraries, the underlying thesauri and ontologies of mathematical objects do not yet exist (or have not yet been given permanence and formal identity), and the agreements on best practices for interoperability and the implementation of linked open data principles in the context of research mathematics repositories have not yet been reached.

Onceptual Tools

General conceptual tools that are used to structure, organize, represent, and share knowledge include the closely related ideas of ontologies, tax­ onomies, and vocabularies. There is considerable debate about the precise definitions and differences among these tools, although ontologies (most commonly viewed as a tool for defining some classes of objects—the attri­ butes that these objects may have and the way in which these objects may be related to each other) are usually seen as the most general formulation (Gruber, 2009). Taxonomies are specific, usually hierarchical, collections of terms that can be used to describe or classify objects in some contexts— examples of these include subject headings or the naming schemes used in biological systematics. “Controlled” vocabularies are collections of values that can be used to populate specific instances of object attributes within an ontology; in a certain sense, they are equivalent to taxonomies in that 19  W3C, “Semantic Web,” http://www.w3.org/standards/semanticweb/.

Eveloping A 21St Century Mathematics Library

they can be used to classify. However, controlled vocabularies are often “flat,” without other internal structure among the possible values, whereas taxonomies commonly include very rich internal hierarchical structure.

Ontologies, vocabularies, and taxonomies work together. As a simple ex­ ample, a part of an ontology might define a specific class of objects called documents; each of these has attributes that include subjects and languages.

One might have a list of possible language values (a controlled vocabulary) associated with the ontology and also a tree structure of subject headings (a taxonomy, though it could also viewed as a simple vocabulary).

For instance, within the mathematical sciences, the widely accepted Bibliographic Ontology20 provides a fairly adequate accounting of the many common relations between objects in categories 1a through 1e listed above.

The BibTeX21 schema that describes the structure of BibTeX ­records defines a similar ontology. The Citation Typing Ontology (CiTO)22 is an ontology for description of the citation relation between documents. The Mathematics Subject Classification (MSC2010)23 provides a very well thought out, largely hierarchical taxonomy for the classification of mathematical documents by subject, and thence for the subjects themselves. OpenMath,24 discussed fur­ ther in Chapter 5, offers a potential standard for representing the semantics of mathematical objects that is very relevant to the DML’s goals.

The application of such ontologies to a mathematical objects data set can create graphical structures of information that can provide new in­ sights. For instance, citations generate a citation graph, and collaborations generate a collaboration graph. Such graphical structures are commonly embedded in the structure of hyperlinked webpages, thereby connecting literature that was not obviously related otherwise.

Development of new ontologies is a complex process requiring a high level of community effort for consensus, even for limited sets of relations. The committee expects that when communities start to curate various digital collections of records of mathematical entities, there will be some “bottom up” development of at least minimal ontologies for these entities, as has already occurred with MSC2010 and OpenMath. The structure of these ontologies will be reflected in the necessary schemas25 for description of the objects they involve, and the graphical relations induced by these 20  The Bibliographic Ontology, “Bibliographic Ontology Specification,” dated November 4, 2009, http://bibliontology.com/specification.

21  BibTeX, http://www.bibtex.org/, accessed January 16, 2014. 22  CiTO, the Citation Typing Ontology, dated March 7, 2013, http://purl.org/spar/cito/. 23  Encoded by the Mathematics Subject Classification (MSC2010), American Mathematical 24  OpenMath Society, OpenMath, http://www.openmath.org/, accessed January 16, 2014.

25  A schema is broadly defined as a representation of a plan or theory in the form of an outline or model.

Ntroduction

ontologies will be of potentially great interest in the process of extracting information and knowledge from mathematical publications.

Urrent Mathematical Resources

The management of formal representations of mathematical concepts is known as mathematics knowledge management (Carette and Farmer, 2009). In this report, this issue is viewed more broadly as the management of mathematical information and concepts, both formal and informal, in­ cluding the bibliographic information and mathematical concepts categories of objects introduced in the previous section, only the latter of which can be usefully regarded as part of mathematics itself.

Bibliographic Resources In Mathematics

Several general bibliographic resources exist, and some of these are ­described in Appendix C. Among them, mathematicians typically use Google26 and Google Scholar27 most often, although CrossRef28 is “­under the hood” whenever a user navigates from one publisher’s site to ­another by a reference link. While many mathematicians heavily utilize these gen­ eral information services because of their power and ubiquity, some math­ ematicians prefer the discipline-specific abstracting and indexing services provided by MathSciNet29 and zbMath.30 This discipline-specific service preference is partly for historical reasons and partly because the focus and quality of metadata provided by these services in mathematics makes it ­easier to find publications of interest. Both services offer bibliographic ­entries in BibTeX,31 which is machine-readable and reusable, for prepara­ tion of reference lists for LaTeX32 documents, and, with more technical ­effort, for publication of online bibliographies in HTML33 or JSON.34 ­Using search engines with access to well-curated bibliographic metadata and full-text indexing is how most mathematicians find mathematical pri­ mary sources today.

26  Google, https://www.google.com/, accessed January 16, 2014. 27  Google Scholar, http://scholar.google.com/, accessed January 16, 2014. 28  CrossRef, http://www.crossref.org/, accessed January 16, 2014.

January 16, 2014. 30  zbMATH, http://www.zentralblatt-math.org/zmath/, accessed January 16, 2014. 31  BibTeX, http://www.bibtex.org/, accessed January 16, 2014.

32  LaTeX—A document preparation system, last revised January 10, 2010, http://www. latex-project.org/. 33  “HTML,” Wikipedia, http://en.wikipedia.org/wiki/HTML, accessed January 16, 2014.

34  “Introducing JSON,” http://www.json.org/, accessed January 16, 2014.

Eveloping A 21St Century Mathematics Library

Services such as MathSciNet, zbMATH, and Google Scholar provide complementary and somewhat overlapping services. One distinct difference is that MathSciNet is organized chronologically and referentially, while Google Scholar is based on “importance” as qualified by page ranks or some variant thereof. Both are important and are used in literature searches.

MathSciNet is great for tasks such as listing all articles by an author and listing all articles in a specific mathematical field, and it has high-quality metadata that are needed for many purposes. Its search capabilities are limited because it only searches over metadata. Google Scholar is often better for searches because it searches over full text, including reference lists, and has better ranking or returns for most purposes. One issue that some mathematicians have with Google Scholar is that it is not possible to limit searches to math or subfields of math. MathSciNet, zbMATH, and Google Scholar combined do a good job providing conventional discovery over the corpus of traditionally published mathematical literature, but no services currently provide a finer-grain search capability that allows a user to search for mathematical objects or ideas that cannot be easily defined by text search, such as an equation or the evolution of a specific notation.

Ideally, a mathematician should have the best of both capabilities through a single interface, but this is challenging because neither MathSciNet nor Google Scholar currently allow their data to be merged with the other’s.

Mathematicians also make extensive use of arXiv as a platform for sharing preprints and keeping up with current research developments. Mathematicians strongly support arXiv in part because the full text is largely indexed and exposed to the Web through search engines. How­ ever, arXiv items are not indexed through services such as MathSciNet or ­zbMATH, which would help connect these items to the rest of the literature. Search tools associated with distinct subsets of the literature, such as arXiv, publisher-based repositories, library catalogs, and academic institutional repositories provide overlapping access to the mathematical lit­ erature. ­Unfortunately, the present configuration of these discipline-specific tools does not provide a single information source where mathematicians can find and access information from diverse sources, and the more general information sources often lack the mathematical metadata and details that make mathematics literature easy to search and browse.

Combining data from multiple information resources (e.g., Google, MathSciNet, zbMATH) is complicated. Partnering organizations would have to allow their data to be collected, reused, or recombined on a large scale, which many services are hesitant to do. Even seemingly open re­ sources (such as arXiv) may have legal restrictions on outside data aggrega­ tion, depending on what is done with the data. This collaboration would have to be negotiated between potential partners with the goal of creating

Ntroduction

a unified view of the mathematics literature. Some approaches toward developing partnerships and relevant examples are discussed in Chapter 3. Given the central importance of bibliographic data searches and the repeated use of bibliographic information by researchers in preparation of research articles, it is essential for the DML to provide adequate biblio­ graphic support tools with access to the best available bibliographic data in mathematics and related fields. Ideally, it should support advanced biblio­ graphic data processing to detect and identify the structure of networks of papers, authors, topics, and the like. The foundations of such bibliographic data processing are provided by the larger existing bibliographic services in mathematics and beyond, especially MathSciNet, zbMATH, and Google Scholar, which are the most commonly used by mathematicians. At ­present, none of these services provides an application programming interface (API) for programmatic access, and none of them allow their data to be down­ loaded in bulk, except with severe restrictions on what can be done with it. To provide the greatest benefit to users of a DML, that would have to change. Both EuDML and Microsoft Academic Search provide steps in a positive direction with more or less open bibliographic data stores with an API for access, which allows tools and services to be built over the corpus.

To seriously engage the mathematics world with a digital library system, extensive coverage of mathematical information is essential. The commit­ tee considered whether the DML could initially focus on out-of-copyright material, but it concluded that there would not be community support or interest in this approach because it is too limited. On the other hand, much progress has been made in digitizing heritage content, and it is essential that this be integrated with the rest of the math literature base.

Specialized Mathematical Information Resources

General bibliographic services provide limited support for navigating and searching mathematical literature below the top five bibliographic classes (documents, people, events, organizations, subjects) discussed above.

Beyond these five universal classes, information storage and retrieval for math-specific entities is fragmented and typically does not have links or

References To The Main Indexing Services.35

Research mathematics literature includes a diverse range of special ­objects—e.g., theorems, lemmas, functions, sequences—that are not repre­ sented adequately, or sometimes at all, in full-text indexing and ­article-level 35  MathSciNet and zbMATH share the MSC2010 subject classification, which provides some basic filtering of bibliographic data by subject. ArXiv uses a coarser classification, which is however easily mapped to sets of top-level MSC 2010 categories.

Eveloping A 21St Century Mathematics Library

expensive and difficult to recognize through machine-based methods alone. Ontologies of objects—such as reference volumes that enumerate classes of functions, sequences, and other objects—have been developed and curated by mathematicians for centuries. These resources include mathematical handbooks, some of the most famous being the following:

•

Abramowitz and Stegun (1972) and the subsequent Digital Library

•

The Princeton Companion to Mathematics (Gowers et al., 2008). There are also examples of more recently developed resources that provide collections of some mathematical objects, including the following:

•

Propositions: Wikipedia’s List of Theorems,38 Mizar39;

•

Proofs: Proofs from the Book (Aigner and Ziegler, 2010), Mizar,

•

Numbers: A Dictionary of Real Numbers (Borwein and Borwein,

•

Sequences: The On-Line Encyclopedia of Integer Sequences (OEIS)42;

•

Functions: Digital Library of Mathematical Functions,43 Wolfram

•

Groups, rings, and fields: Wikipedia’s List of Simple Lie Groups,46 Wikipedia’s List of Finite Simple Groups,47 Centre for Inter­ 36  NIST Digital Library of Mathematical Functions, 2013, http://dlmf.nist.gov/.

37  “Bateman Manuscript Project,” Wikipedia, last modified July 24, 2013, http://en. wikipedia.org/wiki/Bateman_Manuscript_Project. 38  “List of Theorems,” Wikipedia, last modified December 9, 2013, http://en.wikipedia.org/ 39  Mizar Home Page, last modified January 8, 2014, http://mizar.org/.

40  The Coq Proof Assistant, http://coq.inria.fr/, accessed January 16, 2014. 41  “Category:Proof assistants,” Wikipedia, last modified September 21, 2011, http://en. wikipedia.org/wiki/Category:Proof_assistants.

42  On-Line Encyclopedia of Integer Sequences® (OEIS®) Wiki, https://oeis.org/wiki/­Welcome, accessed January 16, 2014. 43  NIST Digital Library of Mathematical Functions, 2013, http://dlmf.nist.gov/.

44  Wolfram MathWorld, http://mathworld.wolfram.com/, accessed January 16, 2014. 45  Wolfram Research, Inc., The Wolfram Functions Site, http://functions.wolfram.com/, accessed January 16, 2014.

46  “List of Simple Lie Groups,” Wikipedia, last modified March 30, 2013, http://en.wikipedia. org/wiki/List_of_simple_Lie_groups. 47  “List of finite simple groups,” Wikipedia, last modified December 18, 2013, http:// en.wikipedia.org/wiki/List_of_finite_simple_groups.

Ntroduction

disciplinary Research in Computational Algebra: Finite Fields,48

•

Inequalities: Wikipedia’s List of Inequalities,51 DasGupta (2008);

•

Formulas: Springer LaTeX Search,52 Hijikata et al. (2009), Kohl­ hase et al. (2012). From a review of these lists, as well as the resources discussed in Appen­dix C, it is clear that authors and editors continue to be motivated to create and publish lists of various kinds of mathematical objects. Some of these lists, especially ones like tables of integrals and lists of sequences, pro­ vide very useful tools for mathematicians and other users of mathe­matics, especially when combined with computational resources. Wikipedia cur­ rently plays a key role in supporting distributed creation and maintenance of numerous lists of serious interest to mathematicians.

Lists and tables have been an essential part of mathematical research throughout history, and the vast majority of working mathematicians have made use of appropriate tables (or, more recently, the equivalent numerical or symbolic software) in the course of their research. The most basic are numerical tables (e.g., values of logarithms, trigonometric functions, vari­ ous special functions, zeros of the zeta function, integer sequences). More sophisticated are lists of mathematical objects (e.g., indefinite and definite integrals, finite simple groups, Fourier transforms, partial differential equa­ tions and their solutions). Or, at even a higher level, lists of theorems, concepts, etc.

At their most basic, tables provide a simple mechanism for speeding up research. Once one identifies that an object under investigation appears in a table, one can make use of prior knowledge about said object, thereby facilitating either applications or new advances in theory. Compiling a table is an important research contribution in its own right, helping codify the knowledge in a field, point out gaps therein, and inspire new research to fill in and extend what is known. Scanning a table often enables one to spot 48  CIRCA, “GAP Instructional Material,” January 2003, http://www-circa.mcs.st-and.ac.uk/ gapfinite.php.

49  Sage Development Team, “Finite Fields,” http://www.sagemath.org/doc/reference/rings_ standard/sage/rings/finite_rings/constructor.html, accessed January 16, 2014. 50  T. Piezas III, A Collection of Algebraic Identities, https://sites.google.com/site/tpiezas/ Home/, accessed January 16, 2014.

51  “List of Inequalities,” Wikipedia, last modified November 28, 2013, http://en.wikipedia. org/wiki/List_of_inequalities. 52  Springer, LaTeX Search, http://www.latexsearch.com/, accessed January 16, 2014.

Eveloping A 21St Century Mathematics Library

otherwise obscure patterns, leading to new theorems and new directions of research. Sara Billey and Bridget Tenner wrote that a database for mathemati­ cal theorems would “enhance experimental mathematics, help researchers make unexpected connections between areas of mathematics, and even im­ prove the refereeing process” (Billey and Tenner, 2013, p. 1093). Extensive lists could also enhance search and retrieval of mathematical information and allow for connections to be made between mathematical topics and objects.

Currently, there are no satisfactory indexes of many mathematical objects, including symbols and their uses, formulas, equations, theorems, and proofs, and systematically labeling them is challenging and, as of yet, unsolved. In many fields where there are more specialized objects (such as groups, rings, fields), there are community efforts to index these, but they are typically not machine-readable, reusable, or easily integrated with other tools and are often lacking editorial efforts. So, the issue is how to identify existing lists that are useful and valuable and provide some central guidance for further development and maintenance of such lists.

Chapter 2 of this report discusses some of the user features that could advance mathematics research by increasing connections, and Chapter 5 discusses what collections of entity lists could start making these features and this connectivity a reality.

Authors:

Peder EZ Larson 1, 2,* , Jenna ML Bernard1, James A Bankson 3, Nikolaj Bøgh 4, Robert A Bok1, Albert P. Chen 5, Charles H Cunningham 6,7, Jeremy Gordon1, Jan-Bernd Hövener 8, Christoffer Laustsen 4, Dirk Mayer 9,10, Mary A McLean11 12, Franz Schilling13, James Slater1, Jean-Luc Vanderheyden5, 14, Cornelius von Morze 15, Daniel B Vigneron1, 2, Duan Xu1, 2, and the HP 13C

94143, Usa.

Denmark. 5 GE Healthcare, Menlo Park, California, USA. 6 Physical Sciences, Sunnybrook Research Institute, Toronto, Ontario, Canada.

8 Section Biomedical Imaging, Molecular Imaging North Competence Center (MOIN CC), Medicine, Baltimore, MD, USA. Cambridge, United Kingdom.

14Jlvmi Consulting Llc, Dousman, Wi, Usa

#See Acknowledgements for a list of all HP 13C MRI Consensus Group Members This work was supported by the ISMRM Hyperpolarized Media MR Study Group, the ISMRM Hyperpolarization Methods & Equipment Study Group, and the Hyperpolarized MRI Technology Resource Center (NIH/NIBIB grant P41EB013598).

Abstract

MRI with hyperpolarized (HP) 13C agents, also known as HP 13C MRI, can measure processes such as localized metabolism that is altered in numerous cancers, liver, heart, kidney diseases, and more. It has been translated into human studies during the past 10 years, with recent rapid growth in studies largely based on increasing availability of hyperpolarized agent preparation methods suitable for use in humans. This paper aims to capture the current successful practices for HP MRI human studies with [1-13C]pyruvate - by far the most commonly used agent, which sits at a key metabolic junction in glycolysis. The paper is divided into four major topic areas: (1) HP 13C-pyruvate preparation, (2) MRI system setup and calibrations, (3) data acquisition and image reconstruction, and (4) data analysis and quantification. In each area, we identified the key components for a successful study, summarized both published studies and current practices, and discuss evidence gaps, strengths, and limitations. This paper is the output of the “HP 13C MRI Consensus Group” as well as the ISMRM Hyperpolarized Media MR and Hyperpolarized Methods & Equipment study groups. It further aims to provide a comprehensive reference for future consensus building as the field continues to advance human studies with this metabolic imaging modality.

Keywords: Hyperpolarized MRI, metabolic imaging, carbon-13, pyruvate, dissolution dynamic

Introduction

MRI with hyperpolarized 13C agents, also known as hyperpolarized (HP) 13C MRI, has shown great potential as a novel imaging modality, particularly for its ability to probe metabolic processes in real time. The first human studies with HP [1-13C]pyruvate were performed in 2011 in prostate cancer patients (1).

Since then, there have been over 60 papers published with imaging results of human subjects from 13 different sites, with applications including prostate cancer, brain tumors, breast cancer, kidney cancer, pancreatic cancer, metastatic disease, liver disease, ischemic heart disease, diabetes and cardiomyopathies. The vast majority of these studies used [1-13C]pyruvate (1–63), where [2-13C]pyruvate (64) and 13C-urea (56) have been demonstrated too.

As clinical HP 13C MRI advances, there is a growing need to build consensus for best practices, which are critical for comparing data across sites, performing multi-site trials,deploying methods to new sites, partnering with vendors, and potentially for obtaining broader regulatory approvals.

In March 2022, we initiated an effort to build consensus within the HP 13C MRI community with this opportunity in mind, and it was greeted with strong enthusiasm. The “HP 13C MRI Consensus Group”, containing over 55 members from 27 sites, identified the area of greatest need and opportunity for consensus building to be HP [1-13C]pyruvate human

●

Pyruvate is the most mature and widely used HP agent and has the most significant translational evidence emphasizing the potential clinical impact.

●

Clinical trials, particularly multi-site trials, have the strongest need for consensus methods to ensure that data can be combined across sites. This work is a Position Paper for which the goal is to describe current successful practices and study methods for HP [1-13C]pyruvate human studies along with justification to support those practices. This is divided into four major topic areas: (1) HP 13C-pyruvate preparation, (2) MRI system setup and calibrations, (3) data acquisition and image reconstruction, and (4) data analysis and quantification (Fig. 1). The current successful practices and study methods include a literature review of published peer-reviewed journal papers showing human HP [1-13C]pyruvate study data, up to September 2022 (1–63), as well as new unpublished information from surveys of HP 13C study sites. Based on this information, we also highlight the evidence gaps, strengths, and limitations of current practices which are summarized at the end of each section.

Figure 1: Illustration of the HP 13C MRI human study process, including the 4 major areas covered in this paper: Hyperpolarized 13C-pyruvate preparation, MRI system setup and calibration, Acquisition and Reconstruction, and Data Analysis and Quantification.

Figure 2: Anatomical targets of HP [1-13C]pyruvate MRI human studies published up to September 2022.

Hyperpolarized 13C-Pyruvate Preparation

This section covers the processes for creating the HP agent, 13C pyruvate, and will include many aspects and considerations that are needed to safely and effectively prepare doses for metabolic imaging studies in human subjects. These include material, personnel, equipment and facility, fluid path preparation, quality control, and release.

It is helpful to understand that the specifications of a dose of 13C pyruvate suitable for in vivo MR HP metabolic imaging were shaped in part by early preclinical studies performed by GE HealthCare summarized in Ref. (65). In short, the safety of the two novel drug components, 13C pyruvate and the electron paramagnetic agent (EPA) AH111501, were demonstrated in those studies. The more precise formulation of the dose suitable for human use was then determined from clinical studies (66) that included two Phase 1 clinical trials in young and elderly healthy volunteers without hyperpolarization of the 13C nuclei and another Phase 1/2a dose escalation and imaging feasibility study with HP 13C pyruvate in 31 prostate cancer patients at the With the exception of the first HP 13C imaging clinical trial, which utilized a prototype device in a cleanroom (1), all HP 13C studies performed in humans to date have utilized the SPINlab polarizer (manufactured by GE HealthCare). Consequently all doses of the HP 13C pyruvate delivered by SPINlab have been produced using the “SPINlab Pharmacy Kit” that serves as the container-closure system for the various drug components (13C pyruvic acid and EPA mixture, dissolution medium, and neutralization and dilution medium) during sample polarization, dissolution and quality control (QC) processes. Thus many aspects of the HP sample preparation considerations discussed below are related to the SPINlab instrument and the consumables designed to be used with it (67).

General Considerations

While more than 860 patients or healthy subjects having been injected with HP 13C pyruvate as of January 2022 without reports of any serious adverse events (68), HP 13C pyruvate injection remains an investigational MR contrast agent and can only be administered by those with Investigational New Drug (IND) exemption from the Food and Drug Administration (FDA) in the USA, a Clinical Trial Application (CTA) in Canada, approval from National Research Ethics Committee Services in the UK, or approval from the relevant local regulatory body. Thus, methods and processes involved to produce a dose should have patient safety as the first priority. Since utilizing dissolution dynamic nuclear polarization (dissolution-DNP) for human use is still a relatively new development, there are no existing published regulatory guidelines specifically for this method.

There are two major production styles that determine how various sites approach the agent preparation. In the US, the most common approach is to rely on a sterilizing filter (“Terminal Sterilization”) to ensure sterility of the final product, akin to PET tracer production, where a starting molecule with a radioisotope is processed using various other ingredients to make the final, desired and injectable contrast agent within a necessarily short amount of time (69). For these sites, sterilization of the components and accessories upstream of this filter are not required, although many of them were manufactured and tested following Good Manufacturing Practice (GMP) or Good Laboratory Practice (GLP) requirements. The filling process is usually performed under an ISO 5 laminar flow hood, but a clean room or an isolator is not required.

This approach is typically accompanied by testing the integrity of the sterilizing filter prior to release of the dose for injection. Typically, post release endotoxin and sterility tests are performed using an aliquot reserved from each released dose.

In the UK and EU, the most common approach is to more-closely follow sterile pharmaceutical compounding guidelines (70), where all components and ingredients are required to be sterile or manufactured under GMP guidelines and are assembled and filled within a clean room environment or an isolator system (“Sterile Preparation”). Typically a batch of Pharmacy Kits for HP 13C pyruvate injection are prepared together. The sterility of the final dose is also ensured by batch validation testing, in addition to the sterility of the ingredients and the sterile compounding process. The endotoxin and sterility testing are performed for the process validation but are not performed for each injected dose.

Some institutions fill and assemble the Pharmacy Kit required for a specific study on the same day or the day prior to polarization, dissolution, and patient administration, but others have also demonstrated the feasibility of preparing a batch of kits, keeping them in a -20ºC freezer and using them over a period of a few months.

Beyond the obvious requirements that the process and the facility has to ultimately produce a dose that is safe to inject into a human, regulatory authorities will also focus on the question “Are you in control of your processes?”. To be in control of your process requires an in-depth and broad understanding of all processes involved in pre, post, and during the production process.

Personnel

It is typical and may be required to have licensed personnel involved in the production process depending on local regulations.Typically a pharmacist, radiopharmacist or other similarly qualified person (QP), in charge of the facility where the Pharmacy Kit filling and preparation is taking place, is responsible for the overall process and the release of the injectable dose.

Qualified cleanroom technicians are often involved in the Pharmacy Kit filling under the supervision of the pharmacist or QP. As is required for pharmaceutical compounding or PET tracer production, training requirements and training records for all personnel need to be maintained and available for audit by the FDA or equivalent.

Equipment And Facility

The facility and all equipment need to have standard operating procedures (SOPs) that describe how equipment is used, maintained, and calibrated to comply with relevant legislation. Currently, almost all the filling of the Pharmacy Kit takes place within a compounding laminar flow hood or isolator (typically ISO 5). At some sites, the filling is conducted within a cleanroom, while at others, it is conducted in a dedicated non-cleanroom space, reflecting differences in cleanroom approach and specifications between regulators worldwide (71). Some equipment or facilities, such as the compounding hood or cleanroom, may require external certified laboratories for testing.

Material Handling

Material handling guidelines (69,70) require SOPs detailing a system to track all of the materials involved in the HP production process for a particular patient dose, similar to current good manufacturing practice (cGMP) requirements for material handling for drug compounding. This includes acceptance standards, storage conditions, amount used in the patient dose for each ingredient and materials used in the assembly of the fluid path and Pharmacy Kit. Currently some users choose to open and inspect and sometimes modify the Pharmacy Kits upon arrival, but some users keep them in the sealed packaging until they are required for dose preparation.

Pharmacy Kit Filling And Assembling

As required by an IND or its equivalent, the preparation of the doses of HP 13C agent are detailed in the Chemistry, Manufacturing, and Control (CMC) section of an applicable regulatory submission; an example of this has been made available (72). It describes the processes of filling the Pharmacy Kit with the different components that make up the final drug product, and of assembling the final kit for either storage or immediate use in the polarizer. Special attention should be given to the laser welding process in order to satisfy installation qualification (IQ) and operational qualification (OQ). Typically, the final developed process is validated by process qualification (PQ) runs, during which 3 or more Pharmacy Kits are filled and used and the final HP 13C products are tested for endotoxin and sterility and to confirm that they meet the dose specifications for injections (usually including pyruvate concentration, residual EPA concentration, pH, liquid state polarization level and dose temperature). The data from 3 consecutive PQ runs are submitted as part of the IND submission (or its equivalent), and are often also reviewed by the Institutional Review Board (IRB) where the studies are conducted.

Quality Control And Dose Release

The quality control (QC) and dose release can be separated into two aspects: one is the QC and release of the filled Pharmacy Kit, and second is the QC and release of the HP 13C agent for injection, after polarization and dissolution. For institutions filling a batch of kits and storing them to use over a period of time, typically the batch can be released based on initial validation, environmental monitoring data from the day of kit production, and if filters are used during preparation of any of the components, filter integrity testing. But in some cases one or more kits are used for validation before the batch of kits are released for future use. For institutions that fill only the kits required for specific studies shortly before the experiment, the filled kits often do not go through separate release tests before they are used.

The quality control of the HP 13C pyruvate solution post dissolution is primarily performed to ensure that the agent meets the dose specifications (Table 1) before it is administered to the subject. These specifications target both safety (pH, residual EPA, temperature) and efficacy (pyruvate concentration, polarization, volume). Typically, the pyruvate concentration, residual EPA concentration, pH, dose temperature, dose volume, and liquid state polarization are measured by the QC accessory associated with the SPINlab polarizer. Some users perform a secondary measurement for one of the parameters, such as pH, using a different instrument or pH paper. For sites that do not go through a separate release testing process for batch filled kits, the integrity of the sterilization assurance filter, a part of the Pharmacy Kit, is typically tested as a part of the dose release. It is also common for these users to preserve an aliquot of the final HP 13C pyruvate solution for post-release endotoxin and sterility testing. This testing cannot be completed fast enough to test an individual dose prior to injection, but this is why other processes such as PQ runs and validation testing are done to minimize the chance a subject could be injected with a contaminated dose.

The Final Dose Release And Injection

should be done under the supervision of a licensed professional, based on local regulations.

Some Key Challenges

Many of the challenges associated with HP 13C pyruvate preparation can be attributed to the conditions required for the dissolution-DNP method of high magnetic field (~3-7 T) and very low temperature (~1 K) during polarization, with pressurized and superheated water necessary for the rapid dissolution event. These extreme conditions are quite challenging for the design of the container-closure and fluid path system. In particular, the cryogenic temperature in the polarizer requires special attention to any moisture or ambient (moist) air introduced into that portion of the fluid path, which can form an ice block at ~1 K. This ice can lead to flow restriction during the dissolution event and reduce the strength of the laser welded bond between the cryovial and its cap. This can ultimately produce failures in the dissolution step, including variations in final pyruvate concentration and pH that may fail to meet QC release criteria as well as fluid path ruptures that provide no available dose and result in polarizer down-time.

The polarization of the HP 13C pyruvate sample decays quickly over the span of a few minutes after dissolution, and thus the process of dissolution, QC for release, and injection should be completed as fast as possible to preserve the high polarization level achieved. Any delays in the preparation process, such as transportation time or equipment malfunction, can significantly reduce the final polarization and result in lower quality imaging data.

Current Practices

A summary of data collected from all sites performing clinical trials with HP 13C-pyruvate is shown in Fig. 3 and Table 1, including the specification of the final dose and how the quality control and release of the final dose are performed. There is a split in the Production Style, described in the General Considerations section above, with 8/13 sites using Sterile Preparation versus 5/13 using Terminal Sterilization. While many of the dose specifications show notable differences in acceptable ranges, all of these variations listed in tables have been successfully and safely been used to perform HP 13C pyruvate studies in humans. Their differences depend on the institutions’ preferences, resources and their particular regulatory situation. There is high similarity in pyruvate ranges, temperature ranges, EPA limits, and volume limits. There is modest variability in pH ranges and large variability in the endotoxin test limit. There is a 3-fold difference in acceptable polarization levels, which are measured to ensure a futile dose is not injected since the polarization is directly proportional to SNR. This reflects the decision by several sites to believe that useful data can be still be obtained with suboptimal polarizations.

Figure 3: Hyperpolarized agent preparation methods reported by sites currently performing HP

In House

Table 1: HP 13C-pyruvate preparation parameters, methods, and dose specifications used for quality control testing and release as well as validation. These were obtained from a survey of all sites performing clinical trials with HP [1-13C]pyruvate. The parameters used for product release are noted in bold text, otherwise these parameters are measured for batch validation or other QC measurements. The endotoxin and sterility testing are performed during process validation of the batch and/or post-injection, and largely depends on the agent production approach.

Summary

The overall safety record of HP 13C-pyruvate has been very strong, and the SPINlab hyperpolarizer has proven to provide high polarizations at human sized doses while meeting numerous QC and release criteria. A weakness remains the failure modes of the SPINlab Phamacy Kits (e.g. ice blocks, path ruptures), which are placed under extreme requirements particularly during dissolution. The preparation process still requires a high degree of expertise.

Therefore, there is a significant need to improve the reliability, robustness, and ease of operation for generating HP 13C-pyruvate doses for human studies. Furthermore, there is a divide between manufacturing and sterile compounding style preparation as well as other site-specific practices, resulting in variations in SOPs and justification required to relevant regulatory bodies. There have also been no comparisons between these approaches. It is also unclear what release criteria and QC parameters are truly required to ensure patient safety.

However, all of the reported methods are acceptable and approved by the appropriate regulatory authorities, and have led to the rapid expansion of successful human studies in recent years.

Mri System Setup And Calibrations

This section covers the MRI system setup, including the imaging system, RF coils, phantoms, and prescan calibration methods.

Imaging System

The main prerequisite for a given MRI scanner to be capable of supporting studies with HP 13C is its “broadband” capability to transmit and receive radiofrequency (RF) signal at the frequency of 13C, which is around 4 times lower than 1H. This does not come as a default on clinical MR devices. The transmit power of the broadband amplifier should also be sufficient to support the intended flip angle and RF pulse shape with the employed transmission RF coil(s) for 13C. Most studies to date use relatively low flip angles (< 90 degrees) for HP 13C in order to preserve polarization for time-resolved imaging. The capability to receive 13C signal on multiple channels is also desirable to increase SNR, as discussed further in the “RF coils” section.

The choice of magnetic field strength is primarily dependent on the metabolites’ frequency separation due to chemical shift dispersion and 1H imaging. High field strengths do not enhance hyperpolarized 13C signal as they do for 1H because the signal strength in a HP experiment relies on manipulating the population of quantum energy states outside of the MRI scanner.

However, the injected HP 13C-pyruvate and its metabolic products have greater frequency separation at higher fields, and it may thus be easier to separate and quantify these resonances at higher fields. This comes at the cost of a reduction in the achievable T2* and often reduced T1. As the initial polarization is independent of the imaging field strength it has been proposed that the increased T2* at 1.5T can potentially be exploited to increase SNR by adapting the acquisition bandwidth or reduce off-resonance imaging effects in cases when the decay of the transverse magnetization is dominated by T2* (73). In practice, 3T has been used in all published human 13C-pyruvate studies surveyed (Supporting Table S1), and comprises the majority of scanners currently in use for human studies (Table 3). A field strength of 3T is well-suited for 1H MRI anatomical reference and correlative imaging.

Stronger and more rapidly slewing magnetic field gradients support more rapid spatial encoding, particularly for metabolite-specific single-shot imaging using echo-planar imaging (EPI) or spiral imaging (See “Acquisition and Reconstruction”). Although the spatial resolution acquired for HP 13C imaging is typically much coarser than for 1H MRI, the factor of ~4 in gyromagnetic ratio leads to the same reduction factor in performance of the gradient system, so 13C experiments are potentially more limited by gradient hardware performance. To date, all human studies have used the commercially-available integrated gradient systems provided in clinical MRI scanners.

Optimization of scanner design has understandably focused on minimization of artifacts in 1H MRI, where devices such as room lights, the gradient amplifiers, and the motors driving the patient bed are checked to ensure that they do not produce RF interference at the 1H frequency, but artifacts may arise at other frequencies. Eddy current compensation is also not always appropriately adjusted for nuclei at other frequencies (74). In order to optimize for 13C, many sites have performed checks on phantoms for RF interference, gradient artifacts, and eddy currents (74), including the use of post-hoc gradient impulse response function characterisation and correction, and some vendors have fixed these issues as well.

Rf Coils

For HP 13C imaging studies in humans, RF coils for both 1H and 13C nuclei are needed, with 1H MRI providing an anatomical reference for registration and optional additional multiparametric MRI readouts. At the Larmor frequency of 13C nuclei, the relative contributions from coil noise compared to sample noise increase compared to 1H (73,75), although sample noise still is likely the dominant contributor for human-sized coils at 32.1MHz - the resonance frequency of 13C nuclei at 3T.

The key requirement for human 13C-pyruvate RF coils are that the coil geometry and sensitive volume must cover the volume of interest in the subject. Table 2 and Figure 4 shows coil configurations that have been used and optimized for applications in different anatomic regions.

Volume resonators are most commonly used for transmit, as they surround the subject to

Provide B1 Transmit Across The Fov (B1

+). While 1H relies on a large birdcage (“body”) coil built into the scanner, 13C transmit coils must be placed inside the bore. This takes up valuable space within the magnet, and also has led to the use of designs with relatively inhomogeneous

B1

+. Many human studies have used Helmholz pair resonators for transmit, including the “clamshell coil”, which has a notably inhomogeneous B1

+ Profile But Has Been Used Because Of

relatively easy integration into the scanner bore. B1

+ Variation Results In Variations In The Flip

angles that control the use of the hyperpolarized magnetization and creates errors in common HP metrics (9,76). The exception are head coils, where birdcage designs with highly

Homogeneous B1

+ can be placed around the head while easily fitting inside the bore. As with 1H MRI, higher SNR can typically be achieved by smaller receive coil elements, such as surface coils or phased arrays, and the majority of 13C receive coils used have layouts similar to 1H phased arrays.

RF coil quality control is important to ensure proper functioning of the coils to provide consistent imaging quality, especially with limited natural abundance 13C signal in vivo. It typically involves 1) a physical integrity check of the coil cables and connectors and 2) phantom SNR tests to check the coil’s performance and to monitor it over time (see Phantoms below). An useful reference for RF coil quality control is outlined in the MRI accreditation program of the American College of Radiology (77) and can be adapted for 13C coils.

Notably, configurations for brain and prostate studies used dual-tuned 1H/13C coil designs, which greatly simplify workflow and registration of 1H and 13C images, as no switching of coils is needed.

Table 2: RF coil configurations reported for human HP [1-13C]pyruvate studies.

Tx = Transmit

coil, RX = receive coil. The commonly used “clamshell” TX coil is a Helmholz pair design. For 1H RF configurations, all used the Body coil for TX unless otherwise noted, and “repositioned” indicates the 13C coil was removed for 1H imaging. One representative reference is listed for each configuration. The RF coil configurations reported in the reviewed papers are shown in Supporting Table S1.

Figure 4: Examples of RF coil configurations used for human HP [1-13C]pyruvate brain studies. (A,B) 13C Clamshell TX (Helmholz pair) and 2× 4-channel paddle RX arrays. (C) 13C Birdcage volume TX and 32-channel RX array (RX array slides into TX coil). (D) 13C Birdcage volume TX and 24-channel RX array, combined with a 1H 8-channel RX array. Image reproduced with permission from Ref (16).

Phantoms

Since hyperpolarized magnetization is non-renewable, phantoms containing 13C nuclei are important to: 1) test the multi-nuclear capabilities of the imaging system, including all parts of the signal excitation and receive chain; 2) perform calibration measurements before a scan with hyperpolarized nuclei; and 3) perform necessary pre-scan adjustments (see “Prescan Calibration” section). The phantoms currently in use are listed in Table 3. Their composition must provide sufficient 13C signal, with additional considerations of conductivity, stability, chemical shift(s) present, potential for dynamic imaging, and cost. The phantom geometries are typically either compact, in order to be used alongside the subject during a HP scan, or large enough to mimic the inner volume of a RF coil for system testing.

One popular compact design contains enriched 13C-urea at high concentration, typically 8 M, which provides a single resonance, placed inside a small container ~1 mL. The most common recipe mixes 13C-urea in a 90% water/10% glycerol solution, with glycerol used to increase the urea solubility and doping with a Gd-based contrast agent to shorten T1 which increases the potential SNR per unit time. For example, when Dotarem is added at a 3:1000 volume ratio the 13C-urea T1 is around 500 ms and T2 is around 100 ms. However, when testing pulse sequences influenced by T1 and T2, doping should be used carefully. This phantom is suitable for frequency calibration, transmit gain calibration, sequence testing, and as a fiducial marker when placed next to a patient. However, enriched 13C-urea has a relatively high cost compared to natural abundance compounds.

For larger volumes (>100 ml), the phantoms most often used contain undiluted ethylene glycol, glycerol, or dimethyl silicone. These compounds have sufficiently high carbon concentrations to provide sufficient 13C signal even with the 1.1% natural abundance of 13C. These larger phantoms matching the inner volume of an RF coil are useful for coil testing, including transmit

+) And Receive (B1

-) coil profile mapping, as well as to mimic acquisitions using in vivo FOV requirements. In this case, size and conductivity should match the expected subject size in order to mimic coil loading and get a realistic estimation of B1+. Large-volume natural abundance urea phantoms have also been used by some sites, but suffer from higher conductivity compared to biological tissues. Typically, it is easier to increase the conductivity and hence coil loading of the non-conductive phantom by adding NaCl to match physiological loading (16,78).

Dynamic phantoms that aim to mimic metabolite kinetics have also been developed (79–81), and have the potential to more closely mimic the HP experiment, but so far these are not widely used.

Prescan Calibration

Prior to performing an MRI acquisition, the so-called prescan procedure is used to set the shim parameters to maximize B0 homogeneity over the field of view (FOV) or a specific region of interest (ROI), the scanner center frequency (CF), the RF transmit gain, and the receiver gain.

While this calibration procedure is usually automated for 1H, the lack of sufficient natural abundance 13C signal prevents use of automated methods. (Although natural abundance 13C lipid signal has been detected, there are so far no reports on using this signal for prescan.) Table 3 shows current practices across sites.

Maximizing B0 homogeneity is independent of the nucleus and is therefore performed prior to 13C imaging using the 1H water signal and existing shimming tools, such as by a standard automated process (“Auto Shimming”) or using high order shimming routines. Similarly, the 13C CF can be calculated from the 1H CF using a predetermined scaling factor that depends on the target chemical shift (82). Another common approach used is to have a small, high-concentration 13C phantom, e.g. 8M 13C-urea, integrated in the RF coil or placed next to the scan subject (1). The reference frequency can also be based on real-time measurements after the HP injection but prior to imaging (83). Both the CF and B0 shimming are critical when using spectrally-selective RF pulses, as inmetabolite-specific imaging methods, where the desired excitation bandwidths are typically very narrow and frequency offsets can lead to a failure mode that is only apparent after injection.

The calibration of the RF transmit power is typically performed on a small, high-concentration 13C phantom placed near the region of interest during the scan or on a large 13C phantom of similar size and coil loading as the subject, prior to the subject scan. Reference power is often done by sweeping the power in a pulse-acquire sequence (53,62), or the Bloch-Siegert method (52,84). When using a small phantom, the location of the phantom, B1

+ Inhomogeneity As Well

as any shielding effects, e.g., when the phantom is integrated into a coil (1), may degrade the accuracy. Other methods include real-time Bloch-Siegert method measurements after the HP injection (83), and using the stronger natural abundance 23Na signal that is close enough to the 13C resonance frequency to be detected by 13C coils (82).

The receiver gain is predetermined, either systematically based on independent phantom measurements and assuming the dose and polarization of the HP compound is known prior to injection, or based on past HP imaging studies.

Power [Kw]

Phantom(s) - during study Phantom(s) - before study 13C Frequency

13C-bicarbonate doped with dimethyl silicone, various

Maximum Values

Table 3: Summary of the imaging systems, phantoms, and prescan procedures used at sites currently performing HP 13C-pyruvate human studies. These were obtained from a survey of all sites performing clinical trials with HP [1-13C]pyruvate. *Previously performed studies with a Siemens 3T Tim Trio. The imaging systems, phantoms, and prescan procedures reported in the reviewed papers are shown in Supporting Table S1.

Summary

Commercially available 3T MRI systems are by far the most commonly used for human HP 13C-pyruvate studies, although a systematic investigation of the impact of B0 has only recently been investigated (73). The multi-nuclear RF transmit and receive chain has proven sufficient for current acquisition strategies, although many sites have observed artifacts due to RF interference, gradient interference, and residual eddy currents when operating at the 13C frequency. A variety of 13C RF coils, tailored for numerous anatomical targets, have been successfully demonstrated, with the main limitation that most transmit coils take up a lot of additional space inside the bore and provide relatively inhomogeneous B1

+ Profiles. The

phantoms used have converged into generally 2 categories - small phantoms containing 13C-enriched compounds that can be used during the study and human-sized phantoms containing compounds with high carbon concentrations but without 13C enrichment that are used to test and calibrate the coils. There are no standardized compositions or geometry, and dynamic phantoms that recapitulate in vivo kinetics would be desirable but are still an emerging area. Prescan calibration procedures were not well defined in most publications, so we surveyed individual sites to determine current practices. Calibration procedures for the B0 field (13C CF and shimming) for most sites take advantage of 1H signal and methods, while methods

For Calibration Of B1

+ is more variable across sites, likely a reflection of remaining challenges in how to perform this calibration. Standardization of both phantoms and calibration procedures would synergistically improve the robustness and reproducibility of HP 13C studies.

Acquisition And Reconstruction

Data acquisition strategies in human HP [1-13C]pyruvate MRI studies must account for multiple chemical shifts, efficiently utilize the non-renewable HP magnetization, and acquire data quickly relative to metabolism and relaxation decay processes. These studies require spectral encoding to separate metabolites, necessitating pulse sequences that efficiently encode up to 5D data (3 spatial + 1 spectral + 1 temporal dimension). RF pulses must efficiently sample without immediately saturating the non-renewable HP magnetization, and sequences must acquire data quickly and be robust to both experimental and physiologic variation (e.g. B1

+ Inhomogeneity,

variation in perfusion) to ensure reproducibility and minimize scan-to-scan variability. This section covers current successful practices for data acquisition in human [1-13C]pyruvate studies, and accompanying 1H imaging, from different anatomic regions, including scan parameters and image reconstruction.

Acquisition And Reconstruction Methods

The acquisition methods used in human [1-13C]pyruvate studies can be classified into 3 categories: 1) MR spectroscopy or MR spectroscopic imaging (“MRS/I”), 2) chemical shift encoding methods, and 3) metabolite-specific imaging (Fig. 5).

Mrs/I Methods Specifically

resolve a spectrum that can be analyzed to extract expected as well as unexpected resonances, making this approach very robust. It was used in many initial studies (1).

Chemical Shift

encoding methods, most commonly the Iterative Decomposition of water and fat with Echo Asymmetry and Least-squares estimation (IDEAL) method, use imaging sequences acquired with multiple TEs and rely on a model-based separation of expected chemical shifts (85).

Metabolite-specific imaging methods use specialized RF pulses that are spatially and spectrally selective to excite individual metabolites which are then typically imaged with fast k-space trajectories such as echo planar imaging (EPI) or spirals (86).

Their Application To Different

organ systems is described below. The image reconstruction methods used in human [1-13C]pyruvate studies have typically been conventional methods (e.g. FFT, non-uniform FFT, or equivalent). The incorporation of accelerated imaging and advanced reconstruction methods including parallel imaging (4,57,87) and compressed sensing (7) has also been applied in human studies for improved spatial resolution, temporal resolution and coverage, but have the potential for additional artifacts as well as SNR losses due to ill-conditioning of the reconstruction (e.g. g-factor).

The Majority Of

published studies do not use accelerated imaging indicating the resolution and coverage achievable without acceleration is currently adequate for successful data collection. Performing coil combination, even with fully sampled data has also been shown to have specific challenges for HP human images: using naive sum-of-squares methods suffer from high noise amplification in the relatively low SNR regime of HP [1-13C]pyruvate (compared to 1H), motivating several HP 13C-specific methods that include data-driven coil sensitivity estimation which have shown obvious improvements over sum-of-squares (11).

More recently denoising techniques have been applied as post-processing of human HP data(41,42,44). The techniques applied are based on spatial-temporal singular value decomposition for unsupervised estimation of signal and noise components. They have shown improvements in apparent SNR in the brain and liver, while care must be taken to choose parameters such as the rank threshold to avoid oversmoothing and overfitting to the estimated signal components.

Prostate Studies

Prostate cancer was the first human application of HP [1-13C]pyruvate (1), and data was acquired with MRS/I methods: 1D dynamic MRS, single-slice 2D dynamic echo-planar spectroscopic imaging (EPSI), and single time point 3D EPSI. Advances in imaging strategies led to the development and application of new acquisition schemes, including undersampled 3D EPSI with compressed-sensing (7), model-based chemical shift encoding methods that use a priori information (47,59), and metabolite-specific EPI (10), all of which can provide volumetric whole-organ coverage and dynamic acquisitions.

The pyruvate bolus arrival in the prostate can vary by ± 10 s between patients, necessitating dynamic imaging to reliably and consistently capture the pyruvate bolus (18). For this reason, all currently ongoing studies acquire dynamic data. While MRS/I, chemical shift encoding, and metabolite-specific imaging can all achieve dynamic imaging, chemical shift encoding and metabolite-specific imaging provide greater dynamic and volumetric coverage (85). For scan prescriptions, the FOV is designed to provide full prostate coverage and typically to match the orientation of the anatomic imaging used for registration. Flip angles used in current studies are constant through time, as quantification with a variable-through-time flip scheme is highly sensitive to bolus timing (8) and errors in the RF transmit (B1 +) field (76).

Heart Studies

Data acquisition methods for 13C imaging in the heart must be designed to meet the demands of significant cardiac motion and blood flow. To cope with the periodic cardiac motion, most human heart studies to date used gating to the diastolic window, the longest cardiac cycle interval, which has reduced motion (2,22,28,30,35,36,38,45,52). The duration of the diastolic window limits the available data sampling time, making cardiac acquisitions the most time-constrained of the HP 13C MRI applications. The most common acquisition approach is metabolite-specific imaging with spiral k-space trajectories (2). Their single-shot imaging capability makes these methods particularly robust to motion effects. Furthermore, spiral k-space trajectories provide rapid k-space coverage and relatively benign flow and motion artifacts. The majority of studies have used 2D multi-slice acquisitions, but 3D encoding has also been used successfully (35).

Brain Studies

For HP 13C MRI of the human brain, the majority of studies have also used 2D (slice selective) acquisitions (10–12,14,16,28,33,40,41,44,51,53,60), with a trend toward volumetric coverage using 2D multi-slice metabolite-specific imaging. 3D metabolite-specific imaging of the whole brain, with phase encoding of the slice direction (34,57), has been shown to provide similar SNR efficiency (88) compared with multislice imaging. A number of studies have employed MRS/I (5,6,29,31–33,50,55) resulting in a spectrum from each voxel, which has the advantage of not requiring a priori information about which peaks to encode. This was important in early brain studies when it was not known which peaks would be detectable. Chemical shift encoding, using a set of images with different echo times and an iterative reconstruction of the individual resonances (i.e. the IDEAL approach (85)), has also been used (12,49,54), with the drawback that coverage in the slice direction was limited due to the time required to acquire multiple echo time images.

Abdomen And Breast Studies

The fundamental approaches to data acquisition and reconstruction in the abdomen and breast are largely similar to the aforementioned applications, but demand attention to particular challenges associated with these anatomic regions, especially relating to respiratory motion.

Although it has been shown that a basic 2D MRSI approach based on phase encoding and FID readout can be successfully applied for HP 13C imaging in breast (15) and kidney (13), major advantages in terms of spatiotemporal resolution and coverage have been realized using tailored approaches based on metabolite-specific imaging (43,62) and chemical shift encoding (43), which have facilitated multi-slice or 3D dynamic acquisitions over large FOVs in the abdomen (4,37,46).

The significant respiratory motion encountered in these regions can directly blur 13C images, and has further favored these rapid acquisition strategies. Motion also degrades B0 homogeneity, which can shift frequency-selective excitation profiles and introduce artifacts into rapid imaging readouts. This makes accurate determination of the acquisition center frequency and shimming essential in these regions which often cover large FOVs. (See “Prescan Calibration” section for more information). In some studies, breath-holding was used to minimize motion effects and enforce frame-to-frame data consistency (42). A pragmatic and reasonably effective approach for dealing with respiratory motion during 13C data acquisition is an initial breath-hold (as long as can be tolerated), followed by free-breathing (46,62).

1H Imaging

Collection of 1H imaging data is essential both for prescribing the 13C acquisition and for interpretation of the resulting 13C data. Multi-planar 1H scouts are acquired prior to 13C acquisition to enable graphical prescription of the 13C imaging region. All human HP 13C-pyruvate imaging studies acquire conventional MRI scans (e.g. T1- and T2-weighted volumes) for anatomic reference, aiming to cover at least the full 13C FOV. Acquiring these anatomic scans as close as possible to the time of 13C imaging (immediately before or after) minimizes potential misregistration between the data sets. Depending on the application, other advanced 1H sequences are also acquired (e.g. diffusion-weighted imaging for cancer imaging).

When contrast-enhanced data is acquired, it is done after 13C imaging, as paramagnetic contrast agents will accelerate 13C relaxation.

Reported Study Parameters

Figures 5 and 6, and Supporting Table S2 shows the reported acquisition study parameters for human HP [1-13C]pyruvate studies published as of September 2022. Figure 5 shows a mixture of MRS/I, metabolite-specific imaging, and chemical shift encoding methods have been successfully used, where spectroscopy-based methods have become less prevalent in recent studies. Figure 6 shows the acquisition timing, including the important start time and interval/temporal resolution, is quite variable across studies.

Figure 5: Acquisition methods used in published HP [1-13C]pyruvate human studies published up to September 2022, classified into: MR spectroscopy and spectroscopy imaging (MRS/I); chemical shift encoding methods, such as IDEAL, that use multiple TEs and model-based reconstructions; and metabolite-specific imaging methods that use spectrally-selective excitation to image a single resonance at a time.

Figure 6: Temporal acquisition characteristics reported in HP [1-13C]pyruvate human studies published up to September 2022. (a) Reported referencing of acquisition start times.

(B)

Acquisition start times reported when using dynamic imaging and when timing was reported relative to the end of the injection. (c) Temporal resolutions. “Not Applicable” indicates dynamic imaging was not used.

Summary

Three general categories of acquisition strategies have been used successfully for human HP 13C-pyruvate studies: MRS/I, model-based chemical shift encoding (e.g. IDEAL) methods, and metabolite-specific imaging methods. These have enabled successful studies in the prostate, heart, brain, abdomen, and breast. Recent studies increasingly have used the imaging-based strategies of metabolite-specific imaging and chemical shift encoding which are the fastest methods, although a heads-to–head comparison between techniques has not been performed.

Metabolite-specific imaging is quite popular because of its speed and compatibility with single-shot imaging, but is sensitive to B0 field variations and thus requires careful calibrations. Nearly all studies surveyed acquired data dynamically, allowing measurement of the bolus and metabolite kinetics. The exact timings and associated flip angles vary quite widely across reported studies, with no consensus yet as to how to choose these parameters. Image reconstruction is typically done directly using Fourier Transform methods, and accelerated imaging strategies are uncommon.

Data Analysis And Quantification

This section covers the analysis of data from human HP [1-13C]pyruvate studies, including modeling and metrics, visualization, as well as considerations for how to store data and metadata. Depending on study design, the analysis may need to give quantitative or semi-quantitative output reflecting a biological process or may just reflect a contrast between different regions of interest for quantitative evaluation.

Metrics

Figure 7: HP [1-13C]pyruvate raw data (A) have typically been quantified using four categories of metrics depending on the acquisition. Data acquired as a single time point are often quantified using normalized metabolite images or metabolite ratios (B). Dynamic data can be quantified using normalized metabolite images or metabolite ratios (B), or with metabolite timings such as time-to-peak (TTP) or pharmacokinetic (PK) models (C). The latter two require the data to be time-resolved. [1-13C]alanine and 13C-bicarbonate are analyzed similarly to [1-13C]lactate but omitted here for display.

Metabolite images are commonly used as summary metrics for HP MRI data, often including some form of normalization as well as summed over time as an area under the time curve (AUC) (17). These are analogous to the visual evaluation that is most used for routine clinical work (89,90). In these metabolite images, we expect that the [1-13C]pyruvate AUC signal is predominantly weighted towards perfusion and uptake, while [1-13C]lactate, [1-13C]alanine and 13C-bicarbonate AUCs represent metabolic conversion. The strength of this approach lies in its simplicity and relatively few underlying assumptions. Limitations to the use of single-metabolite images or AUCs include sensitivity to inhomogeneous coil profiles (57,87,91), the acquisition strategy and acquisition parameters, pyruvate polarization and concentration level, and signal relaxation rates (92). Further, the reader must be careful to interpret all the images in conjunction to better understand the underlying biology; for example, increased [1-13C]lactate in the presence of decreased [1-13C]pyruvate delivery can have a very different meaning compared to increased [1-13C]lactate with increased [1-13C]pyruvate delivery.

In an attempt to address variations in coil sensitivity, polarization level, and pyruvate delivery, AUC images are often computed by normalizing to a specified parameter, such as the maximum pyruvate or average lactate signals, or presented as a ratio such as lactate/pyruvate or divided by “total Carbon” - the sum total of HP 13C signal observed across all metabolites. The AUC ratios between metabolites and pyruvate are proportional to the corresponding forward kinetic rates (81,93), but are not directly comparable to rate constants when magnetization loss rates (e.g. relaxation and losses due to signal excitation) differ between studies. Similarly, the ratios between the produced metabolites (e.g. bicarbonate/lactate) can reflect the balance between downstream metabolic pathways (12,55). Care must be taken to consider how AUC images are calculated and normalized before comparing values between studies.

To further quantify the interpretation, pharmacokinetic (PK) modeling approaches were developed to compute the apparent kinetics of pyruvate-to-metabolite exchange (92,94–99). These yield semi-quantitative to quantitative apparent rate constants, given in s-1. Some models require a vascular input function, while others avoid this requirement (95). PK models can explicitly account for acquisition-specific details such as excitation angle and repetition time, and thus may reduce the effects of these details on quantification. An input-less model, provided in the Hyperpolarized-MRI-Toolbox (https://github.com/LarsonLab/hyperpolarized-mri-toolbox) (100) and thus frequently employed for human data, has been shown to fit well and robustly to prostate and brain data (8,20). PK models are quantitative in nature, arguably provide more relevant biological information (8,20), and appear to be reproducible across sites (51). However, rate constants derived from PK models are still apparent rates, and likely do not reflect a single biological characteristic.

Some additional considerations include whether complex or magnitude data is used, as the noise behaviors will impact the analysis differently. Additionally, cut-off thresholds or other criteria may be used to identify and avoid voxels with insufficient SNR before analysis to improve robustness (20,41).

Regardless of the analysis approach, the underlying biology is not always clearly represented by the data; instead, the metrics may be influenced by perfusion, barrier permeability, intercellular shuttles, enzyme activities, co-substrate concentrations, or combinations thereof, depending on the organ and disease of interest (19,43,94,101–103). This may be addressed by incorporating complementary information. As an example, HP 13C pyruvate data is influenced by perfusion, and thus addition of perfusion MRI could be important for interpretation (98,104,105).

All the methods outlined above have been explored in clinical studies, described in Supporting Table 3 and summarized in Figure 8. As of September 2022, approximately 52% of studies involving human subjects report rate constants derived from a PK model with a few different models reported. A nearly equal fraction (51%) of the studies report AUC ratio values.

Approximately 66% of these studies report metabolite-specific images or AUC values. About 40% report SNR values; this metric is particularly frequent in manuscripts that describe technical developments for clinical HP MRI. Approximately 16% of these studies summarize model-free metrics, and 10% report measurements from a single timepoint. Most studies report a combination of quantities.

Figure 8: Reported metrics used for analysis in HP [1-13C]pyruvate human studies published up to September 2022.

Visualization

A wide variety of approaches have been used for visualizing data from human HP 13C-MRI studies. The challenges and practical considerations are: 1) choosing the appropriate metrics to display, 2) how to encode the parameters (e.g. the colormap), and 3) choosing how to provide anatomical context and other multi-parametric data. The choice of visualization also depends on the goal which could be for diagnostic interpretation, but also quality control, reproducibility among readers and publication.

Metrics

The choice of HP 13C metrics is described in detail above. At this stage in HP 13C development where there is no standardized metric, often a combination of metabolite images and ratios or PK model parameters are shown.

Parameter Encoding

The mapping function chosen should provide an adequate, often quantitative, impression of the parameter mapped. There is a consensus in the visualization field that perceptually uniform maps are best suited to visualize continuous parameters, like the greyscale typically used by radiologists as well as other monochrome (black to blue) and color ranges (fire-type, rainbow-type) (106,107). Multi-color heatmaps have been the most frequently employed method for HP 13C data, while greyscale has infrequently been used but it ensures there is no coloring-based bias as well as facilitating later reuse (Fig. 9a). Among the color schemes employed in the clinical HP 13C literature, fire-type scheme seems to be the most common [similar to “Plasma” or “Inferno” in matplotlib.org]. Next most commonly employed is the rainbow-type scheme [similar to “Rainbow” in matplotlib.org].

Anatomical Context

HP MRI faces the challenge that it does not necessarily depict the anatomical features, similar to PET, and thus requires an anatomical reference. Most often, a grayscale anatomical image is overlaid with a HP colormap (Fig. 9c,d). This approach is very intuitive, but can skew perception as the grey-scale anatomical reference may affect the brightness of the HP data (e.g. signal in the skull). This bias does not occur when showing adjacent maps (Fig. 9a, b). Here, anatomical outlines may help to provide reference (Fig. 9b).

Related Journal Articles & DOI Links

Selected peer-reviewed publications relevant to 12 Lead ECG Acquisition. Click the DOI to access the full paper (may require institutional access).

Why Choose Us?

Bangalore guidance for robotics, Spectre and autonomous systems projects.