Vericut Cnc Composite Simulation

Eeds, Ls2 9Jt, Uk

Agents offer a new and exciting way of understanding the world of work. In this paper we describe the development of agent-based simulation models, designed to help to understand the relationship between people management practices and retail performance. We report on the current development of our simulation models which includes new features concerning the evolution of customers over time. To test the features we have conducted a series of experiments dealing with customer pool sizes, standard and noise reduction modes, and the spread of customers’ word of mouth. To validate and evaluate our model, we introduce new performance measure specific to retail operations. We show that by varying different parameters in our model we can simulate a range of customer experiences leading to significant differences in performance measures. Ultimately, we are interested in better understanding the impact of changes in staff behavior due to changes in store management practices.

Our multi-disciplinary research team draws upon expertise from work psychologists and computer scientists. Despite the fact we are working within a relatively novel and complex domain, it is clear that intelligent agents offer potential for fostering sustainable organizational capabilities in the future.

Keywords: agent-based modeling, agent-based simulation, retail performance, management practices, shopping behavior, customer satisfaction, word of mouth

Ntroduction

The retail sector has been identified as one of the biggest contributors to the productivity gap, whereby the productivity of the UK lags behind that of France, Germany and the USA [1, 2]. A recent report into UK productivity asserted that, ‘...the key to productivity remains what happens inside the firm and this is something of a ‘black box’…’ . A recent literature review of management practices and organizational productivity concluded that management practices are multidimensional constructs that generally do not demonstrate a straightforward relationship with productivity variables , and that both management practices and productivity measures must be context specific to be (respectively) effective and meaningful.

Many attempts have been made to link management practices to an organization’s productivity and performance (for a review, see ), however research findings have been so far been mixed, creating the opportunity for the application of different techniques to advance our understanding.

Simulation can be used to analyze the operation of dynamic and stochastic systems showing their development over time. There are many different types of simulation, each of which has its specific field of application. Agent-Based Simulation (ABS) is particularly useful when complex interactions between system entities exist such as autonomous decision making or proactive behavior. Agent-Based Modeling (ABM) shows how micro-level processes affect macro level outcomes; macro level behavior is not explicitly modeled, it emerges from the micro-decisions made by the individual entities .

There has been a fair amount of modeling and simulation of operational management practices, but people management practices have often been neglected although research suggests that they crucially impact upon an organization’s performance . One reason for this relates to the key component of people management practices, an organization’s people, who may often be unpredictable in their individual behavior. Previous research into retail productivity has typically focused on consumer behavior and efficiency evaluation (e.g. [8, 9]), and we seek to build on this work and address the neglected area of people management practices in retail .

The overall aim of our project is to investigate the link between different management practices and productivity. This strand of work focuses on simulating various in-store scenarios grounded in empirical case studies with a leading UK retailer. In this paper we aim to understand and predict how Agent-Based Modeling and Simulation (ABMS) can assess and optimize the impact of people management practices on customer satisfaction and Word-Of-Mouth (WOM) in relation to the performance of a service-oriented methods to collect both qualitative and quantitative data. In summary, we have worked with a leading UK two retail stores, forty semi-structured interviews with employees including a sixty-three-item questionnaire on the effectiveness of retail management practices, and drawn upon a variety of established information sources internal to the company. This approach has enabled us to acquire a valid and reliable understanding of how the real system operates, revealing insights into the working of the system as well as the behavior of and interactions between the different individuals and their complementary roles within the experiments to investigate the effects of different management scenarios.

So far we have only studied the impact of people management practices (e.g. training and empowerment) on a customer base that is not influenced by any external or internal stimuli, and hence does not evolve [11, 12]. In order to be able to investigate the impact of management practices on customer satisfaction in a more realistic way we need to consider the cues that stimulate customers to respond to these practices.

Therefore our focus is currently on building the capabilities to model customer evolution as a consequence of the implementation of management practices. Consumer decision making, and consequently changes in the behavior of customers over time, are driven by an integration of each consumer’s cognitive and affective skills . At any given point in time a customer’s behavior, as the product of an individual’s cognitions, emotions, and attitudes, may be attributable to an external social cue such as a friend’s recommendation or in-store stimuli , or an internal cue such as memory of one’s own previous shopping experiences . Changing customer requirements may in turn alter what makes a successful management practice (because these are context specific, and customers are a key component of any retail context). In order to enable such studies we enhanced our original Management Practices Simulation (ManPraSim) model and introduced some new features.

In this paper we discuss the key features we have implemented in order to allow the investigation of these kinds of behavioral dynamics. What we are learning here about modeling human behavior has implications for modeling any complex system that involves many human interactions and where the people function with some degree of autonomy.

Related Work

When investigating the behavior of complex systems the choice of an appropriate modeling technique is very important . To inform the choice of modeling technique, the relevant literature spanning the fields of Economics, Social Science, Psychology, Retail, Marketing, OR, Artificial Intelligence, and Computer Science has been reviewed. Within these fields a wide variety of approaches are used which can be classified into three main categories: analytical methods, heuristic methods, and simulation. Often a combination of these are used within a single model (e.g. [17, 18]. After a thorough investigation of the relevant literature we have identified simulation as being the most appropriate approach for our purposes.

Simulation introduces the possibility of a new way of thinking about social and economic processes, based on ideas about the emergence of complex behavior from relatively simple activities . The purpose of simulation is either to better understand the operation of a target system, or to make predictions about a target system’s performance . It allows the testing and evaluation of a theory, and investigation of its implications. Whereas analytical models typically aim to explain correlations between variables measured at one single point in time, simulation models are concerned with the development of a system over time . They allow us to study the transient conditions during the process, along the way to the eventual outcome . The outcome of simulation study is the acquisition of more knowledge about the dynamic behavior of a system, i.e. the state changes of the model as time advances . There are many different approaches to OR simulation, amongst them Discrete-Event Simulation (DES), System Dynamics (SD), and ABS, which is sometimes also referred to as individual-based simulation . The choice of the most suitable approach depends on the issues investigated, the input data available, the required level of analysis, and the type of answers that are sought .

Although computer simulation has been used widely since the 1960s, ABMS only became popular in the early 1990s . It is described by as a mindset as much as a technology: ‘It is the perfect way to view things and understand them by the behavior of their smallest components’. In agent-based models a complex system is represented by a collection of agents that are programmed to follow simple behavioral rules. Agents can interact with each other and with their environment to produce complex collective behavioral patterns. The main characteristics of agents are their autonomy and their ability to take flexible action in reaction to their environment, both determined by motivations generated from their internal states.

They are designed to mimic the attributes and behaviors of their real-world counterparts. ABS is still a relatively new simulation technique and its principal application has been in academic research. There are, however, a wide range of existing application domains that are making use of the agent paradigm and developing agent-based systems, for example in software technology, robotics, and complex systems, and an ever increasing number of computer games use the ABMS approach. draw a distinction between two central Multi-Agent System (MAS) paradigms: multi-agent decision systems and make joint decisions as a group. Mechanisms for joint decision-making can be based on economic mechanisms, such as an auction, or alternative mechanisms, such as argumentation. Multi-agent simulation systems are used as a model to simulate some real-world domain. The typical application is in domains which involve many different components, interacting in diverse and complex ways and where the system- level properties are not readily inferred from the properties of the components. ABMS is extensively used by the game and film industry to develop realistic simulations of individual characters and societies. It is used in computer games, for example The SIMS™ , or in films when diverse heterogeneous characters animations are required, for example the Orcs in Lord of the Rings™ .

Due to the characteristics of the agents, the ABMS approach (which is often implemented as a process interaction DES) appears to be more suitable than the event scheduling modeling approach (which is the fact that it supports the modeling of autonomous behavior. ABMS seems to promote a natural form of modeling these systems . There is a structural correspondence between the real system and the model representation, which makes them more intuitive and easier to understand than for example a system of differential equations as used in SD. emphasizes that one of the key strengths of ABMS is that the system as a whole is not constrained to exhibit any particular behavior as the system properties emerge from its constituent agent interactions. Consequently, assumptions of linearity, equilibrium and so on, are not needed.

mentioned is the high demand in computational power compared to other simulation techniques, most notably in that all of the interactions between agents and between the agent and the environment have to be considered during the simulation run. However, rapid growth in affordable computer power makes this problem less significant . There is consensus in the literature that it is difficult to evaluate agent-based models, because of the heterogeneity of the agents and because the behavior of the system emerges from the interactions between the individual entities. argues that for ‘intellective models’ (e.g. models that illustrate the relative impact of basic explanatory mechanisms) validation is somewhat less critical and the most important thing is to maintain a balance between keeping a model simple and maintaining veridicality (the extent to which a knowledge structure accurately reflects the information environment it represents).

More recently some attempts have been made to develop a common approach to assuring the credibility (verification and validation) of ABS models (see for example [33 - 35]. A two stage approach for empirical validation has been proposed by . The first stage comprises micro- validation of the behavior of the individual agents and the authors recommend this is conducted with reference to real data on individual behavior. The second stage covers macro-validation of the aggregate or emergent behavior of the model when individual agents interact and is advised to be conducted with reference to aggregate time series data. Further, remark that at the macro level only qualitative validation judgments may be possible. Also, problems often occur through the lack of adequate empirical data. state that most quantitative research has concentrated on ‘variable and correlation’ models that do not cohere well with process-based simulation that is inherent in agent-based models. Finally, point out the danger that people new to ABMS may expect too much from the models, particularly regarding their predictive ability. To mitigate this problem it is important to be clear with individuals about what this modeling technique can really offer, in order to guide realistic expectations.

Odel Design And Data Collection

Before building a simulation model one needs to understand the particular problem domain . In order to gain this understanding we have conducted some case studies. What we have learned during these case studies is reflected in the conceptual models presented in this chapter. Furthermore we explain how we intend to use the data we have gathered during our case studies. Throughout the rest of the paper we will use the term 'actor' to refer to a person in the real system, whereas the term 'agent' will be reserved for their counterparts in the simulation model.

Knowledge Gathering

study work involved extensive data collection techniques, spanning: participant observation, semi- structured interviews with team members, management and personnel, completion of survey questionnaires and the analysis of company data and reports (for further information, see ). Research findings were consolidated and fed back (via report and presentation) to employees with extensive experience and has enabled us to acquire a valid and reliable understanding of how the real system operates, revealing insights into the working of the system as well as the behavior of and interactions between the different actors within it.

In order to make sure that our results regarding the application of management practices are applicable for a different not only in their way of operating but also their customer type split and staff setup. We collected study branches.

customer service time in A&TV is significantly longer, and the average purchase is significantly more expensive than in WW. The likelihood of a customer seeking help in A&TV is also much higher than in WW. Out of customers who have received advice, those in WW have a higher likelihood of making a purchase (indeed customers’ questions tend to be very specific to a desired purchase) than in A&TV.

Considering customer types, A&TV tends to attract more solution demanders and service seekers, whereas WW customers tend to be shopping enthusiasts. Finally, it is important to note that the conversion rate (the likelihood of customers making a purchase) is higher in WW than in A&TV.

Onceptual Modeling

A conceptual model captures only the essential and relevant characteristics of a system. It is formulated from the initial problem statement, informal user requirements, and data and knowledge gathered from analysis of previous development models . We have used the knowledge gained from the case studies to inform our conceptual models of the system to be investigated, the actors within the system, and their behavioral changes due to certain stimuli.

Ain Concepts For The Simulation Model

Our initial ideas for the simulation model and its components are display below in Figure 1. Here we can see a bird’s eye view of a shopping centre with customers (red dots) visiting a range of shops. Based on this conceptual overview, we determined that we required three types of agents (customers, sales staff and managers), each with a different set of relevant attributes. Global parameters can influence any aspect of the system. The core of our conceptual simulation model consists of an ABS model with a user interface to allow some form of user interaction (change of parameters) before and during runtime. Regarding system outputs, we aim to find some emergent behavior on a macro level. Visual representation of the simulated system and its actors allows us to monitor and better understand the interactions of entities within the system. Coupled with the standard DES performance measures, we can then identify bottlenecks to assist with optimization of the modeled system.

Oncepts For The Agents

We have used state charts for the conceptual design of our agents. State charts show the different states an entity can be in and define the events that cause a transition from one state to another. This is exactly the information we need in order to represent our agents at a later stage within the simulation environment. We have found this form of graphical representation a useful part of the agent design process because it is easier for an expert in the real system (who is not an expert in ABMS) to quickly take on board the model conceptualization and provide useful validation of the model structure and content.

Designing and building a model is to some extent subjective, and the modeler has to selectively simplify and abstract from the real scenario to create a useful model . A model is always a restricted copy of the real system, and an effective model consists of only the most important components of the real system. In our case, the case studies indicated that the key system components take the form of the behaviors of an actor and the triggers that initiate a change from one behavior to another. We have developed state charts for all of the agents in our simulation model. Figure 2 shows as an example the conceptual template model of a customer agent. The transition rules have been omitted to keep the chart comprehensible. They are explained in detail in the Section 3.3.

Oncepts For A Novel Performance Measure

We have introduced a customer satisfaction level index as a novel performance measure using satisfaction weightings. This new measure is required because existing indices such as queuing times or service times are less useful in modeling services than manufacturing activities. In essence, purely quantitative measures fail to capture the quality of service, which is arguably the most important potential trade-off with retail productivity. The inevitable trade-off between quality and quantity is particularly salient when customers and staff come face-to-face and therefore we consider this measure of quality in conjunction with others of quantity.

Historically customer satisfaction has been defined and measured in terms of customer satisfaction with a purchased product . The development of more sophisticated measures has moved on to incorporate customers’ evaluations of the overall relationship with the retail organization, and a key part of this is the service interaction. Indeed, empirical evidence suggests that quality is more important for customer satisfaction than price or value-for-money , and extensive anecdotal evidence indicates that customer- staff service interactions are an important determinant of quality as perceived by the customer.

The index allows customer service satisfaction to be recorded throughout the simulated lifetime. The idea is that certain situations can have a bigger impact on customer satisfaction than others, and therefore weights can be assigned to events to account for this. Satisfaction indices do not change the likelihood that a evidence supports the notion that regardless of a customer’s changing shopping intentions he or she will tend to repeat shopping habits . Applied in conjunction with an ABS approach, we expect to observe interactions with individual customer differences; variations which have been empirically linked to differences in customer satisfaction (e.g. ). This helps the analyst to find out to what extent customers underwent a positive or negative shopping experience and it also allows the analyst to put emphasis on different operational aspects and try out the impact of different strategies.

Oncepts For Modeling Customer Evolution

There are two different ways in which we consider customer evolution: external stimulation attributable to the WOM and internal stimulation triggered by memory of one’s own previous shopping experiences (this is still work in progress). Sharing information with other individuals (referred to as WOM), significantly affects the performance of retail businesses . An important source of WOM results from customer experiences of retail outlets, and a customer’s judgment about whether or not the experience left them feeling satisfied (or dissatisfied). We incorporate WOM in our simulation model by using the number of satisfied customers at the end of the day to calculate the number of additional customers visiting on the next day. The calculation takes into account that only a fraction of people act upon received WOM.

Our concept for representing internal stimulation comprises the exertion of influence on picking certain customer types more often than others. An enthusiastic shopper with a high satisfaction score is much more likely to go shopping more frequently than a disinterested shopper. Therefore, we have introduced some constraints (e.g. out of all customers picked 50% have to be enthusiastic shoppers, 30% normal shoppers, and 20% disinterested shoppers, or, customers with a higher satisfaction score are more likely to be picked act as consumer cues for triggering impulse buying. Internal cues include respondents’ positive and negative feeling states, and environmental cues include atmospheric cues in retail settings, marketer- controlled cues, and marketing mix stimuli. When modeling internal stimulation we specifically examine performance measures.

Empirical Data

Often agents are based on analytical models or heuristics and, in the absence of adequate empirical data, theoretical models are employed. However, we use frequency distributions to model state change delays and probability distributions to model decision making processes because statistical distributions are the best way in which we can represent the numerical data we have gathered during our case study work. In this way a population is created with individual differences between agents, mirroring the variability of attitudes and behaviors of their real human counterparts.

The frequency distributions are modeled as triangular distributions defining the time that an event lasts, using the minimum, mode, and maximum duration and these figures are based on our own observations and expert estimates in the absence of objective numerical data.

The probability distributions are partly based on company data (e.g. the rate at which each shopping visit results in a purchase) and partly on informed estimates (e.g. the patience of customers before they leave a queue). Table 1 and 2 show some of the distributions we have defined for our simulation models. We also gathered some company data about work team numbers and work team composition, varying opening hours and peak times, along with other operational details.

Mplementation Of The Main Concepts

Our ManPraSim model has been implemented in AnyLogic™ (version 5.5) which is a Java™ based multi- paradigm simulation software . During the implementation we have applied the knowledge, experience and data accumulated through our case study work. Within the simulation model we can represent the following actors: customers, service staff (with different levels of expertise) and different types of managers. Figure 3 shows a screenshot of the customer and staff agent logic in AnyLogic™ as it has been implemented in the latest version of our ManPraSim model (v4). Boxes represent states, arrows transitions, arrows with a dot on top entry points, circles with a B inside branches, and numbers satisfaction weights.

At the beginning of each simulation run the main customer pool is created which represents a population of simulation has started customers are chosen at a specified rate (customer arrival rate) and released into the something and customers who require a refund. If a refund is granted, a customer decides whether his or template consists of four main blocks which all use a very similar logic. In each block, in the first instance, customers try to obtain service directly from an employee and if they cannot obtain it (i.e. no suitable staff member is available) he or she has to queue. The customer is then either served as soon as the suitable staff member becomes available, or leaves the queue if they do not want to wait any longer (an autonomous decision). A complex queuing system has been implemented to support different queuing rules. Once are returned back to the main customer pool where he or she rests until they are picked again.

weights attached to the transitions that take place during the customer’s visit. For example, a customer starts browsing and then requires some help. If he or she obtains help immediately his or her satisfaction to the till. If he or she has to wait for help and then leaves the queue because he or she is fed up waiting his overall satisfaction score of +4 due to the good and prompt advisory service. The actual customer satisfaction weights shown in Figure 3 were derived based on our experience during the data collection period and validated via interviews with the store management staff.

In comparison to the customer agent state chart, the staff agent state chart is relatively simple. Whenever a customer requests a service and the staff member is available and has the right level of expertise for the task requested, the staff member commences this activity until the customer releases the staff member.

Whereas the customer is the active component of the simulation model, the staff member is currently passive, simply reacting to requests from the customer (although both are technically implemented as active objects).

Key Features

There are some additional key features that the simulation model possesses which we describe in the following two sections. First we provide an overview of some important features which have already been implemented in an older version of our simulation model (ManPraSim model v2) but are still relevant to modeling the evolution of customers. This sets the scene for the subsequent description of the new features we have implemented in the latest two versions of our simulation model (ManPraSim model v3 + v4).

Key Features Of The Manprasim Model V2

In the ManPraSim model v2 we introduced realistic footfall, customer types, a finite population of overview of these features because they are described in more detail elsewhere (see ). Realistic footfall (introduced in v2): There are certain peak times where a relatively high number of ManPraSim model v2 we have implemented hourly fluctuations through the addition of realistic footfall (based on some automatically recorded sales transaction data) reflecting different patterns (peaks and troughs) of customer footfall during the day and across different days of the week. In addition we can model the varying opening hours on different days of the week.

Customer types (introduced in v2): In real life customers display certain shopping behaviors which can be categorized. Hence we enhance the realism of our agents’ behavior in the ManPraSim modelv2 by introducing customer types to create a heterogeneous customer base, thereby allowing us to test customer populations closer to what we would find in reality. Customer types have been introduced based on three identified by the case study organization’s own market research (shopping enthusiasts, solution demanders, service seekers) and have been expanded by the addition of two further types (disinterested shoppers, and internet shoppers who are customers that only seek advice but are likely to buy only from the cheapest source i.e. the internet). The three customer types have been identified by the case study organization as the customers who make biggest contribution to their business, in terms of both value and frequency of sales.

In order to avoid artificially inflating the amount of sales that we model we have introduced the two additional types which use services but do not tend to make purchases. The definition of each type is based on the customer’s likelihood to perform a certain action, classified as either: low, moderate, or high. The definitions can be found in Table 3.

Ustomer Type

Distribution adaptation (introduced in v2): In the ManPraSim model v2 we have two algorithms which have been developed to imitate the influence of the customer type attributes mentioned above on customer behavior. They have been implemented as methods that are invoked when defining the state change delays modeled by triangular frequency distributions, and when supporting decision making modeled by probability distributions. Basically the methods define new threshold values for the distributions based on the likelihood values mentioned above. Program 1 shows an example of the pseudo code for the probability distribution threshold correction algorithm. If the customer is a shopping enthusiast and is about to make chart, second branch after leaving the browse state) a corrected threshold value (probability) for this decision is calculated. For this calculation the original threshold of 0.37 (see Table 2) is taken into account and, for a shopping enthusiast where there is a high likelihood to buy (see Table 3), the corrected threshold value is calculated as follows: 0.37+0.37/2 = 0.56. Consequently the likelihood that a shopping enthusiast 18.5%.

Program 1. Pseudo code for the probability distribution threshold correction algorithm

(

Finite customer population (introduced in v2): A key aspect to consider is that the most interesting system outcomes evolve over time and many of the goals of the retail company (e.g. service standards) are planned strategically over the long-term. In the ManPraSim model v2 we have therefore introduced a finite population of customers (main customer pool) where each customer agent is assigned certain characteristics based on the customer types mentioned above. The customer type split in the main customer pool can be defined via an initialization file before the execution of the simulation. The shopping experience of each visit (satisfaction index) is stored in the long term memory of the agent after he or she has left the time span.

Quick exit at closing time (introduced in v2): In the ManPraSim model v2 we have added transitions that emulate the behavior of customers when the store is closing. These transitions result in an immediate exit of each customer for his or her current state (i.e. the equivalent to a customer running out of shopping time and promptly exiting the store). Not all customer states have these additional transitions because it is for example very unlikely that a customer will leave the store immediately when they are already queuing to fifteen minute period, which conforms to what we have observed in the real system.

4.2.2 Key Features of the ManPraSim Models v3 + v4 In the ManPraSim model v3 we have introduced a staff pool with an additional staff type, a new operation mode to support sensitivity analyses and the first implementation of customers’ WOM featuring a static main customer pool. Furthermore, we have created some new performance measures that allow us to implementation of customers’ WOM featuring a dynamic main customer pool.

Staff pool and additional staff types (introduced in v3): Retail trends reflect that shops are now open for longer hours over more days of the week, and our case study organization is no exception. To accurately incorporate this source of system variability, we have introduced a staff pool in the ManPraSim model v3 to allow different staffing on different days of the week. The simulation uses Full Timers (FT) to cover all staff shifts required during weekdays. Additional staff who are required to cover busy weekend shifts are modeled by Part Timers (PT). The maximum number of staff required of each type is calculated during the simulation initialization. A staff pool is then created to include weekday FT staff of different types, and the required generic PT to fill the gaps left in the staff shifts which need to be covered. At the beginning of each day the simulation checks how many workers are required and picks the required amount of staff out of the pool at random. The selection process is ordered as follows: FT first and then PT, if required. PT workers have been defined as a generic staff type and can take over any required role. We have tried to model a staff rota with more complex constraints, for example FT staff working five days followed by two days off, however this has as yet proved unsuccessful. Therefore the currently modeled shifts do not incorporate days off work for FT staff.

Noise reduction mode (introduced in v3): In the ManPraSim model v3 a noise reduction mode has been implemented which allows us to conduct a sensitivity analysis with constant customer arrival rates, constant staffing throughout the week and constant opening hours. We have used the average values of real world case study data to define the constant values. This has resulted in different values for the different way at least we can reduce the system noise to clearly see the impact of the parameter under investigation.

When we progress to reintroduce the system noise, the knowledge we have accumulated when experimenting in noise reduction mode helps us to better understand patterns in systems outcomes, and be better able to attribute causation to the introduction of a particular variable (bearing in mind that in the end we are particularly interested in patterns between variables when they are interacting with one another and not in isolation).

Word of mouth (introduced in v3+v4): We have developed two different strategies for modeling WOM (which have been implemented in the ManPraSim model v3 (static main customer pool) and in v4 (dynamic main customer pool). Both strategies use the same algorithm to calculate the additional customers They differ in the way they pick these customers. In the first case we have used a static main customer pool which is defined with the customer split at the beginning of the simulation run. During the simulation run, the population of the main customer pool stays constant and all customers in the pool are available to be randomly picked to go shopping. In the second case we allow the main customer pool size to change; rather than picking customers from the existing pool we create new customers and add them to the main customer pool. When the main customer pool grows or declines the customer arrival rate is also adjusted, i.e. if the pool grows by 10% due to positive WOM so does the customer arrival rate and vice versa for negative growth.

The algorithm to calculate the number of additional new customers and the lost customers is the same for both strategies. We count the number of customers from the previous day who were satisfied and those who were dissatisfied with the service provided during their shopping visit(s). Satisfied customers are more advise others not to visit. The adoption rate (i.e. the success rate of convincing others to commit to either action) depends on the proportion of people who act upon the received WOM (the adoption fraction) and how many contacts a customer has (the contact rate). The equation behind this calculation is shown below in Equation 1.

−

At the beginning of each trading day we create a daily customer pool that contains randomly chosen customers from the main customer pool according to the required number of customers defined by the customer arrival rates for the day which might differ throughout the day (e.g. considering peak times).

For the first strategy (static customer pool) we simply pick the core number of customers (defined by the original customer arrival rates) plus an additional number of customers which is calculated through Equation 1. These customers are also selected from the main customer pool at random. If the number is negative we release customers from our daily customer pool back into the main customer pool. If we have to release all of our customers (due to customer dissatisfaction, which may result from bad service due to a large number of customers visiting on the previous day) the simulation terminates. This means that in a sense that the store is closed for good because it cannot recover from this situation in the current implementation.

The second strategy (dynamic customer pool) is slightly more complex. Here we expand or decrease the main customer pool by the number of additional or lost customers (respectively) calculated at the beginning of each trading day. This differs from the static strategy because we do not use the existing customers to model direct WOM influence (i.e. the ones resulting from the service satisfaction of the previous day). This seems to be closer to reality because the WOM in most cases carries positive messages and therefore attracts new visitors rather than motivating existing customers to come more often. Negative WOM created considering the hourly customer arrival rates (derived from our case study) as well as the actual size of the main customer pool (see Equation 2).

=

Once the daily pool size has been established, customers are chosen from the main customer pool according to the requested customer split and customer pool size. Then additional customers are created as new agents. The type of each new customer is determined at random. They are created with a positive will already have a positive attitude toward the shop (otherwise they would not have been attracted to it).

Through these additional (or lost) agents the daily customer pool size can expand (or contract). If the negative WOM causes an overall reduction in customers then we choose customers at random from the daily customer pool. Whether or not a particular customer is permanently removed from the main customer pool is determined as follows, depending on the satisfaction score of the chosen customer. If he or she has a positive satisfaction score then the score is neutralized and the customer is returned to the daily customer pool (i.e. he or she can come back for further shopping visits but the shop must start to satisfy the customer’s needs for good service to secure further visits from the customer). If he or she has a neutral or negative score he or she is permanently eliminated from the simulation (i.e. he or she will not come back for any more shopping trips). In the second case the main customer pool size will shrink. If the negative WOM results in more agents being deleted than there are customers available in the daily customer pool the For both strategies, when it reaches shop closing time all customers from the daily customer pool are released back into the main customer pool where they are available for the next day. For the second strategy (dynamic customer pool) it is very likely that the main pool size will then change due to WOM- triggered additions or losses of customers whereas for the first strategy (static customer pool) the main pool size remains constant.

New performance measures: With the introduction of a finite population (represented by our customer pool) we have had to rethink the way in which we collect statistics about the satisfaction of customers. visit, the individual’s satisfaction score (direction and value) has been recorded. Now the life span of a customer lasts the full runtime of the simulation and he or she can be picked several times to visit the satisfaction scores considering customers’ satisfaction history. These measures do not reflect individuals’ satisfaction with the current service experience but instead the satisfaction with the overall service experience during the lifetime of the agent. Furthermore, they are biased to some extent in that an indifferent rating quickly shifts into satisfaction or dissatisfaction (arguably this is realistic because most people tend to make a judgment one way or the other).

Whilst the above is still a valuable piece of information we would also like to know how current service is perceived by each customer. For this reason we have introduced a new set of performance measures to record the experience of each customer’s individual visit (referred to as Customer Satisfaction Measure – Experienced Per Visit or CSM-EPV). These are the same measures as before but on a day-to-day basis they are not anchored by the customer’s previous experiences. We also examine the sum of these ‘per visit’ scores across the lifetime of all customers (referred to as Customer Satisfaction Measure – Accumulated Historical Data or CSM-AHD).

Another new measure tracks the satisfaction growth for customers’ current and overall service experience. With the incorporation of varying customer arrival rates, opening hours and staffing we have brought in a set of performance measures that capture the impact of these variations on a daily basis. These measures satisfaction growth per each individual customer visit. At the end of the simulation run the simulation model produces a frequency distribution which informs us about how satisfied or dissatisfied individual customers have been with the service provided. Furthermore, all forms of customer queue (cashier, normal help, expert help, and refund decision) are now monitored through new performance measures that record how many people have been queuing in a specific queue, and how many of these lost their patience and left the queue prematurely. This measure helps us to understand individual customers’ needs because it tells us what individual customers think about the service provided. Finally, we have added some methods for writing all parameters and performance measures into files to support documentation and analysis of the experiments.

Odel Validation

Validation ensures that the model meets its intended requirements in terms of the methods employed and the results obtained. In order to test the operation of the ManPraSim model and ascertain face validity we have completed several experiments.

We have completed validation exercises on the micro and macro level, as proposed by . Micro simulation comprises tracing an individual entity through the system by building a trajectory of an individual’s behavior throughout the runtime. We have conducted this test for our two different agent types (customer and staff agents) considering all of the different roles these can take. Furthermore we have validated who triggers which process and for what reason. We have then extensively tested the underlying queuing system. On the macro level we have tried to establish simulation model performance outputs that are sound. It has turned out that conducting the experiments with the input data we collected during our case studies did not provide us with a satisfactory match to the performance output data of the real system.

we use here have been derived from real staff rotas. On paper these real rotas suggested that all workers are engaged in exactly the same (one) type of work throughout the day but we know from working with, and observing workers in, the case study organization that in reality each role includes a variety of activities.

Staff members in the real organization allocate their time proactively between competing tasks such as customer service, stock replenishment, and processing purchases. Proactive behavior refers to the individual staff members’ self-started, long term oriented, and persistent service behavior which goes beyond explicit job demands .

So far our simulation models incorporate only one type of work per staff member. For example, the A&TV staff rota indicates that only one dedicated cashier works on weekdays. When we have attempted to model this arrangement, customer queues have become extremely long, and the majority of customers ended up system we observed other staff members working flexibly to meet the customer demand, and if the queue of customers grew beyond a certain point then one or two would step in and open up further tills to take customers’ money before they became dissatisfied with waiting. Furthermore, we observed that a service staff member, when advising a customer, would often continue to close the sale (e.g. filling in guarantee forms and receiving payment from the customer) rather than asking the customer to queue at the till for a cashier whilst moving on to the next customer.

This means that currently our abstraction level is too high and we do not model the real system in an appropriate way. We hope to be able to fix this in a later version. For now we do not consider this to be a big problem so long as we are aware of it. We model as an exercise to gain insights into key variables and their causes and effects and to construct reasonable arguments as to why events can or cannot occur based on the model; we model for insights, not precise numbers.

In our experiments we have modulated the staffing levels to allow us to observe the effects of changing key changed).

Experiments

The purpose of the experiments described below is to further test the behavior of the simulation models as they become increasingly sophisticated, rather than to investigate management practices per se. We have defined a set of standard settings (including all probabilities, staffing levels and the customer type split) for added to the set of standard settings for subsequent experiments. For the experiments that involved only a static main customer pool the run length has been 10 weeks. For the experiments that involved a dynamic main customer pool or both types the run length has been 52 weeks. We have conducted 20 replications (in most cases) to allow the application of rigorous statistical tests.

Omparing Customer Pool Sizes

The first experiment is a sensitivity analysis. We want to find out what impact the customer pool size has on the simulation results and which of our performance measures are affected by it. For this experiment we have used our new noise reduction mode (described in Section 4.2.2). It allows us to focus our attention on the impact of the variable to be investigated. We have run the experiments separately for both case study increments of 2,000. We will interpret the results in terms of customer-related performance measures, which can be separated out between the standard measures which are defined explicitly in terms of at which dissatisfaction.

Hypothesis 1: We predict that varying the customer pool size will not significantly change standard customer measures nor those customer satisfaction measures related measured in terms of experience per visit (CSM-EPV).

Hypothesis 2: We predict that the customer satisfaction measures which accumulate historical data (CSM- AHD) will result in significantly more customers being categorised as either ‘satisfied’ or ‘dissatisfied’ when compared to the number of customers in these categories for the customer satisfaction measure CSM-

Epv.

Hypothesis 3: We predict that the customer pool size will significantly impact on all CSM-AHD measures, specifically the greater the customer pool size, the higher the count of neutral customers, and the lower the count of satisfied and dissatisfied customers.

Our results in percentage terms (see Table 4) appear to provide support for all hypotheses. In order to rigorously test our hypotheses, statistical analyses were applied to the raw customer counts as follows. To investigate Hypothesis 1, a series of one-way between groups ANOVAs were conducted separately for A&TV and WW. Levene’s test for homogeneity of variances was violated by CSM-EPV for satisfied customers in A&TV only, therefore a more stringent significance value was set for this variable (p<.01).

Given that 8 simultaneous comparisons were to be drawn, a Bonferroni adjustment was made to the alpha level, tightening the significance levels to p<.00625 (from p<.05) and p<.00125 (from p<.01). For A&TV, no significant differences were found in the standard performance measures or CSM-EPV between different customer pool sizes. For WW, no significant differences were found apart from the number of customer leaving before receiving normal help (p=.0059). Inspecting mean differences revealed that the mean customer count with a customer pool of 8,000 was higher than for 4,000, 6,000, and 10,000 customers. However, upon examination of supporting data, no single paired comparison was significantly different. Across customer pools, the mean customer count varied between just 4.6 and 7.5 customers leaving before receiving normal help, and so we can explain this pattern in terms of the relative infrequency of this event and conclude that the difference is of little practical significance. This leads us to conclude that overall the evidence supports Hypothesis 1. We predicted this pattern because customer decisions are driven by customer types and the proportional mix of each type is kept constant throughout the experiment.

To test Hypothesis 2, a series of independent samples T-tests were conducted to evaluate the differences between the two methods of measuring customer satisfaction. For A&TV, a significantly higher count of customers were categorized as satisfied on CSM-AHD (M=18,031.18, SD=824.11) as compared to their counterparts on the CSM-EPV measure (M=15,071.12, SD=108.47), t(102.43)=35.61, p<.000 (equal variances not assumed). The same pattern was reflected in a significantly higher count of dissatisfied customers (M=17,134.13, SD=895.06) on CSM-AHD as compared to CSM-EPV (M=11,404.65, SD=221.34), t(111.06)=62.14, p<.000 (equal variances not assumed). The effect sizes are very large in both cases, yielding .86 and .95 respectively.

For WW, a significantly higher count of customers were categorized as satisfied on CSM-AHD (M=54,161.66, SD=3,559.94) as compared to on CSM-EPV (M=33,508.33, SD=82.76), t(99.11)=58.00, p<.000 (equal variances not assumed). The inverse occurred for the count of dissatisfied customers, whereby a smaller number were counted on CSM-AHD (M=3,447.49, SD=1,049.66) than CSM-EPV (M=4,626.00, SD=128.59), t(101.97)=-11.14, p<.000 (equal variances not assumed). The effect size was very large for A&TV at .94, and medium-to-large at .39 for WW.

Overall as expected CSM-AHD generally results in a significantly higher count of satisfied and dissatisfied customers than CSM-EPV. This phenomenon has been discussed in Section 4.2.2. The evidence for A&TV unequivocally supports this hypothesis, whereas it is mixed for WW. The surprisingly relatively low count of dissatisfied of CSM-AHD in WW recurs in evaluation of the next hypothesis (see below).

Table 4. Descriptive statistics for Experiment 1 (all to 2 d.p.)

%

… leaving dissatisfied (accumulated historical data)

,000

To investigate Hypothesis 3, a series of one-way between groups ANOVAs were conducted. Levene’s test for homogeneity of variances was violated by CSM-AHD for satisfied and dissatisfied customers in A&TV and only dissatisfied customers in WW, therefore a more stringent significance value was set for these variables (p<.01). Further to this a Bonferroni adjustment was implemented because of the multiple comparisons, tightening the significance value to p< .0167 (from p<.05) and p<.0033 (from p<.01).

Separately for A&TV and WW, all 3 ANOVAs for each CSM-AHD variable exhibited significant differences (p<.000). Examining mean differences for A&TV: for satisfied customers, significant differences were present between all paired pool sizes except for between 2,000 and 4,000; 6,000 and 8,000; and 8,000 and 10,000. The mean customer count uniformly decreased as the customer pool size increased, with significant differences focused on comparisons between paired customer pool sizes with a greater size differential. For neutral customers, all paired comparisons were significantly different (p<.000), and examination of mean differences reveals a uniformly increasing trend in the customer count as the customer pool size increases. For dissatisfied customers, significant differences persisted (at p<.0033) between the following pairs: 2,000 and all other pool sizes; 4,000 and 8,000; and 4,000 and 10,000. As customer pool size increased, the count of dissatisfied customers uniformly decreased.

Looking at mean differences for WW: for satisfied customers, all paired comparisons were significantly different (p<.000), and an examination of mean differences revealed a uniformly decreasing trend in the count of satisfied customers as customer pool size increases. For neutral customers, again all paired comparisons exhibited significant differences (p<.000), and this time the mean customer counts revealed a uniformly increasing trend as customer pool size increases. Finally, for dissatisfied customers, all paired comparisons were significantly different (p<.000), and the mean customer counts behaved according to a uniformly increasing trend as customer pool size increases. These results follow a similar pattern to that presented in A&TV, apart from the increasing number of dissatisfied customers. We expect that this finding can be explained in terms of an accumulation of the impact of a number of subtle differences in the produce significant effects, see evaluation of Hypothesis 1).

Looking at the percentages presented in Table 4, we can see that the bigger the customer pool size the more the CSM-AHD values tend to the CSM-EPV values. This can be explained by the fact that with a larger accumulates some historical data) reduces. If we sufficiently increase the customer population we expect both customer satisfaction measures to show similar results because the majority of customers are only neutral satisfaction score rather than with an accumulated score. In summary for Hypothesis 3 we can conclude that the evidence largely supports our prediction, with the exception of dissatisfied customer counts in WW for which we have presented an explanation.

The results for Experiment 1 demonstrate that it is therefore important to select and maintain one customer pool size to ensure that all performance measures are providing comparable information for different experiments. Our case study organization does not collect or hold data on customer pool size so we have day (585 for A&TV and 915 for WW) and an estimate of customers’ average inter-arrival time (two weeks for A&TV and one week for WW). These values have been estimated considering the standard customer We use these customer pool sizes for all subsequent experiments.

Omparing Normal And Noise Reduction Mode

In our second experiment we want to investigate the importance of considering hourly differences in customer arrival rates and daily differences in staffing and opening hours (normal mode). These features and specific patterns that occur on a day-to-day basis. Modeling this level of detail can be problematic when we conduct a sensitivity analysis where we want to be able to attribute causation to the introduction of a particular variable. For this kind of experiment we prefer to control some of the system noise to clearly see the impact of the parameter under investigation (noise reduction mode).

Hypothesis 4: We predict that the standard customer performance measures will not significantly differ across the two modes. Hypothesis 5: We predict the two modes will produce significantly different values when looking at the daily performance measures on a day-to-day basis (rather than using averages).

Table 5. Descriptive statistics for Experiment 2 (all to 2 d.p.)

… Leaving Before Finding Anything [Runtime]

20,182.00 134.56 20,661.65 145.08 20,789.75 176.33

… Leaving Whilst Waiting To Pay

6,291.95 165.73 13,148.10 200.16 18,050.20 164.96 -224.88

… Leaving Before Finding Anything

27,662.80 166.79 33,120.80 198.49 37,173.90 196.79 -164.89

Af = 1

Looking at Figure 4 it is clear that there is very little variation between the two different operation modes across the range of standard performance measures. Examining the descriptives (see Table 5), in most cases the runtime performance measures in both modes are approximately the same with a small number of exceptions. Contrary to hypothesis 4, in WW approximately 25% more customers leave whilst waiting to pay in the normal mode (which consequently influences both customer satisfaction measures), as opposed to the noise reduction mode. This apparent cashier bottleneck appears to be exacerbated by any combination of the three factors which are held constant in noise reduction mode. Further analysis is required to isolate the precise cause. We can also see that, as predicted, the smaller the values the more they differ (on an absolute basis) between the two modes as they are accumulated from a smaller number of events and therefore the influence of different random number streams is more apparent.

Examining the daily measures on a day-to-day basis for A&TV in Figure 5, we can observe clear differentiation between the number of customers and transactions across weekdays, Saturdays and Sundays in normal mode whereas the noise reduction mode, as expected, shows no clear patterns. This additional information can be very useful for optimizing the system. For example, we have the lowest number of transactions on Sundays (day 1, 8, etc.) although we do not have the lowest number of customers on this day, therefore we must have a problem with the optimizing the staffing arrangement on Sundays, because on average more customers leave the shop on Sundays without buying anything, despite the fact that the probability of this remains the same for all days of the week.

Overall the experiment has shown that it is legitimate to use the noise reduction mode within the scope of a sensitivity analysis to test the impact of a specific factor on system behavior. It is only when we are interested in specific features that are not available in noise reduction mode (e.g. when we want to study differences between particular days of the week or when we need to obtain quantitative data to optimize a real system) that we need to choose the normal mode.

Authors:

Peder EZ Larson 1, 2,* , Jenna ML Bernard1, James A Bankson 3, Nikolaj Bøgh 4, Robert A Bok1, Albert P. Chen 5, Charles H Cunningham 6,7, Jeremy Gordon1, Jan-Bernd Hövener 8, Christoffer Laustsen 4, Dirk Mayer 9,10, Mary A McLean11 12, Franz Schilling13, James Slater1, Jean-Luc Vanderheyden5, 14, Cornelius von Morze 15, Daniel B Vigneron1, 2, Duan Xu1, 2, and the HP 13C

94143, Usa.

Denmark. 5 GE Healthcare, Menlo Park, California, USA. 6 Physical Sciences, Sunnybrook Research Institute, Toronto, Ontario, Canada.

8 Section Biomedical Imaging, Molecular Imaging North Competence Center (MOIN CC), Medicine, Baltimore, MD, USA. Cambridge, United Kingdom.

14Jlvmi Consulting Llc, Dousman, Wi, Usa

#See Acknowledgements for a list of all HP 13C MRI Consensus Group Members This work was supported by the ISMRM Hyperpolarized Media MR Study Group, the ISMRM Hyperpolarization Methods & Equipment Study Group, and the Hyperpolarized MRI Technology Resource Center (NIH/NIBIB grant P41EB013598).

Abstract

MRI with hyperpolarized (HP) 13C agents, also known as HP 13C MRI, can measure processes such as localized metabolism that is altered in numerous cancers, liver, heart, kidney diseases, and more. It has been translated into human studies during the past 10 years, with recent rapid growth in studies largely based on increasing availability of hyperpolarized agent preparation methods suitable for use in humans. This paper aims to capture the current successful practices for HP MRI human studies with [1-13C]pyruvate - by far the most commonly used agent, which sits at a key metabolic junction in glycolysis. The paper is divided into four major topic areas: (1) HP 13C-pyruvate preparation, (2) MRI system setup and calibrations, (3) data acquisition and image reconstruction, and (4) data analysis and quantification. In each area, we identified the key components for a successful study, summarized both published studies and current practices, and discuss evidence gaps, strengths, and limitations. This paper is the output of the “HP 13C MRI Consensus Group” as well as the ISMRM Hyperpolarized Media MR and Hyperpolarized Methods & Equipment study groups. It further aims to provide a comprehensive reference for future consensus building as the field continues to advance human studies with this metabolic imaging modality.

Keywords: Hyperpolarized MRI, metabolic imaging, carbon-13, pyruvate, dissolution dynamic

Introduction

MRI with hyperpolarized 13C agents, also known as hyperpolarized (HP) 13C MRI, has shown great potential as a novel imaging modality, particularly for its ability to probe metabolic processes in real time. The first human studies with HP [1-13C]pyruvate were performed in 2011 in prostate cancer patients (1).

Since then, there have been over 60 papers published with imaging results of human subjects from 13 different sites, with applications including prostate cancer, brain tumors, breast cancer, kidney cancer, pancreatic cancer, metastatic disease, liver disease, ischemic heart disease, diabetes and cardiomyopathies. The vast majority of these studies used [1-13C]pyruvate (1–63), where [2-13C]pyruvate (64) and 13C-urea (56) have been demonstrated too.

As clinical HP 13C MRI advances, there is a growing need to build consensus for best practices, which are critical for comparing data across sites, performing multi-site trials,deploying methods to new sites, partnering with vendors, and potentially for obtaining broader regulatory approvals.

In March 2022, we initiated an effort to build consensus within the HP 13C MRI community with this opportunity in mind, and it was greeted with strong enthusiasm. The “HP 13C MRI Consensus Group”, containing over 55 members from 27 sites, identified the area of greatest need and opportunity for consensus building to be HP [1-13C]pyruvate human

●

Pyruvate is the most mature and widely used HP agent and has the most significant translational evidence emphasizing the potential clinical impact.

●

Clinical trials, particularly multi-site trials, have the strongest need for consensus methods to ensure that data can be combined across sites. This work is a Position Paper for which the goal is to describe current successful practices and study methods for HP [1-13C]pyruvate human studies along with justification to support those practices. This is divided into four major topic areas: (1) HP 13C-pyruvate preparation, (2) MRI system setup and calibrations, (3) data acquisition and image reconstruction, and (4) data analysis and quantification (Fig. 1). The current successful practices and study methods include a literature review of published peer-reviewed journal papers showing human HP [1-13C]pyruvate study data, up to September 2022 (1–63), as well as new unpublished information from surveys of HP 13C study sites. Based on this information, we also highlight the evidence gaps, strengths, and limitations of current practices which are summarized at the end of each section.

Figure 1: Illustration of the HP 13C MRI human study process, including the 4 major areas covered in this paper: Hyperpolarized 13C-pyruvate preparation, MRI system setup and calibration, Acquisition and Reconstruction, and Data Analysis and Quantification.

Figure 2: Anatomical targets of HP [1-13C]pyruvate MRI human studies published up to September 2022.

Hyperpolarized 13C-Pyruvate Preparation

This section covers the processes for creating the HP agent, 13C pyruvate, and will include many aspects and considerations that are needed to safely and effectively prepare doses for metabolic imaging studies in human subjects. These include material, personnel, equipment and facility, fluid path preparation, quality control, and release.

It is helpful to understand that the specifications of a dose of 13C pyruvate suitable for in vivo MR HP metabolic imaging were shaped in part by early preclinical studies performed by GE HealthCare summarized in Ref. (65). In short, the safety of the two novel drug components, 13C pyruvate and the electron paramagnetic agent (EPA) AH111501, were demonstrated in those studies. The more precise formulation of the dose suitable for human use was then determined from clinical studies (66) that included two Phase 1 clinical trials in young and elderly healthy volunteers without hyperpolarization of the 13C nuclei and another Phase 1/2a dose escalation and imaging feasibility study with HP 13C pyruvate in 31 prostate cancer patients at the With the exception of the first HP 13C imaging clinical trial, which utilized a prototype device in a cleanroom (1), all HP 13C studies performed in humans to date have utilized the SPINlab polarizer (manufactured by GE HealthCare). Consequently all doses of the HP 13C pyruvate delivered by SPINlab have been produced using the “SPINlab Pharmacy Kit” that serves as the container-closure system for the various drug components (13C pyruvic acid and EPA mixture, dissolution medium, and neutralization and dilution medium) during sample polarization, dissolution and quality control (QC) processes. Thus many aspects of the HP sample preparation considerations discussed below are related to the SPINlab instrument and the consumables designed to be used with it (67).

General Considerations

While more than 860 patients or healthy subjects having been injected with HP 13C pyruvate as of January 2022 without reports of any serious adverse events (68), HP 13C pyruvate injection remains an investigational MR contrast agent and can only be administered by those with Investigational New Drug (IND) exemption from the Food and Drug Administration (FDA) in the USA, a Clinical Trial Application (CTA) in Canada, approval from National Research Ethics Committee Services in the UK, or approval from the relevant local regulatory body. Thus, methods and processes involved to produce a dose should have patient safety as the first priority. Since utilizing dissolution dynamic nuclear polarization (dissolution-DNP) for human use is still a relatively new development, there are no existing published regulatory guidelines specifically for this method.

There are two major production styles that determine how various sites approach the agent preparation. In the US, the most common approach is to rely on a sterilizing filter (“Terminal Sterilization”) to ensure sterility of the final product, akin to PET tracer production, where a starting molecule with a radioisotope is processed using various other ingredients to make the final, desired and injectable contrast agent within a necessarily short amount of time (69). For these sites, sterilization of the components and accessories upstream of this filter are not required, although many of them were manufactured and tested following Good Manufacturing Practice (GMP) or Good Laboratory Practice (GLP) requirements. The filling process is usually performed under an ISO 5 laminar flow hood, but a clean room or an isolator is not required.

This approach is typically accompanied by testing the integrity of the sterilizing filter prior to release of the dose for injection. Typically, post release endotoxin and sterility tests are performed using an aliquot reserved from each released dose.

In the UK and EU, the most common approach is to more-closely follow sterile pharmaceutical compounding guidelines (70), where all components and ingredients are required to be sterile or manufactured under GMP guidelines and are assembled and filled within a clean room environment or an isolator system (“Sterile Preparation”). Typically a batch of Pharmacy Kits for HP 13C pyruvate injection are prepared together. The sterility of the final dose is also ensured by batch validation testing, in addition to the sterility of the ingredients and the sterile compounding process. The endotoxin and sterility testing are performed for the process validation but are not performed for each injected dose.

Some institutions fill and assemble the Pharmacy Kit required for a specific study on the same day or the day prior to polarization, dissolution, and patient administration, but others have also demonstrated the feasibility of preparing a batch of kits, keeping them in a -20ºC freezer and using them over a period of a few months.

Beyond the obvious requirements that the process and the facility has to ultimately produce a dose that is safe to inject into a human, regulatory authorities will also focus on the question “Are you in control of your processes?”. To be in control of your process requires an in-depth and broad understanding of all processes involved in pre, post, and during the production process.

Personnel

It is typical and may be required to have licensed personnel involved in the production process depending on local regulations.Typically a pharmacist, radiopharmacist or other similarly qualified person (QP), in charge of the facility where the Pharmacy Kit filling and preparation is taking place, is responsible for the overall process and the release of the injectable dose.

Qualified cleanroom technicians are often involved in the Pharmacy Kit filling under the supervision of the pharmacist or QP. As is required for pharmaceutical compounding or PET tracer production, training requirements and training records for all personnel need to be maintained and available for audit by the FDA or equivalent.

Equipment And Facility

The facility and all equipment need to have standard operating procedures (SOPs) that describe how equipment is used, maintained, and calibrated to comply with relevant legislation. Currently, almost all the filling of the Pharmacy Kit takes place within a compounding laminar flow hood or isolator (typically ISO 5). At some sites, the filling is conducted within a cleanroom, while at others, it is conducted in a dedicated non-cleanroom space, reflecting differences in cleanroom approach and specifications between regulators worldwide (71). Some equipment or facilities, such as the compounding hood or cleanroom, may require external certified laboratories for testing.

Material Handling

Material handling guidelines (69,70) require SOPs detailing a system to track all of the materials involved in the HP production process for a particular patient dose, similar to current good manufacturing practice (cGMP) requirements for material handling for drug compounding. This includes acceptance standards, storage conditions, amount used in the patient dose for each ingredient and materials used in the assembly of the fluid path and Pharmacy Kit. Currently some users choose to open and inspect and sometimes modify the Pharmacy Kits upon arrival, but some users keep them in the sealed packaging until they are required for dose preparation.

Pharmacy Kit Filling And Assembling

As required by an IND or its equivalent, the preparation of the doses of HP 13C agent are detailed in the Chemistry, Manufacturing, and Control (CMC) section of an applicable regulatory submission; an example of this has been made available (72). It describes the processes of filling the Pharmacy Kit with the different components that make up the final drug product, and of assembling the final kit for either storage or immediate use in the polarizer. Special attention should be given to the laser welding process in order to satisfy installation qualification (IQ) and operational qualification (OQ). Typically, the final developed process is validated by process qualification (PQ) runs, during which 3 or more Pharmacy Kits are filled and used and the final HP 13C products are tested for endotoxin and sterility and to confirm that they meet the dose specifications for injections (usually including pyruvate concentration, residual EPA concentration, pH, liquid state polarization level and dose temperature). The data from 3 consecutive PQ runs are submitted as part of the IND submission (or its equivalent), and are often also reviewed by the Institutional Review Board (IRB) where the studies are conducted.

Quality Control And Dose Release

The quality control (QC) and dose release can be separated into two aspects: one is the QC and release of the filled Pharmacy Kit, and second is the QC and release of the HP 13C agent for injection, after polarization and dissolution. For institutions filling a batch of kits and storing them to use over a period of time, typically the batch can be released based on initial validation, environmental monitoring data from the day of kit production, and if filters are used during preparation of any of the components, filter integrity testing. But in some cases one or more kits are used for validation before the batch of kits are released for future use. For institutions that fill only the kits required for specific studies shortly before the experiment, the filled kits often do not go through separate release tests before they are used.

The quality control of the HP 13C pyruvate solution post dissolution is primarily performed to ensure that the agent meets the dose specifications (Table 1) before it is administered to the subject. These specifications target both safety (pH, residual EPA, temperature) and efficacy (pyruvate concentration, polarization, volume). Typically, the pyruvate concentration, residual EPA concentration, pH, dose temperature, dose volume, and liquid state polarization are measured by the QC accessory associated with the SPINlab polarizer. Some users perform a secondary measurement for one of the parameters, such as pH, using a different instrument or pH paper. For sites that do not go through a separate release testing process for batch filled kits, the integrity of the sterilization assurance filter, a part of the Pharmacy Kit, is typically tested as a part of the dose release. It is also common for these users to preserve an aliquot of the final HP 13C pyruvate solution for post-release endotoxin and sterility testing. This testing cannot be completed fast enough to test an individual dose prior to injection, but this is why other processes such as PQ runs and validation testing are done to minimize the chance a subject could be injected with a contaminated dose.

The Final Dose Release And Injection

should be done under the supervision of a licensed professional, based on local regulations.

Some Key Challenges

Many of the challenges associated with HP 13C pyruvate preparation can be attributed to the conditions required for the dissolution-DNP method of high magnetic field (~3-7 T) and very low temperature (~1 K) during polarization, with pressurized and superheated water necessary for the rapid dissolution event. These extreme conditions are quite challenging for the design of the container-closure and fluid path system. In particular, the cryogenic temperature in the polarizer requires special attention to any moisture or ambient (moist) air introduced into that portion of the fluid path, which can form an ice block at ~1 K. This ice can lead to flow restriction during the dissolution event and reduce the strength of the laser welded bond between the cryovial and its cap. This can ultimately produce failures in the dissolution step, including variations in final pyruvate concentration and pH that may fail to meet QC release criteria as well as fluid path ruptures that provide no available dose and result in polarizer down-time.

The polarization of the HP 13C pyruvate sample decays quickly over the span of a few minutes after dissolution, and thus the process of dissolution, QC for release, and injection should be completed as fast as possible to preserve the high polarization level achieved. Any delays in the preparation process, such as transportation time or equipment malfunction, can significantly reduce the final polarization and result in lower quality imaging data.

Current Practices

A summary of data collected from all sites performing clinical trials with HP 13C-pyruvate is shown in Fig. 3 and Table 1, including the specification of the final dose and how the quality control and release of the final dose are performed. There is a split in the Production Style, described in the General Considerations section above, with 8/13 sites using Sterile Preparation versus 5/13 using Terminal Sterilization. While many of the dose specifications show notable differences in acceptable ranges, all of these variations listed in tables have been successfully and safely been used to perform HP 13C pyruvate studies in humans. Their differences depend on the institutions’ preferences, resources and their particular regulatory situation. There is high similarity in pyruvate ranges, temperature ranges, EPA limits, and volume limits. There is modest variability in pH ranges and large variability in the endotoxin test limit. There is a 3-fold difference in acceptable polarization levels, which are measured to ensure a futile dose is not injected since the polarization is directly proportional to SNR. This reflects the decision by several sites to believe that useful data can be still be obtained with suboptimal polarizations.

Figure 3: Hyperpolarized agent preparation methods reported by sites currently performing HP

In House

Table 1: HP 13C-pyruvate preparation parameters, methods, and dose specifications used for quality control testing and release as well as validation. These were obtained from a survey of all sites performing clinical trials with HP [1-13C]pyruvate. The parameters used for product release are noted in bold text, otherwise these parameters are measured for batch validation or other QC measurements. The endotoxin and sterility testing are performed during process validation of the batch and/or post-injection, and largely depends on the agent production approach.

Summary

The overall safety record of HP 13C-pyruvate has been very strong, and the SPINlab hyperpolarizer has proven to provide high polarizations at human sized doses while meeting numerous QC and release criteria. A weakness remains the failure modes of the SPINlab Phamacy Kits (e.g. ice blocks, path ruptures), which are placed under extreme requirements particularly during dissolution. The preparation process still requires a high degree of expertise.

Therefore, there is a significant need to improve the reliability, robustness, and ease of operation for generating HP 13C-pyruvate doses for human studies. Furthermore, there is a divide between manufacturing and sterile compounding style preparation as well as other site-specific practices, resulting in variations in SOPs and justification required to relevant regulatory bodies. There have also been no comparisons between these approaches. It is also unclear what release criteria and QC parameters are truly required to ensure patient safety.

However, all of the reported methods are acceptable and approved by the appropriate regulatory authorities, and have led to the rapid expansion of successful human studies in recent years.

Mri System Setup And Calibrations

This section covers the MRI system setup, including the imaging system, RF coils, phantoms, and prescan calibration methods.

Imaging System

The main prerequisite for a given MRI scanner to be capable of supporting studies with HP 13C is its “broadband” capability to transmit and receive radiofrequency (RF) signal at the frequency of 13C, which is around 4 times lower than 1H. This does not come as a default on clinical MR devices. The transmit power of the broadband amplifier should also be sufficient to support the intended flip angle and RF pulse shape with the employed transmission RF coil(s) for 13C. Most studies to date use relatively low flip angles (< 90 degrees) for HP 13C in order to preserve polarization for time-resolved imaging. The capability to receive 13C signal on multiple channels is also desirable to increase SNR, as discussed further in the “RF coils” section.

The choice of magnetic field strength is primarily dependent on the metabolites’ frequency separation due to chemical shift dispersion and 1H imaging. High field strengths do not enhance hyperpolarized 13C signal as they do for 1H because the signal strength in a HP experiment relies on manipulating the population of quantum energy states outside of the MRI scanner.

However, the injected HP 13C-pyruvate and its metabolic products have greater frequency separation at higher fields, and it may thus be easier to separate and quantify these resonances at higher fields. This comes at the cost of a reduction in the achievable T2* and often reduced T1. As the initial polarization is independent of the imaging field strength it has been proposed that the increased T2* at 1.5T can potentially be exploited to increase SNR by adapting the acquisition bandwidth or reduce off-resonance imaging effects in cases when the decay of the transverse magnetization is dominated by T2* (73). In practice, 3T has been used in all published human 13C-pyruvate studies surveyed (Supporting Table S1), and comprises the majority of scanners currently in use for human studies (Table 3). A field strength of 3T is well-suited for 1H MRI anatomical reference and correlative imaging.

Stronger and more rapidly slewing magnetic field gradients support more rapid spatial encoding, particularly for metabolite-specific single-shot imaging using echo-planar imaging (EPI) or spiral imaging (See “Acquisition and Reconstruction”). Although the spatial resolution acquired for HP 13C imaging is typically much coarser than for 1H MRI, the factor of ~4 in gyromagnetic ratio leads to the same reduction factor in performance of the gradient system, so 13C experiments are potentially more limited by gradient hardware performance. To date, all human studies have used the commercially-available integrated gradient systems provided in clinical MRI scanners.

Optimization of scanner design has understandably focused on minimization of artifacts in 1H MRI, where devices such as room lights, the gradient amplifiers, and the motors driving the patient bed are checked to ensure that they do not produce RF interference at the 1H frequency, but artifacts may arise at other frequencies. Eddy current compensation is also not always appropriately adjusted for nuclei at other frequencies (74). In order to optimize for 13C, many sites have performed checks on phantoms for RF interference, gradient artifacts, and eddy currents (74), including the use of post-hoc gradient impulse response function characterisation and correction, and some vendors have fixed these issues as well.

Rf Coils

For HP 13C imaging studies in humans, RF coils for both 1H and 13C nuclei are needed, with 1H MRI providing an anatomical reference for registration and optional additional multiparametric MRI readouts. At the Larmor frequency of 13C nuclei, the relative contributions from coil noise compared to sample noise increase compared to 1H (73,75), although sample noise still is likely the dominant contributor for human-sized coils at 32.1MHz - the resonance frequency of 13C nuclei at 3T.

The key requirement for human 13C-pyruvate RF coils are that the coil geometry and sensitive volume must cover the volume of interest in the subject. Table 2 and Figure 4 shows coil configurations that have been used and optimized for applications in different anatomic regions.

Volume resonators are most commonly used for transmit, as they surround the subject to

Provide B1 Transmit Across The Fov (B1

+). While 1H relies on a large birdcage (“body”) coil built into the scanner, 13C transmit coils must be placed inside the bore. This takes up valuable space within the magnet, and also has led to the use of designs with relatively inhomogeneous

B1

+. Many human studies have used Helmholz pair resonators for transmit, including the “clamshell coil”, which has a notably inhomogeneous B1

+ Profile But Has Been Used Because Of

relatively easy integration into the scanner bore. B1

+ Variation Results In Variations In The Flip

angles that control the use of the hyperpolarized magnetization and creates errors in common HP metrics (9,76). The exception are head coils, where birdcage designs with highly

Homogeneous B1

+ can be placed around the head while easily fitting inside the bore. As with 1H MRI, higher SNR can typically be achieved by smaller receive coil elements, such as surface coils or phased arrays, and the majority of 13C receive coils used have layouts similar to 1H phased arrays.

RF coil quality control is important to ensure proper functioning of the coils to provide consistent imaging quality, especially with limited natural abundance 13C signal in vivo. It typically involves 1) a physical integrity check of the coil cables and connectors and 2) phantom SNR tests to check the coil’s performance and to monitor it over time (see Phantoms below). An useful reference for RF coil quality control is outlined in the MRI accreditation program of the American College of Radiology (77) and can be adapted for 13C coils.

Notably, configurations for brain and prostate studies used dual-tuned 1H/13C coil designs, which greatly simplify workflow and registration of 1H and 13C images, as no switching of coils is needed.

(1)

Table 2: RF coil configurations reported for human HP [1-13C]pyruvate studies.

Tx = Transmit

coil, RX = receive coil. The commonly used “clamshell” TX coil is a Helmholz pair design. For 1H RF configurations, all used the Body coil for TX unless otherwise noted, and “repositioned” indicates the 13C coil was removed for 1H imaging. One representative reference is listed for each configuration. The RF coil configurations reported in the reviewed papers are shown in Supporting Table S1.

Figure 4: Examples of RF coil configurations used for human HP [1-13C]pyruvate brain studies. (A,B) 13C Clamshell TX (Helmholz pair) and 2× 4-channel paddle RX arrays. (C) 13C Birdcage volume TX and 32-channel RX array (RX array slides into TX coil). (D) 13C Birdcage volume TX and 24-channel RX array, combined with a 1H 8-channel RX array. Image reproduced with permission from Ref (16).

Phantoms

Since hyperpolarized magnetization is non-renewable, phantoms containing 13C nuclei are important to: 1) test the multi-nuclear capabilities of the imaging system, including all parts of the signal excitation and receive chain; 2) perform calibration measurements before a scan with hyperpolarized nuclei; and 3) perform necessary pre-scan adjustments (see “Prescan Calibration” section). The phantoms currently in use are listed in Table 3. Their composition must provide sufficient 13C signal, with additional considerations of conductivity, stability, chemical shift(s) present, potential for dynamic imaging, and cost. The phantom geometries are typically either compact, in order to be used alongside the subject during a HP scan, or large enough to mimic the inner volume of a RF coil for system testing.

One popular compact design contains enriched 13C-urea at high concentration, typically 8 M, which provides a single resonance, placed inside a small container ~1 mL. The most common recipe mixes 13C-urea in a 90% water/10% glycerol solution, with glycerol used to increase the urea solubility and doping with a Gd-based contrast agent to shorten T1 which increases the potential SNR per unit time. For example, when Dotarem is added at a 3:1000 volume ratio the 13C-urea T1 is around 500 ms and T2 is around 100 ms. However, when testing pulse sequences influenced by T1 and T2, doping should be used carefully. This phantom is suitable for frequency calibration, transmit gain calibration, sequence testing, and as a fiducial marker when placed next to a patient. However, enriched 13C-urea has a relatively high cost compared to natural abundance compounds.

For larger volumes (>100 ml), the phantoms most often used contain undiluted ethylene glycol, glycerol, or dimethyl silicone. These compounds have sufficiently high carbon concentrations to provide sufficient 13C signal even with the 1.1% natural abundance of 13C. These larger phantoms matching the inner volume of an RF coil are useful for coil testing, including transmit

+) And Receive (B1

-) coil profile mapping, as well as to mimic acquisitions using in vivo FOV requirements. In this case, size and conductivity should match the expected subject size in order to mimic coil loading and get a realistic estimation of B1+. Large-volume natural abundance urea phantoms have also been used by some sites, but suffer from higher conductivity compared to biological tissues. Typically, it is easier to increase the conductivity and hence coil loading of the non-conductive phantom by adding NaCl to match physiological loading (16,78).

Dynamic phantoms that aim to mimic metabolite kinetics have also been developed (79–81), and have the potential to more closely mimic the HP experiment, but so far these are not widely used.

Prescan Calibration

Prior to performing an MRI acquisition, the so-called prescan procedure is used to set the shim parameters to maximize B0 homogeneity over the field of view (FOV) or a specific region of interest (ROI), the scanner center frequency (CF), the RF transmit gain, and the receiver gain.

While this calibration procedure is usually automated for 1H, the lack of sufficient natural abundance 13C signal prevents use of automated methods. (Although natural abundance 13C lipid signal has been detected, there are so far no reports on using this signal for prescan.) Table 3 shows current practices across sites.

Maximizing B0 homogeneity is independent of the nucleus and is therefore performed prior to 13C imaging using the 1H water signal and existing shimming tools, such as by a standard automated process (“Auto Shimming”) or using high order shimming routines. Similarly, the 13C CF can be calculated from the 1H CF using a predetermined scaling factor that depends on the target chemical shift (82). Another common approach used is to have a small, high-concentration 13C phantom, e.g. 8M 13C-urea, integrated in the RF coil or placed next to the scan subject (1). The reference frequency can also be based on real-time measurements after the HP injection but prior to imaging (83). Both the CF and B0 shimming are critical when using spectrally-selective RF pulses, as inmetabolite-specific imaging methods, where the desired excitation bandwidths are typically very narrow and frequency offsets can lead to a failure mode that is only apparent after injection.

The calibration of the RF transmit power is typically performed on a small, high-concentration 13C phantom placed near the region of interest during the scan or on a large 13C phantom of similar size and coil loading as the subject, prior to the subject scan. Reference power is often done by sweeping the power in a pulse-acquire sequence (53,62), or the Bloch-Siegert method (52,84). When using a small phantom, the location of the phantom, B1

+ Inhomogeneity As Well

as any shielding effects, e.g., when the phantom is integrated into a coil (1), may degrade the accuracy. Other methods include real-time Bloch-Siegert method measurements after the HP injection (83), and using the stronger natural abundance 23Na signal that is close enough to the 13C resonance frequency to be detected by 13C coils (82).

The receiver gain is predetermined, either systematically based on independent phantom measurements and assuming the dose and polarization of the HP compound is known prior to injection, or based on past HP imaging studies.

Power [Kw]

Phantom(s) - during study Phantom(s) - before study 13C Frequency

8

13C-bicarbonate doped with dimethyl silicone, various

Power [Kw]

Phantom(s) - during study Phantom(s) - before study 13C Frequency

Maximum Values

Table 3: Summary of the imaging systems, phantoms, and prescan procedures used at sites currently performing HP 13C-pyruvate human studies. These were obtained from a survey of all sites performing clinical trials with HP [1-13C]pyruvate. *Previously performed studies with a Siemens 3T Tim Trio. The imaging systems, phantoms, and prescan procedures reported in the reviewed papers are shown in Supporting Table S1.

Summary

Commercially available 3T MRI systems are by far the most commonly used for human HP 13C-pyruvate studies, although a systematic investigation of the impact of B0 has only recently been investigated (73). The multi-nuclear RF transmit and receive chain has proven sufficient for current acquisition strategies, although many sites have observed artifacts due to RF interference, gradient interference, and residual eddy currents when operating at the 13C frequency. A variety of 13C RF coils, tailored for numerous anatomical targets, have been successfully demonstrated, with the main limitation that most transmit coils take up a lot of additional space inside the bore and provide relatively inhomogeneous B1

+ Profiles. The

phantoms used have converged into generally 2 categories - small phantoms containing 13C-enriched compounds that can be used during the study and human-sized phantoms containing compounds with high carbon concentrations but without 13C enrichment that are used to test and calibrate the coils. There are no standardized compositions or geometry, and dynamic phantoms that recapitulate in vivo kinetics would be desirable but are still an emerging area. Prescan calibration procedures were not well defined in most publications, so we surveyed individual sites to determine current practices. Calibration procedures for the B0 field (13C CF and shimming) for most sites take advantage of 1H signal and methods, while methods

For Calibration Of B1

+ is more variable across sites, likely a reflection of remaining challenges in how to perform this calibration. Standardization of both phantoms and calibration procedures would synergistically improve the robustness and reproducibility of HP 13C studies.

Acquisition And Reconstruction

Data acquisition strategies in human HP [1-13C]pyruvate MRI studies must account for multiple chemical shifts, efficiently utilize the non-renewable HP magnetization, and acquire data quickly relative to metabolism and relaxation decay processes. These studies require spectral encoding to separate metabolites, necessitating pulse sequences that efficiently encode up to 5D data (3 spatial + 1 spectral + 1 temporal dimension). RF pulses must efficiently sample without immediately saturating the non-renewable HP magnetization, and sequences must acquire data quickly and be robust to both experimental and physiologic variation (e.g. B1

+ Inhomogeneity,

variation in perfusion) to ensure reproducibility and minimize scan-to-scan variability. This section covers current successful practices for data acquisition in human [1-13C]pyruvate studies, and accompanying 1H imaging, from different anatomic regions, including scan parameters and image reconstruction.

Acquisition And Reconstruction Methods

The acquisition methods used in human [1-13C]pyruvate studies can be classified into 3 categories: 1) MR spectroscopy or MR spectroscopic imaging (“MRS/I”), 2) chemical shift encoding methods, and 3) metabolite-specific imaging (Fig. 5).

Mrs/I Methods Specifically

resolve a spectrum that can be analyzed to extract expected as well as unexpected resonances, making this approach very robust. It was used in many initial studies (1).

Chemical Shift

encoding methods, most commonly the Iterative Decomposition of water and fat with Echo Asymmetry and Least-squares estimation (IDEAL) method, use imaging sequences acquired with multiple TEs and rely on a model-based separation of expected chemical shifts (85).

Metabolite-specific imaging methods use specialized RF pulses that are spatially and spectrally selective to excite individual metabolites which are then typically imaged with fast k-space trajectories such as echo planar imaging (EPI) or spirals (86).

Their Application To Different

organ systems is described below. The image reconstruction methods used in human [1-13C]pyruvate studies have typically been conventional methods (e.g. FFT, non-uniform FFT, or equivalent). The incorporation of accelerated imaging and advanced reconstruction methods including parallel imaging (4,57,87) and compressed sensing (7) has also been applied in human studies for improved spatial resolution, temporal resolution and coverage, but have the potential for additional artifacts as well as SNR losses due to ill-conditioning of the reconstruction (e.g. g-factor).

The Majority Of

published studies do not use accelerated imaging indicating the resolution and coverage achievable without acceleration is currently adequate for successful data collection. Performing coil combination, even with fully sampled data has also been shown to have specific challenges for HP human images: using naive sum-of-squares methods suffer from high noise amplification in the relatively low SNR regime of HP [1-13C]pyruvate (compared to 1H), motivating several HP 13C-specific methods that include data-driven coil sensitivity estimation which have shown obvious improvements over sum-of-squares (11).

More recently denoising techniques have been applied as post-processing of human HP data(41,42,44). The techniques applied are based on spatial-temporal singular value decomposition for unsupervised estimation of signal and noise components. They have shown improvements in apparent SNR in the brain and liver, while care must be taken to choose parameters such as the rank threshold to avoid oversmoothing and overfitting to the estimated signal components.

Prostate Studies

Prostate cancer was the first human application of HP [1-13C]pyruvate (1), and data was acquired with MRS/I methods: 1D dynamic MRS, single-slice 2D dynamic echo-planar spectroscopic imaging (EPSI), and single time point 3D EPSI. Advances in imaging strategies led to the development and application of new acquisition schemes, including undersampled 3D EPSI with compressed-sensing (7), model-based chemical shift encoding methods that use a priori information (47,59), and metabolite-specific EPI (10), all of which can provide volumetric whole-organ coverage and dynamic acquisitions.

The pyruvate bolus arrival in the prostate can vary by ± 10 s between patients, necessitating dynamic imaging to reliably and consistently capture the pyruvate bolus (18). For this reason, all currently ongoing studies acquire dynamic data. While MRS/I, chemical shift encoding, and metabolite-specific imaging can all achieve dynamic imaging, chemical shift encoding and metabolite-specific imaging provide greater dynamic and volumetric coverage (85). For scan prescriptions, the FOV is designed to provide full prostate coverage and typically to match the orientation of the anatomic imaging used for registration. Flip angles used in current studies are constant through time, as quantification with a variable-through-time flip scheme is highly sensitive to bolus timing (8) and errors in the RF transmit (B1 +) field (76).

Heart Studies

Data acquisition methods for 13C imaging in the heart must be designed to meet the demands of significant cardiac motion and blood flow. To cope with the periodic cardiac motion, most human heart studies to date used gating to the diastolic window, the longest cardiac cycle interval, which has reduced motion (2,22,28,30,35,36,38,45,52). The duration of the diastolic window limits the available data sampling time, making cardiac acquisitions the most time-constrained of the HP 13C MRI applications. The most common acquisition approach is metabolite-specific imaging with spiral k-space trajectories (2). Their single-shot imaging capability makes these methods particularly robust to motion effects. Furthermore, spiral k-space trajectories provide rapid k-space coverage and relatively benign flow and motion artifacts. The majority of studies have used 2D multi-slice acquisitions, but 3D encoding has also been used successfully (35).

Brain Studies

For HP 13C MRI of the human brain, the majority of studies have also used 2D (slice selective) acquisitions (10–12,14,16,28,33,40,41,44,51,53,60), with a trend toward volumetric coverage using 2D multi-slice metabolite-specific imaging. 3D metabolite-specific imaging of the whole brain, with phase encoding of the slice direction (34,57), has been shown to provide similar SNR efficiency (88) compared with multislice imaging. A number of studies have employed MRS/I (5,6,29,31–33,50,55) resulting in a spectrum from each voxel, which has the advantage of not requiring a priori information about which peaks to encode. This was important in early brain studies when it was not known which peaks would be detectable. Chemical shift encoding, using a set of images with different echo times and an iterative reconstruction of the individual resonances (i.e. the IDEAL approach (85)), has also been used (12,49,54), with the drawback that coverage in the slice direction was limited due to the time required to acquire multiple echo time images.

Abdomen And Breast Studies

The fundamental approaches to data acquisition and reconstruction in the abdomen and breast are largely similar to the aforementioned applications, but demand attention to particular challenges associated with these anatomic regions, especially relating to respiratory motion.

Although it has been shown that a basic 2D MRSI approach based on phase encoding and FID readout can be successfully applied for HP 13C imaging in breast (15) and kidney (13), major advantages in terms of spatiotemporal resolution and coverage have been realized using tailored approaches based on metabolite-specific imaging (43,62) and chemical shift encoding (43), which have facilitated multi-slice or 3D dynamic acquisitions over large FOVs in the abdomen (4,37,46).

The significant respiratory motion encountered in these regions can directly blur 13C images, and has further favored these rapid acquisition strategies. Motion also degrades B0 homogeneity, which can shift frequency-selective excitation profiles and introduce artifacts into rapid imaging readouts. This makes accurate determination of the acquisition center frequency and shimming essential in these regions which often cover large FOVs. (See “Prescan Calibration” section for more information). In some studies, breath-holding was used to minimize motion effects and enforce frame-to-frame data consistency (42). A pragmatic and reasonably effective approach for dealing with respiratory motion during 13C data acquisition is an initial breath-hold (as long as can be tolerated), followed by free-breathing (46,62).

1H Imaging

Collection of 1H imaging data is essential both for prescribing the 13C acquisition and for interpretation of the resulting 13C data. Multi-planar 1H scouts are acquired prior to 13C acquisition to enable graphical prescription of the 13C imaging region. All human HP 13C-pyruvate imaging studies acquire conventional MRI scans (e.g. T1- and T2-weighted volumes) for anatomic reference, aiming to cover at least the full 13C FOV. Acquiring these anatomic scans as close as possible to the time of 13C imaging (immediately before or after) minimizes potential misregistration between the data sets. Depending on the application, other advanced 1H sequences are also acquired (e.g. diffusion-weighted imaging for cancer imaging).

When contrast-enhanced data is acquired, it is done after 13C imaging, as paramagnetic contrast agents will accelerate 13C relaxation.

Reported Study Parameters

Figures 5 and 6, and Supporting Table S2 shows the reported acquisition study parameters for human HP [1-13C]pyruvate studies published as of September 2022. Figure 5 shows a mixture of MRS/I, metabolite-specific imaging, and chemical shift encoding methods have been successfully used, where spectroscopy-based methods have become less prevalent in recent studies. Figure 6 shows the acquisition timing, including the important start time and interval/temporal resolution, is quite variable across studies.

Figure 5: Acquisition methods used in published HP [1-13C]pyruvate human studies published up to September 2022, classified into: MR spectroscopy and spectroscopy imaging (MRS/I); chemical shift encoding methods, such as IDEAL, that use multiple TEs and model-based reconstructions; and metabolite-specific imaging methods that use spectrally-selective excitation to image a single resonance at a time.

Figure 6: Temporal acquisition characteristics reported in HP [1-13C]pyruvate human studies published up to September 2022. (a) Reported referencing of acquisition start times.

(B)

Acquisition start times reported when using dynamic imaging and when timing was reported relative to the end of the injection. (c) Temporal resolutions. “Not Applicable” indicates dynamic imaging was not used.

Summary

Three general categories of acquisition strategies have been used successfully for human HP 13C-pyruvate studies: MRS/I, model-based chemical shift encoding (e.g. IDEAL) methods, and metabolite-specific imaging methods. These have enabled successful studies in the prostate, heart, brain, abdomen, and breast. Recent studies increasingly have used the imaging-based strategies of metabolite-specific imaging and chemical shift encoding which are the fastest methods, although a heads-to–head comparison between techniques has not been performed.

Metabolite-specific imaging is quite popular because of its speed and compatibility with single-shot imaging, but is sensitive to B0 field variations and thus requires careful calibrations. Nearly all studies surveyed acquired data dynamically, allowing measurement of the bolus and metabolite kinetics. The exact timings and associated flip angles vary quite widely across reported studies, with no consensus yet as to how to choose these parameters. Image reconstruction is typically done directly using Fourier Transform methods, and accelerated imaging strategies are uncommon.

Data Analysis And Quantification

This section covers the analysis of data from human HP [1-13C]pyruvate studies, including modeling and metrics, visualization, as well as considerations for how to store data and metadata. Depending on study design, the analysis may need to give quantitative or semi-quantitative output reflecting a biological process or may just reflect a contrast between different regions of interest for quantitative evaluation.

Metrics

Figure 7: HP [1-13C]pyruvate raw data (A) have typically been quantified using four categories of metrics depending on the acquisition. Data acquired as a single time point are often quantified using normalized metabolite images or metabolite ratios (B). Dynamic data can be quantified using normalized metabolite images or metabolite ratios (B), or with metabolite timings such as time-to-peak (TTP) or pharmacokinetic (PK) models (C). The latter two require the data to be time-resolved. [1-13C]alanine and 13C-bicarbonate are analyzed similarly to [1-13C]lactate but omitted here for display.

Metabolite images are commonly used as summary metrics for HP MRI data, often including some form of normalization as well as summed over time as an area under the time curve (AUC) (17). These are analogous to the visual evaluation that is most used for routine clinical work (89,90). In these metabolite images, we expect that the [1-13C]pyruvate AUC signal is predominantly weighted towards perfusion and uptake, while [1-13C]lactate, [1-13C]alanine and 13C-bicarbonate AUCs represent metabolic conversion. The strength of this approach lies in its simplicity and relatively few underlying assumptions. Limitations to the use of single-metabolite images or AUCs include sensitivity to inhomogeneous coil profiles (57,87,91), the acquisition strategy and acquisition parameters, pyruvate polarization and concentration level, and signal relaxation rates (92). Further, the reader must be careful to interpret all the images in conjunction to better understand the underlying biology; for example, increased [1-13C]lactate in the presence of decreased [1-13C]pyruvate delivery can have a very different meaning compared to increased [1-13C]lactate with increased [1-13C]pyruvate delivery.

In an attempt to address variations in coil sensitivity, polarization level, and pyruvate delivery, AUC images are often computed by normalizing to a specified parameter, such as the maximum pyruvate or average lactate signals, or presented as a ratio such as lactate/pyruvate or divided by “total Carbon” - the sum total of HP 13C signal observed across all metabolites. The AUC ratios between metabolites and pyruvate are proportional to the corresponding forward kinetic rates (81,93), but are not directly comparable to rate constants when magnetization loss rates (e.g. relaxation and losses due to signal excitation) differ between studies. Similarly, the ratios between the produced metabolites (e.g. bicarbonate/lactate) can reflect the balance between downstream metabolic pathways (12,55). Care must be taken to consider how AUC images are calculated and normalized before comparing values between studies.

To further quantify the interpretation, pharmacokinetic (PK) modeling approaches were developed to compute the apparent kinetics of pyruvate-to-metabolite exchange (92,94–99). These yield semi-quantitative to quantitative apparent rate constants, given in s-1. Some models require a vascular input function, while others avoid this requirement (95). PK models can explicitly account for acquisition-specific details such as excitation angle and repetition time, and thus may reduce the effects of these details on quantification. An input-less model, provided in the Hyperpolarized-MRI-Toolbox (https://github.com/LarsonLab/hyperpolarized-mri-toolbox) (100) and thus frequently employed for human data, has been shown to fit well and robustly to prostate and brain data (8,20). PK models are quantitative in nature, arguably provide more relevant biological information (8,20), and appear to be reproducible across sites (51). However, rate constants derived from PK models are still apparent rates, and likely do not reflect a single biological characteristic.

Some additional considerations include whether complex or magnitude data is used, as the noise behaviors will impact the analysis differently. Additionally, cut-off thresholds or other criteria may be used to identify and avoid voxels with insufficient SNR before analysis to improve robustness (20,41).

Regardless of the analysis approach, the underlying biology is not always clearly represented by the data; instead, the metrics may be influenced by perfusion, barrier permeability, intercellular shuttles, enzyme activities, co-substrate concentrations, or combinations thereof, depending on the organ and disease of interest (19,43,94,101–103). This may be addressed by incorporating complementary information. As an example, HP 13C pyruvate data is influenced by perfusion, and thus addition of perfusion MRI could be important for interpretation (98,104,105).

All the methods outlined above have been explored in clinical studies, described in Supporting Table 3 and summarized in Figure 8. As of September 2022, approximately 52% of studies involving human subjects report rate constants derived from a PK model with a few different models reported. A nearly equal fraction (51%) of the studies report AUC ratio values.

Approximately 66% of these studies report metabolite-specific images or AUC values. About 40% report SNR values; this metric is particularly frequent in manuscripts that describe technical developments for clinical HP MRI. Approximately 16% of these studies summarize model-free metrics, and 10% report measurements from a single timepoint. Most studies report a combination of quantities.

Figure 8: Reported metrics used for analysis in HP [1-13C]pyruvate human studies published up to September 2022.

Visualization

A wide variety of approaches have been used for visualizing data from human HP 13C-MRI studies. The challenges and practical considerations are: 1) choosing the appropriate metrics to display, 2) how to encode the parameters (e.g. the colormap), and 3) choosing how to provide anatomical context and other multi-parametric data. The choice of visualization also depends on the goal which could be for diagnostic interpretation, but also quality control, reproducibility among readers and publication.

Metrics

The choice of HP 13C metrics is described in detail above. At this stage in HP 13C development where there is no standardized metric, often a combination of metabolite images and ratios or PK model parameters are shown.

Parameter Encoding

The mapping function chosen should provide an adequate, often quantitative, impression of the parameter mapped. There is a consensus in the visualization field that perceptually uniform maps are best suited to visualize continuous parameters, like the greyscale typically used by radiologists as well as other monochrome (black to blue) and color ranges (fire-type, rainbow-type) (106,107). Multi-color heatmaps have been the most frequently employed method for HP 13C data, while greyscale has infrequently been used but it ensures there is no coloring-based bias as well as facilitating later reuse (Fig. 9a). Among the color schemes employed in the clinical HP 13C literature, fire-type scheme seems to be the most common [similar to “Plasma” or “Inferno” in matplotlib.org]. Next most commonly employed is the rainbow-type scheme [similar to “Rainbow” in matplotlib.org].

Anatomical Context

HP MRI faces the challenge that it does not necessarily depict the anatomical features, similar to PET, and thus requires an anatomical reference. Most often, a grayscale anatomical image is overlaid with a HP colormap (Fig. 9c,d). This approach is very intuitive, but can skew perception as the grey-scale anatomical reference may affect the brightness of the HP data (e.g. signal in the skull). This bias does not occur when showing adjacent maps (Fig. 9a, b). Here, anatomical outlines may help to provide reference (Fig. 9b).

Related Journal Articles & DOI Links

Selected peer-reviewed publications relevant to 12 Lead ECG Acquisition. Click the DOI to access the full paper (may require institutional access).

Why Choose Us?

Bangalore guidance for robotics, Spectre and autonomous systems projects.