Ansys Needle Penetration Simulation

1

Simulating Customer Experience and Word-Of-Mouth in

Leeds, Ls2 9Jt, Uk

Agents offer a new and exciting way of understanding the world of work. In this paper we describe the development of agent-based simulation models, designed to help to understand the relationship between people management practices and retail performance. We report on the current development of our simulation models which includes new features concerning the evolution of customers over time. To test the features we have conducted a series of experiments dealing with customer pool sizes, standard and noise reduction modes, and the spread of customers’ word of mouth. To validate and evaluate our model, we introduce new performance measure specific to retail operations. We show that by varying different parameters in our model we can simulate a range of customer experiences leading to significant differences in performance measures. Ultimately, we are interested in better understanding the impact of changes in staff behavior due to changes in store management practices.

Our multi-disciplinary research team draws upon expertise from work psychologists and computer scientists. Despite the fact we are working within a relatively novel and complex domain, it is clear that intelligent agents offer potential for fostering sustainable organizational capabilities in the future.

Keywords: agent-based modeling, agent-based simulation, retail performance, management practices, shopping behavior, customer satisfaction, word of mouth

1. Introduction

The retail sector has been identified as one of the biggest contributors to the productivity gap, whereby the productivity of the UK lags behind that of France, Germany and the USA [1, 2]. A recent report into UK productivity asserted that, ‘...the key to productivity remains what happens inside the firm and this is something of a ‘black box’…’ . A recent literature review of management practices and organizational productivity concluded that management practices are multidimensional constructs that generally do not demonstrate a straightforward relationship with productivity variables , and that both management practices and productivity measures must be context specific to be (respectively) effective and meaningful.

Many attempts have been made to link management practices to an organization’s productivity and performance (for a review, see ), however research findings have been so far been mixed, creating the opportunity for the application of different techniques to advance our understanding.

Simulation can be used to analyze the operation of dynamic and stochastic systems showing their development over time. There are many different types of simulation, each of which has its specific field of application. Agent-Based Simulation (ABS) is particularly useful when complex interactions between system entities exist such as autonomous decision making or proactive behavior. Agent-Based Modeling

2

(ABM) shows how micro-level processes affect macro level outcomes; macro level behavior is not explicitly modeled, it emerges from the micro-decisions made by the individual entities . There has been a fair amount of modeling and simulation of operational management practices, but people management practices have often been neglected although research suggests that they crucially impact upon an organization’s performance . One reason for this relates to the key component of people management practices, an organization’s people, who may often be unpredictable in their individual behavior. Previous research into retail productivity has typically focused on consumer behavior and efficiency evaluation (e.g. [8, 9]), and we seek to build on this work and address the neglected area of people management practices in retail .

The overall aim of our project is to investigate the link between different management practices and productivity. This strand of work focuses on simulating various in-store scenarios grounded in empirical case studies with a leading UK retailer. In this paper we aim to understand and predict how Agent-Based Modeling and Simulation (ABMS) can assess and optimize the impact of people management practices on customer satisfaction and Word-Of-Mouth (WOM) in relation to the performance of a service-oriented methods to collect both qualitative and quantitative data. In summary, we have worked with a leading UK two retail stores, forty semi-structured interviews with employees including a sixty-three-item questionnaire on the effectiveness of retail management practices, and drawn upon a variety of established information sources internal to the company. This approach has enabled us to acquire a valid and reliable understanding of how the real system operates, revealing insights into the working of the system as well as the behavior of and interactions between the different individuals and their complementary roles within the experiments to investigate the effects of different management scenarios.

So far we have only studied the impact of people management practices (e.g. training and empowerment) on a customer base that is not influenced by any external or internal stimuli, and hence does not evolve [11, 12]. In order to be able to investigate the impact of management practices on customer satisfaction in a more realistic way we need to consider the cues that stimulate customers to respond to these practices.

Therefore our focus is currently on building the capabilities to model customer evolution as a consequence of the implementation of management practices. Consumer decision making, and consequently changes in the behavior of customers over time, are driven by an integration of each consumer’s cognitive and affective skills . At any given point in time a customer’s behavior, as the product of an individual’s cognitions, emotions, and attitudes, may be attributable to an external social cue such as a friend’s recommendation or in-store stimuli , or an internal cue such as memory of one’s own previous shopping experiences . Changing customer requirements may in turn alter what makes a successful management practice (because these are context specific, and customers are a key component of any retail context). In order to enable such studies we enhanced our original Management Practices Simulation (ManPraSim) model and introduced some new features.

In this paper we discuss the key features we have implemented in order to allow the investigation of these kinds of behavioral dynamics. What we are learning here about modeling human behavior has implications for modeling any complex system that involves many human interactions and where the people function with some degree of autonomy.

2. Related Work

When investigating the behavior of complex systems the choice of an appropriate modeling technique is very important . To inform the choice of modeling technique, the relevant literature spanning the fields of Economics, Social Science, Psychology, Retail, Marketing, OR, Artificial Intelligence, and Computer Science has been reviewed. Within these fields a wide variety of approaches are used which can be classified into three main categories: analytical methods, heuristic methods, and simulation. Often a

3

combination of these are used within a single model (e.g. [17, 18]. After a thorough investigation of the relevant literature we have identified simulation as being the most appropriate approach for our purposes. Simulation introduces the possibility of a new way of thinking about social and economic processes, based on ideas about the emergence of complex behavior from relatively simple activities . The purpose of simulation is either to better understand the operation of a target system, or to make predictions about a target system’s performance . It allows the testing and evaluation of a theory, and investigation of its implications. Whereas analytical models typically aim to explain correlations between variables measured at one single point in time, simulation models are concerned with the development of a system over time . They allow us to study the transient conditions during the process, along the way to the eventual outcome . The outcome of simulation study is the acquisition of more knowledge about the dynamic behavior of a system, i.e. the state changes of the model as time advances . There are many different approaches to OR simulation, amongst them Discrete-Event Simulation (DES), System Dynamics (SD), and ABS, which is sometimes also referred to as individual-based simulation . The choice of the most suitable approach depends on the issues investigated, the input data available, the required level of analysis, and the type of answers that are sought .

Although computer simulation has been used widely since the 1960s, ABMS only became popular in the early 1990s . It is described by as a mindset as much as a technology: ‘It is the perfect way to view things and understand them by the behavior of their smallest components’. In agent-based models a complex system is represented by a collection of agents that are programmed to follow simple behavioral rules. Agents can interact with each other and with their environment to produce complex collective behavioral patterns. The main characteristics of agents are their autonomy and their ability to take flexible action in reaction to their environment, both determined by motivations generated from their internal states.

They are designed to mimic the attributes and behaviors of their real-world counterparts. ABS is still a relatively new simulation technique and its principal application has been in academic research. There are, however, a wide range of existing application domains that are making use of the agent paradigm and developing agent-based systems, for example in software technology, robotics, and complex systems, and an ever increasing number of computer games use the ABMS approach. draw a distinction between two central Multi-Agent System (MAS) paradigms: multi-agent decision systems and multi-agent simulation systems. In multi-agent decision systems, agents participating in the system must make joint decisions as a group. Mechanisms for joint decision-making can be based on economic mechanisms, such as an auction, or alternative mechanisms, such as argumentation. Multi-agent simulation systems are used as a model to simulate some real-world domain. The typical application is in domains which involve many different components, interacting in diverse and complex ways and where the system- level properties are not readily inferred from the properties of the components. ABMS is extensively used by the game and film industry to develop realistic simulations of individual characters and societies. It is used in computer games, for example The SIMS™ , or in films when diverse heterogeneous characters animations are required, for example the Orcs in Lord of the Rings™ .

Due to the characteristics of the agents, the ABMS approach (which is often implemented as a process interaction DES) appears to be more suitable than the event scheduling modeling approach (which is typically associated with the term DES) when it comes to modeling human-centered systems. This is due to the fact that it supports the modeling of autonomous behavior. ABMS seems to promote a natural form of modeling these systems . There is a structural correspondence between the real system and the model representation, which makes them more intuitive and easier to understand than for example a system of differential equations as used in SD. emphasizes that one of the key strengths of ABMS is that the system as a whole is not constrained to exhibit any particular behavior as the system properties emerge from its constituent agent interactions. Consequently, assumptions of linearity, equilibrium and so on, are not needed.

On the other hand, some difficulties have been linked to use of ABMS. One disadvantage that is often mentioned is the high demand in computational power compared to other simulation techniques, most notably in that all of the interactions between agents and between the agent and the environment have to be considered during the simulation run. However, rapid growth in affordable computer power makes this

4

problem less significant . There is consensus in the literature that it is difficult to evaluate agent-based models, because of the heterogeneity of the agents and because the behavior of the system emerges from the interactions between the individual entities. argues that for ‘intellective models’ (e.g. models that illustrate the relative impact of basic explanatory mechanisms) validation is somewhat less critical and the most important thing is to maintain a balance between keeping a model simple and maintaining veridicality (the extent to which a knowledge structure accurately reflects the information environment it represents).

More recently some attempts have been made to develop a common approach to assuring the credibility (verification and validation) of ABS models (see for example [33 - 35]. A two stage approach for empirical validation has been proposed by . The first stage comprises micro- validation of the behavior of the individual agents and the authors recommend this is conducted with reference to real data on individual behavior. The second stage covers macro-validation of the aggregate or emergent behavior of the model when individual agents interact and is advised to be conducted with reference to aggregate time series data. Further, remark that at the macro level only qualitative validation judgments may be possible. Also, problems often occur through the lack of adequate empirical data. state that most quantitative research has concentrated on ‘variable and correlation’ models that do not cohere well with process-based simulation that is inherent in agent-based models. Finally, point out the danger that people new to ABMS may expect too much from the models, particularly regarding their predictive ability. To mitigate this problem it is important to be clear with individuals about what this modeling technique can really offer, in order to guide realistic expectations.

3. Model Design And Data Collection

Before building a simulation model one needs to understand the particular problem domain . In order to gain this understanding we have conducted some case studies. What we have learned during these case studies is reflected in the conceptual models presented in this chapter. Furthermore we explain how we intend to use the data we have gathered during our case studies. Throughout the rest of the paper we will use the term 'actor' to refer to a person in the real system, whereas the term 'agent' will be reserved for their counterparts in the simulation model.

3.1 Knowledge Gathering

study work involved extensive data collection techniques, spanning: participant observation, semi- structured interviews with team members, management and personnel, completion of survey questionnaires and the analysis of company data and reports (for further information, see ). Research findings were consolidated and fed back (via report and presentation) to employees with extensive experience and has enabled us to acquire a valid and reliable understanding of how the real system operates, revealing insights into the working of the system as well as the behavior of and interactions between the different actors within it.

In order to make sure that our results regarding the application of management practices are applicable for a different not only in their way of operating but also their customer type split and staff setup. We collected study branches.

customer service time in A&TV is significantly longer, and the average purchase is significantly more expensive than in WW. The likelihood of a customer seeking help in A&TV is also much higher than in WW. Out of customers who have received advice, those in WW have a higher likelihood of making a purchase (indeed customers’ questions tend to be very specific to a desired purchase) than in A&TV.

Considering customer types, A&TV tends to attract more solution demanders and service seekers, whereas WW customers tend to be shopping enthusiasts. Finally, it is important to note that the conversion rate (the likelihood of customers making a purchase) is higher in WW than in A&TV.

3.2 Conceptual Modeling

A conceptual model captures only the essential and relevant characteristics of a system. It is formulated from the initial problem statement, informal user requirements, and data and knowledge gathered from analysis of previous development models . We have used the knowledge gained from the case studies to inform our conceptual models of the system to be investigated, the actors within the system, and their behavioral changes due to certain stimuli.

3.2.1 Main Concepts For The Simulation Model

Our initial ideas for the simulation model and its components are display below in Figure 1. Here we can see a bird’s eye view of a shopping centre with customers (red dots) visiting a range of shops. Based on this conceptual overview, we determined that we required three types of agents (customers, sales staff and managers), each with a different set of relevant attributes. Global parameters can influence any aspect of the system. The core of our conceptual simulation model consists of an ABS model with a user interface to allow some form of user interaction (change of parameters) before and during runtime. Regarding system outputs, we aim to find some emergent behavior on a macro level. Visual representation of the simulated system and its actors allows us to monitor and better understand the interactions of entities within the system. Coupled with the standard DES performance measures, we can then identify bottlenecks to assist with optimization of the modeled system.

3.2.2 Concepts For The Agents

We have used state charts for the conceptual design of our agents. State charts show the different states an entity can be in and define the events that cause a transition from one state to another. This is exactly the information we need in order to represent our agents at a later stage within the simulation environment. We have found this form of graphical representation a useful part of the agent design process because it is easier for an expert in the real system (who is not an expert in ABMS) to quickly take on board the model conceptualization and provide useful validation of the model structure and content.

Designing and building a model is to some extent subjective, and the modeler has to selectively simplify and abstract from the real scenario to create a useful model . A model is always a restricted copy of the real system, and an effective model consists of only the most important components of the real system. In our case, the case studies indicated that the key system components take the form of the behaviors of an actor and the triggers that initiate a change from one behavior to another. We have developed state charts for all of the agents in our simulation model. Figure 2 shows as an example the conceptual template model of a customer agent. The transition rules have been omitted to keep the chart comprehensible. They are explained in detail in the Section 3.3.

3.2.3 Concepts For A Novel Performance Measure

We have introduced a customer satisfaction level index as a novel performance measure using satisfaction weightings. This new measure is required because existing indices such as queuing times or service times are less useful in modeling services than manufacturing activities. In essence, purely quantitative measures fail to capture the quality of service, which is arguably the most important potential trade-off with retail productivity. The inevitable trade-off between quality and quantity is particularly salient when customers and staff come face-to-face and therefore we consider this measure of quality in conjunction with others of quantity.

Historically customer satisfaction has been defined and measured in terms of customer satisfaction with a purchased product . The development of more sophisticated measures has moved on to incorporate customers’ evaluations of the overall relationship with the retail organization, and a key part of this is the service interaction. Indeed, empirical evidence suggests that quality is more important for customer

6

satisfaction than price or value-for-money , and extensive anecdotal evidence indicates that customer- staff service interactions are an important determinant of quality as perceived by the customer. The index allows customer service satisfaction to be recorded throughout the simulated lifetime. The idea is that certain situations can have a bigger impact on customer satisfaction than others, and therefore weights can be assigned to events to account for this. Satisfaction indices do not change the likelihood that a evidence supports the notion that regardless of a customer’s changing shopping intentions he or she will tend to repeat shopping habits . Applied in conjunction with an ABS approach, we expect to observe interactions with individual customer differences; variations which have been empirically linked to differences in customer satisfaction (e.g. ). This helps the analyst to find out to what extent customers underwent a positive or negative shopping experience and it also allows the analyst to put emphasis on different operational aspects and try out the impact of different strategies.

3.2.4 Concepts For Modeling Customer Evolution

There are two different ways in which we consider customer evolution: external stimulation attributable to the WOM and internal stimulation triggered by memory of one’s own previous shopping experiences (this is still work in progress). Sharing information with other individuals (referred to as WOM), significantly affects the performance of retail businesses . An important source of WOM results from customer experiences of retail outlets, and a customer’s judgment about whether or not the experience left them feeling satisfied (or dissatisfied). We incorporate WOM in our simulation model by using the number of satisfied customers at the end of the day to calculate the number of additional customers visiting on the next day. The calculation takes into account that only a fraction of people act upon received WOM.

Our concept for representing internal stimulation comprises the exertion of influence on picking certain customer types more often than others. An enthusiastic shopper with a high satisfaction score is much more likely to go shopping more frequently than a disinterested shopper. Therefore, we have introduced some constraints (e.g. out of all customers picked 50% have to be enthusiastic shoppers, 30% normal shoppers, and 20% disinterested shoppers, or, customers with a higher satisfaction score are more likely to be picked act as consumer cues for triggering impulse buying. Internal cues include respondents’ positive and negative feeling states, and environmental cues include atmospheric cues in retail settings, marketer- controlled cues, and marketing mix stimuli. When modeling internal stimulation we specifically examine performance measures.

3.3 Empirical Data

Often agents are based on analytical models or heuristics and, in the absence of adequate empirical data, theoretical models are employed. However, we use frequency distributions to model state change delays and probability distributions to model decision making processes because statistical distributions are the best way in which we can represent the numerical data we have gathered during our case study work. In this way a population is created with individual differences between agents, mirroring the variability of attitudes and behaviors of their real human counterparts.

The frequency distributions are modeled as triangular distributions defining the time that an event lasts, using the minimum, mode, and maximum duration and these figures are based on our own observations and expert estimates in the absence of objective numerical data.

The probability distributions are partly based on company data (e.g. the rate at which each shopping visit results in a purchase) and partly on informed estimates (e.g. the patience of customers before they leave a queue). Table 1 and 2 show some of the distributions we have defined for our simulation models. We also gathered some company data about work team numbers and work team composition, varying opening hours and peak times, along with other operational details.

4.1 Implementation Of The Main Concepts

Our ManPraSim model has been implemented in AnyLogic™ (version 5.5) which is a Java™ based multi- paradigm simulation software . During the implementation we have applied the knowledge, experience and data accumulated through our case study work. Within the simulation model we can represent the following actors: customers, service staff (with different levels of expertise) and different types of managers. Figure 3 shows a screenshot of the customer and staff agent logic in AnyLogic™ as it has been implemented in the latest version of our ManPraSim model (v4). Boxes represent states, arrows transitions, arrows with a dot on top entry points, circles with a B inside branches, and numbers satisfaction weights.

At the beginning of each simulation run the main customer pool is created which represents a population of simulation has started customers are chosen at a specified rate (customer arrival rate) and released into the something and customers who require a refund. If a refund is granted, a customer decides whether his or template consists of four main blocks which all use a very similar logic. In each block, in the first instance, customers try to obtain service directly from an employee and if they cannot obtain it (i.e. no suitable staff member is available) he or she has to queue. The customer is then either served as soon as the suitable staff member becomes available, or leaves the queue if they do not want to wait any longer (an autonomous decision). A complex queuing system has been implemented to support different queuing rules. Once are returned back to the main customer pool where he or she rests until they are picked again.

weights attached to the transitions that take place during the customer’s visit. For example, a customer starts browsing and then requires some help. If he or she obtains help immediately his or her satisfaction score goes up (+2) and after he or she received the help the score goes up again (+4). He or she then moves to the till. If he or she has to wait for help and then leaves the queue because he or she is fed up waiting his overall satisfaction score of +4 due to the good and prompt advisory service. The actual customer satisfaction weights shown in Figure 3 were derived based on our experience during the data collection period and validated via interviews with the store management staff.

In comparison to the customer agent state chart, the staff agent state chart is relatively simple. Whenever a customer requests a service and the staff member is available and has the right level of expertise for the task requested, the staff member commences this activity until the customer releases the staff member.

Whereas the customer is the active component of the simulation model, the staff member is currently passive, simply reacting to requests from the customer (although both are technically implemented as active objects).

4.2 Key Features

There are some additional key features that the simulation model possesses which we describe in the following two sections. First we provide an overview of some important features which have already been implemented in an older version of our simulation model (ManPraSim model v2) but are still relevant to modeling the evolution of customers. This sets the scene for the subsequent description of the new features we have implemented in the latest two versions of our simulation model (ManPraSim model v3 + v4).

4.2.1 Key Features Of The Manprasim Model V2

In the ManPraSim model v2 we introduced realistic footfall, customer types, a finite population of overview of these features because they are described in more detail elsewhere (see ). Realistic footfall (introduced in v2): There are certain peak times where a relatively high number of ManPraSim model v2 we have implemented hourly fluctuations through the addition of realistic footfall (based on some automatically recorded sales transaction data) reflecting different patterns (peaks and troughs) of customer footfall during the day and across different days of the week. In addition we can model the varying opening hours on different days of the week.

Customer types (introduced in v2): In real life customers display certain shopping behaviors which can be categorized. Hence we enhance the realism of our agents’ behavior in the ManPraSim modelv2 by introducing customer types to create a heterogeneous customer base, thereby allowing us to test customer populations closer to what we would find in reality. Customer types have been introduced based on three identified by the case study organization’s own market research (shopping enthusiasts, solution demanders, service seekers) and have been expanded by the addition of two further types (disinterested shoppers, and internet shoppers who are customers that only seek advice but are likely to buy only from the cheapest source i.e. the internet). The three customer types have been identified by the case study organization as the customers who make biggest contribution to their business, in terms of both value and frequency of sales.

In order to avoid artificially inflating the amount of sales that we model we have introduced the two additional types which use services but do not tend to make purchases. The definition of each type is based on the customer’s likelihood to perform a certain action, classified as either: low, moderate, or high. The definitions can be found in Table 3.

Customer Type

Distribution adaptation (introduced in v2): In the ManPraSim model v2 we have two algorithms which have been developed to imitate the influence of the customer type attributes mentioned above on customer behavior. They have been implemented as methods that are invoked when defining the state change delays modeled by triangular frequency distributions, and when supporting decision making modeled by probability distributions. Basically the methods define new threshold values for the distributions based on the likelihood values mentioned above. Program 1 shows an example of the pseudo code for the probability distribution threshold correction algorithm. If the customer is a shopping enthusiast and is about to make chart, second branch after leaving the browse state) a corrected threshold value (probability) for this decision is calculated. For this calculation the original threshold of 0.37 (see Table 2) is taken into account and, for a shopping enthusiast where there is a high likelihood to buy (see Table 3), the corrected threshold value is calculated as follows: 0.37+0.37/2 = 0.56. Consequently the likelihood that a shopping enthusiast 18.5%.

9

Program 1. Pseudo code for the probability distribution threshold correction algorithm

(

Finite customer population (introduced in v2): A key aspect to consider is that the most interesting system outcomes evolve over time and many of the goals of the retail company (e.g. service standards) are planned strategically over the long-term. In the ManPraSim model v2 we have therefore introduced a finite population of customers (main customer pool) where each customer agent is assigned certain characteristics based on the customer types mentioned above. The customer type split in the main customer pool can be defined via an initialization file before the execution of the simulation. The shopping experience of each visit (satisfaction index) is stored in the long term memory of the agent after he or she has left the time span.

Quick exit at closing time (introduced in v2): In the ManPraSim model v2 we have added transitions that emulate the behavior of customers when the store is closing. These transitions result in an immediate exit of each customer for his or her current state (i.e. the equivalent to a customer running out of shopping time and promptly exiting the store). Not all customer states have these additional transitions because it is for example very unlikely that a customer will leave the store immediately when they are already queuing to fifteen minute period, which conforms to what we have observed in the real system.

4.2.2 Key Features of the ManPraSim Models v3 + v4 In the ManPraSim model v3 we have introduced a staff pool with an additional staff type, a new operation mode to support sensitivity analyses and the first implementation of customers’ WOM featuring a static main customer pool. Furthermore, we have created some new performance measures that allow us to implementation of customers’ WOM featuring a dynamic main customer pool.

Staff pool and additional staff types (introduced in v3): Retail trends reflect that shops are now open for longer hours over more days of the week, and our case study organization is no exception. To accurately incorporate this source of system variability, we have introduced a staff pool in the ManPraSim model v3 to allow different staffing on different days of the week. The simulation uses Full Timers (FT) to cover all staff shifts required during weekdays. Additional staff who are required to cover busy weekend shifts are modeled by Part Timers (PT). The maximum number of staff required of each type is calculated during the simulation initialization. A staff pool is then created to include weekday FT staff of different types, and the required generic PT to fill the gaps left in the staff shifts which need to be covered. At the beginning of each day the simulation checks how many workers are required and picks the required amount of staff out of the pool at random. The selection process is ordered as follows: FT first and then PT, if required. PT workers have been defined as a generic staff type and can take over any required role. We have tried to model a staff rota with more complex constraints, for example FT staff working five days followed by two days off, however this has as yet proved unsuccessful. Therefore the currently modeled shifts do not incorporate days off work for FT staff.

10

Noise reduction mode (introduced in v3): In the ManPraSim model v3 a noise reduction mode has been implemented which allows us to conduct a sensitivity analysis with constant customer arrival rates, constant staffing throughout the week and constant opening hours. We have used the average values of real world case study data to define the constant values. This has resulted in different values for the different way at least we can reduce the system noise to clearly see the impact of the parameter under investigation.

When we progress to reintroduce the system noise, the knowledge we have accumulated when experimenting in noise reduction mode helps us to better understand patterns in systems outcomes, and be better able to attribute causation to the introduction of a particular variable (bearing in mind that in the end we are particularly interested in patterns between variables when they are interacting with one another and not in isolation).

Word of mouth (introduced in v3+v4): We have developed two different strategies for modeling WOM (which have been implemented in the ManPraSim model v3 (static main customer pool) and in v4 (dynamic main customer pool). Both strategies use the same algorithm to calculate the additional customers They differ in the way they pick these customers. In the first case we have used a static main customer pool which is defined with the customer split at the beginning of the simulation run. During the simulation run, the population of the main customer pool stays constant and all customers in the pool are available to be randomly picked to go shopping. In the second case we allow the main customer pool size to change; rather than picking customers from the existing pool we create new customers and add them to the main customer pool. When the main customer pool grows or declines the customer arrival rate is also adjusted, i.e. if the pool grows by 10% due to positive WOM so does the customer arrival rate and vice versa for negative growth.

The algorithm to calculate the number of additional new customers and the lost customers is the same for both strategies. We count the number of customers from the previous day who were satisfied and those who were dissatisfied with the service provided during their shopping visit(s). Satisfied customers are more advise others not to visit. The adoption rate (i.e. the success rate of convincing others to commit to either action) depends on the proportion of people who act upon the received WOM (the adoption fraction) and how many contacts a customer has (the contact rate). The equation behind this calculation is shown below in Equation 1.

−

−

At the beginning of each trading day we create a daily customer pool that contains randomly chosen customers from the main customer pool according to the required number of customers defined by the customer arrival rates for the day which might differ throughout the day (e.g. considering peak times).

For the first strategy (static customer pool) we simply pick the core number of customers (defined by the original customer arrival rates) plus an additional number of customers which is calculated through Equation 1. These customers are also selected from the main customer pool at random. If the number is negative we release customers from our daily customer pool back into the main customer pool. If we have to release all of our customers (due to customer dissatisfaction, which may result from bad service due to a large number of customers visiting on the previous day) the simulation terminates. This means that in a sense that the store is closed for good because it cannot recover from this situation in the current implementation.

11

The second strategy (dynamic customer pool) is slightly more complex. Here we expand or decrease the main customer pool by the number of additional or lost customers (respectively) calculated at the beginning of each trading day. This differs from the static strategy because we do not use the existing customers to model direct WOM influence (i.e. the ones resulting from the service satisfaction of the previous day). This seems to be closer to reality because the WOM in most cases carries positive messages and therefore attracts new visitors rather than motivating existing customers to come more often. Negative WOM created considering the hourly customer arrival rates (derived from our case study) as well as the actual size of the main customer pool (see Equation 2).

=

=

=

=

=

Once the daily pool size has been established, customers are chosen from the main customer pool according to the requested customer split and customer pool size. Then additional customers are created as new agents. The type of each new customer is determined at random. They are created with a positive will already have a positive attitude toward the shop (otherwise they would not have been attracted to it).

Through these additional (or lost) agents the daily customer pool size can expand (or contract). If the negative WOM causes an overall reduction in customers then we choose customers at random from the daily customer pool. Whether or not a particular customer is permanently removed from the main customer pool is determined as follows, depending on the satisfaction score of the chosen customer. If he or she has a positive satisfaction score then the score is neutralized and the customer is returned to the daily customer pool (i.e. he or she can come back for further shopping visits but the shop must start to satisfy the customer’s needs for good service to secure further visits from the customer). If he or she has a neutral or negative score he or she is permanently eliminated from the simulation (i.e. he or she will not come back for any more shopping trips). In the second case the main customer pool size will shrink. If the negative WOM results in more agents being deleted than there are customers available in the daily customer pool the For both strategies, when it reaches shop closing time all customers from the daily customer pool are released back into the main customer pool where they are available for the next day. For the second strategy (dynamic customer pool) it is very likely that the main pool size will then change due to WOM- triggered additions or losses of customers whereas for the first strategy (static customer pool) the main pool size remains constant.

New performance measures: With the introduction of a finite population (represented by our customer pool) we have had to rethink the way in which we collect statistics about the satisfaction of customers. visit, the individual’s satisfaction score (direction and value) has been recorded. Now the life span of a customer lasts the full runtime of the simulation and he or she can be picked several times to visit the satisfaction scores considering customers’ satisfaction history. These measures do not reflect individuals’ satisfaction with the current service experience but instead the satisfaction with the overall service experience during the lifetime of the agent. Furthermore, they are biased to some extent in that an indifferent rating quickly shifts into satisfaction or dissatisfaction (arguably this is realistic because most people tend to make a judgment one way or the other).

12

Whilst the above is still a valuable piece of information we would also like to know how current service is perceived by each customer. For this reason we have introduced a new set of performance measures to record the experience of each customer’s individual visit (referred to as Customer Satisfaction Measure – Experienced Per Visit or CSM-EPV). These are the same measures as before but on a day-to-day basis they are not anchored by the customer’s previous experiences. We also examine the sum of these ‘per visit’ scores across the lifetime of all customers (referred to as Customer Satisfaction Measure – Accumulated Historical Data or CSM-AHD).

Another new measure tracks the satisfaction growth for customers’ current and overall service experience. With the incorporation of varying customer arrival rates, opening hours and staffing we have brought in a set of performance measures that capture the impact of these variations on a daily basis. These measures satisfaction growth per each individual customer visit. At the end of the simulation run the simulation model produces a frequency distribution which informs us about how satisfied or dissatisfied individual customers have been with the service provided. Furthermore, all forms of customer queue (cashier, normal help, expert help, and refund decision) are now monitored through new performance measures that record how many people have been queuing in a specific queue, and how many of these lost their patience and left the queue prematurely. This measure helps us to understand individual customers’ needs because it tells us what individual customers think about the service provided. Finally, we have added some methods for writing all parameters and performance measures into files to support documentation and analysis of the experiments.

4.3 Model Validation

Validation ensures that the model meets its intended requirements in terms of the methods employed and the results obtained. In order to test the operation of the ManPraSim model and ascertain face validity we have completed several experiments.

We have completed validation exercises on the micro and macro level, as proposed by . Micro simulation comprises tracing an individual entity through the system by building a trajectory of an individual’s behavior throughout the runtime. We have conducted this test for our two different agent types (customer and staff agents) considering all of the different roles these can take. Furthermore we have validated who triggers which process and for what reason. We have then extensively tested the underlying queuing system. On the macro level we have tried to establish simulation model performance outputs that are sound. It has turned out that conducting the experiments with the input data we collected during our case studies did not provide us with a satisfactory match to the performance output data of the real system.

We identified the staffing setup used in the simulation models as the root cause of the problems. The data we use here have been derived from real staff rotas. On paper these real rotas suggested that all workers are engaged in exactly the same (one) type of work throughout the day but we know from working with, and observing workers in, the case study organization that in reality each role includes a variety of activities.

Staff members in the real organization allocate their time proactively between competing tasks such as customer service, stock replenishment, and processing purchases. Proactive behavior refers to the individual staff members’ self-started, long term oriented, and persistent service behavior which goes beyond explicit job demands .

So far our simulation models incorporate only one type of work per staff member. For example, the A&TV staff rota indicates that only one dedicated cashier works on weekdays. When we have attempted to model this arrangement, customer queues have become extremely long, and the majority of customers ended up system we observed other staff members working flexibly to meet the customer demand, and if the queue of customers grew beyond a certain point then one or two would step in and open up further tills to take customers’ money before they became dissatisfied with waiting. Furthermore, we observed that a service staff member, when advising a customer, would often continue to close the sale (e.g. filling in guarantee forms and receiving payment from the customer) rather than asking the customer to queue at the till for a cashier whilst moving on to the next customer.

13

This means that currently our abstraction level is too high and we do not model the real system in an appropriate way. We hope to be able to fix this in a later version. For now we do not consider this to be a big problem so long as we are aware of it. We model as an exercise to gain insights into key variables and their causes and effects and to construct reasonable arguments as to why events can or cannot occur based on the model; we model for insights, not precise numbers.

In our experiments we have modulated the staffing levels to allow us to observe the effects of changing key changed).

5. Experiments

The purpose of the experiments described below is to further test the behavior of the simulation models as they become increasingly sophisticated, rather than to investigate management practices per se. We have defined a set of standard settings (including all probabilities, staffing levels and the customer type split) for added to the set of standard settings for subsequent experiments. For the experiments that involved only a static main customer pool the run length has been 10 weeks. For the experiments that involved a dynamic main customer pool or both types the run length has been 52 weeks. We have conducted 20 replications (in most cases) to allow the application of rigorous statistical tests.

5.1 Comparing Customer Pool Sizes

The first experiment is a sensitivity analysis. We want to find out what impact the customer pool size has on the simulation results and which of our performance measures are affected by it. For this experiment we have used our new noise reduction mode (described in Section 4.2.2). It allows us to focus our attention on the impact of the variable to be investigated. We have run the experiments separately for both case study increments of 2,000. We will interpret the results in terms of customer-related performance measures, which can be separated out between the standard measures which are defined explicitly in terms of at which dissatisfaction.

Hypothesis 1: We predict that varying the customer pool size will not significantly change standard customer measures nor those customer satisfaction measures related measured in terms of experience per visit (CSM-EPV).

Hypothesis 2: We predict that the customer satisfaction measures which accumulate historical data (CSM- AHD) will result in significantly more customers being categorised as either ‘satisfied’ or ‘dissatisfied’ when compared to the number of customers in these categories for the customer satisfaction measure CSM-

Epv.

Hypothesis 3: We predict that the customer pool size will significantly impact on all CSM-AHD measures, specifically the greater the customer pool size, the higher the count of neutral customers, and the lower the count of satisfied and dissatisfied customers.

Our results in percentage terms (see Table 4) appear to provide support for all hypotheses. In order to rigorously test our hypotheses, statistical analyses were applied to the raw customer counts as follows. To investigate Hypothesis 1, a series of one-way between groups ANOVAs were conducted separately for A&TV and WW. Levene’s test for homogeneity of variances was violated by CSM-EPV for satisfied customers in A&TV only, therefore a more stringent significance value was set for this variable (p<.01).

Given that 8 simultaneous comparisons were to be drawn, a Bonferroni adjustment was made to the alpha level, tightening the significance levels to p<.00625 (from p<.05) and p<.00125 (from p<.01). For A&TV, no significant differences were found in the standard performance measures or CSM-EPV between

14

different customer pool sizes. For WW, no significant differences were found apart from the number of customer leaving before receiving normal help (p=.0059). Inspecting mean differences revealed that the mean customer count with a customer pool of 8,000 was higher than for 4,000, 6,000, and 10,000 customers. However, upon examination of supporting data, no single paired comparison was significantly different. Across customer pools, the mean customer count varied between just 4.6 and 7.5 customers leaving before receiving normal help, and so we can explain this pattern in terms of the relative infrequency of this event and conclude that the difference is of little practical significance. This leads us to conclude that overall the evidence supports Hypothesis 1. We predicted this pattern because customer decisions are driven by customer types and the proportional mix of each type is kept constant throughout the experiment.

To test Hypothesis 2, a series of independent samples T-tests were conducted to evaluate the differences between the two methods of measuring customer satisfaction. For A&TV, a significantly higher count of customers were categorized as satisfied on CSM-AHD (M=18,031.18, SD=824.11) as compared to their counterparts on the CSM-EPV measure (M=15,071.12, SD=108.47), t(102.43)=35.61, p<.000 (equal variances not assumed). The same pattern was reflected in a significantly higher count of dissatisfied customers (M=17,134.13, SD=895.06) on CSM-AHD as compared to CSM-EPV (M=11,404.65, SD=221.34), t(111.06)=62.14, p<.000 (equal variances not assumed). The effect sizes are very large in both cases, yielding .86 and .95 respectively.

For WW, a significantly higher count of customers were categorized as satisfied on CSM-AHD (M=54,161.66, SD=3,559.94) as compared to on CSM-EPV (M=33,508.33, SD=82.76), t(99.11)=58.00, p<.000 (equal variances not assumed). The inverse occurred for the count of dissatisfied customers, whereby a smaller number were counted on CSM-AHD (M=3,447.49, SD=1,049.66) than CSM-EPV (M=4,626.00, SD=128.59), t(101.97)=-11.14, p<.000 (equal variances not assumed). The effect size was very large for A&TV at .94, and medium-to-large at .39 for WW.

Overall as expected CSM-AHD generally results in a significantly higher count of satisfied and dissatisfied customers than CSM-EPV. This phenomenon has been discussed in Section 4.2.2. The evidence for A&TV unequivocally supports this hypothesis, whereas it is mixed for WW. The surprisingly relatively low count of dissatisfied of CSM-AHD in WW recurs in evaluation of the next hypothesis (see below).

Table 4. Descriptive statistics for Experiment 1 (all to 2 d.p.)

0.3%

… leaving dissatisfied (accumulated historical data)

0.2%

… leaving dissatisfied (accumulated historical data)

15

To investigate Hypothesis 3, a series of one-way between groups ANOVAs were conducted. Levene’s test for homogeneity of variances was violated by CSM-AHD for satisfied and dissatisfied customers in A&TV and only dissatisfied customers in WW, therefore a more stringent significance value was set for these variables (p<.01). Further to this a Bonferroni adjustment was implemented because of the multiple comparisons, tightening the significance value to p< .0167 (from p<.05) and p<.0033 (from p<.01).

Separately for A&TV and WW, all 3 ANOVAs for each CSM-AHD variable exhibited significant differences (p<.000). Examining mean differences for A&TV: for satisfied customers, significant differences were present between all paired pool sizes except for between 2,000 and 4,000; 6,000 and 8,000; and 8,000 and 10,000. The mean customer count uniformly decreased as the customer pool size increased, with significant differences focused on comparisons between paired customer pool sizes with a greater size differential. For neutral customers, all paired comparisons were significantly different (p<.000), and examination of mean differences reveals a uniformly increasing trend in the customer count as the customer pool size increases. For dissatisfied customers, significant differences persisted (at p<.0033) between the following pairs: 2,000 and all other pool sizes; 4,000 and 8,000; and 4,000 and 10,000. As customer pool size increased, the count of dissatisfied customers uniformly decreased.

Looking at mean differences for WW: for satisfied customers, all paired comparisons were significantly different (p<.000), and an examination of mean differences revealed a uniformly decreasing trend in the count of satisfied customers as customer pool size increases. For neutral customers, again all paired comparisons exhibited significant differences (p<.000), and this time the mean customer counts revealed a uniformly increasing trend as customer pool size increases. Finally, for dissatisfied customers, all paired comparisons were significantly different (p<.000), and the mean customer counts behaved according to a uniformly increasing trend as customer pool size increases. These results follow a similar pattern to that presented in A&TV, apart from the increasing number of dissatisfied customers. We expect that this finding can be explained in terms of an accumulation of the impact of a number of subtle differences in the produce significant effects, see evaluation of Hypothesis 1).

Looking at the percentages presented in Table 4, we can see that the bigger the customer pool size the more the CSM-AHD values tend to the CSM-EPV values. This can be explained by the fact that with a larger accumulates some historical data) reduces. If we sufficiently increase the customer population we expect both customer satisfaction measures to show similar results because the majority of customers are only neutral satisfaction score rather than with an accumulated score. In summary for Hypothesis 3 we can conclude that the evidence largely supports our prediction, with the exception of dissatisfied customer counts in WW for which we have presented an explanation.

The results for Experiment 1 demonstrate that it is therefore important to select and maintain one customer pool size to ensure that all performance measures are providing comparable information for different experiments. Our case study organization does not collect or hold data on customer pool size so we have day (585 for A&TV and 915 for WW) and an estimate of customers’ average inter-arrival time (two weeks for A&TV and one week for WW). These values have been estimated considering the standard customer We use these customer pool sizes for all subsequent experiments.

5.2 Comparing Normal And Noise Reduction Mode

In our second experiment we want to investigate the importance of considering hourly differences in customer arrival rates and daily differences in staffing and opening hours (normal mode). These features and specific patterns that occur on a day-to-day basis. Modeling this level of detail can be problematic

16

when we conduct a sensitivity analysis where we want to be able to attribute causation to the introduction of a particular variable. For this kind of experiment we prefer to control some of the system noise to clearly see the impact of the parameter under investigation (noise reduction mode).

Hypothesis 4: We predict that the standard customer performance measures will not significantly differ across the two modes. Hypothesis 5: We predict the two modes will produce significantly different values when looking at the daily performance measures on a day-to-day basis (rather than using averages).

Table 5. Descriptive statistics for Experiment 2 (all to 2 d.p.)

… Leaving Before Finding Anything [Runtime]

20,182.00 134.56 20,661.65 145.08 20,789.75 176.33

… Leaving Whilst Waiting To Pay

6,291.95 165.73 13,148.10 200.16 18,050.20 164.96 -224.88

… Leaving Before Finding Anything

27,662.80 166.79 33,120.80 198.49 37,173.90 196.79 -164.89

Af = 1

Looking at Figure 4 it is clear that there is very little variation between the two different operation modes across the range of standard performance measures. Examining the descriptives (see Table 5), in most cases the runtime performance measures in both modes are approximately the same with a small number of exceptions. Contrary to hypothesis 4, in WW approximately 25% more customers leave whilst waiting to pay in the normal mode (which consequently influences both customer satisfaction measures), as opposed to the noise reduction mode. This apparent cashier bottleneck appears to be exacerbated by any combination of the three factors which are held constant in noise reduction mode. Further analysis is required to isolate the precise cause. We can also see that, as predicted, the smaller the values the more they differ (on an absolute basis) between the two modes as they are accumulated from a smaller number of events and therefore the influence of different random number streams is more apparent.

Examining the daily measures on a day-to-day basis for A&TV in Figure 5, we can observe clear differentiation between the number of customers and transactions across weekdays, Saturdays and Sundays in normal mode whereas the noise reduction mode, as expected, shows no clear patterns. This additional information can be very useful for optimizing the system. For example, we have the lowest number of transactions on Sundays (day 1, 8, etc.) although we do not have the lowest number of customers on this day, therefore we must have a problem with the optimizing the staffing arrangement on Sundays, because on average more customers leave the shop on Sundays without buying anything, despite the fact that the probability of this remains the same for all days of the week.

Overall the experiment has shown that it is legitimate to use the noise reduction mode within the scope of a sensitivity analysis to test the impact of a specific factor on system behavior. It is only when we are interested in specific features that are not available in noise reduction mode (e.g. when we want to study differences between particular days of the week or when we need to obtain quantitative data to optimize a real system) that we need to choose the normal mode.

Related Journal Articles & DOI Links

Selected peer-reviewed publications relevant to 12 Lead ECG Acquisition. Click the DOI to access the full paper (may require institutional access).

Why Choose Us?

Bangalore guidance for robotics, Spectre and autonomous systems projects.