Equations need to be viewed in Intenet Explorer. Symbol fonts are not available in Firefox.
The goal of market-share analysis is to assess the effectiveness of marketing actions in competitive environments. The data-collection principle which derives from this goal is to measure the causal variables and performance indicators so as to give as clear a view of marketing effectiveness as possible. There are two main threats to achieving clarity. The first threat comes from inaccuracies in the record of transactions and market conditions. The second threat comes from problems in aggregating even accurate records over time, space, and transaction units. We will discuss these threats in the next two sections. Then we will review the kind of data currently being collected by national tracking services and discuss how these data can be combined into a market information systems .
A data feast has been laid on the table for market researchers to enjoy.We wish to acknowledge Penny Baron for showing us how this feast is prepared.It all arrives on the analyst's plate so well prepared and presented that it is easy to forget that the ingredients may be less than perfect to begin with.
In optically scanned, point-of-sale systems the shopping basket is the basic transaction unit. If the focus of a competitive analysis or a marketing-effectiveness study is on a single category such as margarine or coffee, then only a component of the basic transaction is of primary interest. This component gets recorded in two ways. First, it is posted to an accumulating record of sales for that particular stock-keeping unit (SKU). And second, if the customer is part of an ongoing scanner panel, the whole transaction is posted to a longitudinal (over time and stores) record of this member's purchases.
For the data to be accurately recorded the many components in the management system (of which the POS system is only a part) all have to work properly. The current advertised specials have to be properly entered into the central price file. When advertised specials involve multiple like items, such as brand sizes, colors, or special packs, the proper parent-child relations need to be recorded for each different Uniform-Product-Code (UPC) description included in the advertised special. In multiple-store retail chains the updated price file has to be properly downloaded to the store controller at each retail location. There has to be integrity in the communications between the store controller and each register. Price is advertised in multiple ways. The price tags on items, shelf markings, in-store displays, newspaper or magazine features, home mailers, television and radio spots all have to report prices which are consistent among themselves and with the central price file. And, of course, the item has to scan properly. If an item does not scan, the cashier is likely to record the price manually without the UPC codes needed to post this part of the transaction to the proper files. The scan rate has a very important effect on the utility of the data.
Store managers are granted varying degrees of autonomy by their respective chains, but many have the flexibility to alter prices in their store's controller in order to meet competitive pressures in the immediate locale, or to substitute like items for out-of-stock advertised specials. There is cause for concern about how these special items are recorded in the local system during a promotion period and what happens to these records when the promotion is concluded. A single program normally reverses all the special prices at the end of a promotional period. It is a complex matter to deal with the general rule and all the exception.
While in most data collection efforts the research staff is responsible for data integrity, with scanner data management is responsible. The buyer/merchandiser function, the store-operations function and the informations-systems function of the firm must all work together to insure the accuracy of scanner data. It is an issue of major importance not only to the utility of the data for further research but also to customer satisfaction and to the firm's compliance with business and professional codes. Fortunately, scanner-based systems provide the records that allow management to trace back to the source of an inaccuracy. Error tracking is an important feedback mechanism - helping to insure price integrity, and thereby insuring the utility of the data for kinds of analyses discussed in this book.
Before the advent of optical scanners, we had a different view of marketing performance. One could plot quarterly or annual sales versus marketing expenditures and get the naive view that more successful companies spent more on their advertising and promotional efforts. This would be a particularly naive view if this year's marketing budget is based on last year's revenues or profits. During sustained periods of growth, this leads to expanding budgets and sometimes a corporate sense of invulnerability as expanding markets lead to sales growth almost regardless of competitive activity. During recessionary times, the macro-economic conditions provide a ready scapegoat for corporate performance. Gross aggregation of this sort masks the victories and defeats in each competitive arena which, in sum, constitute firm performance.
The basic analytical principles involved here are similar to those discussed in section 2.8 on the relation between market share and choice probabilities. The issue there concerned whether individual choice probabilities are accurately reflected in aggregated market shares. There we saw that special attention is needed only when choice probabilities are heterogeneous and purchase frequencies are heterogeneous and correlated with choice probabilities - Case 4(b). In the present context we are collecting data to reflect how the causal conditions of the marketplace relate to market shares. Causal condition are most likely to be the elements of the marketing mix such as prices and promotion but also could include regional or seasonal differences, or differences among consumer segments. Table 4.1 displays the four cases.
Table 4.1: Aggregating Market Shares and Causal Conditions
| Causal Conditions | ||
| Market Shares | Homogeneous | Heterogeneous |
| Case 1: | Case 3: | |
| Homogeneous | E(si) = pi | E(si) = pi |
| [`(f(Xk))] = f([`X]k) | [`(f(Xk))] ¹ f([`X]k) | |
| Case 2: | Case 4 (a) Uncorrelated: | |
| Heterogeneous | E(si) = [`(p)]i | E(si) = [`(p)]i |
| [`(f(Xk))] = f([`X]k) | [`(f(Xk))] ¹ f([`X]k) | |
| Case 4 (b) Correlated: | ||
| E(si) = [`(p)]i + cov(m, pi)/ [`(m)] | ||
| [`(f(Xk))] ¹ f([`X]k) | ||
The analogy to Case 1 would be when the market shares are homogeneous and the causal conditions are homogeneous. This might be the case when multiple stores in a retail chain offer the same promotion in a particular week and get essentially the same market response in different areas. Then no insight will be lost through aggregation. Case 2 refers to when market shares are heterogeneous, but causal conditions are homogeneous. This is a situation in which essentially identical stores (perhaps stores in a chain), in essentially identical areas and time periods, offer the same promotional package, and get different market responses. It sounds like a missing-variables problem (i.e., there is something we could measure which could explain the differences). If one can find the missing variable then the causal conditions would no longer be homogeneous and Case 2 is no longer relevant. But if Case 2 describes the situation, investigating the average market shares (or other performance measures) will not distort the known relations to the causal conditions. The primary loss will be degrees of freedom. In Case 3 the market shares are homogeneous and the casual conditions are heterogeneous. Promotions differ over stores but the market shares do not change. While this might seem unlikely, it can happen. Marketing efforts can be ineffective. Case 3 is like Case 4(a) because if market shares do not change they can not be correlated with changes in casual conditions. Here the concerns involve how the aggregation over causal conditions is achieved. Any variable, Xk , we measure as an indicator of a causal condition will be represented in the models as f(Xk) . Whenever causal conditions are heterogeneous it is probably true that the function of the average measure (i.e., f([`X]k)) is not equal to the average of the functions (i.e., [`(f(Xk))] ). Given a choice, it should be the function values which are aggregated rather than the original variables.Often the log-centered form of the variable is a proper and convenient form to aggregate.Since there is no variation in the market shares, the variation lost through aggregation of the causal conditions is unexplainable, and probably therefore not a major loss to our understanding. In Case 4(a) there is heterogeneity in both causal conditions and market shares, but there is no correlation between the variability in market shares and the variability in causal conditions. This is like Case 3, in that aggregation should be done carefully, but there will be little, if any, loss in explanatory power. Finally, in Case 4(b), we have the heterogeneity in market share being correlated with heterogeneity in causal conditions. If differences across regions are correlated with differences in market shares, then the model should be expanded to reflect the role of regions. Similarly, seasonal effects could be incorporated into the model, if they were correlated with differences in market shares. Explanatory power will be lost if cases of these types are aggregated.
While market-share analysis is applicable in broader arenas, the packaged-goods industry will be used as the basis around which the general principles of data collection and aggregation will be discussed. The diffusion of scanner technology has had its greatest impact on the packaged-goods industry. In our estimation it is no accident that the growth in promotion budgets relative to advertising budgets in packaged-good firms has coincided with the availability of data showing the surges in sales which correspond to short-term promotions. But to assess if these promotional expenditures are worthwhile, we need data which are aggregated to correspond to the promotional event. This is a classic Case 4(b) situation. There are huge swings (heterogeneity) in market shares which are correlated with the changes (heterogeneity) in causal conditions (the promotional environment). Aggregation that combines promotion and nonpromotion periods can obscure the relations we wish to study.
The basic time unit of a promotion is a week. This does not mean that promotions only last one week. But in any given week a promotion could begin or end. Temporal aggregation beyond a week would virtually assure that the onset or termination of a promotion would be averaged in with nonpromotion sales, as was the case with Nielsen's bimonthly store-audit data. One could argue for representing each day, but since the promotional conditions are essentially homogeneous over days in a week, we have either Case 1 or 2 and sacrifice at most degrees of freedom rather than explanatory power with such aggregation.One problem is that promotions could begin on different weekdays in different trading areas. This will create at least a small aggregation bias in a weekly database.
The basic spatial unit is either the grocery chain in a trading area or the stores within a grocery chain. A trading area is conveniently defined by the local newspapers. Newspaper features announcing a promotion on a brand for all the local stores of a particular grocery chain help specify the boundaries of a trading area. In each trading area one day has typically evolved as Food Day (i.e., the day on which the special food ads are printed reporting the promotions which are available for the coming week). The basic principle is to capture the events as close to the place of occurrence as possible. Retail scanner data record the events as they occur, whereas warehouse-withdrawal data capture events further from the place of transaction. If we decide to aggregate over stores within a grocery chain, we are averaging possibly heterogeneous market shares over a homogeneous promotional environment - a Case 2 situation, losing degrees of freedom, but not explanatory power.
The basic transaction unit is the brand. Brands come in different sizes (or weights) and versions (e.g., conditioners for dry, normal, or oily hair, coffee ground for drip, electric or regular percolators or automatic coffee makers). With every variation in size, version, special pack, etc., there is a unique UPC code. This can translate into dozens of separate UPC codes for a single brand - typically far too much detail to utilize in market-share analysis. At the other extreme, national advertising typically lumps all the versions and sizes under a single brand banner. The best level of aggregation lies somewhere between these two extremes. But exactly where is difficult to judge. It depends on the business decisions being studied, on practical matters such as computer capacity, and on experience issues such as how familiar one is with this style of analysis. For scanner data one guideline comes from the causal data. If separate sizes or versions of a brand are promoted together, they probably can be lumped together in the data set. As with the matter of industry definition discussed in Chapter 2, we can make a tentative definition, perform the analysis and see if the competitive map portrays substantively different roles to the different sizes or versions of a brand. Our experience in the coffee market led to aggregating all sizes together, but distinguishing between ground and soluble, and between caffeinated and noncaffeinated versions of a brand. In an Australian household-products category (Carpenter, Cooper, Hanssens & Midgley [1988]) various sizes of a brand were aggregated into an 11-competitor market. In a proprietary study of another household-products category the various sizes of each brand were also differentiated, leading to a 66-competitor market - too unwieldy for most purposes.
Aggregating minor brands is also a very judgmental issue. In the Australian household-products study all the wet versions of minor brands were aggregated into AW4 and all the dry versions of minor brands were aggregated into AD4. There were so many minor brands in the market that these aggregates became the largest brands in the study. Since combining small brands together is always a Case 4(b) aggregation, creating large-seeming competitors out of many tiny brands is to be avoided whenever possible. It is probably preferable to allow many more brands as separate units of observation, but restrict the variables which describe these brands to simple-effects, rather than differential effects or fully extended representations. This topic will be taken up in Chapter 5.
We now turn to a description of the kinds of data which are being collected from thousands of stores each week.
This section will describe the variables recorded in the major retail-scanner databases. The data are reported in each store in each week for each UPC code, so that aggregation into brands and grocery chains are separate issues. The prototypes are based on InfoScan/PromotionScan from Information Resources, Inc. (IRI), and a new database, Monitor, from A.C. Nielsen (a division of Dun & Bradstreet).In mid-1987 Dun & Bradstreet reached a tentative agreement to acquire IRI and form Nielsen Information Resources. This move was blocked in November 1987 by the unanimous vote of the SEC. The industry consequences of this blocked attempt are still being played out at the time of this writing. This chapter tries to reflect the types of data which will be available for market-share analysis regardless of the outcomes.
The emphasis in these databases is on descriptive tracking - staying as close as possible to the data and using analytical modeling as little as possible, but creating a database which can be aggregated to fit the analytical needs of any client. In each store-week a core set of measures is reported.
Along with codes for the geographic market area (e.g., the Chicago market) and the grocery chain (e.g., Jewel), these are the basic set of measures which can be gathered directly from POS systems. Note the emphasis on only those basic measures which will add together sensibly over related UPCs. One reports sales volumes, not market shares, until the question, ``Shares of what?'' can be answered.
Nielsen's Monitor is like the combination of IRI's InfoScan and PromotionScan. PromotionScan audits the stores and newspapers to record the promotional environment. Displays and features are separately collected as zero-one measures, gathered either by people in the stores or outside agents and then integrated into the store database each week. The Majer's classification of newspaper features (A - major ads, B - coupon-sized ads, C - line ads) is becoming the standard. A measure reflecting in-ad coupon should be included, rather than merely being reflected as an A- or B-feature in Majer's terms. The displays are sometimes broken down into big displays, little displays (such as shelf talkers), end-of-aisle (gondola-end) displays, or dump bins in products categories such as bar soap.
When display, store-coupon, and feature measures are incorporated with the data from the POS system, it becomes straightforward to track indices such as volume sold on feature, display or any trade deal, or average price or price discount on any style promotion. Over weeks we can track duration of promotions to investigate stores' promotion policies or promotional wearout.
Manufacturers' coupons have been the slowest to be integrated into tracking services. While the purchase price is reported net of coupons redeemed, a simple zero-one measure for the redemption of a manufacturers' coupons is not very helpful. The volume sold on manufacturers' coupons is not reported in either InfoScan or Monitor, but is recorded in the BehaviorScan panels mentioned below. In BehaviorScan panels the coupons used in each transaction of a panelist are put in a separate plastic bag at the check stand, then hand-keyed into the panelist's computer file each week. Given the effort involved, it is not surprising that recording of manufacturers' coupons has lagged behind other developments.
A member of a household scanner panel does little that is conspicuously different from any other shopper. When purchases are made the cashier scans the panel member's bar-coded card. Thus, regardless of the store in which the purchases were made, the organization maintaining the panel can accumulate a comprehensive record of transactions for the household. The panel members are offered minor appliances or similar gifts, and lotteries for more major gifts, such as vacations. Demographic information is collected at the time of recruitment. Very little in the way of traditional survey-research questioning is conducted on panel members. It is generally felt that the more you question these people the less typical their shopping behavior becomes.
The role of household scanner panels in tracking databases such as InfoScan is partly different than the role of IRI's BehaviorScan, Nielsen's Erim or SAMI/Burke's Adtel panels which were organized to support controlled market tests for new products or advertising campaigns. BehaviorScan panels consist of up to 3,000 households in each of 10 mid-sized, media-isolated markets. The media isolation is needed for control in advertising tests. To do national tracking requires that panels to be organized in major urban markets. Major urban markets such as New York, Chicago, and Los Angeles are not conducive to controlled store tests because the geographically broad shopping patterns make it organizationally difficult to capture all purchases of a household, and because the market-research organization lacks the ability to control exposure to advertising. Both kinds of panels must reflect the total purchase profile of the market area. But the BehaviorScan panels must also be partitionable into a number of parallel subgroups for various test and control conditions. BehaviorScan panels help assess new-product sales potential, media weight, media copy, price and promotions, and shelf location. The primary focus of a panel for a national tracking service is simply on providing nationally representative data which help to explain the store-level performance. On a store-week level household scanner panels readily provide penetration measures such as the percent of households using the category or brand (down to the UPC level). Over time one can observe the aggregate path of brand diffusion. Average interpurchase time and average purchase quantity can be reported for the category or the brand. These could be combined to reflect the aggregate household inventory of each brand, leading to interesting studies of how household inventories interact with the promotional environment. One can also report aggregate demographic characterization of the purchasers of different brands.
Panels for market-share analysis must provide the necessary data to address the heterogeneity issue discussed in Chapter 2, ``Is there a correlation between brand-choice probabilities and usage frequency?'' This suggests partitioning the panel into heavy users and light users, and looking for any systematic differences in choice probabilities across brands. If systematic differences appear, a segmentation is indicated as is discussed later in this chapter.
Warehouse-withdrawal or bimonthly store-audit data used to be the primary methods for tracking sales. But for assessing marketing effectiveness these data have obvious shortcomings. We can know what is withdrawn, and can combine these data with the retail promotional environment. But the lag between withdrawal and sales has to be established and may vary over retail locations. There are also the potential for early withdrawal in anticipation of the retail promotion, transshipment of a good trade deal or stockpiling by the stores late in the trade deal to take advantage of discounted wholesale prices even after the retail promotion is over. While warehouse-withdrawal data are useful for understanding behavior in the channels of distribution, their utility for market-share analysis has diminished with the availability of data which track transactions at the retail outlet. Bimonthly store-audit data are temporally aggregated to an extent which masks the weekly pulsing of sales.
Diary panels have also waned in popularity with the emergence of scanner panels. Diary panels suffer from problems of the accuracy of recall and recording which are eliminated by scanner panels. Even when the accuracy issue is attacked by the use of in-home UPC readers, insuring that all transactions are scanned in-home is still problematic. Any in-home measurement or diary maintenance is more obtrusive than transaction scanning at the point of purchase. How long panelists are unaffected by their special status is a concern in any panel, but the more obtrusive the panel measurement, the more of a problem this becomes.
Some of the data which now flow from controlled test-market scanner panels (e.g., BehaviorScan, Erim, or Adtel), should become available for tracking services. The television-exposure data now collected from panel slices in BehaviorScan contain important information for evaluating the effectiveness of media campaigns. Some BehaviorScan households electronically collect viewing data every five seconds and dump the record automatically via an early morning telephone call which is answered just before the bell rings. If a store-level solution to the problem of recording manufacturers' coupons is not found, scanner panels can provide estimates. Subpartitions of panels have also been made available for more traditional market-research surveys of brand perceptions and preferences, as well as attitudes, opinions, and interests. Using standardizations such as zeta-scores or exp(z-scores) makes it straightforward to integrate interval-scale consumer ratings into market-share analysis.
The idea that we can relate all of the measures described above to market performance, and do so in a way that allows us to assess the unique marketing effectiveness of each brand, is a captivating challenge to market analysts and a unique opportunity for managers. While the models are oriented to relating the partial impact of each marketing instrument on volume sold (and the consequent revenues), the costs associated with the marketing effort of the firm and the retailer are also obtainable (or estimable). The models serve as an inference engine, which, combined with data and decision-support software, become an information system capable of simulating the profit consequences of any competitive scenario. The market-wide coverage of the data and the models is why we term these market information systems .
The models which form the inference engine collectively constitute the system of models for competitive analysis depicted in Figure 4.1. The ``Competitive Structure Information (Longitudinal)'' refers to a database of the style developed in this chapter, where the focus is on market-share data for each store each week and the causal variables presumed to influence the transactions. Note that the ``Standardized Data'' feed into the market-share model while the ``Raw Data'' feed the category-volume model. While the modeling can be done in other ways, we feel the raw prices and promotion levels drive the total size of the pie, while the distinctiveness of marketing efforts reflected in the standardized data influences how the pie is shared. The ``Segment Structure Information'' which also feeds these two models refers to the data from household panels which provide for additional understanding of store-level results and also indicate if segmentation of the store-level data is advisable. If a partition of heavy users versus light users is indicated, the panel provides estimates of the population market shares for each segment and the store sales provide sum constraints which should be useful in developing good estimates.
Figure 4.1: A System of Models for Competitive Analysis
The products of the market-shares times category-volume estimates are sales forecasts for each brand in the market. These forecasts must be diagnostically rich to fulfill their role in this system. The diagnostic value of the sales models comes primarily from the elasticities. We know from equation (2.19) that the sum of the market-share elasticity and the category-volume estimates is the sales elasticity.The equation assumes no systematic competitive reaction, but this assumption is relaxed in Chapter 6.A univariate time-series approach to forecasting category volume will not provide the required elasticities. We need to know how the internal conditions of the market affect total category volume. Basically this means we need terms in the category-volume model which correspond to the differential effects in the market-share model. With such variables in a sales model we can run any kind of simulations required for brand planning, and obtain the elasticities needed for investigating competitive structure and for feeding the strategic-analysis model.
The main method for investigating market structure is through competitive maps. The procedures, described fully in Chapter 6, look at the structure underlying the asymmetric, brand-by-brand arrays of cross elasticities over store-weeks, and highlight the events which produce systematic changes in competitive structure. A map corresponding to each structural change can be constructed to assess the threats and opportunities associated with market events. Chapter 6 will also introduce the ``Logit Ideal-Point Model''Cooper, Lee G. & Masao Nakanishi [1983b], ``Two Logit Models for External Analysis of Preferences,'' Psychometrika , 48, 4 (December), 607-20.which can localize the most preferred regions in the competitive maps. Competitive maps can help direct the inquiry into the market which is essential to brand planning.
Firms have data on their own costs and should be able to provide estimates of competitors' costs as well as the costs borne by the retailers for features, displays, and store coupons. The close tying of forecasted revenues with estimated costs enables managers to assess the effectiveness of marketing actions and facilitates brand planning. With panel data on media exposure, one can see how advertising expenditures translate into exposures and how exposures translate into sales. When cost data are combined with the elasticities from the sales model the basic ingredients are present for a strategic-analysis model. Carpenter, Cooper, Hanssens, and Midgley [1988] show how elasticities from attraction models can be used to investigate optimal price and advertising policies under the boundary conditions of no competitive reaction and optimal competitive reaction. The role of these kinds of analyses is discussed in Chapter 7.
Brand planning, in general, and promotion planning, in particular, are directly affected by such a market information system. While management is required to be explicit about the competitive environment it expects to face, plans can be tested for both their profits and robustness to competitive efforts. Chapter 7 presents a brand-planning exercise which uses such a system.
