Equations need to be viewed in Intenet Explorer. Symbol fonts are not available in Firefox.

Chapter 5
Parameter Estimation

5.1  Calibrating Attraction Models

In Chapter 3 we presented market-share attraction models in detail. As we tried to describe realistically the market and competitive structures, more and more complex models had to be introduced - ending at a cross-effects model which has a unique role for each piece of information (e.g., each price or each feature) on the brand to which it refers as well as on every other competitor. From the practical point of view, however, these complex models are not useful unless it is possible for one to calibrate them from the actual market performance of brands. Calibration establishes the value or importance of each of these roles in determining the market performance of each brand. In this chapter we will review the techniques to estimate the parameters of attraction models. We will begin with the most basic models, i.e., the simple-effects form of MCI and MNL models, and then proceed to more complex models such as differential-effects and cross-effects models. To remind the reader, the general specification of simple-effects attraction models is given below.

where:

s_i = the market share of brand i

A_i = the attraction of brand i

m = the number of brands

X_ki = the value of the k^th explanatory variable (X_k) for brand i (e.g., prices, product attributes, expenditures for advertising, distribution, sales force)

K = the number of explanatory variables

f_k = a positive, monotone transformation of X_k

e_i = the specification-error term

a_i, b_k (i = 1, 2, ¼,m; k = 1, 2, ¼, K) = parameters to be estimated.

We may choose either MCI or MNL models, depending on whether f_k is an identity transformation or an exponential transformation. We will often use the MNL model below in order to simplify our presentation, but the corresponding derivations for the MCI model would be straightforward. Before presenting the use of regression analysis, we will first discuss other estimation techniques applicable to model (5.1).

5.1.1  Maximum-Likelihood Estimation

The maximum-likelihood approach to parameter estimation assumes that the data are obtained from a random sample (sample size n) of individuals who are asked to choose one brand from a set of brands (i.e., choice set ).See Haines, George, H., Jr., Leonard S. Simon & Marcus Alexis [1972], ``Maximum Likelihood Estimation of Central-City Food Trading Areas,'' Journal of Marketing Research , IX (May), 154-59. Also see McFadden [1974]. The resultant data consist of the number of individuals who selected object i , n_i (i = 1, 2, ¼, m). This describes a typical multinomial choice process. In order for us to use this type of data, we must modify the definition of the model (5.1) slightly. We assume that the probability, p_i , rather than the market share s_i , that an individual chooses brand i , is specified asSee sections 2.8 and 4.1 for discussions of when market shares and choice probabilities are interchangeable.

Clearly p_i is a function of the parameters of the model, that is, the a 's and b 's. We may write the likelihood for a set of observed choices n₁, n₂,¼, n_m as

and the logarithm of the likelihood function as

By maximizing L or logL with respect to the parameters of the model, we obtain the maximum-likelihood estimates of them. The maximum-likelihood technique may be extended to the cases where observations are taken at more than one choice situation (multiple time periods, locations, customer groups, etc.) provided that an independent sample of individuals is drawn at each choice situation. For example, if a series of independent samples is drawn over time, the log-likelihood function may be written as

where n_it and p_it are the number of individuals who chose brand i in period t and the probability that an individual chooses brand i in period t , respectively, and T is the number of periods under observation.

The maximum-likelihood procedure is a useful technique for parameter estimation in that the properties of estimated parameters are well known,See Haines, et al. [1972].ut we choose not to use it in this book for several reasons. First, since the likelihood and log-likelihood functions are nonlinear in parameters a 's and b 's, the maximum-likelihood procedure requires a nonlinear mathematical-programming algorithm to obtain parameter estimates. Besides being cumbersome to use, such an algorithm does not ensure that the global maximum for the likelihood function is always found. Second, we will be using POS data primarily in calibrating the model. Since POS data generated at a store include multiple purchases in a period made by the same customers, the observed n_i 's may not follow the assumptions of a multinomial distribution which underlie the likelihood function. Third, we will be in most cases using observed market shares, that is, the proportions of purchases of brand i , p_i , based on an unknown but large total number of purchases.Neither multiple purchases in a single shopping trip, nor purchases of a brand on each of multiple shopping trips within a single reporting period (e.g., a week), fit well with the multinomial-sampling assumptions. Yet both such occurrences can be common in POS data. When analyzing POS data at the store-week level the market shares are not subject to the sampling variation with which maximum-likelihood procedures deal so well. Only the specification error requires special treatment. Section 5.4 presents generalized least-squares (GLS) procedures to cope with the issues.The regression techniques developed in the next section are more easily adaptable to this type of data than the maximum-likelihood procedure.

5.1.2  Log-Linear Estimation

We will be presenting estimation procedures based on regression analysis in the next section, but the fact that logit models could be estimated by first applying a log-linear transformation and then applying a regression procedure has been known for a long time. We will review some of these procedures before we turn to the approach which we believe is the most convenient.

Over thirty-five years ago BerksonBerkson, Joseph [1953], ``A Statistically Precise and Relatively Simple Method of Estimating the Bioassay with Quantal Response, Based on the Logistic Function,'' Journal of the American Statistical Association , 48 (September), 565-99.showed that a logistic model of binary choice becomes linear in parameters by the so-called logit transformation . Suppose that each individual in a sample (of size n) independently chooses object 1 with probability p₁ , given by

where:

p₁ = the probability that object 1 is chosen in a binary choice

X_k1 = the k^th characteristic of object 1

b₀, b₁, ¼, b_K = the parameters to be estimated.

If the logit transformation is applied to the above model, we have

That equation (5.3) is linear in parameters logb₀ and b_k (k = 1, 2, ¼, K) suggests the use of regression analysis. But, since the probability p₁ is unobservable, it must be replaced in the left-hand side of (5.3) by p₁ which is the proportion of individuals in the sample who selected object 1. The final estimating equation is in the following form.

The subscript t indicates the t^th subgroup from which the p₁ 's are calculated. The error term e_t is the difference between logit transforms of p₁ and p₁ , and known to be a function of p₁ and the sample size per subgroup from which p₁ is calculated.To examine the property of the error term, first expand the left-hand side of (5.4) by the Taylor expansion, keep the first two terms, and apply the mean-value theorem to obtain

where p₁^* is a value between p₁ and p₁ . The error term is clearly a function of p₁ and therefore heteroscedastic (i.e. unequal variance). If we assume a simple binomial process for each individual selecting object 1 and a reasonably large sample size (n > 100 , say), the variance of e is approximately equal to 1/np₁(1 - p₁) . The use of a generalized least-squares procedure is called for.

Berkson's method has been extended to the estimation of parameters of multinomial logit (MNL) models by Theil.Theil, Henri [1969], ``A Multinomial Extension of the Linear Logit Model,'' International Economics Review , 10 (October), 251-59.Assume a multinomial choice process in which each individual independently selects object i with probability p_i from a set of m objects in a single trial, and let p_i be specified by an MNL model

This model differs from (5.1) in that a single parameter a is specified instead of m parameters, a₁, a₂, ¼, a_m . Theil noted that

where 1 is an arbitrarily chosen object, and suggested the following estimation equation which is linear in parameters b₁, b₂,¼, b_K .

where p_i is the proportion of individuals who chose object i in sample, and e_it^* is the combined error term. Subscript t indicates the t^th subsample. It is obvious that equation (5.4) is a special case of (5.5) for which the number of objects in the choice set, m , equals 2. The total degrees of freedom for this estimation equation is (m - 1)T where T is the number of subsamples. It is known that the variances of e_it^* 's are unequal, and McFadden [1974] studied a method for correcting for this problem. The estimation technique which we will propose in the next section is a variant of Theil's method. It is true that both Theil's method and our method yield identical estimates of parameters and their properties are also identical, but we believe that our method has an advantage in its ease of interpretation.

5.2  Log-Linear Regression Techniques

As we have noted in Chapter 2, model (5.1) becomes linear in its parameters by applying the log-centering transformation. Take the MNL model, for example. First, take the logarithm of both sides of (5.1).

If we sum the above equation over i (i = 1, 2, ¼, m) and divide by m , we have

where [s\tilde] is the geometric mean of s_i and [`(a)], [`X]_k and [`(e)] are the arithmetic means of a_i , X_ki and e_i , respectively, over i . Subtracting the above equation from the preceding one, we obtain the following form which is linear in its parameters.

Similarly, the application of the log-centering transformation to the MCI model results in

where [X\tilde]_k is the geometric mean of X_ki . Since those two equations are linear in parameters a_i^* = (a_i - [`(a)]) (i = 1, 2, ¼, m) and b_k (k = 1, 2, ¼, K ), one may estimate those parameters by regression analysis.

Suppose that we obtain market-share data for T choice situations . In the following, we often let subscript t indicate the observations in period t , but this is simply an example. Needless to say, the data do not have to be limited to time-series data, and choice situations may be stores, areas, customer groups, or combinations such as store-weeks. Applying the log-centering transformation to the market shares and the marketing variables for each situation t creates the following variables:

 

s_it^* = log(s_it/ [s\tilde]_t) (i = 1, 2, ¼, m)

[s\tilde]_t = the geometric mean of s_it

X_kit^* = log(X_kit / [X\tilde]_kt) ( i = 1, 2, ¼,m; k = 1, 2, ¼, K)

[X\tilde]_kt = the geometric mean of X_kit .

Using the above notation, the regression models actually used to estimate the parameters are specified as follows.

MNL Model:

MCI Model:

where e_it^* = (e_it - [`(e)]<sub>t</sub>) and [`(e)]_t is the arithmetic mean of e_it over i in period t . Variable d_j is a dummy (binary-valued) variable which takes value of 1 if j = i and 0 otherwise. Note that estimated values of a_i^¢ (i = 2, 3, ¼, m) from (5.6 - 5.7) are not the estimates of original parameters a_i , but the estimates of difference (a_i - a₁) where brand 1 is an arbitrarily chosen brand. Thus we have shown that the parameters of attraction model (5.1) are estimable by simple log-linear regression models (5.6 - 5.7). However, as was surmised from the discussion of Berkson's and Theil's methods, the error term e_it^* in those regression models may not have an equal variance for all i and t . We will turn to this problem in a later section.

In earlier workNakanishi, Masao & Lee G. Cooper [1982], ``Simplified Estimation Procedures for MCI Models,'' Marketing Science , 1, 3 (Summer), 314-22.we showed that the regression models (5.6 - 5.7) are in turn equivalent to the following regression models.

MNL Model:

MCI Model:

Variable D_u is another dummy variable which takes value of 1 if u = t and 0 otherwise. The corresponding models (5.6 - 5.7) and (5.8 - 5.9) yield an identical set of estimates of a_i^¢'s and b_k 's , and in this sense they are redundant. But one of the advantages of (5.8 - 5.9) is that it is not necessary to apply the log-centering transformation to market shares and marketing variables before regression analysis can be performed, and therefore reduces the need for pre-processing of data. If the number of choice situations, T , is reasonably small, it is perhaps easier to use (5.8 - 5.9). If T is so large that the specification of dummy variables D_u (u = 2, 3, ¼, T) becomes cumbersome, then the use of (5.6 - 5.7) is recommended. In addition, the properties of the error term e_it in (5.8 - 5.9) are easier to analyze than those of e_it^* in (5.6 - 5.7).

5.2.1  Organization of Data for Estimation

Leaving theoretical issues aside for a while, let us look at the actual procedures one must follow for parameter estimation. Given a standardized statistical-program package, such as SAS^(R), the first thing one must do is to arrange the data so that the regression analysis program in such a package may handle regression models (5.6 - 5.7) and (5.8 - 5.9).

Suppose that we have market-share data for m brands in T choice situations (periods, areas, customer groups, etc.), and accompanying marketing activities data. Market-share data may be in the form of percentages (or proportions) or in absolute units. If one ignores for the moment the heteroscedasticity (i.e., unequal variances and nonzero covariances) problems associated with the error terms in regression models (5.6 - 5.9), whether the market-share data are in absolute units or in percentages is immaterial, because the log-centering transformation yields identical parameter estimates regardless of whether it is applied to proportions or the actual numbers of units sold.This property of log-centering is called the homogeneity of the 0^th degree. The estimated values of a₁ in (5.8 - 5.9) are the only terms affected by the choice between proportions and actual numbers, but it does not influence the values of market shares estimated by the inverse log-centering transformation. able 5.1 is an example of market-share data generated by a POS system.

These data were actually obtained at a single store in 14 weeks (i.e., T = 14 ). There are five national and two regional brands of margarine ( m = 7 ). Brand 2 is the same as brand 1 and brand 4 is the same as brand 3, but in larger packages. All brands are half-pound (225g) packages except brands 2 and 4 which are one-pound (450g) packages. The market shares do not sum to one presumably due to private-label brands not listed here. Market shares are volume shares computed by first converting the numbers of units sold to weight volumes and then computing the weight-volume share of each brand. Inspection of the table will show that the market is obviously very price-sensitive.

We will now try to estimate the price elasticity of market shares based on attraction model (5.1). Also given are average daily sales volumes of margarine in this store expressed in units of half-pound package equivalents. The first step in estimation is to create a data set which includes dummy variables d_j (j = 2, 3, ¼, m) and D_u (u = 2, 3, ¼, T) so that regression model (5.8 - 5.9) may be used. We chose (5.8 - 5.9) because the number of periods T is reasonably small (= 14). Table 5.2 shows a partial listing of the data set arranged for estimation with the REG procedure in the SAS^(R) statistical package.

Market share and price data are taken from Table 5.1, and the logarithms of shares and prices are added. In addition two sets of dummy variables - week dummies and brand dummies - are put in the data set. The dummy variables (D1-D5) for only the first five weeks are reported to save space. If the reader examines the pattern of two sets of dummy variables, their meaning should be self-explanatory. The dummy

Table 5.1: POS Data Example (Margarine)

Table 5.2: Data Set for Estimation

variables for weeks graphically reflect that the influence of a particular week is constant over brands. The dummy variables for brands graphically reflect that the baseline level of attraction for each brand is constant over weeks, and thus independent of variations in market conditions.

5.2.2  Reading Regression-Analysis Outputs

Now we are in a position to estimate the parameters of attraction model (5.1), in which the only marketing variable is price. Letting P_it be the price of brand i in week t , there is only one attraction component for the MCI version of (5.1) which may be written as

which in turn shows that the regression model (5.8) is applicable here.

Table 5.3 gives the estimation results from the SAS^(R) REG procedure.

The dependent variable is, of course, the logarithm of market share. The first part of the output gives the analysis of variance results. The most important summary statistic for us is, of course, the R² figure of 0.735 (or the adjusted R² value of 0.65) which suggests that almost 75% of the total variance in the dependent variable (log of share) has been explained by the independent (=exploratory) variables (log of price, in this case) and dummy variables d₂ through d₇ and D₂ through D₁₄ . The F-test with the ``Prob > F'' figure of 0.0001 shows that the R² value is high enough for us to put our reliance on the regression results.This test is really against a null hypothesis that all the parameters are zero. There is less than a one-in-ten-thousand chance that this null hypothesis is true. So we can be confident that something systematic is going on, but it takes a much closer look to understand the sources and meaning of these systematic influences.Note that the total degrees of freedom (i.e., the available number of observations -1) is not 97 but 83. This is because there are observations in the data set (see Table 5.2) for which the market share is zero. Since one cannot take the logarithm of zero, the program treats those observations as missing, decreasing the total degrees of freedom. The problems associated with zero market shares will be discussed in section 5.11.

The second part of the output gives the parameter estimates; the intercept gives the estimate of a₁ ; D2 through D7 give estimates of

Table 5.3: Regression Results for MCI Equation (5.8)

a₂^¢, ¼, a₇^¢ ; DD2 through DD14 give the estimates of g₂, g₃, ¼, g₁₄ ; the value next to LPRICE gives the estimate of b_p , and so forth. From this table several important facts concerning the competitive structure of margarine in this store are learned.

First, the estimated price parameter is a large negative value, -8.34 , indicating that the customers of this store are highly price-sensitive. The statistical significance for the estimate is shown by the T-value and ``Prob > |T|'' column, both of which show that the estimate is highly significant.To be precise it is significantly different from zero. It should also be noted that the reported probability levels are for two-tailed tests. While nondirectional hypotheses are appropriate for time-period and brand dummy variables, we often have directional hypotheses about the influences of prices or other marketing instruments. The reported probabilities should be cut in half to assess the level of significance of one-sided tests.Recall from Chapter 2 that the parameter value is not the same as the share elasticity for a specific brand. In the case of an MCI model, the latter is given by b_p(1 - s_it) . For example, if a brand has a 20% share, its share elasticity with respect to price is approximately -8.34 ×(1 - 0.2) = -6.67 , indicating a 10% price cut should lead to a 66.7% increase in share (from 20% to 33%).

Second, the estimates of brand specific parameters, a₂^¢,¼, a_m^¢ , are all negative and statistically significant. The true values of a₂^¢, ¼, a_m^¢ are estimated by adding the corresponding regression estimates to the estimated value of a₁ . Since a₁ is estimated at 44.8, we know that brand 1 has the strongest attraction if other things are equal. Brand 5 has the weakest attraction with a₁ + a₅ = (44.8-3.55) = 41.25 . This implies that, other things being equal, brand 1 is 35 times (= exp 3.55) as attractive as brand 5. It is rather interesting to note that brand 2 (which is one-pound package of brand 1) has approximately one-half the attraction (exp-.62 » 0.54) of brand 1. Even within a brand a weaker size has to resort to lower unit prices than the stronger size to gain a larger share .

Third, the estimates of g₂, g₃, ¼, g_T are with few exceptions (weeks 6, 7, and 11) statistically insignificant. This normally suggests that dummy variables D₂, D₃, ¼ , D_T may be deleted from the regression model, which in turn suggests that a multiplicative model of market share (discussed in Chapter 2) probably would have done as well as the attraction (MCI) model in analyzing the data in Table 5.1. However, we chose an attraction model not only because of how well it fits the data but because it represents a more logically consistent view of the market

Table 5.4: Regression Results for MNL Equation (5.9)

and competition.The parameters for the time periods merely serve the role of insuring that the other parameters are identical to those of the original nonlinear model. This structure guarantees that the model will produce market-share estimates which are always non-negative and always sum to one over estimates for all alternatives in a choice situation. ince our purpose is to estimate the parameters of an attraction model correctly, it is not justified for us to drop those dummy variables from the regression equation.

Table 5.4 gives the estimation results by equation (5.8) of the MNL version of attraction model (5.1). The independent variables are the same as those of (5.9), except that price itself is used instead of the logarithm of price. The overall pattern of estimated parameters is very similar to those from (5.9). The estimated value of the price elasticity parameter, b_p , is -0.054 . Recall that the share elasticity with respect to a marketing variable (price in this case) is given by b_p P_it(1 -s_it) . If s_it is 0.2 and price is 150 yen for a brand, the price elasticity is approximately -6.5 , which agrees well with the estimated elasticity value from equation (5.9).

5.2.3  The Analysis-of-Covariance Representation

It may added that regression models (5.8 - 5.9) are equivalent to an analysis-of-covariance (ANCOVA) model of the following form.

MNL Model:

or

MCI Model:

where:

m = the grand mean

m_i = the brand main effects (i = 1, 2, ¼, m)

m_t = the period main effects (t = 1, 2, ¼, T).

There is no brand-by-period interaction term because there is one observation per brand-period combination. The ANCOVA models yield parameter estimates that are identical to those obtained from models (5.8 - 5.9). This ANCOVA representation clarifies the characteristics of (5.8); an attraction model requires that the period main effects be taken out before the parameters of marketing variables are to be estimated. If we ignore the properties of the error term (discussed in the next section), the ANCOVA model may be convenient to use in practice since it does not require cumbersome specification of brand and period dummy variables.

5.3  Properties of the Error Term

We have deferred the discussion of the analysis of the error term up to this point, though it has been suggested that the error terms in regression models (5.6 - 5.7) and (5.8 - 5.9) are known to have unequal variances and non-zero covariances in some cases and may require special care in estimation. Before we show this, we will have to make some assumptions as to the composition of the error term with respect to the sources of error.

It is important to recognize two sources of errors inherent in the estimation of market-share models. The variability due to sampling is clearly one source of error, but there is another source of error we must consider. Recall that attraction model (5.1) includes an error term, e_i , which arises due to the omission of some relatively minor factors from its specification of explanatory variables, the X_kit 's, in (5.1). We will call this source of error the specification error . Considering those sources of error, the error terms in regression models (5.8 - 5.9) may be expressed as

where e_i1t is the specification-error term and e_2it is the sampling-error term.To be precise, the error term in attraction model (5.1) should be written as e_1it , but we will not change the notation at this point for the reasons that will become apparent later. he error term in regression model (5.6) is given by subtracting [`(e)]_t , the means of e_it over i in period t , from e_it . Hence we may write

where [`(e)]<sub>1t</sub> and [`(e)]_2t are respective means of e_1it and e_2it over i in period t .

5.3.1  Assumptions on the Specification-Error Term

We will make the following assumptions regarding the specification-error term, e_1it , throughout the remainder of this book.

1. e_1it is normally distributed with mean 0 and variance s_i² ,
2. the covariance between e_1it and e_1jt is s_ij for all t ,
3. there is no correlation between e_1it and e_1ju if u ¹ t ,
4. e_1it is uncorrelated with the sampling-error term, e_2it .

We have so far made no assumption about the sampling-error term (except that it is uncorrelated with the specification-error term) because the method of data collection greatly affects the properties of sampling errors. Two basic methods of data collection will be distinguished.

One is the survey method in which a sample is randomly drawn from a universe of consumers/buyers. In this case the unit of analysis is the individuals in the sample. One may ask the respondent which brand he/she selected or how many times he/she purchased each brand in a period. Individual selections or purchases are then aggregated over the sample to yield market-share estimates. It may be noted that the so-called consumer panels - diary or optical-scanner - share essentially the same characteristics as the survey method as a data collection technique because the unit of analysis is an individual consumer or household.

Another basic method concerns data gathered from POS system. It should be emphasized that POS-generated market-share data are based on all purchases made in a store in a period and not on the responses obtained from a sample of customers to the store. This means that we need not be concerned with the normal sources of sampling variations (i.e., sampling variations among customers within a store). Our only concern is with sampling variations between stores, since POS data currently available to syndicated users are usually based on a sample of stores. We will deal with each type of data collection method in turn.

5.3.2  Survey Data

Let us assume that a series of samples of consumers or buyers is obtained by a simple random sampling. We assume that an independent sample is drawn for each period (or choice situation). Since the following analysis is limited within a period, time subscript t is dropped for simplicity. As noted above, one may ask the respondent either which brand he/she chose or how many times he/she bought each brand in a period. We will have to treat those two questioning techniques separately.

First consider the case in which each respondent is asked which single brand he/she chose from a set of available brands (= choice set). In this case we may assume that the aggregated responses to the question follow a multinomial choice process. Formally stated, given a sample size n and the probability that a respondent chose brand i is p_i (i = 1, 2, ¼,m) (m is the number of available brands), the joint probability that brand i is chosen by n_i individuals (i = 1, 2, ¼, m) is given by

The market-share estimates are p_i = n_i/n (i = 1, 2, ¼, m). These estimates are subject to sampling variations.

Let us now turn to the properties of the sampling-error term

when market-share estimates, the p_i 's, are generated by the multinomial process described above. It is well known that for a reasonably large sample size (n > 30 , say), p_i is approximately normally distributed with mean p_i and variance p_i(1 - p_i)/n . Given this approximate distribution, we want to know how e_2i is distributed. We will use the same technique as that used by Berkson. First, expand logp_i by the Taylor expansion around logp_i and retain only the first two terms. Then apply the mean-value theorem to obtain

where p_i^* is a value between p_i and p_i . This shows that for a reasonably large sample size, logp_i is approximately normally distributed with mean logp_i and variance p_i(1 -p_i)/np_i^*2 . The approximation will improve with the increase in sample size, n . Thus the sampling error is also approximately normally distributed with mean zero and variance p_i(1 - p_i)/np_i^*2 . Furthermore, due to the nature of a multinomial process, it is known that e_2i and e_2j ( j ¹ i) in the same period are correlated and have an approximate covariance -p_ip_j/np_i^* p_j^* where p_j^* is a value between p_j and p_j . For a reasonably large sample size, we may take

Clearly the variance of the error term is a function in p_i and takes a minimum value 1/n for p_i = 0.5 and a large value for very small values of p_i . For example, if p_i = 0.01 , the variance of e_2i is approximately equal to 99/n . This phenomenon is called heteroscedasticity in the variance of e_2i . But we must also be concerned with the covariance between e_2i and e_2j to the extent 1/n is not negligible.

The above properties of the error term are based on the assumptions that each respondent is asked which brand he/she chose in a given choice situation. The properties change considerably if the respondent is asked how many units of each brand he/she purchased in a period. The individual responses are aggregated over the sample to yield the number of units of brand i bought by the entire sample, x_i (i = 1, 2, ¼, m ). The estimate of market share of brand i is given by [^s]_i = x_i/ x where x is the sum of the x_i 's over i . What are the properties of the error term when the logarithm of [^s]_i is used as the dependent variable in regression model (5.8 - 5.9) or the log-centered value of [^s]_i is used in (5.6 - 5.7)? The answer depends on the assumption we make on the process which generates the x_i 's. In general the derivation of the properties of the error term is a complicated task since [^s]_i is a ratio of two random variables x_i and x , the latter including the former as a part of it. Luckily for us, however, the estimated value of parameters of (5.6 - 5.7) will not change if we used the log-centered value of [`x]_i , the mean of x_i , in place of the log-centered value of [^s]_i in (5.6 - 5.7), since

where [x\tilde] and [[^s]\tilde] are the geometric means of x_i and [^s]_i over i in a given period. This in turn suggests that in regression model (5.8 - 5.9) we may use log(x_i) as the dependent variable without changing the estimated values of parameters other than a₁ . This reduces our task in analyzing the properties of the error term considerably.

Suppose that the x_i 's are generated by an arbitrary multivariate process with means m₁, m₂, ¼, m_m and covariance matrix Q with elements {q_ij}. Note that the true market share is given by s_i = m_i/ m where m is the sum of the m_i 's over i . The sample mean of x_i , [`x]_i , is an estimate of m_i . We obtain the linear approximation of logx_i by the usual method, that is,

where x_i^* is a value between [`x]<sub>i</sub> and m<sub>i</sub> . If we replace log([^s]<sub>i</sub>) in the equations leading to (5.8 - 5.9) by log[`x]_i , the sampling-error term becomes

When the sample size is reasonably large, the approximate variances and covariances among the e_2i 's are given by

These results agree with those for the multinomial process, if we note that m_i = p_i and [`x]_i = p_i in the latter process. The variance and covariances of the sampling error term are clearly functions of m_i and may take a large value if m_i or m_j are near zero. The existence of heteroscedasticity is obvious.

We now combine the above results with our assumptions on the specification-error term. Under the assumptions of a multinomial choice process and a single choice per individual, the approximate variances and covariances among the e_i 's in a same period are given by

where Var(e_2i) and Cov(e_2i, e_2j) are given either by (5.10). Because of the heteroscedasticity of the error term, it is known that the estimated parameters of regression models (5.6 - 5.9) based on the ordinary least-squares (OLS) procedure do not have the smallest variance among the class of linear regression estimators. Nakanishi and Cooper [1974] suggested the use of a two-stage generalized least-squares (GLS) procedure in the case of a multinomial choice situation to reduce the estimation errors associated with regression models (5.6 - 5.9). The interested reader is referred to Appendix 5.14 for more details of this GLS procedure.

5.3.3  POS Data

When the market-share estimates are obtained from POS systems, it is not necessary for us to consider the sampling errors within a store, but, if our market-share data are obtained by aggregating market-share figures for a number of stores, we should expect that there are variations between stores. This presents us the heteroscedasticity problem similar to what we encountered with survey data. But there are additional problems as well. Each store tends to offer its customers a uniquely packaged marketing activities. If we aggregate market-share figures from several stores, we will somehow have to aggregate marketing variables over the stores. As discussed in Chapter 4, aggregation is safe if the causal condition (i.e., promotional variables) are homogeneous over the stores - as might be the case when stores within a grocery chain are combined. One should avoid the ambiguity which results from aggregation, either by explicitly recognizing each individual store or by aggregating only over stores (within grocery chains) with relatively homogeneous promotion policies. We will take this approach in the remainder of this book.

Stated more formally, let s_iht be the market share of brand i in store h in period t , and X_kiht be the value of the k^th marketing variable in store h in period t . Regression model (5.6 - 5.7) may be rewritten with the new notation as

MNL Model:

MCI Model:

where s^*_iht is the log-centered value of s_iht in store h in period t , and [`X]_kht and [X\tilde]_kht are the arithmetic mean and geometric mean of X_kiht over i in store h in period t .

The main advantage of a disaggregated model such as (5.11 - 5.12) is that we do not have to deal with sampling errors in estimation. Similar expressions may be obtained for (5.7) or (5.9), but in actual applications there will be too many dummy variables which have to be included in the model. It will be necessary to specify (H ×T - 1) dummy variables, where H is the number of stores, which replaces the (T - 1) period dummy variables in (5.8 - 5.9). With only a moderate number of stores and periods it may become impractical to try to include all necessary dummy variables for estimation, in which case the use of models (5.11 - 5.12) is recommended.

5.4  *Generalized Least-Squares Estimation

In the preceding section we noted that the error terms in regression models for estimating parameters of market-share models tend to be heteroscedastic, i.e., have unequal variances and nonzero covariances. If market-share figures are computed from POS data, the error terms in regression models (5.6 - 5.12) involve only what we call specification errors . Let S be the variance-covariance matrix of specification errors with variances s_i² (i = 1, 2, ¼, m) on the main diagonal and covariances s_ij (j ¹ i) as off-diagonal elements. Because matrix S is heteroscedastic, Bultez and NaertBultez, Alain V. & Philippe A. Naert [1975], ``Consistent Sum-Constrained Models,'' Journal of the American Statistical Association, 70, 351 (September) 529-35.proposed an iterative GLS procedure. The steps of an iterative GLS procedure are as follows.

1. The OLS procedure is used to estimate the parameters in one of the regression models (5.6 - 5.12), and S is estimated from the residual errors.One can simply sort the OLS residuals by brand and time period, compute the variance of each brand's residuals and compute the covariance between ordered residuals for each pair of brands.
2. The data for each period are re-weighted by the estimated [^(S)]^-1/2 .
3. The first two steps are repeated until the estimated values of the regression parameters converge.

There is one minor problem in applying this iterative procedure. It may be remembered that, in regression model (5.6), the log-centering transformation is applied to the dependent variable, the variance-covariance matrix for the e^*_it 's is given by

where I is an identity matrix and J is a matrix, all elements of which are equal to 1. The dimensions of both I and J is m ×m , where m is the number of available brands. [^(S)]^* computed from OLS residuals is therefore singular and not invertible. Since regression models (5.8 - 5.9) are equivalent to (5.6 - 5.7), the residuals estimated from the former are identical to those estimated from the latter, and hence the estimated covariance matrices are also identical. In general, if both brand-dummy variables and period- (or store-) dummy variables are inserted in a regression model, the estimated residual covariance matrix becomes singular. This certainly is an impediment to the GLS estimation procedure which requires the inverses of estimated covariance matrices.

There are three methods of circumventing this problem. One is to delete one row and corresponding column from [^(S)]^* and invert it. One observation (which corresponds to the deleted row/column of [^(S)]^*) per period is deleted and the parameters are estimated on the remaining data. The drawback of this technique is that estimated parameters will be transformations of original parameters, and hence will have to be transformed back to the original, a process which is rather cumbersome. A second method is to set to zero those off-diagonal elements of an estimated residual covariance matrix which are nearly zero. Though theoretically less justifiable, it has its merit in simplicity. Usually it is sufficient to set just a few elements to zero before the inverse may be obtained.If

one wishes to be more formal in this method, one may set to zero those elements which are not significantly different from zero statistically. On the other hand, by setting all off-diagonal elements to zero we obtain an easily implemented, weighted least-squares () procedures which compensates only for differences the variance of specification errors between brands.The third method is to find the generalized inverse of [^(S)]^* .

5.4.1  Application of GLS to the Margarine Data

As an illustration of the GLS technique consider the data set given in Table 5.1. The OLS estimation technique applied to regression model (5.8) yielded the parameter estimates in Table 5.3. Residual errors were then computed from the above OLS results and S was estimated. The estimated S and its inverse are shown below. Those elements of the estimated S which were less than 0.3 were set to zero before the matrix was inverted.

Covariance Matrix

Inverse Covariance Matrix

The square-root of the above inverse matrix was pre-multiplied by the data matrix for each week, and the estimates of the following form are obtained.

where X_t is the independent variable matrix and y_t is the vector of the dependent variable for period t . The re-estimated parameter values are shown in Table 5.5.

Table 5.5: GLS Estimates for Table 5.3

Table 5.5 gives the so-called two-stage GLS estimates. If necessary, residual errors and S may be computed from the above results again and another GLS estimates may be obtained. But, since the parameter estimates in Table 5.3 are extremely close to those in Table 5.5, further iterations seem unnecessary. In fact it has been our experience that OLS and GLS estimates are very similar in many cases. The OLS procedure appears satisfactory in many applications.

So far we have reviewed estimation techniques applicable to relatively simple attraction models (5.1). We have shown in Chapter 3 that attraction models may be extended to include differential effects and cross effects between brands. In the following sections we will discuss more advanced issues related to the parameter estimation of differential-effects and cross-effects (fully extended) models.

5.5  Estimation of Differential-Effects Models

The differential-effects version of attraction model (5.1) is expressed as follows.

where either an identity or exponential transformation may be chosen for f_k , depending on whether an MCI or MNL model is desired. The chief difference between (5.1) and (5.13) is the fact that parameter b_ki has an additional subscript i , suggesting that the effectiveness (and hence the elasticity) of a marketing variable may differ from one brand to the next. This is certainly a plausible model in some situations and worth calibrating.

The estimation of parameters b_ki (i = 1, 2, ¼, m) is not extremely complicated. Only a slight modification of regression models (5.6 - 5.9) achieves the result. Using the previous definitions for dummy variables d_j and D_u , the differential-effects versions of regression models (5.6 - 5.7) are given by

MNL Model:

MCI Model:

In regression models (5.14 - 5.15) the independent variables are replaced by each variable multiplied by (d_j - 1/m), which equals (1 - 1/m) if j = i , and -1/m otherwise. Thus the number of independent variables is (m×K) + m - 1 . Note that regression models (5.14 - 5.15) will have to be estimated without the intercept term. Most regression programs provide us with this option.If an intercept term is included, its estimated value will be zero. We cannot obtain the estimate of a₁ from (5.14) or (5.15), but this poses no problem in computing market shares since the estimated value of a_i is actually the difference between true a_i and a₁ .Rather than automatically assigning a₁ as the brand intercept to drop, one can run the regression with all brand intercepts (which will be a singular model) and find the intercept closest to zero as the one to drop.Similarly regression models (5.8 and 5.9) may be modified as follows for their respective differential-effect versions.

MNL Model:

MCI Model:

Regression models (5.14 - 5.15) and (5.16 - 5.17) yield identical estimates of parameters a 's (except a₁) and b 's. If the number of periods (or choice situations) is large, (5.14 - 5.15) will be preferred.

The reader may feel that the following regression models are more straightforward modifications of (5.6 - 5.7), but it is not the case.

MNL Model:

MCI Model:

Models (5.18 - 5.19) do not represent an attraction model, but a log-linear market-share model in which the share of brand i is specified as

where X^*_ki is a centered value of X_ki , that is, ( X_kit -[`X]_kt) if f_k is an exponential transformation and (X_kit/ [X\tilde]_kt) if f_k is an identity transformation. While these models themselves may have desirable features as market-share models, models (5.18 - 5.19) are not the estimating equations for (5.13).The difference here is that (5.14 - 5.15) log-center the differential-effect variable, while (5.18 - 5.19) log-center the simple-effect variable and then multiply these log-centered variables by the brand-specific dummy variables.

Let us see what those modifications mean from the illustrative data of Table 5.1. The independent variable in this case is price. In order to estimate regression model (5.17) (for an MCI version), data must be arranged as in Table 5.6. Only the dependent variable and a part of explanatory variables (log(price) × brand dummy variables) are shown. The week and brand dummy variables are the same style as in Table 5.2.

The estimation results are shown in Table 5.7. The fit of model, as measured by R² , improved from 0.736 to 0.826. The gain from adding six more independent variables (LPD1 through LPD7 instead of LOG(PRICE)) may be measured by the incremental F-ratio 4.9386 ( = (86.8406-77.3339)/(6 × .32083)), which is significant at the .99 level (df = 6, 57). This shows that the differential-effect model is a significant improvement over the explanatory power of the simple-effects model. The estimated parameter values are markedly different from one brand to the next. Looking at the price-parameter estimates, we note that a larger size tends to be more price sensitive than a smaller size even within a brand. Brands 2 and 4 have greater (in absolute values) values than brands 1 and 2. Brand 5 is most price sensitive with the estimated value of -24.08, but this may reflect the fact that this brand's share was zero and hence not available for estimation for 10 weeks out of 14. We shall discuss this issue in a later section. Two brands, 6 and 7, are not price sensitive. Their price parameters are not statistically different from zero as indicated by their respective ``Prob. > |T|'' values. As to the estimates of a 's, we may note that they are negatively correlated with price-parameter estimates over brands, but we will not attempt to make generalizations on the basis of this single example.

The arrangement of data for estimating model (5.15) is given in Table 5.8. Only the dependent variable and the price × brand dummy

Table 5.6: Data Set for Differential-Effects Model

Table 5.7: Regression Results for Differential-Effects Model (MCI)

Table 5.8: Log-Centered Differential-Effects Data