Automated online design and analysis of marketing research activity and data

[0001] This invention relates to a method, system and program product for dynamic online automation of the design of Marketing Research activity, its deployment and the analysis and reporting of the data gathered from it. Marketing research may include one or more research approaches, e.g., marketing experiments, surveys, interviews, focus group discussions and the like.

[0002] Marketing research initiates from the problem that the merchant is trying to answer. The information required to address the problem is determined, the existing information and the available techniques are examined and the information gap is analyzed. Thereafter, a research approach is designed based on available data and analytical techniques to collect the missing information. The drawback is that this process is mostly manual and off-line in nature.

[0003] In the brick-and-mortar world, customer opinions and actions cannot be compared, since only aggregate data is available. Customers, whose opinions are collected, may not be contacted later to check whether they acted on the opinions expressed or if they behaved according to their disclosed preferences, and if they did not, what were the reasons. To overcome this problem, marketers have created panels of households who are paid regular incentives to disclose their preferences and product purchase information. The high cost of maintaining such a panel, however, limits the size and the scope of such studies.

[0004] Offering incentives as mentioned above is an established practice in marketing research. Often, like the marketing research process, the incentive determination is done manually and is not personalized for each participant.

[0005] On the other hand, even though the merchant can observe customer's actions on the Internet, he/she has no specific means of concluding about customer intentions. The merchant, therefore, tries to infer about preferences and opinions from customer's observed actions. Association rules and discovery of sequential patterns based on purchase data is one such attempt. These approaches have the limitation that they try to find patterns in the collected data and draw conclusions rather than answer a specific problem given by the merchant.

[0006] U.S. Pat. Nos. 5,893,098 and 6,175,833 have discussed the method and apparatus to conduct or-line customer surveys and opinion polls respectively. These methods are bound in the manner that they do automate the administration of customer surveys and opinion polls, but they do not automate the process of their design.

[0007] U.S. Pat. No. 6,195,646 describes a pricing model for selling information, pay per view, pay by the hour, etc., and dimensions along which the pricing model could be based.

[0008] The object of the present invention is to overcome the above limitations by providing a method, system and program product to conduct the design process automatically and dynamically in an on-line environment.

[0009] The second object of the invention is to compare the value of information with the cost of acquisition to decide on the method and process of information acquisition.

[0010] Further objective of the invention is to control the parameters of the information.

[0011] Yet another objective of the invention is to have a method, system and program product to determine the value and type of incentives for each customer.

[0012] To achieve the said objectives this invention provides a method for automating the design of online marketing research activity, its deployment and the analysis and reporting of the obtained data comprising:

[0013] identifying the desired marketing research objectives,

[0014] determining the marketing information required to meet said marketing research objectives,

[0015] selecting the most appropriate research approach design,

[0016] deciding the target group and costs for effectively deploying said marketing research approach design,

[0017] deploying said marketing research approach design including dynamically modifying said research approach design and/or target group of participants,

[0018] collecting and analyzing data, and

[0019] reporting findings.

[0020] The said identification of desired marketing research objective is by selecting from predefined objectives and/or their combinations and or inter-relationships.

[0021] The said marketing information is determined by mapping each said objective to a set of data analysis techniques, identifying associated data requirement, determining which required information is not available in existing data repositories, and the desired format of such missing data.

[0022] The said research approach design is selected by choosing from a repository of research designs based on said marketing information, computed cost of deployment including cost of personalization and incentivisation, and value of marketing information to be obtained.

[0023] The said deployment is by adaptively adjusting the number and selection of participants and/or dynamically modifying said research design to suit an individual participant such that maximum information value is obtained at minimal cost.

[0024] The said predefined objectives include sales forecast, product preference test, advertising campaign effectiveness studies, new product acceptance and potential, pricing studies, study of business trends, desired feature enhancement to a product, market share analysis, studies of coupons, marketing campaign design, and market structure study to spot new opportunities.

[0025] The said data analysis techniques include conjoint analysis, segmentation using clustering based on attribute importance of product features, and collaborative filtering or demographic segmentation.

[0026] The above method includes an objective-technique map for using a supervised learning technique for analysis of participant information.

[0027] The above method includes incentive, utility and gain values for each participant associated with each said research design.

[0028] The said method includes the use of previous customer history of purchases, responses to research designs, click-stream information and any other available data.

[0029] The said incentivisation is based on the value of information from the customer and the cost of collection of said information including the cost of said incentive.

[0030] The above method includes selecting said participants based on cost of reaching the participant, cost of incentives offered to ensure participation, the probability of obtaining a response and the cost of analysing and using the information collected.

[0031] The above method includes the use of adaptive re-sampling technique for predicting a new participants' class and computing the uncertainty index for each participant as well as the total uncertainty index for all the participants.

[0032] The above method includes determining said incentives optimally, based on the history of said participants responses to said research designs, probability of said participant responding to a given incentive, value of information expected to be received and cost of conducting the research on said participant.

[0033] The participant selection is performed incrementally until a specified termination condition is reached.

[0034] The above method includes computation of the value of the information expected from said participant using Bayes decision theory to compute expected incremental payoff from each participant, the total expected payoff, and the decline in uncertainty.

[0035] The said mapping is performed iteratively with merchant interaction.

[0036] The said mapping is implemented using an incremental learning algorithm to improve output.

[0037] The said method uses a selected one or more out of a plurality of approaches to determine required data, said selection being performed by said merchant based on cost and merit of each approach.

[0038] The above method includes learning techniques for identifying desired marketing research objective, determining marketing information required, and selecting the most appropriate research approach design, based on findings and merchant feedback.

[0039] The said existing data repositories are available online either at all times or periodically or at specified time frames.

[0040] The said method incorporates participant preferences in responding to various research designs and/or participant profile as a basis for selection of participants including expected response time, previous history of purchases, usage of coupons, response to advertisements and research designs, click-stream information and any other participant specific information.

[0041] The above method incorporates the value of information as a function of the best prediction model from among several prediction models.

[0042] The said payoff is computed based on one or more repetitive game(s).

[0043] The said deployment is over multiple participant sets each set being determined by selection techniques.

[0044] The participant selection uses incremental learning algorithms and/or predictive techniques and/or learning tools to improve output.

[0045] The incentivisation uses incremental learning algorithms to improve output.

[0046] The above method uses a plurality of marketing research approach designs, a different marketing research approach design being used for each participant based on ranking according to expected gains.

[0047] The said termination condition is based on a threshold on uncertainty or rate of change of uncertainty or information value or rate of change of information value

[0048] The research approach design consists of one or a combination of an experimental design to study the response to an advertisement, coupon or product recommendation and/or classification of participants and/or survey(s).

[0049] The research approach design consists of a set of independent sub-sets of research approach designs.

[0050] The research approach design is deployed incrementally in stages with each stage consisting of one or more independent sub-sets.

[0051] The incentive offered to the participant is contingent on the timing of response and/or quality of response.

[0052] The present invention also provides a system for automating the design of online marketing research activity, its deployment and the analysis and reporting of the obtained data comprising:

[0053] marketing research objectives identification means,

[0054] marketing information requirement determination means to meet said marketing research objectives,

[0055] research approach design selection means,

[0056] target group selection means for effectively deploying said marketing research approach design,

[0057] deployment means for deploying said marketing research approach design including dynamically modifying said research approach design and/or target group of participants,

[0058] means for collecting and analyzing data, and

[0059] means for reporting findings.

[0060] The marketing research objective identification means is a merchant objective specification tool for selecting from predefined objectives or their combination and/or interrelationships.

[0061] The said requirement determination means is information requirement analysis tool for mapping each said objective to a set of data analysis techniques, identifying associated data requirements and determining which required information is not available in existing data repositories and the desired format of such missing data.

[0062] The said research approach design selection means is a research approach design selection tool for choosing from a repository of research designs based on said marketing information, computed cost of deployment including cost of personalization and incentivisation, and value of marketing information to be obtained.

[0063] The said deployment means is a mechanism for adaptively adjusting the number and selection of participants and/or dynamically modifying said research design to suit an individual participant such that maximum information value is obtained at minimal cost.

[0064] The said merchant objectives specification tool provides means for selecting from predefined objectives including sales forecast, product preference test, advertising campaign effectiveness studies, new product acceptance and potential, pricing studies, study of business trends, desired feature enhancement to a product, market share analysis, studies of coupons, marketing campaign design, and market structure study to spot new opportunities.

[0065] The said information requirement analysis tool incorporates a mechanism for implementing data analysis techniques including conjoint analysis, segmentation using clustering based on attribute importance of product features, and collaborative filtering or demographic segmentation.

[0066] The above system further includes a mechanism for implementing an objective map technique for using a supervised learning technique for analysis of participant information.

[0067] The above system includes means for associating incentive, utility and gain values for each participant with each said research design used.

[0068] The said system includes a mechanism for using previous participant history of purchases, responses to research designs, click-stream information and any other available data.

[0069] The above system includes means for determining said incentives based on the value of information from the participant and the cost of collection of said information including the cost of said incentive.

[0070] The above system further comprises mechanism for selecting said participants based on cost of reaching the participant, cost of incentives offered to ensure participation, the probability of obtaining a response and the cost of analysing and using the information collected.

[0071] The said system further comprises means for using adaptive resampling technique for predicting a new participants' class and computing the uncertainty index for each participant.

[0072] The above system further comprises mechanism for determining said incentives optimally, based upon the history of said participants responses to said research designs, probability of said participant responding to a given incentive, value of information expected to be received and cost of conducting the research on said participant.

[0073] The above system further includes means for incremental participant selection until a specified termination condition is reached.

[0074] The said system includes means for computing the value of the information expected from said participant using Bayes decision theory to compute expected incremental payoff from each participant, the total expected payoff, and the decline in uncertainty.

[0075] The said means for mapping is a mechanism that operates iteratively with merchant interaction.

[0076] The said means for mapping uses an incremental learning mechanism to improve output.

[0077] A selected one or more out of a plurality of approach means are used to determine required data, said approach means being selected by said merchant based on cost and merit of each approach.

[0078] The said system further includes learning means for identifying desired marketing research objective, determining marketing information required, and selecting the most appropriate research approach design, based on findings and merchant feedback.

[0079] The said existing data repositories are available online either at all times or periodically or at specified time frames.

[0080] The above system further includes means for incorporating participant preferences in responding to various research designs and/or participant profile as a basis for selection of participants including expected response time, previous history of purchases, usage of coupons, response to advertisements and research designs, click-stream information and any other participant specific information.

[0081] The above system further includes means for incorporating the value of information as a function of the best prediction model from among several prediction models.

[0082] The said system includes means for computing said payoff based on one or more repetitive game(s).

[0083] The above system further includes means for deploying said marketing research approach design over multiple participant sets each set being determined by selection techniques.

[0084] The said participant selection uses incremental learning means and/or predictive mechanism and/or learning tools to improve output.

[0085] The above system incorporates incremental learning means to improve incentivisation.

[0086] The said system includes a mechanism for using a plurality of marketing research approach designs, a different marketing research approach design being used for each participant based on ranking according to expected gains.

[0087] The said system further includes means for using a threshold on uncertainty or rate of change of uncertainty or information value or rate of change of information value for identifying said termination condition.

[0088] The research approach design consists of one or a combination of experimental designs to study the response to an advertisement, coupon or product recommendation and/or classification of participants and/or survey(s).

[0089] The said research approach design consists of a set of independent sub-sets of research approach designs.

[0090] The above system further includes means for deploying said research approach design incrementally in stages with each stage consisting of one or more independent sub-sets.

[0091] The incentive offered to the participant is contingent on the timing of response and/or quality of response.

[0092] The instant invention further provides a computer program product comprising computer readable program code stored on a computer readable storage medium embodied therein for automating the design of online marketing research activity, its deployment and the analysis and reporting of the obtained data comprising:

[0093] computer readable program code means configured for identifying the desired marketing research objectives,

[0094] computer readable program code means configured for determining the marketing information required to meet said marketing research objectives,

[0095] computer readable program code means configured for selecting the most appropriate research approach design,

[0096] computer readable program code means configured for deciding the target group and costs for effectively deploying said marketing research approach design,

[0097] computer readable program code means configured for deploying said marketing research approach design including dynamically modifying said research approach design and/or target group of participants,

[0098] computer readable program code means configured for collecting and analyzing data, and

[0099] computer readable program code means configured for reporting findings.

[0100] The said computer readable program code means configured for identifying the desired marketing research objectives is a merchant objective specification program code means for selecting from predefined objectives or their combination and/or interrelationships.

[0101] The said computer readable program code means configured for determining the marketing information is an information requirement analysis program code means for mapping each said objective to a set of data analysis techniques, identifying associated data requirements and determining which required information is not available in existing data repositories and the desired format of such missing data.

[0102] The said computer readable program code means configured for selecting the research approach design is research approach design tool program code means for choosing from a repository of research designs based on said marketing information, computed cost of deployment including cost of personalization and incentivisation, and value of marketing information to be obtained.

[0103] The said computer readable program code means configured for deploying is a program code means for adaptively adjusting the number and selection of participants and/or dynamically modifying said research design program code means to suit an individual participant such that maximum information value is obtained at minimal cost.

[0104] The said merchant objectives specification program code means is configured for selecting from predefined objectives including sales forecast, product preference test, advertising campaign effectiveness studies, new product acceptance and potential, pricing studies, study of business trends, desired feature enhancement to a product, market share analysis, studies of coupons, marketing campaign design, and market structure study to spot new opportunities.

[0105] The said information requirement analysis program code means is configured for implementing data analysis techniques including conjoint analysis, segmentation using clustering based on attribute importance of product features, and collaborative filtering or demographic segmentation.

[0106] The above computer program product includes computer readable program code means configured for implementing an objective map technique for using a supervised learning technique for analysis of participant information.

[0107] The above computer program product includes computer readable program code means configured for associating incentive, utility and gain values for each participant with each said research design program code means used.

[0108] The above computer program product further includes computer readable program code means configured for using previous participant history of purchases, responses to research designs program code means, click-stream information and any other available data.

[0109] The above computer program product further includes computer readable program code means configured for determining said incentives based on the value of information from the participant and the cost of collection of said information including the cost of said incentive.

[0110] The above computer program product further comprises computer readable program code means configured for selecting said participants based on cost of reaching the participant, cost of incentives offered to ensure participation, the probability of obtaining a response and the cost of analysing and using the information collected.

[0111] The above computer program product further comprises computer readable program code means configured for using adaptive resampling technique for predicting a new participants' class and computing the uncertainty index for each participant.

[0112] The above computer program product further comprises computer readable program code means configured for determining said incentives optimally, based upon the history of said participants responses to said research design program code means, probability of said participant responding to a given incentive, value of information expected to be received and cost of conducting the research on said participant.

[0113] The above computer program product further includes computer readable program code means configured for incremental participant selection until a specified termination condition is reached.

[0114] The above computer program product includes computer readable program code means configured for computing the value of the information expected from said participant using Bayes decision theory to compute expected incremental payoff from each participant, the total expected payoff, and the decline in uncertainty.

[0115] The said computer readable program code means configured for mapping is a mechanism that operates iteratively with merchant interaction.

[0116] The said computer readable program code means configured for mapping uses an incremental learning mechanism to improve output.

[0117] The above computer program product further incorporates program code means for selecting one or more out of a plurality of computer readable program code approach means configured for determining required data, said approach means being selected by said merchant based on cost and merit of each approach.

[0118] The above computer program product further includes computer readable program code learning means configured for identifying desired marketing research objective, determining marketing information required and selecting the most appropriate research approach design program code means, based on findings and merchant feedback.

[0119] The said existing data repositories are available online either at all times or periodically or at specified time frames.

[0120] The above computer program product further includes computer readable program code means configured for incorporating participant preferences in responding to various research designs and/or participant profile program code means as a basis for selection of participants wherein profile includes expected response time, previous history of purchases, usage of coupons, responses to advertisements and research designs, click-stream information and any other participant specific information.

[0121] The above computer program product further includes computer readable program code means configured for incorporating the value of information as a function of the best prediction model from among several production models.

[0122] The above computer program product includes computer readable program code means configured for computing said payoff based on one or more repetitive game(s).

[0123] The above computer program product further includes computer readable program code means configured for deploying said marketing research approach design program code means over multiple participant sets each set being determined by selection techniques.

[0124] The said program code means for participant selection uses incremental learning algorithms and/or predictive techniques and/or learning tools to improve output.

[0125] The said computer program product incorporates computer readable program code means configured to use incremental learning to improve incentivisation.

[0126] The plurality of marketing research approach design program code are used, a different marketing research approach design being used for each participant based on ranking according to expected gains.

[0127] The said computer readable program code means is configured for using a threshold on uncertainty or rate of change of uncertainty or information value or rate of change of information value for identifying said termination condition

[0128] The said research approach design program code means consists of one or a combination of experimental design program code means for studying the response to an advertisement, coupon or product recommendation and/or classification of customers and/or survey(s).

[0129] The said research approach design program code means consists of a set of independent sub-sets of research approach designs.

[0130] The said research approach design program code means is deployed incrementally in stages with each stage consisting of one or more independent sub-sets.

[0131] The above computer program product further includes program code means for offering said incentive to the participant contingent on the timing of response and/or quality of response.

[0132] The invention will now be described with reference to the accompanying drawings:

[0133] FIG. 1 is a flowchart giving an overview of Dynamic Online Marketing Research

[0134] FIG. 2 is a flowchart describing the operation of the Information Requirement Analysis Tool.

[0135] FIG. 3 is a flowchart describing the operation of the Research Approach Design Tool.

[0136] FIG. 1 represents an overview of Dynamic Online Marketing Research. The merchant specifies one or more objectives to be achieved by the system, using a Merchant objective specification tool (1.1). The information requirement analysis tool (1.2) takes as input the merchant objective(s), data analysis tool repository (1.3) and available secondary data (1.4), and determines the missing (primary) data to be gathered (1.5). The next step is to determine and design the approach to gather the primary data (1.6). This tool takes as input the repository of research designs (1.7). The research approach design tool outputs a set of approaches with associated incentive, utility and gain values. Merchant may be allowed to choose from this, or the system automatically chooses the approach that maximizes the total gain. Some part of the selected approach may be executed by the system in an on-line manner and the rest may be done by the merchant through off-line mechanism (1.8). The data collected is analyzed (1.9) and the findings (1.10) are presented to the merchant.

[0137] For the purpose of online marketing research, the merchant specifies a plurality of objectives to be achieved by the system, with the help of Merchant Objective Specification Tool (1.1). The objective could be sales forecast for a product, service or category of products, product preference tests, advertisement campaign effectiveness study, new product acceptance and potential, pricing studies, study of business trends in a particular market, desired feature enhancements to a product, market share analysis, studies of coupons, marketing campaign design, market structure study to spot new product opportunities, etc. The tool, as shown in FIG. 1, provides a graphical user interface to the merchant to specify the objective. The graphical user interface asks merchant to select one of the predefined objectives or a combination of these objectives with an interface to define interrelationships between each of the sub-objectives. For example, the objective of launching a new product can be broken down in sub-objectives of new product design, identification of the market segment, and finding the target customers for the new product.

[0138] The secondary data (1.4), as shown in FIG. 1, is the set of existing information from data banks, government publications, periodicals and books, third-party information resources, prior research reports, past transaction data, etc. It may comprise on-line as well as off-line data, for example, the secondary data may include one or more of the following:

[0139] Information about customer demographics, off-line sales transactions and off-line coupon usage records.

[0140] Online information on customer demographics, purchase history, coupon usage history, and click-stream.

[0141] Customer information regarding usage of products. For example, for a car, the mileage, condition of car at different points in time (say, at the time of servicing), its features, usage occasions, number of travelers in the car relative to the seating capacity.

[0142] History of customer responses to a research design including but not limited to, the presence or absence of response, quality of response and timing of response.

[0143] The data analysis tools repository (1.3) comprises of tools and techniques used for data mining, prediction, learning (supervised and unsupervised), classification, statistical analysis like multivariate regression, maximum likelihood functions, Bayesian estimators and neural network classifiers, analytical tools such as conjoint analysis, discriminant analysis, multidimensional scaling, perceptual maps and brand switching matrix.

[0144] The repository has information about input requirements, output result and the performance and/or predictive accuracy of each tool. The repository may also contain an initial map of merchant objectives and the techniques that can be used to achieve them. The mapping could be stored in the form of a database table, logical rules, decision tree or even a neural network.

[0145] FIG. 2 gives a flowchart describing the operation of Information Requirement Analysis Tool (1.2). For each customer of interest, the information requirement analysis tool finds the missing data using the selected data analysis technique. The objective(s) (2.1) specified by the merchant is broken down into a set of sub-objectives such that each sub-objective can be mapped to a set of data analysis techniques (2.4), that are present in the data analysis tools repository (2.2). The objectives can be classified along different dimensions, for example, product category, product attributes, timing of purchase, time analysis (pre-purchase or post-purchase), old or new product. Several classification methods or learning algorithms, for example decision trees, can be used for this purpose.

[0146] For example, the objective of launching a new product can be broken down into sub-objectives as shown in Table 1. For each sub-objective, a data analysis can be selected using a mapping.

TABLE 1
Example of an objective-technique map
Sub-objective	Data analysis technique
New product design	Conjoint Analysis
Identification of the market segment	Segmentation using clustering based on
	attribute importance of product features
Finding the target customers for the new	Collaborative filtering or demographic segmentation
product

[0147] The mapping is done using the objective-technique map available in the tools repository (2.2).

[0148] Next, the form and nature of data required by selected techniques to satisfy the merchant objectives is determined (2.4). This uses the input requirements, accuracy and output result information of each technique available in the tools repository. For example, conjoint analysis for new product design requires prospective customers' opinions about product features. The amount they are willing to pay for each additional attribute can be collected from a set of questions addressed to them or through the purchase transactions data. Each data analysis technique can only use specific data available in a pre-defined format. Desired level of accuracy for meeting the objective and performance/predictive accuracy of the analysis technique is used to determine the quantum of data required. Once the information required is determined, the system checks if the merchant's question can be answered with the existing set of data which includes secondary data (2.5). The existing information may not include the data set required for the selected data analysis tool, or the amount of data may be insufficient. If yes then the process is over. Then gaps in the information are determined (2.6). For example, if the merchant is planning to design a large campaign for coupons, the merchant may want to collect information about the behavior of customers for different coupon characteristics. The system decides using the objective-technique map to use a supervised learning technique such as a decision tree for analysis for customer information. The system discovers that the information is insufficient for a certain class of customers, say, high redemption customers. The merchant or the system may define a threshold on the number of redemptions in order for a customer to be classified as high coupon redeemer. The number of high redeemers may not be large or the variation in coupon redemption patterns may not be significant for a specific class of customers.

[0149] FIG. 3 gives a flowchart describing the operation of Research Approach Design Tool (1.6). The research approach design tool takes as input the information required (3.2), the research approach repository (3.1) and creates a set of one or more research designs (3.3), where each resultant design has an associated incentive, utility and gain values for each customer. For each potential research approach (3.4), using the existing set of information including customer profile, history of response to different designs and depending on the missing information, target customer selection is done, approach is personalized (3.5) and incentives are determined. Keeping in mind the merchant's objectives (3.6), the value of information is computed (3.7).

[0150] The customer selection and incentive determination (3.4), both use the value of information from the customer and the cost of collecting the information from the customer. The incentive for each customer affects the cost of information collection. Hence, an iterative loop may be required between customer selection and incentive determination before the final design. The final outcome is a set of approaches, targeted to a customer set, with associated incentive, utility, and gain values.

[0151] For a given a research design, the selection of participants (3.4) is done so that the required information can be gathered, with minimal cost.

[0152] The selection of participant customers is based on two parameters, the value of the information obtained from the customer and the cost incurred by including the customer in the approach design. The cost of the customer depends on the cost of reaching the customer C_r (cost of an advertisement inviting participation), any incentive offered to ensure participation D, the probability of the customer responding and the cost of analyzing and using information collected C_a. The various costs can be computed using secondary data and merchant's domain knowledge. The participant customer selection evaluates the potential set of participants along these two parameters and selects the optimal set.

[0153] The accuracy of the research output increases incrementally with additional information gathered from each participant. The slope of the gain chart at any specific point specifies the increase in output accuracy with the added participant's information. In an instance, the optimal customer selection can be based on incremental adaptive resampling with an information theoretic criterion. The sample size is incrementally increased till the desired output accuracy is achieved. For example, the system may be able to map a given merchant objective to a customer classification problem. One such merchant objective can be determination of coupon discount value for each customer, where different discount values can be treated as different classes.

[0154] The system needs to design a research approach which, when applied to the customer, provides the correct class to which the customer belongs. The well-known adaptive re-sampling technique can be used as a potential research approach. The design of this research approach requires training of a committee of learners on different samples of the training data T, where T represents labeled customer data. Once the learners are trained on sample T, they can be used to predict a new customer's class using a majority rule. The prediction of each learner for the new customer is polled and an uncertainty index is computed. Given a training sample T, the total uncertainty is sum of the uncertainty indices for each customer. As the training sample is increased by T, the total uncertainty declines. $\begin{array}{cc}\mathrm{Uncertainty}\mathrm{index},U_{j,T}=,\frac{1-{}_{1<=i<=L}\left(N_{\left(i,j\right)}\right)^2}{K^2}& \left(1\right)\end{array}$

[0155] Total classification uncertainty for sample T, $\begin{array}{cc}{U}_T={}_j{U}_{j,T}\mathrm{for}\mathrm{all}\mathrm{customers}jC,& \left(2\right)\end{array}$

[0156] Reduction in total classification uncertainty,

[0157] where

[0158] 1. N_(i,j)number of learners classifying the customer j as class i,

[0159] 2. Number of learners, K is more than number of classes L, L<K, and

[0160] 3. ${}_{1<=i<=L}\left(N_{\left(i,j\right)}\right)=K,\mathrm{the}\mathrm{total}\mathrm{number}\mathrm{of}\mathrm{learners}.$

[0161] the total number of learners.

[0162] The system selects customers in order of declining U_T. Thus, customer selection for this research approach design keeps adding customers to the selected set, till the termination condition is satisfied. For example, the merchant may specify a threshold on the uncertainty and/or the rate of change in uncertainty. The system may assume that decline in uncertainty relative to number of samples collected is a continuous curve and the slope of the curve at any instant provides an estimate for reduction in uncertainty for the next customer. For small value of T₁-T₂, the rate of decline of uncertainty is $\begin{array}{cc}\frac{U_T}{T}=\frac{U_{\mathrm{T1}}-U_{\mathrm{T2}}}{\mathrm{T1}-\mathrm{T2}}& \left(4\right)\end{array}$

[0163] Incentive determination (3.4) involves determining the incentive, if any, to be offered to the customer to increase the probability and authenticity of his/her response. The objective is to determine a set of incentives for the customers so as to maximize the total expected gain from the information.

[0164] The tool may use one or more of the following parameters:

[0165] 1. History of the customer responses to research design(s) including, but not limited to, the presence or absence of response, quality of response and timing of response.

[0166] 2. Probability of the customer responding to the research design for a given incentive. This may depend on the history of the customer responses to research designs.

[0167] 3. Value of the information from the customer.

[0168] 4. Cost of conducting the research on the customer: The total cost of administering the research design is the cost of reaching the customer C_r (cost of advertisement inviting participation), any incentive offered to ensure participation D, and the cost of analyzing and using information collected C_a.

[0169] For example, the optimization expression can be written as: Max G(j, D) over D for all customers j C.

[0170] Expected gain from customer j,

[0171] where R (j, D) represents the probability of response of customer j, when he/she is offered an incentive D to participate in a research design.

[0172] Once the optimal incentive for each customer is computed, gain for the customer can be computed from Equation (5) above. The total gain from research approach is the sum of the gain from individual customers.

[0173] The approach is personalized (3.5) for the selected set of target customers. Personalization of research approach refers to the modification of the research design, i.e., the experiment, the survey questionnaire or campaign parameters to suit an individual customer, such that maximum information value with minimal inputs is obtained from the customer. It uses customer's previous purchase history, previous responses to research designs, click-stream information and any other available data for the purpose. For example, if the customer has already purchased the product, the research approach would remove the question about whether he/she has made a purchase or not. Instead the research design might focus on feedback, comparison with other products etc., depending on the merchant's objective.

[0174] For example, Table 2 represents a scenario of missing information and the potential research approach. Missing information from Joe, Jane and George can be collected either from responses to advertisement(s) or by specifically asking a survey question. Since surveys can handle larger number of attributes with relative ease, a survey might be the preferred research approach for Jane, and advertisements may be preferred for Joe and George. Also, advertisements shown to Maria may have different values of the attribute of the product but excluding the attribute value for which her preference is known.

TABLE 2
Missing information of a customer for a potential research approach
Customer	Missing information	Potential research approach
Joe	Importance of 2	Response to an advertisement highlighting the two
	product attributes	attributes
Jane	Importance of 4	Survey question with attribute names and rating on a
	product attributes	10-point scale
George	Importance of 1	Response to an advertisement highlighting the attribute
	attribute
Maria	Preferred value of an	Response to an advertisements highlighting product
	attribute	with specific value of that attribute.
Jack	Product usage behavior	Survey question with different usage occasions with the
		option Other than mentioned here (please specify).
Senorita	Product purchase	Survey question depicting steps in purchase process in
	process	which respondent is asked to pick the option that
		closely resembles his/her process.
Nicholas	Inclination to buy	Response to an advertisement of a similar product or
		the new product.

[0175] The value or the utility of information (3.7) depends on the nature and the content of information, the customer from whom it is received and what action the merchant proposes to take based on the information. The merchant's objective function therefore, plays a significant role in determining the value of information. In the preferred embodiment of this invention, Bayes decision theory is used to compute the value of information. The value of information is equivalent to value of action taken with the information less the value of action taken without the information. The value of information from the customer can be computed by constructing a payoff matrix of possible merchant actions with and without the information. The payoff matrix is a game theoretic representation of interaction between the customer and the merchant.

[0176] For the merchants objective, the decision about different discount values impacts the purchase decision of the customer for a particular product. Say there are L discount classes with discount values m₁, m₂, m₃, . . . m_L, arranged in order of increasing value. Our hypothesis is that if customer j truly belongs to class i, he/she would redeem any coupon of value more than m_i and purchase the product. If the customer is offered a discount lower than m_i, he/she would not purchase the product. The system can assume without loss of generalization that coupons are arranged in increasing order of discount value, i.e., m_i<m_i1, for all values of i, 1<i<L. Given a training sample T, the system constructs a payoff matrix _i,j,T for each merchant action as shown in Table 3.

TABLE 3
Pay-off matrix for the merchant (without repetition)
		Correctly
	Incorrectly	classified class I	Incorrectly classified
	classified class I < Io	Io	class I > Io
Product sales	No sale	Sale	Sale
Coupon redeemed	No	Yes	Yes
Cost of distribution	Cd	Cd	Cd
Cost of redemption	R	R
Merchant's profit	-Cd	Margin - (Cd R)-	Margin - (Cd R)-mi
		mi

[0177] At any given set of information, the system classifies each customer j C in class i, where 1<i<L. The uncertainty index gives the accuracy of prediction. To maximize the merchant's payoff, the system computes the best course of action specific to a customer. The following set of equations represent the payoffs,

[0178] Maximize Expected payoff, $\begin{array}{cc}{}_{j,T}={}_{1<=i<=L}{P}_{i,j,T^{*}}{}_{i,j,T}& \left(6\right)\end{array}$

[0179] where $P_{i,j,T}=\frac{N_{i,j,T}}{K},$

[0180] fraction of learners classifying the customer j as class i.

[0181] Total expected payoff, $\begin{array}{cc}{}_T={}_{jj,T}\left(\max \right)& \left(7\right)\end{array}$

[0182] Incremental increase in payoff,

[0183] Based on a decision for each customer, the total expected payoff, as in Equation (7), is computed. To each customer j C, the system computes incremental payoff _T and decline in uncertainty U_T.

[0184] Target participant selection may also be based on a preference of participants who are opinion leaders or customers who have large influence on other customers. Customers, for whom the pattern of purchase behavior precedes other customers in time, are classified as opinion leaders or trendsetters. Pattern recognition in time is a well-researched field and such patterns can be easily identified.

[0185] Target participant selection can also be based on the expected response time of the participant in determining customer cost. It is quite possible that some customers may not log into the site for a long time, resulting in a long time lag in data collection.

[0186] The research approach repository (1.7) comprises a plurality of experiment designs and survey questionnaires. Associated with each approach is input requirement and information about what the approach can gather. Different research approaches such as experiments and/or surveys can be used to collect the missing information. The research approach could be one or a combination of the following forms:

[0187] An experimental design to study the response to an advertisement, coupon or product recommendation

[0188] Classification of customers

[0189] Survey

[0190] The experiment and/or a survey can in turn consist of a set of smaller experiments and/or surveys.

[0191] Each research approach can be characterized by a plurality of parameters. For example, survey queries can be classified based on a set of characteristics. We can define characteristics in terms of the type of information and form of query. Type of information refers to attitudinal information, product preference, top of the mind brand recall, most purchased brand, etc. Form of query refers to the form of questions, open ended questions, multiple choice, multiple choice with none of the above and a form for text entry, and rating and ranking scale of data.

[0192] This complete set of information about research approaches, their input and output parameters is a part of the repository.

[0193] The selected research approach may be executed in different ways. For example, in a survey, there can be two ways of giving the questions to the user. a) All the questions are given to the user in one single form. b) The questions are presented to the user one at a time. At each stage he can choose to either continue answering or quit. Here the next question may be based on the answer given for the previous question. In this case the number of questions asked to a user can be decreased and more useful information can be captured.

[0194] The research approach parameters such as the value of the participant incentive may change during the approach execution or participant may receive the incentive only if he/she answers all survey questions.

[0195] Further, the participant in an experiment and/or a survey may or may not be made aware of what incentive he/she has earned until the end of execution.