US 8,224,686 B1Grant
Methods, systems, and articles of manufacture for analyzing initiatives for a business network
Issue Date:2012-07-17
•18 Claims
•19 Drawing Sheets
Abstract
A system, method, and article of manufacture is disclosed for analyzing a business initiative for a business network including business locations separated into control group sites and test sites that have implemented the business initiative for a predetermined test period. Each of the sites have an associated set of attributes reflecting various characteristics corresponding to the respective site, such as geographical location, size of business location, number of employees, etc. In one aspect of the invention, a process is performed that collects a performance value for each of the test and control group sites reflecting a level of performance of each respective sites during the test period. The performance of the test sites is then measured relative to the performance of the control sites over the same time period. The process may segment the performance values for each test site attribute to identify those attributes that have a greater impact on the performance values of the test sites than other attributes. Further, the process configures a model for predicting the performance values of the test sites using the identified attributes and determines whether the model accurately predicts these performance values. If so, the process applies the model to the non-tested sites to predict the performance values of these sites. Based on the predicted performance values, a user may select one or more of the sites to implement the business initiative.
Metadata
Assignee
- Applied Predictive Technologies
Inventors
- Anthony Bruce
- Jim Manzi
Application Information
Application Number:US 13/185,559
Filing Date:2011-07-19
Priority Date:2004-01-30
Art Unit:3623
Classifications
IPC:
G06Q10/00G06Q30/00
Patent Drawings (19 sheets)
Description
Cross-Reference to Related Applications
[0001] This application is a continuation of U.S. patent application Ser. No. 10/767,191, entitled “METHODS, SYSTEMS, AND ARTICLES OF MANUFACTURE FOR ANALYZING INITIATIVES FOR A BUSINESS NETWORK,” filed Jan. 30, 2004, now U.S. Pat. No. 8,010,399 which is hereby incorporated by reference in its entirety.
Field of the Invention
[0002] This invention relates to business initiative analysis systems, and more particularly, to methods, systems, and articles of manufacture for performing a segmented initiative analysis for initiatives implemented at selected business locations in order to identify in which other locations to implement the initiative.
Background of the Invention
[0003] For as long as retailers have been selling products and services, they have been seeking ways to increase profits. Accordingly, many retailers create new initiatives that they believe will have a positive impact on sales. These initiatives usually cover various aspects of business operations that drive profits. For example, retailers may change product prices, move products to different locations of a sales floor, change the amount of space allocated to each product, test new products, add or reduce sales staff, introduce new concepts (e.g., in-store kiosks), remodel older stores, and test new marketing campaigns. Retailers may test these new initiatives in selected test locations (i.e., certain stores) and subsequently determine whether to introduce the initiatives to remaining business locations based on the success of the initiatives at the selected test locations. Historically, retailer management have used business instinct and/or anecdotal evidence to assess the success of the initiatives in order to make a decision whether to implement the initiatives at the rest of its business locations.
[0004] In recent years, however, some retailers have become more structured and analytical in their set up and analysis of tests. These retailers collect performance metrics, such as sales and gross profit data from the test locations and analyze the data using conventional software products, such as spreadsheet or statistical software packages. Retailers may measure the change in the performance metrics at the locations that introduced the new initiatives relative to a change in the same metrics at a control group of locations that did not implement the initiatives. In doing so, the retailers attempt to identify the impact of the initiatives on the performance metrics in order to determine whether they provide a positive return on investment. Accordingly, these retailers can make an informed decision whether to extend the concept to remaining locations in a business network.
[0005] Even the most advance retailers, however, are only able to assess the average impact of the tested initiative. In other words, a retailer that tests a new store layout in 20 stores, can only interpret the average sales impact of the new layout in those 20 stores. Further, they may lack the ability to understand how that average impact varied among different types of stores and which of the other non-tested stores in their business network would benefit from implementation of the new layout. For example, the retailer will not be able to determine whether, based on the 20-store test, the new layout works better in certain demographic or competitive environments, in certain locations (e.g., shopping centers), or in larger or smaller stores. Because of these shortcomings, a retailer may not be able to make an informed business decision regarding which non-tested stores should implement the new layout. Instead, the retailer must make a global decision that affects the entire business network. That is, either roll out the tested initiative to all stores in the network or eliminate the initiative altogether.
[0006] Often, an initiative will work well in some location types and not in others. It would therefore be preferable if the retailer could only extend the initiative to those stores most likely to benefit from the initiative instead of extending the imitative to either all stores or none at all.
Summary of the Invention
[0007] It is therefore desirable to have an integrated system that allows a retailer to focus investments on locations that are most likely to succeed through the implementation of a tested initiative. Accordingly, there is a need for a system and method that segments the results of a tested initiative to identify types of business locations that have responded well to the initiative and use the segmented results to assess the potential application of the tested initiative at other locations.
[0008] In one aspect of the invention, a method is provided for analyzing a business initiative for a business network including business locations separated into test sites that have implemented the business initiative for a predetermined test period and control sites that did not have the initiative implemented. Each of the sites have an associated set of attributes reflecting various characteristics corresponding to the respective site, such as geographical location, size of business location, number of employees, etc. The method includes collecting a performance value for each of the test sites and control group sites during a predetermined period of time when the business initiative was tested at the test sites, wherein the performance value represents a level of performance associated with the respective site. The performance value of the test sites is measured relative to the performance in the control stores over the same time period. Also, the method includes segmenting the performance values of the test sites for each attribute to identify those store attributes most strongly related to performance during the initiative and estimating the performance values for every other location if each were to implement the tested initiative based on the relevant attributes of each site. The method may also include determining a set of non-test sites to implement the tested business initiative based on the estimated performance values and displaying the determined set of non-test group sites.
[0009] In another aspect of the invention, a system is provided for analyzing a business initiate for a business network including business locations having sites including test sites that have implemented the business initiative and non-test group sites that have not implemented the business initiative. Each site is associated with a set of attributes and the system includes a client associated with the business network and a server. Based on requests from the client, the server is configured to identify a set of test sites that have higher business levels of performance than other test sites during the test period. Also, the server may configure a model that accurately predicts the performance of the non-test group sites if they were to implement the business initiative based on selected attributes associated with the set of test sites and rank the non-test group sites based on their respective predicted performance. A user operating the client may then select a set of the non-test group sites to actually implement the business initiative.
[0010] Further, a method is provided for analyzing a business initiative for a business network including business locations including test sites that have implemented the business initiative during a predetermined test period and non-test group sites that have not implemented the initiative. Each site is associated with a set of attributes. The method may include receiving, from a server, a list of the attributes ranked based on each attribute's impact on performance values associated with the test sites during the test period. The method further includes configuring a model to predict the performance value of the sites based on the ranked list of attributes and instructing the server to execute the model for the test sites. Moreover, the method may include receiving results of the executed model, whereby the results include quantitative measures of the model's ability to accurately predict the performance levels of the test sites. Based on a determination that the model accurately predicts the performance levels of the test sites, the method instructs the server to apply the model to the non-test group sites to predict the performance levels of the non-test group sites. Also, the method includes receiving a list of non-test group sites ranked based on each non-test group site's predicted performance level.
[0011] In yet another aspect of the invention, a system is provided for analyzing a business initiative for a business network including business locations having sites including test sites that have implemented the business initiative during a predetermined test period and non-test group sites that have not implemented the business initiative, whereby each of the sites is associated with a set of attributes. The system may include a memory having program code including program code for determining the impact of each of the attributes on performance values of the test sites during the test period. Further, the memory includes program code for configuring a model, based on user input associated with the attributes, that predicts performance values of the test sites in relation to actual performance values of the test sites during the test period. The memory also includes program code for executing the model for the test sites to produce results reflecting the model's ability to accurately predict the performance values of the test sites. Also, the memory includes program code for reconfiguring the model based on additional user input associated with the attributes and program code for executing the model for the non-test group sites to produce predicted performance values for the non-test group sites when implementing the business initiative. The system further includes a processor for executing the program code.
[0012] Moreover, a method is provided by analyzing a business initiative for a business network including a first set of test business locations that have implemented the business initiative for a predetermined test period and a second set of non-test business locations that have not implemented the business initiative. The method includes predicting a performance of each of the test business locations using actual performance values collected from the test business locations and attributes associated with each of the test business locations. Also, the method includes determining whether the predicted performance accurately reflects actual performance of the test business locations and predicting a performance of each of the non-test business locations based on a determination that the performance of the test business locations are accurately predicted. Based on the predicted performance of those business locations, the method selects a subset of the non-test business locations for implementing the business initiative.
[0013] In another aspect of the invention, a method is provided for analyzing a business initiative for a business network including business locations including test sites that have implemented the business initiative for a predetermined test period and non-test group sites that did not implement the business initiative. Each of the sites have an associated set of attributes reflecting various characteristics. The method includes determining a set of control group sites from the non-test group sites and collecting a performance value for each of the test and non group sites reflecting a level of performance of each respective sites. Further, the method includes separating, for each attribute, the test site performance values into fragments based on values for the attribute for each test site. Also, the method includes separating, for each attribute, the control group site performance values into fragments based on values for the attribute for each control group site. The method further includes analyzing, for each attribute, the performance value of each test site fragment with the performance value of a corresponding control group site fragment and determining a strength of the relationship between the attribute and the performance of the test sites based on the analysis. Moreover, the method includes configuring a model using the attributes and at least one of the determined strength relationships and applying the model to the test site performance values to produce first results reflecting estimates of the performance values of the test sites. Additionally, the method includes reexecuting the model for non-test group sites reflecting sites in the business network that have not implemented the initiative to produce second results reflecting estimates of performance values for the non-test group sites if these sites were to implement the tested initiative and determining which non-test group sites to actually implement the tested initiative based on the second results.
[0014] In another aspect of the invention, a method is provided for predicting the performance of a target business location when implementing a tested business initiative. The method includes receiving feedback associating an implementation of the business initiative at a test business location. Further, the method includes predicting a likelihood of the business initiative having a positive impact on the target business location based on a model that produces quantitative performance data associated with the business initiative and is constructed using one or more attributes associated with the test business location. Also, the method includes determining whether the target business location should implement the business initiative based on the predicted likelihood, whereby the target business location is included in a set of business locations that may be selected to implement the business initiative.
[0015] Additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of methods, systems, and articles of manufacture consistent with features of the present invention. The aspects of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Brief Description of the Drawings
[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate several aspects of the invention and together with the description, serve to explain the principles of the invention. In the drawings,
[0017] FIG. 1 illustrates an exemplary system environment in which methods, systems, and articles of manufacture, consistent with features and principles of the present invention may be implemented;
[0018] FIG. 2 illustrates a flowchart of an exemplary initiative segmentation process, consistent with certain aspects of the present invention;
[0019] FIGS. 3A and 3B illustrate flowcharts of an exemplary initiative analysis process, consistent with certain aspects of the present invention;
[0020] FIG. 4 shows a screen shot of an exemplary test site selection web page, consistent with certain aspects of the present invention;
[0021] FIG. 5 shows a screen shot of an exemplary analysis set up web page, consistent with certain aspects of the present invention;
[0022] FIG. 6 shows a screen shot of an exemplary initial analysis results web page, consistent with certain aspects of the present invention;
[0023] FIG. 7 illustrates a flowchart of an exemplary segmentation process, consistent with certain aspects of the present invention;
[0024] FIG. 8 shows an exemplary test site attribute table, consistent with certain aspects of the present invention;
[0025] FIG. 9 shows an exemplary unsegmented attribute quartile result graph, consistent with certain aspects of the present invention;
[0026] FIG. 10 shows a block diagram of a segmented attribute quartile comparison process, consistent with certain aspects of the present invention;
[0027] FIG. 11 shows a screen shot of an exemplary attribute table web page, consistent with certain aspects of the present invention;
[0028] FIG. 12 shows a screen shot of an exemplary attribute details web page, consistent with certain aspects of the present invention;
[0029] FIG. 13 shows a screen shot of an exemplary model selection web page, consistent with certain aspects of the present invention;
[0030] FIG. 14 shows a screen shot of an exemplary model results web page, consistent with certain aspects of the present invention;
[0031] FIG. 15 shows a screen shot of an exemplary model results graph web page, consistent with certain aspects of the present invention;
[0032] FIG. 16 shows an exemplary business location analysis table, consistent with certain aspects of the present invention; and
[0033] FIG. 17 shows an exemplary business region analysis table, consistent with certain aspects of the present invention.
Detailed Description
[0034] Methods, systems and articles of manufacture consistent with certain aspects of the present invention identify business locations that respond most positively to a tested initiative and build a model based on the common attributes of those identified test sites to predict the impact of the tested initiative on other locations in a business network.
[0035] In one aspect of the invention, a server is disclosed for performing one or more processes for analyzing a business initiative for a business network including business locations having a set of test sites that have implemented the business initiative for a predetermined test period and a set of control group sites that did not have the initiative implemented. Each of the sites have an associated set of attributes reflecting various characteristics corresponding to the respective site, such as geographical location, size of business location, number of employees, etc. In one aspect of the invention, the server collects from a client a performance value for each of the test and control group sites reflecting a level of performance of each site during the test period. The server then calculates the change in performance in the test sites during the test period, relative to the change in performance in the control group sites over the same time period. The server may segment the performance values for each test site attribute to identify those attributes that have a greater impact on the performance values of the test sites than other attributes. Further, the system configures a model for predicting the performance values of the test sites using the identified attributes and determines whether the model accurately predicts these performance values. If so, the server applies the model to all non-tested sites (i.e., all sites in business network 110 excluding the test sites) to predict the performance values of each site. Based on the predicted performance values, a user operating the client or associated with the business network may select one or more of those sites to implement the business initiative.
[0036] Reference will now be made in detail to the exemplary aspects of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.
[0037] The above-noted features and other aspects and principles of the present invention may be implemented in various environments. Such environments and related applications may be specially constructed for performing the various processes and operations of the invention or they may include a general purpose computer or computing platform selectively activated or reconfigured by program code to provide the necessary functionality. The processes disclosed herein are not inherently related to any particular computer or other apparatus, and may be implemented by a suitable combination of hardware, software, and/or firmware. For example, various general purpose machines may be used with programs written in accordance with teachings of the invention, or it may be more convenient to construct a specialized apparatus or system to perform the required methods and techniques.
[0038] The present invention also relates to computer readable media that include program instructions or program code for performing various computer-implemented operations based on the methods and processes of the invention. The program instructions may be those specially designed and constructed for the purposes of the invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of program instructions include for example machine code, such as produced by a compiler, and files containing a high level code that can be executed by the computer using an interpreter.
I. System Overview
[0039] FIG. 1 illustrates an exemplary system environment 100 in which certain aspects related to the present invention may be implemented. As shown, system environment 100 may include a business network 110, network 120, and a server 130.
[0040] Business network 110 represents a network of businesses that provide similar products and/or services to customers. The businesses may be geographically dispersed. Business network 110 may include one or more business regions 112-1 to 112-N that represent different geographical locations, such as cities, counties, states, country regions, virtual boundaries having a predetermined bounded distance (e.g., 100 mile radius), and any other type of defined region within a particular location. Each region 112-1 to 112-N may include one or more business locations 114 that represent retailers that sell products and/or services associated with business network 110. For example, business network 110 may represent a grocery store chain that includes a plurality of stores geographically dispersed among a state of the United States. Each region 112-1 to 112-N may represent locations that include grocery stores in the chain (e.g., business locations 114) that are within a predetermined distance of one another (e.g., 100 mile radius).
[0041] Also included in business network 110 is business management unit 116, which represents a business entity that manages business network 110. For example, business management unit 116 may be an organization headquarters or a marketing division for business network 110. Unit 116 may include one or more clients 118 that are operated by one or more users associated with business network 110 that perform various management and/or marketing duties for business network 110. Following the above example, business management unit 110 may be a marketing division of the exemplary grocery store chain that determines which products and/or services each retailer location (e.g., 114) should provide.
[0042] In one aspect of the invention, client 118 may be one or more computer systems such as a desktop computer, workstation, laptop, personal digital assistant or any other similar computing system operated by a user. Client 118 may include a processor, associated memory, and other elements and functionalities available in computer systems (all not shown). These elements may include input/output devices, such as a keyboard, mouse, and display, although these input means may take other forms. Also, client 118 may include a network interface for communicating with external resources, such as network 120 and server 130. Further, client 118 may execute browser software that enables a user to request, receive, and render information from network 120.
[0043] Although FIG. 1 shows business network 110 including a single business management unit 116 (and accompanying client 118), each business location 114 may include a client 118 that is capable of accessing network 120 and server 130 to access and perform processes consistent with certain aspects related to the present invention. Further, although business network 110 is shown including a plurality of regions 112-1 to 112-N, network 110 may include business locations 114 that are all included within a single region.
[0044] Network 120 may be a shared, public, or private network and encompasses a wide area or local area. Network 120 may be implemented through any suitable combination of wired and/or wireless communication networks. For example, network 120 may be implemented through a Wide Area Network (“WAN”), Local Area Network (“LAN”), an intranet and/or the Internet. Further, network 120 may represent multiple networks, such as a wireless carrier network connected to the Internet.
[0045] Server 130 may be a computer system such as a desktop computer, workstation, or any other similar server side computing system that performs one or more server-side processes. In one aspect of the invention, server 130 may include, among other things, interface unit 132, processing unit, 134, memory 136, and database 140. Although server 130 is shown including elements 132-140, it may include additional computing elements, such as an input/output device (e.g., display, mouse, keyboard) that enables a user to access the data and software in server 130. In addition, server 130 may be a group of servers.
[0046] Interface unit 132 may be one or more hardware and/or software components that are configured to receive and send information from/to network 120. Interface unit 132 may include, or leverage, different types of software to control access to information maintained by server 130. For example, interface unit 132 may execute firewall software that controls access to information and processes provided by server 130 using one or more security protocols (e.g., password processing, encryption technologies, etc. The above listed functionalities are not intended to be limiting and interface unit 132 may perform other types of interface processes to facilitate the exchange of information between server 130 and network 120.
[0047] Processing unit 134 may be one or more processing devices that execute software processes stored in one or more memory devices, such as memory 136. Further, processing unit 134 may execute software processes stored internally, such as internal flash memory configured within processing unit 134.
[0048] Memory 136 may be one or more memory devices that store data and software programs that, when executed by processing unit 134, perform processes consistent with certain aspects related to the present invention. These programs may include segmentation engine 137, and model processing engine 138. Further, memory 136 may include server-side software applications that enable server 130 to create and provide content including information produced by other programs executed by processing unit 134, such as web server software. Alternatively, server 130 may leverage web server software that is located in another memory device (not shown) internal or external to server 130. The above software programs and functionalities are not intended to be limiting and memory 136 may be configured to store additional or fewer types of data and/or software programs that perform other processes consistent with server-side computing systems.
[0049] Database 140 may be one or more memory devices that store data and/or executable software that is used by server 130 to perform processes consistent with certain aspects related to the present invention. For example, database 140 may include one or more data structures (e.g., tables, maps, arrays, directories, etc.) that include data that is used by one or more of the software programs stored in memory 136 and executed by processing unit 134. Although server 130 shows database 140 and memory 136 as separate elements, the information and software stored in these elements may be combined in a single memory device. Also, memory 136 and/or database 140 may be located external to server 130 and accessible through network 120 or a dedicated back-end communication path, such as a Local Area Network (LAN).
Ii. Initiative Analysis Process
[0050] Methods, systems, and articles of manufacturer consistent with aspects of the present invention enable one or more users of client 118 to access functionalities offered by server 130 to determine retail strategies associated with business network 110. For example, a user in business management unit 116 may execute browser software in client 118 to request analysis information regarding one or more initiatives implemented at one or more business locations 114. FIG. 2 shows an exemplary initiative segmentation process that may be performed by methods, systems, and/or articles of manufacture consistent with aspects of the present invention.
[0051] Initially, a user or users at business management unit 116 may create and select an initiative that is to be applied to a predetermined number of test sites reflecting a set of business locations 114 (e.g., thirty stores) included in business network 110 (Step 210). The proposed initiative may be a change in store layouts, offering different products and/or services, different marketing campaigns (e.g., advertising, etc.), and any other type of idea that may be implemented at the selected test sites.
[0052] The user of business management unit 116 may then direct the selected test sites to implement the proposed initiative (Step 220). The initiative may be implemented for a predetermined amount of time (e.g., three months, six months, etc.), which is also known as a test period. Alternatively, the initiatives may be implemented at the test sites indefinitely. During the test period, business management unit 116 may collect feedback results from each test site based on one or more performance metrics selected by unit 116 (Step 230). For example, business management unit 116 may request sales information, foot traffic (i.e., number of persons entering a business location), etc. from each test site during the test period and may request additional performance data for each test site for a period before the test period.
[0053] Once the appropriate feedback results are collected, a user at business management unit 116 may use client 118 to perform an initiative analysis process based on the results (Step 240). The initiative analysis process provides information regarding the effectiveness of the initiative at the test sites. Further, the initiative process provides information regarding a proposed effectiveness of the initiative if implemented in other non-tested business locations in business network 110 (i.e., retailer locations that did not implement the initiative). These other business locations may be referred to as non-test group sites. In one aspect of the invention, the initiative analysis process includes the exchange of information between client 118 (under the operation of a user) and server 130. For example, a user operating client 118 may use browser software to access a web site that is provided or leveraged by server 130 to perform the initiative analysis process. Accordingly, server 103 may provide content to client 118 that includes queries, data, and analysis information associated with the initiative analysis process.
[0054] A user, such as a manager at business management unit 116, or software processes may analyze the information provided by server 130 and selects one or more applicable non-test group sites to implement the initiative (Step 250). Accordingly, business management unit 116 determines and implements one or more strategies associated with the tested initiative in a selected number of business locations 114, and/or groups of locations, such as regions 112-1 to 112-N.
[0055] In certain aspects, client 118 may download information to server 130 associated with the results of the test initiatives at the test sites. Further, server 130 may receive, or retrieve, attributes associated with each business location 114 from client 118, such as store size, locations, sale data, types of products and/or service offered, location and distances of competing merchant sites, age of the business location, amount of a lease (if any), weather during the test period, staff information, etc. Alternatively, or in addition to the information provided by client 118, server 130 may execute software that determines, collects, and/or retrieves attribute information associated with business network 110, regions 112-1 to 112-N (e.g., population and demographic data, etc.), and business locations 114. The attribute information may be stored in database 140 for subsequent use by server 130 and/or client 118.
[0056] FIGS. 3A and 3B show a flowchart of an exemplary initiative analysis process, consistent with certain aspects of the present invention. As explained above in connection with FIG. 2 , a user operating client 118 may access server 130 through browser software. In one aspect of the invention, server 130 may leverage web server software that creates and renders web page content that includes information associated with the initiative analysis process. For example, the initiative analysis process may begin with a user of client 118 receiving a web page including a list of target sites that implemented the initiative and provided feedback results to client 118. FIG. 4 shows an exemplary screen shot of a test site selection web page including a list of test sites 410, the location of each test site 420, the type of each test site (not shown), and a query section for proposed dates the user may select to perform an analysis 440. From this information, the user may select the test sites from which server 130 is perform an initiative analysis (Step 305).
[0057] Once the test sites are selected, server 130 may provide client 118 with information that enable the user to set up the initiative analysis (Step 310). FIG. 5 shows an exemplary screen shot of an analysis setup page that allows the user to define one or more items associated with the initiative analysis process. For example, in setting up the analysis, the user may select one or more performance metrics 510 from which the user wishes to use in its analysis. A performance metric may include sales data in dollars (change in average sales during the predetermined test time period), gross profit margins, number of products sold, and any other type of metric associated with the performance of the business locations 114. Further, server 130 may allow the user to select a product/service category 520 for the analysis, such as a type of product, all products, etc.
[0058] In addition to performance metrics and categories, the user may select time periods 540 from which the initiatives were implemented by the test sites. Also, the user may request that server 130 eliminates any initiative feedback data that appear to be anomalous by removing any outlier sites (selection 550). An outlier site has performance values that are too different than the rest of its respective test or control group. The user determines a threshold performance value beyond which a site is considered to be too different than its peers and thus deemed an outlier by server 130. By removing outlier test sites from consideration, the initiative analysis process does not consider the information for this test site during its analysis operations.
[0059] Additionally, or alternatively, the user may select a control group 560 of business locations in the network 110 that have not implemented the initiative. For example, the user may select to have all or a portion of the remaining business locations considered in the analysis process, or request that the server select a group of sites that are most similar to the test group of sites in which the initiative was implemented. This group of stores selected by the user or server 130 is the control group.
[0060] Once the user has set the appropriate performance metrics, categories, etc. for the initiative analysis process, server 130 may perform an initial analysis on the test site initiative result data (Step 315). In one aspect, server 130 may determine a performance value for each test site as well as the control group sites for each performance metric and category selected by the user. The performance value of a test site may be measured as a change in a performance metric/category combination for each test site relative to the change in the control group sites over a common time period. Once server 130 determines the performance values for the test sites, it may provide the results of the initial analysis to user via client 118. FIG. 6 shows an exemplary screen shot of an initial analysis results web page including data reflecting the results of the initial analysis process.
[0061] As shown in FIG. 6 , server 130 may provide information associated with the performance (e.g., performance values) of the test sites relative to the control group sites for the selected performance metric. In the exemplary screen shot of FIG. 6 , server 130 provides the user with performance metric value changes in absolute values (e.g., actual dollars in sales) (e.g., graph 610), and in percentage values (e.g., graph 620). Further, server 130 may determine the average change in the selected performance metric for the test sites relative to the change in the same performance metric for the control group sites. This may be displayed in graph form as well. Additionally, server 130 may determine the likelihood that the performance of the test group relative to the control group of sites is not random chance. Server 130 may provide such confidence values to the user as a 1-p value (e.g., graph 640). The confidence value may be determined based on information reflecting a difference between test site performance metric data and control group site performance metric data.
[0062] Once server 130 completes the initial analysis process and provides the results of this process to the user, it may execute a segmentation process consistent with certain aspects related to the present invention (Step 320). The segmentation process may direct processor unit 134 to access the attributes associated with business locations 114 stored in database 140. Once accessed, the segmentation process sorts through the attributes to identify those that most strongly relate to the different test site's performance values during implementation of the initiative. In one aspect, server 130 may determine the strength of the relationship between each attribute and differential performance by test site (e.g., the attributes that are most in common among the top performing test sites and most different than the worst performing test sites).
Iii. Segmentation Process
[0063] FIG. 7 shows a flowchart of an exemplary segmentation process consistent with certain aspects of the present invention. To segment the test site results, server 130 may execute segmentation engine 137 that initially sorts the test sites based on their corresponding performance metric results (Step 705). For example, if the selected performance metric is an average percentage of sales difference over a predetermined test period, then segmentation engine 137 may sort the test sites from highest to lowest average sales difference. FIG. 8 shows an exemplary test site attribute table including a sorted list of selected test sites 801 based on their corresponding performance metric performance values 802. Each test site is associated with one or more attributes 803 that correspond to some characteristic related to the test sites. Accordingly, segmentation engine 137 may determine a measurement of the strength of the relationship between each attribute 803 and the performance metric values 802 of test sites 801 (Step 710). In one aspect, segmentation engine 137 will use statistical processes to determine the strength of the relationship between the attributes 803 and performance metrics 802. For example, segmentation engine 137 may analyze the correlation and R-squared of each attribute to the performance of the test group sites. In such an example, the server 130 would create a scatter plot graph for each attribute. Data points in the scatter plot are associated with individual test sites. Server 130 plots the site's value for a selected attribute 803 on the x-axis and plots the value for the selected performance metric 802 on the y-axis. Server 130 then determines and places a regression line through the scatter plot to calculate the correlation and r-squared values of that regression line. These values (i.e., correlation and r-squared values) indicate the strength of the relationship between the selected attribute 803 and the performance metric 802. Server 130 repeats this process for every attribute. In one aspect, server 130 determines positive and/or negative correlations. A positive correlation means that as the value of an attribute increases, the value of the selected performance metric also increases. A negative correlation means that as the attribute value increases, the value of the selected performance metric decreases.
[0064] Once determined, segmentation engine 137 may sort the attributes based on the strength relationship values (Step 715). The sorting allows segmentation engine 137 to have a prioritized list of attributes that are ranked based on their relationship with the performance of the test sites. Thus, referring to FIG. 8 , if segmentation process 137 determines that attribute 1 has the strongest relationship with the average sales difference of test sites 801 when compared to other attributes, that attribute may be ranked first in the list.
[0065] In certain aspects, segmentation engine 137 may perform an unsegmented or segmented process on the attributes and test sites. Unsegmented processing involves analyzing the performance of each test site against the performance of all control group sites. Segmented processing involves analyzing the performance of each test site against the performance of control group test sites having similar attribute relationships. Steps 725-750 and 760-785 demonstrate the differences between the two processes.
[0066] If segmentation engine 137 is directed or configured to perform unsegmented processing (Step 720; UNSEGMENTED), an initial attribute may be selected for processing (Step 725). Segmentation engine 137 may analyze the performance metric values associated with the selected attribute and splits the test sites into fragments that represent sets of test sites that are divided based on the statistical rank of their performance metric values (Step 730). In one aspect of the invention, segmentation engine 137 separates the test sites into quartiles, quintiles, or any number of fragments. Segmentation engine 137 may then analyze the performance metric values of each test site attribute fragment against the performance of all control group test sites (Step 735). Based on the analysis, segmentation engine 137 may generate unsegmented attribute fragment results, which may include a resultant map of fragments representing the difference between the performance metrics of each test site fragment and all of the control group sites (Step 740). This information may be stored in memory 136 or database 138.
[0067] In one aspect of the invention, segmentation engine 137 may also determine a value representing a relationship between the performance of the test sites (i.e., performance metric) and the selected attribute. For example, segmentation engine 137 may determine the 1-p value of the relationship between the performance of the test sites (i.e., performance metric) and the selected attribute. The 1-p value represents the probability that the relationship between the attribute and the performance metric (either positive or negative) is not random chance.
[0068] FIG. 9 shows a graph 900 reflecting a relationship between the performance of the test sites (i.e., performance metric) and a selected attribute. As shown, graph 900 includes a number of fragments 901-905 reflecting the unsegmented attribute fragment results computed by segmentation engine 137. For this exemplary attribute, line 910 shows a positive correlation between the performance metric values and the attribute values.
[0069] If there are more attributes 803 to process (Step 745; YES), then segmentation engine 137 may select another attribute (Step 750) and the unsegmented process continues at Step 730. On the other hand, if there are no more attributes 803 to process (Step 745; NO), server 130 may end the segmentation process and proceed to Step 325 of the initiative analysis process of FIG. 3 (Step 790).
[0070] If segmentation engine 137 is directed or configured to perform segmented processing on the attributes (Step 720; SEGMENTED), an initial attribute 803 is selected for processing (Step 760). Segmentation engine 137 may split the test site performance metric results into fragments that represent sets of test sites that are divided based on the statistical rank of their values for the selected attribute (Step 765). In addition, segmentation engine 137 may split the control group sites into fragments based on their values for the same selected attribute (Step 770). Segmentation engine 137 may then analyze each test site fragment against a corresponding control group fragment (Step 775). The results of the analysis are processed into segmented attribute fragment results that reflect the differences in the performance metric values for the test sites and corresponding control group sites for the selected attribute (Step 777). To better understand this segmented analysis, FIG. 10 shows a block diagram of an exemplary segmented attribute fragment comparison process consistent with certain aspects related to the present invention.
[0071] As shown, memory 137 may store a set of control group fragment information 1010-1040 and a set of test site fragments 1050-1080 for a particular attribute under analysis. During a segmented process operation, segmentation engine 137 may compare corresponding fragments from each set to generate segmented attribute fragment information 1090-1096. For example, control group fragment 1010 may be analyzed against test site fragment 1050 to produce segmented attribute fragment 1090, while control group fragment 1020 is analyzed against test fragment 1060 to produce segmented attribute fragment 1092. The segmented attribute fragment information 1090-1096 is stored in memory 137 for subsequent processing. Accordingly, at Step 775 of FIG. 7 , segmentation engine 137 generates and stores the segmented attribute fragment results in a memory device, such as memory 136 or database 140.
[0072] Also, segmentation engine 137 may determine a correlation value and a 1-p value in a manner similar to that described above in connection with Step 740 and FIG. 9 . That is, segmentation engine 137 may determine a correlation and 1-p values that represent the strength of the relationship between the performance of the test sites (i.e., performance metric) and the selected attribute. The correlation values, 1-p values, and associated information may be stored in memory 136 or database 140.
[0073] If there are more attributes 803 to process (Step 780; YES), then segmentation engine 137 may select another attribute (Step 785) and the segmented process continues at Step 765. On the other hand, if there are no more attributes 803 to process (Step 780; NO), server 130 may end the segmentation process and proceed to Step 325 of the initiative analysis process of FIG. 3 (Step 790).
[0074] In addition to the processes performed during the segmentation process, server 130 may also generate pointers, software links, etc. between each attribute and its corresponding unsegmented and/or segmented attribute fragment information. Using these links, server 130 may generate content and data that allows the user to receive, view, and analyze the results of the segmentation processes (Step 325). For example, FIG. 11 shows an exemplary screen shot of an attribute table 1100 generated by server 130 consistent with aspects of the present invention. Table 1100 may be produced by server 130 using the attribute information generated during the segmentation process performed in Step 320 of FIG. 3 . A user may request to view the attribute information by instructing server 130, via client 118, using queries, hyperlinks, and any other form of request technique through the browser software executing at client 118.
[0075] As shown, attribute table 1100 may include a list of all of the attributes 1110 associated with the test sites that have implemented the proposed initiative. These attributes may correspond to attributes 803 described above in connection with FIG. 8 . Further, table 1100 may include a strength value 1120 representing a strength of the relationship between each attribute and the performance metric value on the test sites. As shown, attributes 1110 are ranked based on their corresponding strength values (e.g., R-squared values). Additionally, table 1100 may include correlation values 1130, and 1-p values for each attribute. These values may reflect the correlation values, 1-p values and information determined in Steps 730 and/or 775 of FIG. 7 . As shown in FIG. 11 , server 130 may determine and present correlation values for the test sites and control group sites.
[0076] In another aspect of the invention, the attributes 1110 listed in table 1100 may be active in that the user may select an attribute (e.g., click) using an input device to view additional information regarding the attribute selected. FIG. 12 shows a screen shot of an exemplary attribute details page associated with segmented attribute quartile results for a selected attribute 1210. In one aspect, the details page may include information representing the fragments generated by segmentation engine 137. For example, the details page may include a graph having segmented fragments 1220 for attribute 1210. On the X-axis of the graph may be the attribute information 1240 and the Y-axis is the performance metric values 1230, determined in this example in segmented form. Further, the details page may include a chart 1250 having information associated with the segmented attribute fragment data reflected in the graph. This information may include data reflecting the number of test sites 1260 and control group sites 1270 in each fragment, the computed performance metric data 1280, and a 1-p value 1290 reflecting a measure of likelihood that the relationship between the attribute and the performance metric values is not random. Segmentation engine 137 may calculate this value using statistical algorithms and/or processes based on the information provided by the segmented attribute fragment results.
[0077] Additionally, or alternatively, server 130 may generate and render a scatter plot graph in which each test site is a data point in the graph, the X-axis is the value for the selected attribute and the Y-axis is the performance metric values. Server 130 may be configured to allow the user to select more than one attribute from attribute table 1100. In response to this selection, server 130 may process and generate attribute graphs (e.g., fragment-based, scatter plot-based, etc.) including information showing the relationship between the multiple attributes and their corresponding performance metric values for the test sites.
[0078] Referring back to FIG. 3A , once the user has analyzed the segmentation process output data (Step 325), server 130 allows the user to select a model type and provide associated parameters for the model to be used for analyzing the initiative performed in the test sites (Step 330). Server 130 is configured to execute the selected model using information related to the test sites to identify the attributes that are deemed most relevant to the success of certain test sites that implemented the initiative. Server 130 uses the test site information to train the selected model. Once the user, or a software program, determines that the model is predicting accurate performance metric data based on the actual initiative feedback data provided by the test sites, server 130 may apply the model to the control group sites.
[0079] In one aspect of the invention, server 130 may offer different types of modeling methodologies that are used to predict performance values of the test and/or control group sites, such as linear regression, staged linear regression, neural network basic train, neural network stepwise regression, decision tree, K-means similar site modeling, and any other type of software model that may be used to analyze the information produced and/or collected by segmentation engine 137 and server 130 during the initiative analysis process. Server 130 may generate a web page that includes a list of modeling methodologies from which the user may select. Also, server 130 may allow the user to program a custom modeling methodology that may be used to analyze the initiative. Once the model is selected, server 130 may query the user for parameters and other type of information used by the selected model. FIG. 13 shows a screen shot of an exemplary model selection web page 1300 that includes menus for allowing the user to select a model and provide one or more parameters for the selected model.
[0080] As shown, web page 1300 allows the user to select the type of performance metric 1310 to run in the selected model 1320. Further, server 130 may allow the user to select which attributes 1330 are available for consideration by the selected model when executed. As shown in FIG. 13 , the exemplary web page 1300 may allow the user to select/deselect an attribute 1330 using corresponding selection boxes 1340 and 1350 to indicate that the particular attribute is to be used by the selected model. Also, the user may provide parameter information, such as selecting the maximum number of attributes 1380 the model should use when completing its analysis of the test sites and the implemented initiative. For example, the user may instruct server 130 to consider only a certain number of the attributes with the strongest relationship to performance (e.g., the ten most important attributes) from among the entire list of attributes 1330. The user may force the model to include a particular attribute by selecting parameter box 1350 for that particular attribute. In this instance, server 130 always considers and uses the attributes selected via parameter 1350 when predicting performance values of the test sites.
[0081] Further, the user may also direct server 130 to consider cross correlations 1360 between attributes when executing the selected model. Cross correlation is a parameter that reflects the similarity between attributes and their affect on the selected model's results. For example, two or more of the selected attributes 1330 may be closely related, such as an attribute for the number of competitive business locations within a five mile radius and another attribute for the number of competitive business locations within a ten mile radius. These two exemplary attributes may affect the model in a similar fashion. The user may not want such similar attributes considered in the selected model. Accordingly, by setting the cross correlation parameter 1360, the user directs server 130 to remove one of the similar attributes (leaving one attribute) from consideration by the selected model. This enables server 130 to consider independent attributes when executing the selected model. Also, the user may select the number of iterations 1370 the model should run when training itself for producing its results.
[0082] Once the model and associated parameters are selected, server 130 may execute the selected model (Step 335). In one aspect, server 130 may leverage model processing engine 138 to process the model. In such an instance, server 130 provides the user's selections and parameters to engine 138 for subsequent processing. Further, model processing engine 138 may re-execute the model based on the number of iterations 1370 selected by user in Step 330. In this regard, engine 138 tests and trains the model to calibrate its results. For example, model processing engine 138 may build a model using a subset of the test sites (e.g., 20 out of 30 test sites) and tests the results of the model on the remaining test sites (e.g., the remaining 10 test sites). The model may compute predicted performance metrics for each of the remaining test sites based on the attributes selected by the model and compares them to the actual performance metrics of the remaining test sites. Server 130 may repeat this process for the number of iterations set by the user in Step 330.
[0083] Once the model is executed, server 130 may generate and render the results to the user, via client 110. The user may then analyze the results of the executed model (Step 340). In one aspect, server 130 may generate a model results web page that includes information pertaining to the results of the executed model. FIG. 14 shows a screen shot of an exemplary model results web page 1400 consistent with certain aspects of the present invention. As shown, model results web page 1400 includes details of the attributes selected by the model. For example, web page 1400 may include a list of attributes 1410 used by the model. For each of these attributes, and based on the type of model selected by the user, server 130 may also provide details regarding the variables, constants, and other data used by the selected model. For example, results web page 1400 shows the results for a selected linear regression model. In this example, server 130 may provide the values for a linear least squares fitting algorithm used to find the best fitting line through a set of points. This algorithm may include a constant and coefficient that are assigned to each attribute. Accordingly, server 130 may provide the co-efficient values 1420 assigned to each attribute used by the model. Further, the constant 1430 used by the model for this type of model algorithm may be included in results web page 1400. Additionally, server 130 may provide information showing the average impact 1440 of each attribute on the model results.
[0084] Moreover, server 130 may provide information showing how accurate the model predicts performance values of the test sites based on the selected parameters and the initiative. In one aspect, server 130 may provide quantitative measures reflecting the fitness of the model for predicting and analyzing the performance values of the test sites. For example, server 130 may provide an R-squared value 1450 reflecting a quantitative value of the model's fitness for predicting the success of the initiative in the control group sites. Alternatively, server 130 may express the fitness value as an error rate percentage value reflecting whether the predicted performance values are within a certain range of actual performance values associated with the test sites.
[0085] Additionally, server 130 may provide confidence measurements 1460 reflecting quantitative values of how well each attribute is actually contributing to producing a model that will successfully predict the performance metrics of the control group sites when implementing the initiative. Also, server 130 may determine and provide a percent populated value 1470 for each attribute 1410 that represents the percentage of test sites that server 130 has information for that particular attribute.
[0086] In another aspect of the invention, model processing engine 138 may generate a graph showing the results of the executed model. FIG. 15 shows an exemplary model results graph 1500 page showing the results of the linear regression model explained above with respect to FIG. 14 . The Y-axis of graph 1500 represents the actual performance metric values for each of the test sites and the X-axis represents the predicted performance metric values for the same test sites. Line 1510 represents a match between predicted performance metric values and actual performance metric values. The model processing engine 138 may determine the number of test sites are within certain ranges of a predetermined benchmark value. This information may be reflected as table 1520 that includes information showing the percentage and number of test sites within various percentages of the benchmark.
[0087] The user of business management unit 116 uses information reflected in graph 1500 and/or model results web page 1400 to determine whether the selected model and/or attributes are acceptable (Step 345). Alternatively, a software program executed by server 130 may analyze the information produced by the executed model to determine whether the model and/or attributes considered by the model are acceptable based on predetermined rules. Various factors may be considered by the user, or software program, when determining whether the model and/or attributes are acceptable. For example, a user may determine that one of the attributes 1410 considered by the selected model was not an appropriate attribute based on the percent populated value 1440 for that attribute. Alternatively, or additionally, the confidence measurement 1460 for an attribute may cause the user or the software program to determine that that attribute is not acceptable for consideration by the model. On a broader scale, the user or software program may determine that the selected model is not appropriate for predicting performance metrics for the non-test sites based on the confidence measurement for that model (e.g., r-squared value 1450). The above exemplary factors are not intended to be limiting and the user and/or software program may use other result information to determine the appropriateness of the selected model and/or attributes.
[0088] If the results are not acceptable (Step 345; NO), the user and/or software program may re-select a model, attributes, and/or parameters for analyzing the initiative (e.g., return to Step 330). For example, the user may remove one or more attributes from consideration prior to re-execution of the selected model. If, however, the results are acceptable (Step 345; YES), the user and/or software program may direct server 130 to apply the model to the non-test group sites to determine a predicted impact of the initiative on these sites (Step 350). The server 130 may do this by applying attributes 1410 and their corresponding coefficients 1420 selected by the model in step 335 to each non-test site's values for those attributes 1410. In doing so, server 130 calculates the predicted performance of the initiative on each non-test site based on that site's relevant attributes 1410.
[0089] Once server 130 applies the tested model to the non-test group sites and the test sites, a determination may be made as to those sites having the highest or more acceptable predicted performance metric values. Server 130 may rank the sites accordingly and provide this information to the user via client 118. FIG. 16 shows a business location analysis table that includes exemplary results from applying the model to the control group sites. As shown, FIG. 16 shows a list of business locations (e.g., control group and test sites) 1610 that are ranked based on each site's predicted performance metric values 1620. In other words, using the sales lift performance metric as an example, server 130 may place those sites that have an estimated larger sales lift when implementing the initiative higher in the list of sites 1610 than those who have lower estimated sale values. Further, server 130 may provide information showing the actual performance metric values 1630 for those test sites that implemented the initiative and provided actual feedback information to server 130. Based on the exemplary results depicted in FIG. 16 , the user may select one or more of the non-test sites to implement the initiative. Accordingly, the user analyzes the information provided by model processing engine 138 and server 130 to make an informed decision on which sites to implement the initiative tested at the test sites. Along the same lines, the user may also select those sites that should not implement the proposed initiative.
[0090] In another aspect of the invention, server 130 may provide results of the processed model based on any grouping of the sites. One such example of a group of sites are regions 1121 to 112-N explained above in connection with FIG. 1 . FIG. 17 shows a table that includes exemplary results from applying the model to the non-test group sites in a region format For example, FIG. 17 shows an exemplary table listing the regions 1710 having those business locations (e.g., control group sites and/or test sites) 1720 and their associated predicted performance metric values 1730 and actual performance metric values 1740 (if applicable). As with the table depicted in FIG. 16 , server 130 may rank the regions 1710 accordingly to their predicted performance metric values. Thus, those regions having business locations with corresponding high estimated performance metric values are ranked above those regions with lower or less attractive performance metric values. Accordingly, instead of selecting individual business locations for implementing the initiatives, the user may determine which regions to apply the proposed initiative based on the information provided by server 130 in this format.
Iv. Conclusion
[0091] Variations of the methods and systems consistent with features of the present invention previously described may be implemented without departing from the scope of the invention. For example, server 130 may be configured with software that automatically performs some or all of the analysis and decisions performed by a user operating client 118. For instance, server 130 may include software stored in memory 136 that, when executed by processing unit 134, analyzes the list of attributes ranked according to their impact on the performance metric values of the test sites (e.g., correlation and/or r-squared values). Server 130 may also execute software that selects the model and associated parameters (e.g., attributes, iterations, etc.) for predicting the performance levels of the test and non-test group sites. Additional, server 130 may execute software that selects one or more of the non-test group sites to implement the tested initiative based on the predicted performance values produced by the executed model. The above processes are not intended to be limiting and other procedures may be performed by software processes executed by server 130 and/or client 118 to supplement and/or compliment those decisions, data input, and analysis performed by a user.
[0092] Also, although the initiative analysis process is described as using web server and browser software for exchanging information with client 118 and server 120, methods and systems consistent with aspects of the invention may use any type of technology to allow a user to send and receive information from server 120. Further, although the above described aspects of the invention include communications between a client 118 and a server 120, methods and systems consistent with aspects of the invention are not limited to client-server network configurations. That is, a user associated with business network 110 may leverage a computing environment that is local (i.e., same computing system) to a system operated by the user.
[0093] Further, methods, systems, and articles of manufacture, consistent with features of the present invention may be implemented using various network models, and is not limited to a particular computer and/or network environment. Furthermore, methods, systems, and articles of manufacture, consistent with features of the present invention are not limited to the implementation of systems and processes compliant any particular type of programming language. Any number of programming languages may be utilized without departing from the scope of the present invention.
[0094] Additionally, although aspects of the present invention are described as being associated with data stored in memory and other storage mediums, one skilled in the art will appreciate that these aspects can also be stored on or read from other types of computer-readable media, such as secondary storage devices, like hard disks, floppy disks, or CD-ROM; a carrier wave from the Internet; or other forms of RAM or ROM. Accordingly, the invention is not limited to the above described aspects of the invention, but instead is defined by the appended claims in light of their full scope of equivalents.
Claims
Certificate of Correction present
The claims shown below may not include correction changes. Use the corrected PDF for authoritative text.
What is claimed is:
1. A method for analyzing a business initiative for a business network including business locations including test sites that have implemented the business initiative during a predetermined test period and non-test group sites that have not implemented the initiative, each of the sites being associated with a set of attributes, the method comprising:
generating test site fragments based on the value of each attribute and performance value for each test site during the predetermined test period for the business initiative;
generating control site fragments based on the value of each attribute and performance value for each control group site during the predetermined test period for the business initiative, where the non-test group sites include a set of control group sites;
determining the strength relationship of the between each attribute and the performance values of the test sites based on an analysis between selected test site fragments and corresponding control site fragments; and
generating, by a server, a list of the attributes ranked based on the strength of the relationship between the attribute and performance value for each control group site.
2. The method of claim 1, further comprising executing a model to the non-test group sites to predict the performance levels of the non-test group sites.
3. The method of claim 2, further comprising ranking a list of the non-test group sites based upon each non-test group site's predicted performance level.
4. The method of claim 3, further comprising selecting a subset of the non-test group sites to implement the business initiative based on the ranked list of those sites.
5. The method of claim 2, further comprising:
selecting the model from a list of models provided by the server; and
selecting one or more parameters for the selected model.
6. The method of claim 2, further comprising:
reconfiguring the model with at least one new parameter based on a determination that the model does not accurately predict the performance levels of the test sites; and
instructing the server to execute the reconfigured model for the test sites.
7. The method of claim 6, further comprising:
repeating the reconfiguring and executing the reconfigured model until a quantitative measure reflects that the model accurately predicts the performance levels of the test sites.
8. The method of claim 2, further comprising selecting a number of the ranked attributes that the model should use.
9. The method of claim 2, further comprising receiving results of the model, wherein the results include quantitative measures of the model's ability to accurately predict the performance levels of the test sites, wherein the quantitative measures includes a ranked list of selected attributes that the model considered during its execution and data values assigned to each of the selected attributes by the model.
10. The method of claim 9, wherein the data values includes a coefficient data value for a mathematical function used by the model to generate the results.
11. The method of claim 1, wherein the list of the attributes ranked based on each attribute's impact on the test site performance values is generated by the server based on comparisons between test site fragments and corresponding control group site fragments.
12. The method of claim 1, further comprising:
instructing the server to execute the model for the test sites; and
receiving results of the executed model, wherein the results include quantitative measures of the model's ability to accurately predict the performance levels of the test sites.
13. A computer program product, comprising a non-transitory computer readable media having computer program code therein, said computer program code adapted to cause a computer to perform the following steps of:
analyzing a business initiative for a business network including business locations having sites including test sites that have implemented the business initiative during a predetermined test period and non-test group sites that have not implemented the initiative, each of the sites being associated with a number of attributes, wherein the non-test group sites includes a set of control group sites;
determining, the impact of each of the attributes on performance values of the test sites during the test period by:
segmenting the test sites into fragments based on a selected attribute and the performance value for the test sites during the predetermined test period for the business initiative,
segmenting the control group sites into fragments based on the selected attribute and the performance value for the control group sites during the predetermined test period for the business initiative, and
determining the strength of the relationship between the selected attribute and the test site performance values based on a comparison of the test site fragments and corresponding control group fragments.
14. The computer program product of claim 13, wherein the computer program code further includes program code for configuring a model, based on user input associated with the attributes, that predicts performance values of the test sites in relation to actual performance values of the test sites during the test period.
15. The computer program product of claim 14, wherein the computer program code further includes program code for executing the model for the test sites to produce results reflecting the model's ability to accurately predict the performance values of the test sites.
16. The computer program product of claim 14, wherein the computer program code further includes program code for reconfiguring the model based on additional user input associated with the attributes.
17. The computer program product of claim 13, wherein the computer program code further includes program code for providing a user with a list of non-test group sites ranked according to each non-test group site's predicted performance value.
18. The computer program product of claim 13, wherein the computer program code further includes program code for executing the model for the test sites includes program code for configuring the model using a subset of the test sites and testing the model's ability to accurately predict performance values by predicting the performance values of test sites not included in the subset of test sites.
Patent Citations (66)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US4972504(A) | 1990-11-01 | Daniel et al. | Applicant |
| US5138638(A) | 1992-08-01 | Frey | Applicant |
| US5227874(A) | 1993-07-01 | Von Kohorn | Applicant |
| US5245533(A) | 1993-09-01 | Marshall | Applicant |
| US5315093(A) | 1994-05-01 | Stewart | Applicant |
| US5406477(A) | 1995-04-01 | Harhen | Applicant |
| US5613217(A) | 1997-03-01 | Hagstrom et al. | Applicant |
| US5692107(A) | 1997-11-01 | Simoudis et al. | Applicant |
| US5826252(A) | 1998-10-01 | Wolters et al. | Applicant |
| US6055524(A) | 2000-04-01 | Cheng | Applicant |
| US6078891(A) | 2000-06-01 | Riordan et al. | Applicant |
| US6321206(B1) | 2001-11-01 | Honarvar | Applicant |
| US6484158(B1) | 2002-11-01 | Johnson et al. | Applicant |
| US6553352(B2) | 2003-04-01 | Delurgio et al. | Applicant |
| US6708156(B1) | 2004-03-01 | Gonten | Applicant |
| US6801818(B2) | 2004-10-01 | Kopcha | Applicant |
| US6934748(B1) | 2005-08-01 | Louviere et al. | Examiner |
| US6966061(B1) | 2005-11-01 | Vance et al. | Applicant |
| US6970830(B1) | 2005-11-01 | Samra et al. | Applicant |
| US7072863(B1) | 2006-07-01 | Phillips et al. | Applicant |
| US7080027(B2) | 2006-07-01 | Luby et al. | Applicant |
| US7092896(B2) | 2006-08-01 | Delurgio et al. | Applicant |
| US7155398(B2) | 2006-12-01 | Thier | Applicant |
| US7213023(B2) | 2007-05-01 | Hadzikadic | Examiner |
| US7308497(B2) | 2007-12-01 | Louviere et al. | Applicant |
| US7412398(B1) | 2008-08-01 | Bailey | Applicant |
| US7451065(B2) | 2008-11-01 | Pednault et al. | Applicant |
| US7526434(B2) | 2009-04-01 | Sharp | Applicant |
| US7546246(B1) | 2009-06-01 | Stamm et al. | Applicant |
| US7548879(B2) | 2009-06-01 | Cash et al. | Applicant |
| US7599848(B2) | 2009-10-01 | Wefers et al. | Applicant |
| US7689456(B2) | 2010-03-01 | Schroeder et al. | Applicant |
| US7689459(B2) | 2010-03-01 | Capurso et al. | Applicant |
| US7729931(B1) | 2010-06-01 | Tobin | Applicant |
| US7895072(B1) | 2011-02-01 | Manzi et al. | Examiner |
| US7904327(B2) | 2011-03-01 | Phelan et al. | Applicant |
| US2001/0032105(A1) | 2001-10-01 | Frye et al. | Applicant |
| US2001/0042003(A1) | 2001-11-01 | Tanaka | Applicant |
| US2002/0069099(A1) | 2002-06-01 | Knox et al. | Applicant |
| US2002/0133250(A1) | 2002-09-01 | Kopcha | Applicant |
| US2002/0169665(A1) | 2002-11-01 | Hughes et al. | Applicant |
| US2003/0018510(A1) | 2003-01-01 | Sanches | Applicant |
| US2003/0023841(A1) | 2003-01-01 | Atherton et al. | Applicant |
| US2003/0046125(A1) | 2003-03-01 | Flores | Applicant |
| US2003/0046126(A1) | 2003-03-01 | Flores et al. | Applicant |
| US2003/0050830(A1) | 2003-03-01 | Troyer | Applicant |
| US2003/0069659(A1) | 2003-04-01 | Wada et al. | Applicant |
| US2003/0069782(A1) | 2003-04-01 | Chrisman et al. | Applicant |
| US2003/0110080(A1) | 2003-06-01 | Tsutani et al. | Applicant |
| US2003/0130883(A1) | 2003-07-01 | Schroeder et al. | Applicant |
| US2003/0176931(A1) | 2003-09-01 | Pednault et al. | Applicant |
| US2003/0177055(A1) | 2003-09-01 | Zimmerman et al. | Applicant |
| US2003/0195793(A1) | 2003-10-01 | Jain et al. | Applicant |
| US2003/0212584(A1) | 2003-11-01 | Flores | Applicant |
| US2004/0054511(A1) | 2004-03-01 | Turner et al. | Applicant |
| US2004/0143477(A1) | 2004-07-01 | Wolff | Applicant |
| US2004/0210471(A1) | 2004-10-01 | Luby et al. | Applicant |
| US2004/0243485(A1) | 2004-12-01 | Borenstein et al. | Applicant |
| US2004/0260624(A1) | 2004-12-01 | Chan et al. | Applicant |
| US2005/0039206(A1) | 2005-02-01 | Opdycke | Applicant |
| US2005/0055275(A1) | 2005-03-01 | Newman et al. | Applicant |
| US2005/0075921(A1) | 2005-04-01 | Hayes-Roth | Applicant |
| US2005/0108082(A1) | 2005-05-01 | Jenkinson | Applicant |
| US2005/0200476(A1) | 2005-09-01 | Forr et al. | Applicant |
| US2006/0105775(A1) | 2006-05-01 | Von Kaenel et al. | Applicant |
| US2006/0195370(A1) | 2006-08-01 | Howarth | Applicant |
Non-Patent Literature (21)
- “An Application of Clustering for Strategic Group Analysis,” Author: Kathryn Rudie Harrigan, Source: Strategic Management Journal, vol. 6, No. 1 (Jan.-Mar. 1985), pp. 55-73.Applicant
- “Idea Management in a Shared Drawing Tool,” Iva M. Lu and Marilyn M. Mantei, Proceedings of the Second European Conference on Computer-Supported Cooperative Work Bannon, L. Robinson, M. & Schmidt, K. (Editors), Sep. 25-27, 1991, Amsterdam, The Netherlands.Applicant
- “A process for evaluating retail store efficiency: a restricted DEA approach,” Thomas, Rhonda, R.; Barr, Richard S.; Cron, William L.; Slocum Jr., John W.; International Journal of Research in Marketing 15, (1998) pp. 487-503.Applicant
- “Marketing's Contribution to the Implementation of Business Strategy: An Empirical Analysis,” Author(s): Stanley F. Slater and Eric M. Olson Source: Strategic Management Journal, vol. 22, No. 11 (Nov. 2001), pp. 1055-1067.Applicant
- “What Have We Learned About Generic Competitive Strategy? A Meta-Analysis,” Author(s): Colin Campbell-Hunt, Source: Strategic Management Journal, vol. 21, No. 2 (Feb. 2000), pp. 127-154.Applicant
- “The New Product Development Process for Commercial Financial Services,” Scott J. Edgett, Industrial Marketing Management 25, pp. 507-515 (1966).Applicant
- “Cluster Analysis in Test Market Selection,” Author(s): Paul E. Green, Ronald E. Frank, Patrick J. Robinson, Source: Management Science, vol. 13, No. 8, Series B, Managerial (Apr. 1967), pp. B387-B400.Applicant
- “The short-term effect of store-level promotions on store choice, and the moderating role of individual variables”, Pierre Voile, Journal of Business Research 53 (2001), pp. 63-73.Applicant
- “Advertising Experiments at the Campbell Soup Company,” Eastlack Jr., Joseph O.; Rao, Ambar G., Marketing Science, vol. 8, No. 1 (Winter, 1989), pp. 57-71.Applicant
- “Selecting Profitable Hotel Sites at La Quinta Motor Inns,” Sheryl E. Kimes and James A. Fitzsimmons, Interfaces, 1990, 20(2), p. 12.Applicant
- “New Product Models: Practice, Shortcomings and Desired Improvements,” Vijay Mahajan, Jerry Wind, Journal of Product Innovation Management, 1992; 9:128-139.Applicant
- “An Empirical Analysis of Strategy Types,” Author(s): Craig Galbraith and Dan Schendel, Source: Strategic Management Journal, vol. 4, No. 2 (Apr.-Jun. 1983). pp. 153-173.Applicant
- “An Investigation into the New Product Process: Steps, Deficiencies, and Impact,” Robert G. Cooper, Elko J. Kleinschmidt, Journal of Product Innovation Management, 1986, 3:71-85.Applicant
- “Pre-Test-Market Models: Validation and Managerial Implications,” Glen L. Urban; Gerald M. Katz, Journal of Marketing Research, vol. 20, No. 3 (Aug. 1983) pp. 221-234.Applicant
- “A Longitudinal Analysis of the Impact of Service Changes on Customer Attitudes,” Ruth N. Bolton and James H. Drew, Journal of Marketing, 1991, 55(1), p. 1.Applicant
- “A Review of Sales Forecasting Models Most Commonly Applied in Retain Site Evaluation,” David Rogers, International Journal of Retail & Distribution Management, 1992, 20(r), p. 3.Applicant
- Applied Predictive Technologies Announces Launch of Adaptive Marketing Suite for Online and Brick-and Mortar Channels, PR Newswire, January, p. 1.Applicant
- Boosting Customer Response: Mercer Management Consulting and Applied Predictive Technologies Form Marketing Alliance, PR Newswire, Mar. 12, 2001, p. 1.Applicant
- “New to You,” Deborah Szynal, Marketing News, Apr. 2002, 36(7), p. 23.Applicant
- “Focus on: Business Analytics,” D. Amato-McCoy, Chain Store Age, Apr. 2010, 86(4), pp. 71-73.Applicant
- “Adding Bricks to Clicks: The Contingencies Driving Cannibalization and Complementarity in Multichannel Retailing,” Jill Avery, Thomas J. Steenburgh, John Deighton and Mary Caravella, Harvard Business School, Feb. 2009.Applicant