Method and system for targeted marketing by leveraging video-based demographic insights

[0001] This application is a continuation of U.S. Ser. No. 11/116,409, filed Apr. 28, 2005, now U.S. 7,267,277 and the complete contents thereof is hereby incorporated by reference.

[0002] 1. Field of the Invention

[0003] The present invention generally relates to the gathering and compilation of data for market research purposes and, more particularly, to the gathering of point-of-sale transaction data for market research purposes.

[0004] 2. Background Description

[0005] Retail enterprises utilize targeted marketing to optimize marketing expense versus revenue (purchase) response. For example, retail marketers may use transaction-related demographic data to construct customer-centric historical transactions that tie in past promotions, purchases, and census demographics to build response models for revenue, profit, delinquency, and other factors. Such response models may be employed to manage ongoing and future marketing campaigns. These models are most commonly employed in connection with retail distribution channels where individual customers can be fully identified, such as direct mail and Internet commerce, because the models require historical data identifying particular customers to particular transactions.

[0006] Because of the need for transaction-related data, response models are not typically employed in the most pervasive and widely used retail channel—the retail store. In a retail store environment:

• • [0007] The customer base tends to be largely unknown;
  • [0008] Shoppers typically make purchases using payment media, such as cash or credit cards, which do not provide demographic data; and
  • [0009] Outside of a few segments such as grocery retailing, retail loyalty programs do not have large-scale subscription or usage.
    Various other factors may also impede the collection of transaction-related demographic data in a retail environment. The fact that a shopper may walk through a store without making a purchase, even while dwelling in certain areas more than others, makes demographic data on overall customer traffic an ineffective substitute for demographic data that has been correlated to transaction data.

[0010] Thus, the present invention recognizes the problem that, without the capability to obtain transaction-related demographic data, it is next to impossible to use response modeling or other data analytic techniques to increase the effectiveness of targeted marketing in a retail store environment.

[0011] It is therefore an object of the present invention to provide a system and method for obtaining transaction-related demographic data in a retail store environment. This is accomplished by leveraging in-store security camera systems to obtain still images or video signal sequences from which demographic information can be extracted.

[0012] The present invention combines and aggregates point-of-sale transactional data with information extracted from still images or video sequences, applies data mining to this aggregated data, and uses insights from the data mining to improve item promotions and targeted marketing on the retail floor.

[0013] Types of data that may be obtained from point-of-sale terminals include, without limitation, the following:

• • [0014] Date and time of day
  • [0015] Item category
  • [0016] Item number (e.g., SKU or other inventory control identifier)
  • [0017] Item cost
  • [0018] Tax
  • [0019] Total amount of purchase
  • [0020] Type of payment (e.g., cash, charge, debit)
  • [0021] And so forth

[0022] Types of data that may be extracted from still images or video signal sequences include, without limitation, the following:

• • [0023] Date and time of day, or other attributes that may be used to correlate demographic data to transaction data
  • [0024] Gender
  • [0025] Ethnicity
  • [0026] Age group (e.g., child, teenager, adult, senior, etc.)
  • [0027] Socio-economic status signals (e.g., well-dressed)
  • [0028] Dwell times (i.e., amount of time spent browsing or lingering in an area, which may signal preferences for certain merchandise categories)
  • [0029] Size of party (e.g., individual or group, small or large party, number of people in party, etc.)
  • [0030] Height (tall, short, etc.)
  • [0031] And so forth

[0032] Insights available from data analytic and data mining techniques that correlate or aggregate point-of-sale transactional data with demographic data extracted from video signal sequences may apply data analytic and data mining techniques to develop clusters, associations, and predictive models. For example, area traffic patterns may be analyzed in terms of gender, ethnicity, age, socio-economic status, and other factors. Point-of-sale transaction data may be aggregated or categorized into “times of day.” In applying such techniques, the focus will typically be on obtaining correlations between time, demographics, and items sold. Examples of the types of conclusions that might be reached as a result of such an application of data analytic or data mining techniques include, without limitation, the following hypothetical examples:

• • [0033] The fastest selling products during the early evening hours may be found to be DVDs and CDs, as it may be found that shoppers during the early evening hours are predominantly teenagers.
  • [0034] It may be found that mothers with children in strollers tend to be the predominant shoppers in the Baby section between 10:00 a.m. and 11:00 a.m.
  • [0035] Male adults may be found to crowd a store's hardware section on Saturday mornings.

[0036] Insights of this nature may be used to improve product promotions and targeted marketing (including, but not limited to, store personnel focus) on the retail sales floor, linking insights may obtained from data mining to general census-based mailing lists for targeted marketing.

[0037] Thus, the present invention provides a system and method for obtaining demographic marketing data comprising a camera apparatus that may be used to capture customer image data at a point-of-sale location and to send the images to a customer image database, in which the image data may be stored and classified by one or more attributes, including at least one attribute (including, but not limited to date and time of day) to identify customer image data to relevant transactions at corresponding point-of-sale locations. The camera apparatus may or may not include a video camera, and the camera apparatus may or may not include more than one camera. Video camera systems may provide more data than still image systems and may be more likely to be found already in place in retail environments in the form of video surveillance systems.

[0038] Demographic data may be extracted from customer image data by an available means. The means for extracting demographic data may be fully automated instead of employing a human operator to input demographic data attributes based on observation of the customer image. When the camera apparatus includes more than one camera, the means for extracting demographic data from customer image data may or may not include a means to enable customer images from one camera to be correlated with customer images from another camera.

[0039] The demographic data thus extracted may or may not include at least one of the following: gender, ethnicity, age group, socioeconomic status signals, dwell time, size of party, height. Some dimensions of demographic data, including but not limited to height and dwell time (i.e., amount of time spent browsing or lingering in an area), may be more readily extracted through automated means. Some dimensions of demographic data may be used as proxies for others, including but not limited to the use of height ranges as proxies for gender.

[0040] Extracted demographic data may be sent to the customer image database as one or more attributes associated with a customer image. The demographic data may or may not be extracted from customer image data before the customer image data is sent to the customer image database from the camera apparatus.

[0041] A transaction data collection apparatus may be used to collect transaction data at a point-of-sale location and to send the transaction data to a transaction database, in which the transaction data may be stored and classified by one or more attributes including attributes to identify transaction data to particular transactions, or aggregations of transactions, at particular times on particular days during which the transaction data were collected at particular point-of-sale locations. The transaction data collection apparatus may or may not be a retail checkout scanner.

[0042] A computer may be connected to the customer image database and the transaction database. The computer may be used to process customer image data, transaction data, and attributes relating to the customer image data and transaction data. Processing performed by the computer may include correlating extracted demographic data to transaction data for items purchased and sending correlated data to a target marketing database. Processing performed by the computer may also include using correlated data to generate a market research report. The computer may or may not generate a market research report on a real time basis in response to an operator request for a report correlating specified categories of demographic data and specified categories of transaction data. Thus, the computer may generate formal marketing reports on a period basis; alternatively, the computer may respond on a real time basis to ad hoc queries from operators engaged in market research, market planning, and other marketing-related activities. The market research report may nor may not comprise a response model.

[0043] A computer implemented system for obtaining a report correlating demographic data and transaction data at a point-of-sale location may comprise: a camera or equivalent imaging device for obtaining customer image data at a point-of-sale location; a transaction data collector for collecting transaction data at said point-of-sale location; and a computer or network of computers which (a) classifies one or more images or portions of said images obtained with the camera or equivalent imaging device, using extracted information from the customer image data, into one or more categories based on one or more attributes that correspond to said extracted information, (b) classifies one or more transactions obtained with said transaction data collector into one or more categories based on one or more attributes, (c) correlates one or more classified images or portions of images with one or more classified transactions, and (d) generates a report of the correlated information from (c).

[0044] The system described in the preceding paragraph may or may not include a printer for printing said report from (d), above. Similarly, the system may or may not include a display for displaying said report from (d), above. The camera or equivalent imaging device employed by the system may or may not be a video camera or a still camera. In addition, the system may or may not employ a scanner as a transaction data collector, and such a scanner may or may not read bar codes or radio frequency identification (RFID) tags. The system may or may not classify images under (a), above, according to: one or more of gender, ethnicity, age, and socio-economic status; one or more of size and height; or dwell time. The computer employed by the system to classify images under (a), above, may or may not operate in an interactive manner or in an automated manner for classifying the images. The categories employed by the system to classify transactions under (b), above, may or may not include one or more of date, time of day, and type of good. Finally, the report generated by the system under (d), above, may or may not include a response model.

[0045] A method for obtaining a report correlating demographic data and transaction data at a point-of-sale location may comprise the steps of: obtaining customer image data at a point-of-sale location using a camera or equivalent imaging device; collecting transaction data at said point-of-sale location; classifying one or more images or portions of images obtained with the camera or equivalent imaging device, using extracted information from said customer image data, into one or more categories based on one or more attributes that correspond to said extracted information; classifying one or more transactions obtained with said transaction data collector into one or more categories based on one or more attributes; correlating one or more classified images or portions of images with one or more classified transactions; and generating a report of the correlated information.

[0046] The method described in the preceding paragraph may or may not employ a scanner in the step of collecting transaction data is obtained using a scanner. In addition, the method may or may not accomplish the step of classifying one or more images by using a computer interactively or automatically.

[0047] Given the challenges of doing effective retail floor targeted marketing, the retail industry needs such new capabilities to improve the effectiveness of retail marketing. Video data mining technology is at the forefront of enabling enhanced retail-store-based customer relations management.

[0051] Referring now to FIG. 1, there is shown a customer 100 at a point-of-sale location being surveilled by a camera apparatus 110 which captures image data of the customer and sends the image data to a customer image database 120. The image data are classified by one or more attributes to enable a customer image, or aggregation of customer images, to be identified as relating to a particular transaction, or aggregation of transactions, for which transaction data is available in the transaction database 150. A means 130 is provided for extracting demographic data from the customer images captured by the camera apparatus 110. The extraction means 130 can be a workstation that is either interactively operated by a technician to extract demographic data or a workstation automatically operable using software, hardware, and/or firmware for extracting the demographic data from customer images. A transaction data collection apparatus 140, in this case a retail checkout scanner, collects transaction data at point-of-sale location and sends the transaction data to a transaction database 150, in which the transaction data are stored and classified by one or more attributes including, without limitation, at least one attribute to identify transaction data as relating to a particular transaction, or aggregation of transactions, for which customer image data is available in the customer image database 120. A computer 160, which is connected to the customer image database 120 and the transaction database 150, processes customer image data, transaction data, and attributes relating to the customer image and transaction data. The processing includes, without limitation, correlating extracted demographic data to transaction data, sending correlated data to a target marketing database 161, and using correlated data to generate a report, such as a market research report, which in this embodiment is a formal report produced by a laser printer 162 or other graphics output device.

[0052] Referring now to FIG. 2, there is shown a step 211 of acquiring customer image data (such as may be done by use of an in-store camera apparatus as shown in FIG. 1). In step 215, demographic data is extracted from the acquired customer image data. In step 219, the data are classified by attributes (including but not limited to date and time of day) which may be used to correlate demographic data to transaction data. Also in FIG. 2, there is shown a step 251 of obtaining transaction data (such as may be done using scanner data from an in-store checkout system as shown in FIG. 1). In step 259, transaction data thus obtained are classified by attributes (including but not limited to date and time of day) which may be used to correlate transaction data to demographic data. Finally in FIG. 2, there is shown a step 291 of correlating demographic data with transaction data. In step 299, the correlated data is used to generate a report.

[0053] While the invention has been described in terms of a single preferred embodiment, those skilled in the art will recognize that the invention can be practiced with modification within the spirit and scope of the appended claims.