US 7,689,432 B2Grant
System and method for influencing recommender system & advertising based on programmed policies
Issue Date:2010-03-30
•46 Claims
•3 Drawing Sheets
Abstract
A system and method for influencing a recommender system and advertising uses a controlled, programmed policy. The recommender system uses the policy to adjust recommendations made to users for particular items, so that a policy holder receives some measure of preference in recommendations. The preference may be implemented in the form of a filter, or a presentation of items. The policy can be applied selectively, and be based on pay for placement type consideration, demographics, time, and other related parameters. Advertising for the users can be similarly adjusted in coordination with the policy.
Metadata
Assignee
- Hayley Logistics LLC
Inventor
- John N. Gross
Application Information
Application Number:US 10/863,743
Filing Date:2004-06-07
Priority Date:2003-06-06
Art Unit:3629
Classifications
IPC:
G06Q99/00
Patent Drawings (3 sheets)
Description
Related Application Data
[0001] The present application claims the benefit under 35 U.S.C. 119(e) of the priority date of Provisional Application Ser. No. 60/476,341 filed Jun. 6, 2003, which is hereby incorporated by reference.
Field of the Invention
[0002] The present invention relates to electronic commerce systems and methods of influencing behavior of automated interactive software agents. The invention has particular applicability to applications in which items are recommended to online users on websites over the Internet.
Background
[0003] Recommender systems are well-known in the art as a form of intelligent software agent, which operates to provide specific “recommendations” to users (typically consumers) of goods and/or services that are predicted to be of interest. A number of commercial implementations can be found in the prior art, including at Internet commerce sites operated by Yahoo!, Amazon, EBay, and Netflix, and by electronic program managers, such as TiVo.
[0004] In general, recommender systems operate using one or more basic methodologies, which can be classified basically as collaborative filtering (CF) based, content based (CN) or a hybrid of the two. In the case of CF, the user's interests are determined through a variety of mechanisms, including demographic profiling, explicit and implicit data gathering, ratings for items, etc., to generate a profile. The user is then compared against other users with a similar profile, to see if there are items which the user may not be aware of, but which are rated highly by others like him/her. Thus, the comparison and analysis for collaborative filtering is basically between individuals.
[0005] In the content based filtering schemes, the primary focus is similarity between items, with little regard for the types of persons who prefer the item. In other words, the recommender merely attempts to correlate pairs of items, so that, for example, if two items are highly correlated (i.e. meaning that the selection of one is often accompanied by the selection of the other) the system can inform the user of such fact. Thus, the comparison is primarily between items.
[0006] One aspect of recommender systems that is not addressed in the prior art is a mechanism for biasing or influencing recommendations based on some external parameter that is unrelated to the individual's correlation with other individuals, and unrelated to the characteristics of the item. For example, it is well-known in the art to implement so-called “pay for placement” features and polices in, among other places, Internet based search engines. In some cases, a company can “buy” a high ranking for one or more keywords, so that, for instance, when a user visits a particular website and enters a query into a search engine (say on topic A), such site provides responsive material to such user from sources in accordance with which companies have paid a premium to be associated with topic A.
[0007] Thus, Apple Computer company may reserve and pay consideration to a search engine provider to make certain that any queries involving the word “apple” will always contain (and in some cases highlight or showcase) a “hit” to web page, ad or other content from Apple. In this fashion, a company can influence or bias a search engine to produce results that favor presentation of content from one source over another, regardless of any other correlations that may exist to the underlying query.
[0008] Accordingly, while such systems exist for influencing a user's selection of content or purchases based on keyword searches, there is an unmet need for a system or method for influencing a recommender system in a similar fashion. Clearly it would be desirable if a company could influence a purchaser while he/she is reviewing content, such as by recommending content that is characterized by some quality such as the origin of the content. In this way, originators/suppliers of such content (be it goods, services, etc.) can benefit from a treatment within a recommender system environment that is similar to that afforded to the aforementioned search engine applications. Again, since the origin of a product/service is not considered by a typical recommender system, the prior art provides no mechanism for this type of functionality.
Summary of the Invention
[0009] An object of the present invention, therefore, is to overcome the aforementioned limitations of the prior art.
[0010] Another object is to implement a recommender system that uses a form of programmed placement policy or bias in providing recommendations to subscribers.
[0011] An aspect of the invention, therefore, concerns a method of operating a recommender system, which recommender is used for recommending items of interest to users interacting with an online service provider. The method generally comprises the steps of: setting up one or more policies to be used for influencing recommendations made by the recommender system to a user; responding to a request for a recommendation; causing the recommender system to use at least one of the one or more policies when presenting one or more policy influenced recommendations to the user in response to the request.
[0012] Generally speaking, the policy is used to provide a preference to a policy holder, the reference being either a preference filter to be used against a list of recommendation items generated by the recommender system, and/or a presentation preference for one or more items associated with the policy holder and which are part of the list of recommendation items. The policy can be defined with reference to a period of time in which the policy is to be used; one or more keywords; one or more demographic variables associated with the user; a type characteristic associated with each of the items recommended by the recommender system; a release date associated with the items recommended by the recommender system; and/or with reference to a brand or source of the items recommended by the recommender system.
[0013] In a preferred approach the one or more policies are considered after a recommender system has generated an initial list of uninfluenced recommendations. However, in some cases it may be more desirable to consider and implement the policy before the initial list is created.
[0014] In some applications, an additional step can be performed of evaluating the one or more policies under a cost—benefit analysis to determine an appropriate policy to be used for the user. A different policy can be used for each request for a recommendation made by the user, so that the one or more policies are utilized on a request by request basis. In some instances it may be desirable to compare a policy influenced recommendation to the user against a recommendation made by the recommender system in the absence of considering the one or more policies.
[0015] A policy record for a selected policy is updated within a policy log when a recommendation is made to the user based on such selected policy. For some embodiments it may be desirable to generate a report of a number of recommendations made by a recommender system broken down according to a type of policy used for such recommendations.
[0016] In some preferred embodiments, multiple policies are evaluated for each recommendation made by the recommender system. Some of the policies may include a fulfillment requirement, so that the recommender system is forced to generate a certain number of recommendations based on the first one of the policies. In other cases, the policies may be selected on a controlled opportunity basis to influence recommendations, such that a first policy can have a higher percentage chance of influencing a recommendation compared to a second policy.
[0017] A related aspect of the invention is a method which controls advertising material presented to the user with the one or more recommendations based on which of the one or more policies is used to generate to the one or more recommendations.
[0018] Another aspect of the invention concerns a recommender system which is programmed with the above policy methods in the form of various software routines. The recommender system is typically implemented at an e-commerce website. However, in some applications the recommender system is implemented at least in part in a client computing device, including a programming receiver. The recommendation is typically associated with a book, a movie, an auction item, or a content item.
[0019] It will be understood from the Detailed Description that the inventions can be implemented in a multitude of different embodiments. Furthermore, it will be readily appreciated by skilled artisans that such different embodiments will likely include only one or more of the aforementioned objects of the present inventions. Thus, the absence of one or more of such characteristics in any particular embodiment should not be construed as limiting the scope of the present inventions.
Description of the Drawings
[0020] FIG. 1 is an illustration of a preferred process utilized to influence behavior and recommendations of a recommender system through a programmed policy in accordance with the present invention;
[0021] FIG. 2 illustrates the types of policies that can be used to influence a recommender system in accordance with the present invention.
Detailed Description
[0022] A preferred process for influencing the behavior and recommendations of a recommender system through a programmed policy, such as a pay-for-placement bias is illustrated in FIG. 1 . The recommender system which can be used with the present invention includes any of a number of well-known types of software agents which operate at an e-commerce website and process a person's explicit and implicit inputs to generate a suggestion to the person for one or more items, including the types referred to above.
[0023] These recommender systems typically interact with a subscriber over a network (such as the Internet), as part of a World Wide Web (WWW) web page on a website, and which is presented within a client device (i.e., some form of computing system) within a browser software program. In some cases, however, a recommender may be built in as part of an electronic programming guide or manager, as in the case of certain television/satellite receivers. The details of such recommender systems, their programming, etc., and the specifics of the hardware which implement the same—namely, servers, client devices, browsers, and WWW page codings are well known, and are not material to the present invention. For examples, see e.g., U.S. Pat. Nos. 6,515,681; 6,064,980 and 5,583,763 which are hereby incorporated by reference. Those skilled in the art will appreciate that any conventional implementation can be used with the present invention.
[0024] Furthermore, as concerns the inventive process below, again, as noted herein, such process is expected to be embodied in a variety of software programs, routines, etc., that run on one more computers coupled to the Internet. In other embodiments these programs and routines will run directly on a computing machine at the subscriber's premises, such as a digital video recorder, receiver, etc. The actual implementation details will vary from platform to platform, can be designed and effectuated by persons skilled in the art from ordinary skill in this field, and are thus not material to the present discussion.
[0025] As seen in FIG. 1 , a recommender system influence policy is defined and provided to a recommender system at step 110. The basic types of influence policies are described further below, and can include, for example, a pay-for-placement policy. Generally speaking, these policies, when implemented in software as part of a filtering mechanism, can be considered a form of influencer engine.
[0026] At step 120, a subscriber to the website interacts with the recommender system in a conventional fashion, and so as to elicit a recommendation. For example, in the context of a movie rental website, the subscriber may request a suggestion for titles in a drama genre.
[0027] A recommendation is then provided to the user at step 130. Unlike a conventional recommendation, however, the recommendation in this instance, however, is based on the recommender system's adoption of a specific policy provided at step 110.
[0028] It should be noted, of course, that the policy may be defined at step 110, but it may in fact only be effectuated later at step 130. In other words, instead of considering the policy to filter the recommendations ab initio as part of the initial selection process, the recommender system might alternatively generate a normal list of recommendations which are examined for compliance with the policy, and then packaged in accordance with the policy for presentation to the user. The latter may be preferable in some instances because an overlay type system can avoid conflicts and re-designs of commercial recommender system software packages.
[0029] In any event, those skilled in the art will appreciate that the implementation details will vary from application to application. Furthermore, it should be noted, of course, that the policy defined at step 110 may in fact be selectively applied only at step 130 during specified times, and on a subscriber by subscriber, or item by item basis.
[0030] At step 140 the final list of recommended items is presented to the user, which, again, as noted above, could be a service, a movie title, a book title, an auction item, a consumer article, a content item, etc. Again, the recommendation may be in the form of a single item and/or a set of items. In the latter case the items may be visually arranged, sized, etc. in a manner that accentuates certain listings, or makes them more prominent in some fashion. The placement and presentation of such recommendations can also be the subject of a policy of the types described below.
[0031] In the context of a website, of course, such recommendations will be presented within a web browser, while, in the context of a home programming receiver the recommendations will be presented within an electronic programming guide interface. For PDAs and phones, of course, the presentations will be made within a conventional display. Again, the particular form and method of delivering the recommendations is not critical, and will vary in accordance with a computing platform used by the user to interact with the recommender system.
[0032] Finally, step 160 notes that the overall performance of the influencer policy is evaluated. For example, a log can be maintained of the number of times the recommendation could have been made under a policy, the number of times the recommendations were made under each influencer policy, and their adoptions by the persons seeing them, again, broken down by policy. An adoption rate then can be calculated to see the persuasiveness of the influencer engine. The influencer adoption rate can then be compared against adoption rates from non-influenced recommendations to see the relative effectiveness of the influencer engine.
[0033] In addition, an evaluation can be made to determine the relative “closeness” in each instance between a proposed recommender item, and an actual item recommended by the influencer engine. In other words, the “cost” to the website operator, in terms of an influencer imposed “error” on the recommender system can be calculated. The error can be determined, for example, by measuring an actual ranking for the policy-influenced recommend item for the person in question, as compared to the ranking for the item that would have been recommended to the person in question.
[0034] Again, in some implementations, the “error” might be evaluated in advance of presenting an actual recommendation as shown at step 170. In this way, if the error cost of presenting the influenced item is too great, the e-commerce operator can elect to veto the recommendation, and perhaps provide either an alternate “influenced” recommendation, or an uninfluenced recommendation.
[0035] In another variation, the recommender system can implement a selection process for identifying an appropriate influence engine at step 170. This would be useful in those cases where an e-commerce site operator has a variety of influence policies that can be used by third parties for use in a recommender system. For example, a first entity may purchase or request a first policy that biases a first type of item in its favor, so that a recommender system preferably identifies and recommends these types of items to users. A second entity may similarly purchase a second policy that biases a second type of item in their favor.
[0036] One method of selecting a policy, as noted below, is merely to randomly select a policy among a set of available policies, or to implement a scheduler based on a desired rate of presentation each policy.
[0037] Another method of deciding on an influence policy is to consider a cost/benefit analysis for each policy. Thus, at step 130, a recommender system may evaluate the relative cost of accommodating either the first policy or the second policy. In those instances where it is possible to consider alternative choices, an e-commerce site may thus elect to go with an option that minimizes an overall “cost” in terms of deviance from an optimal recommended item. This selection can be logged at step 180 to identify the recommendation, and the policy that is associated with the same. In this manner, the e-commerce operator can tabulate a number of recommendations provided pursuant to a policy to demonstrate compliance to a purchaser of the policy. The log can include a number of recommendations provided pursuant to each policy, as well as a total number of recommendations made. Again, in some cases an e-commerce operator may chose to not use any influence engine, to avoid potentially giving a poor recommendation.
[0038] This type of selection process, therefore, can minimize the cost to the recommender system operator, because it can elect, on a user by user, and item by item basis, to provide a recommendation that satisfies at least one given policy, and yet is not unacceptably expensive. In many cases, for example, a particular item mandated by a policy will be the same as a baseline recommendation that otherwise suggested by the recommender system. The baseline recommendation for a particular user may also, by default, incorporate demographic, geographic and similar data. If the e-commerce operator has the option of selecting between choices, and can pick the lowest cost item, this feature helps to improve the profitability of such site. Moreover in some instances, certain websites will identify common popular selections made by persons in the immediate geographic vicinity of the user (i.e., Amazon Purchase Circles). A baseline recommendation therefore, can include a number of other active influences under the control of the e-commerce operator which are outside the scope of any specific policy which is then used for comparison.
[0039] FIG. 2 identifies a number of policy or influencer inputs that can be provided to an influence engine policy manager 210. The policies and influences may be conditioned or qualified, of course, so that a recommender system does not have to implement a particular policy in each instance. In fact, a policy might only need to be implemented based on satisfying a certain policy percentage threshold that is negotiated between the policy requester and the policy implementer (i.e., the e-commerce operator). For example, the policy percentage threshold may specify that the policy must be implemented in at least 50% of the situations in which it could be presented. The figure will vary of course from situation to situation, and the invention is not limited in this respect. Alternatively, a policy may be implemented or purchased based simply on a guaranteed bulk number of recommendations over an indefinite period. A number of other conventional options are easily integratable as well, and the present invention is not limited in this respect.
[0040] The policies which could be implemented include, for example: time based policies 215, keyword based policies 220, demographic based policies 225, type based policies 230, release data based 235, origin based 240, and other based 245. These policies can be described basically as follows.
[0041] First, a time based policy 215 may specify, for example, that a supplier of certain items will receive preferential treatment by a recommender system over a predetermined period of time, or during certain hours of the day. A keyword based policy 220 may specify, for example, that a supplier of certain items will receive preferential treatment by a recommender system for queries made subscribers which include certain key words. The use of pay for placement based on keywords is well-known in the art of search engine technologies; for further details, see U.S. Pat. No. 6,269,361: System and method for influencing a position on a search result list generated by a computer network search engine—which is hereby incorporated by reference. Many of the techniques discussed in such reference could be adopted here as well for influencing both a type and position of a recommendation provided by a recommender system.
[0042] A demographics based policy 225 could be based on the characteristics/profile of the user receiving the recommendation. For example, a policy might be triggered only when a subscriber meets certain criteria, such as an age range, an income range, a domain range, a geographic region, etc. Thus, when such policy is in place, a recommender system will give preference to items originating from the policy holder.
[0043] A type based policy 230 could be based on the characteristics of the item. For example, some media items may be classified by author, genre, topic, etc. A type based policy could specify that if the recommender generates an initial list of recommended items having a common characteristic (i.e., such as the same topic), the policy holder will be given a preference for their items meeting such characteristic.
[0044] In another instance, a release date based policy 235 could be based on the date of introduction of the item. For example, a policy holder of this type may be given a preference so that, to the extent new items may satisfy a recommendation requirement, only new items from such entity are recommended. Other more limited preferences could also be specified.
[0045] Another basic policy is an origin based policy 240, which could specify that a particular entity should receive priority over other content providers for items which otherwise meet certain criteria. Thus, for example, if a user requests a recommendation on a certain type of book, a recommender system could give a preference to content from certain policy holders.
[0046] Other policies will be apparent to those skilled in the art. Again, each of these policies may further be qualified by some absolute number of recommendations, so that a policy may be in force for a certain selected number of recommendations.
[0047] In all of the above, the preference given to the policy holder may be in the form of a filter (so that only the policy holder's items are displayed) or a priority in time of presentation, or positional preference on a screen, or ranking, or display characteristic (sizing or other visual enhancement). Other examples of providing a preference to effectuate a policy will be apparent to those skilled in the art.
[0048] It will be further apparent to those skilled in the art that multiple policies may exist at the same time, and that the influence engine policy manager must identify and evaluate such policies for a given set of recommendations to be presented to a subscriber. Furthermore, when a recommendation is made in accordance with a policy, a log of such must be made as well, so that a policy compliance report can be generated.
[0049] Thus in some cases it will be possible that one or more different types of policies may be implemented by an influence engine manager. As an example, both time based and type based policies may be in force at the same time. Furthermore, even within a single policy type, more than one policy may be in force. As an example, one entity may purchase one set of keywords for a keyword based policy 220, while another entity may purchase another set of keywords for a second keyword based policy.
[0050] When more than one policy is in place, these policies may be “eligible” for accommodation by an influence engine manager, and are considered, preferably, on a cost basis and/or a fulfillment basis. In other words, an e-commerce operator can evaluate the cost of supplying a particular recommendation in accordance with a policy, as well as the outstanding fulfillment obligation that may exist. A fulfillment obligation can arise from the fact that an e-commerce operator may be required to present either an absolute number of recommendations to subscribers in accordance with a policy, or a certain percentage over time, or in accordance with a given capacity. In other instances there may be no fulfillment obligation of course.
[0051] Thus, at any given moment, a set of policies (P1, P2 . . . PN) is associated with a set of separate entities (E1, E2 . . . EN), and a set of costs per recommendation charged by the e-commerce provider (C1, C2, . . . CN), which can vary. For example, an e-commerce operator may charge more for a time based policy than a keyword based policy. The policies may, in some applications, be further associated with a set of corresponding fulfillment criteria (F1, F2, . . . FN) which specify an existing compliance with a fulfillment obligation for the policy.
[0052] To ensure that certain policies are not left unsatisfied, a scheduling routine can be implemented for the policies as part of the influence manager. For example, a policy ranking can be done, based on examining the compliance log noted earlier, to examine which policies are most in need of being met. At any moment in time, the influence engine policy manager may select only the top X or top N % of eligible policies through such ordering process. Thus, the available policies are reduced by some number to a smaller subset of eligible policies based on a fulfillment requirement and a suitable conventional planning algorithm.
[0053] It is this smaller subset of eligible policies, therefore, which is evaluated for any particular recommendation by a scheduling routine. In one simple approach, policies can be rotated continuously in a round-robin fashion from highest to lowest. When a particular policy is satisfied with a given recommendation, this policy is then moved to the bottom of the eligible policy list.
[0054] Other examples and techniques will be apparent to those skilled in the art for scheduling a pool of policies. The only constraint, of course, is an e-commerce operator cannot use a set of policies that are inconsistent and/or impossible to satisfy at the same time. The manner in which the policies are satisfied, including an acceptable planning algorithm, can be determined on an application by application basis using techniques well known in the art of capacity planning.
[0055] In some instances an e-commerce operator may provide an influence placement per item that is not constrained by a fulfillment obligation, but is merely compensated for on a per item presentation basis. In these cases, of course, the policies may or may not be rotated, depending on the fee per presentation charged by the e-commerce operator. In such cases the e-commerce operator may factor in not only the error cost, but also the benefit (i.e. the cost charged to the policy requester for the recommendation) to maximize revenue for the e-commerce operator. That is, if the total estimated value for adopting a policy for a particular recommendation is CN−EN, where CN is a value to recommender for using policy N, and EN is a cost of an error of recommendation for policy N, then an e-commerce operator can select from a variety of policies for any given recommendation to maximize a recommendation based revenue.
[0056] In other instances, if the benefit of each recommendation is approximately the same, the e-commerce operator can use a round robin schedule noted above. Alternatively, the e-commerce operator can use a random number generator, combined with an individual weighting value for each policy, so that at any given moment in time, a set of policies that are randomly selected are considered eligible for consideration. Other examples will be apparent to those skilled in the art, and the invention is not limited to any particular variant.
[0057] Again, for each recommendation made, an e-commerce operator can note the policy used, and thus charge a particular policy requester in an appropriate amount. In some instances, an e-commerce operator may elect to tabulate an adoption rate for the recommendations as demonstrated by the subscribers. From such data, an e-commerce operator can detect, for example, a frequency by which policies are adopted, and a listing of the same from lowest to greatest frequency. This adoption frequency, in turn, can be used in a feedback manner to control the selection and consideration of policies on the front end of the recommender system, so that policies which are more likely to be adopted are given preference in consideration. This strategy has the effect of being beneficial to both the policy requester and the e-commerce site operator, since both are achieving greater recognition from the recommender system.
[0058] Finally, it is apparent that recommendations and advertisements can be correlated at certain e-commerce websites. For example, if a book title pertaining to automobiles is recommended to an online user, an advertising placement engine running at the website can also direct an automobile related ad to the user at the same time. These kinds of context based advertising methods are well-known in the art. In some instances, for example, if an advertiser is associated both with recommendation content and advertising content, the two can be correlated and reconciled during a recommendation request so that a supplier of a recommended item is also used for a source of advertising material.
[0059] It can be seen from the above that the present invention, through its influence on a recommender system, in fact can also significantly affect an advertising delivery campaign, and the type of ad that will be delivered in any instance to an online user. Consequently in some variations of the invention an advertising delivery system and a recommender system policy can be coordinated. Thus, the recommender system policy can also indirectly drive and control an advertising delivery system.
[0060] Consequently, as shown in FIG. 1 , an optional step 175 determines whether to deliver an ad that is correlated to the recommender system policy. It should be noted that the selection of a particular ad can utilize essentially the same logic, strategy and process as noted above for selecting a recommendation policy. Accordingly, in instances where there is flexibility in providing a choice of ads, an e-commerce provider may elect to provide an ad that provides the highest revenue for given impression to such user.
[0061] As noted earlier, the above referenced policy based recommender system operated may be implemented in at an e-commerce site server, or they may be executed within a local computing device used by the subscriber, such as a PDA, an electronic program guide receiver (such as TiVo box), etc. In some instances, the recommender system processes and databases described above will be distributed between a client and server device to optimize performance and flexibility.
[0062] In some embodiments it may be desirable to allow individual users to opt in or out of recommender systems which use policy based recommendations. This election may be implemented on an application by application basis across different types of e-commerce sites. Where appropriate, conventional discounts, inducements, etc., known in the art could also be used to persuade persons to opt-in to such arrangements. For some applications it may be useful to display a table or list of comparative recommendations made by separate recommender systems, including policy based recommendations such as those of the present invention, compared to recommendations made by friends of the user, recommendations associated with persons living in the same area, recommendations associated with persons with similar demographics, etc. Other examples will be apparent to those skilled in the art.
[0063] It will be understood by those skilled in the art that the above is merely an example of a policy based recommender system and that countless variations on the above can be implemented in accordance with the present teachings. A number of other conventional steps that would be included in a commercial application have been omitted, as well, to better emphasize the present teachings.
[0064] The routines for implementing the policy identification, scheduling, selection and logging operations as described above can be implemented using any conventional software programs and routines available to skilled artisans, and the present invention is not limited to any specific embodiment thereof. Furthermore, it is expected that such routines will vary from computer to computer in actual commercial practice. To better illustrate the features of the present invention, other aspects of the exchange system have been omitted, and it will be understood that such system could include other options, features, entries, etc., that are not specifically shown.
[0065] The above descriptions are intended as merely illustrative embodiments of the proposed inventions. It is understood that the protection afforded the present invention also comprehends and extends to embodiments different from those above, but which fall within the scope of the present claims.
Claims
What is claimed is:
1. A method of operating an online recommender system for recommending items of interest to users interacting with an online service provider, comprising:
defining a pay-for-placement influence policy applicable to a third party advertiser;
establishing a biasing policy based on the influence policy for influencing recommendations made by the recommender system to a user of the recommender system,
wherein the biasing policy is representative of preferences of the third party advertiser, and biases the recommender system to recommend items associated with the third party advertiser;
responding to a request for a recommendation; and
applying the biasing policy to the recommender system when presenting a policy influenced recommendation in response to the request.
2. The method of claim 1, further comprising prior to said applying:
evaluating a plurality of biasing policies under a cost-benefit analysis to determine an appropriate policy to apply,
wherein cost in the cost-benefit analysis is an error cost, the error cost being a difference in ranking of a non-biased proposed recommendation and a pay-for-placement policy influenced recommendation, and a benefit is a cost charged to the third party advertiser for the policy influenced recommendation.
3. The method of claim 2, further comprising:
controlling advertising material presented to the user with the policy influenced recommendation based on which of the plurality of biasing policies is used to generate the policy influenced recommendation.
4. The method of claim 2, wherein a different biasing policy can be used for each request for a recommendation made by the user, so that the plurality of biasing policies are utilized on a request by request basis.
5. The method of claim 1, further comprising:
comparing a policy influenced recommendation to the user against a recommendation made by the recommender system in the absence of considering the biasing policy.
6. The method of claim 2, further comprising:
updating a policy record for a selected biasing policy within a policy log when a policy influenced recommendation is made to the user based on said selected biasing policy.
7. The method of claim 6, further comprising:
generating a report of a number of policy influenced recommendations made by the recommender system broken down according to a type of biasing policy used for the policy influenced recommendations.
8. The method of claim 1, wherein the preferences comprise at least one of a preference filter to be used against a list of recommendation items generated by the recommender system, and a presentation preference for a plurality of items associated with the third party advertiser and which are part of the list of recommendation items.
9. The method of claim 1, wherein the biasing policy is defined with reference to a period of time in which the biasing policy is to be used.
10. The method of claim 1, wherein the biasing policy is defined with reference to a keyword determining if and when the biasing policy is to be used.
11. The method of claim 1, wherein multiple biasing policies are evaluated for each policy influenced recommendation made by the recommender system.
12. The method of claim 1, wherein the recommender system is implemented at an e-commerce website.
13. The method of claim 1, wherein the recommender system is implemented at least in part in a client computing device, including a programming receiver.
14. The method of claim 1, wherein the biasing policy is considered after a recommender system has generated an initial list of uninfluenced recommendations.
15. The method of claim 1, wherein the biasing policy is utilized by a recommender system while formulating an initial list of recommendations.
16. The method of claim 1, wherein the biasing policy includes a fulfillment requirement, so that the recommender system is forced to generate a number of pay-for-placement policy influenced recommendations based on the biasing policy.
17. The method of claim 2, wherein the biasing policies are selected on a controlled opportunity basis to influence recommendations, such that a first policy can have a higher percentage chance of influencing a recommendation compared to a second policy.
18. A method of presenting advertising via an electronic device in conjunction with recommendations made by a recommender system, comprising:
defining a pay-for-placement influence policy applicable to a third party advertiser;
implementing a biasing policy based on the influence policy for influencing recommendations made by the recommender system to a user of the recommender system,
wherein said biasing policy is representative of preferences of the third party advertiser, and biases the recommender system to recommend items associated with the third party advertiser;
responding to a request for a recommendation; and
presenting advertising content based on the biasing policy while presenting policy influenced recommendation to the user in response to the request.
19. The method of claim 18, wherein the request is made by the user during an online Internet session.
20. The method of claim 18, wherein the request is made by the user from a home based content receiver.
21. The method of claim 18, wherein advertising content is based on recommendation content presented in the policy influenced recommendation.
22. The method of claim 21, wherein the third party advertiser associated with policy influenced recommendation is automatically selected as a source of the advertising for the advertising content.
23. The method of claim 18, wherein the advertising content is presented based on a pay for placement request from the third party advertiser.
24. A recommender system operating on a website and programmed to provide recommendations to users for items in response to a request for a recommendation, comprising:
a first software routine for defining a pay-for-placement influence policy applicable to a third party advertiser;
a second software routine for establishing a biasing policy based on the influence policy for influencing recommendations made by the recommender system to a user of the recommender system,
wherein the biasing policy is representative of preferences of the third party advertiser, and biases the recommender system to recommend items associated with the third party advertiser, and
wherein the recommender system applies the biasing policy to present policy influenced recommendations to the user in response to the request for a recommendation.
25. A method of recommending items of interest to a user of an online service provider, comprising:
maintaining a profile for a user at the online service provider, wherein the profile includes prior selections and/or ratings of items made by the user;
providing one or more recommendations to the user using a recommender system which uses account profiles of users similar to the user to generate such recommendations;
defining a pay-for-placement influence policy applicable to a third party advertiser;
defining a first item which is to receive a preference from the recommender system based on the influence policy,
wherein the preference biases the recommender system to recommend items associated with the third party advertiser; and
selectively influencing the recommender system on a subscriber by subscriber basis, so that the first item is presented to a user in a policy influenced recommendation based on the preference when the user's profile meets criteria of the influence policy,
wherein the influence policy can be used to present recommendations of items to the user which would not otherwise be presented by the recommender system to the user.
26. The method of claim 25, wherein the policy influenced recommendation is presented during an interactive online session.
27. The method of claim 26, wherein the policy influenced recommendation is positioned and/or displayed more prominently than other items presented in response to a recommendation request.
28. The method of claim 25, wherein the policy influenced recommendation is provided based on the first item being a new item.
29. The method of claim 25, wherein the policy influenced recommendation is provided only during a certain predefined time period and/or during certain times of the day.
30. The method of claim 25, wherein the criteria includes demographic characteristics which must be associated with the profile for the user.
31. A method of recommending items of interest to a user of an online service provider, comprising:
maintaining a profile for a user at the online service provider, wherein the profile includes prior selections and/or ratings of items made by the user;
providing recommendations in a first list to the user over an electronic communications network using a recommender system which uses account profiles of other users similar to the user to generate such recommendations;
defining a pay-for-placement influence policy applicable to a third party advertiser;
defining a first item which is to be recommended by the recommender system based on the influence policy and preference factors unrelated to the account profiles of other users,
wherein the preference factors bias the recommender system to recommend items associated with the third party advertiser; and
selectively influencing the recommender system such that the first item is presented, based on the influence policy and the preference factors, only to selected users residing within predefined geographic regions,
wherein the influence policy can be used to present recommendations of items to the user which would not otherwise be presented by the recommender system over the electronic communications network to the user in the first list.
32. The method of claim 31, wherein the first item is a newly released movie.
33. The method of claim 31, wherein the first item is recommended in response to keywords used in requests for recommendations by users in the predefined geographic regions.
34. The method of claim 31, further comprising:
tabulating user selections of the first item made in response to the preference factors.
35. The method of claim 31, wherein the preference factors include an age of the user, such that the first item is presented only to selected users residing within predefined geographic regions who satisfy an age range criterion.
36. The method of claim 31, further comprising:
defining a second item which is to be recommended by the recommender system based on the influence policy and second preference factors unrelated to the account profiles of other users;
defining a schedule to be used in connection with recommending the first item and the second item, the schedule being based on a cost-benefit analysis performed on a subscriber by subscriber basis; and
presenting one or both of the first item and the second item to the selected users residing within the predefined geographic regions based on the schedule.
37. A method of recommending items of interest to users of an online service provider, comprising:
defining a pay-for-placement influence policy applicable to a third party advertiser;
providing an influenced recommendation based on a biasing policy derived from the influence policy for a first item to a user over a network using a collaborative filtering based recommendation engine,
wherein the biasing policy is representative of preferences of the third party advertiser, and biases the recommender system to recommend items associated with the third party advertiser, and
wherein the first item is characterized by a first content associated with a first identifiable source of content; and
providing an advertisement to the user which is correlated to the influenced recommendation, the advertisement being generated by an advertising delivery system and describing a second item to the user over the network,
wherein the second item is characterized by a second content that is also associated with the first identifiable source of content, and
wherein presentation of recommendations and advertisements are coordinated for users the user of the online service provider.
38. The method of claim 37, wherein the recommendation and the advertisement are presented at the same time.
39. The method of claim 38, wherein the recommendation and the advertisement are presented within a single web page.
40. A method of recommending items of interest to users of an online service provider, comprising:
defining a pay-for-placement influence policy applicable to a third party advertiser;
providing an influenced recommendation based a biasing policy derived from the influence policy for a first item to a user over a network using a collaborative filtering based recommendation engine,
wherein the biasing policy is representative of preferences of the third party advertiser, and biases the recommender system to recommend items associated with the third party advertiser; and
providing an advertisement to the user which is correlated to the recommendation, the advertisement being generated by an advertising delivery system and describing a second item to the user over the network,
wherein the second item is related logically to the first item and is associated with a context presented the user in which the first item is presented,
wherein presentation of recommendations and advertisements are coordinated for the user of the online service provider.
41. The method of claim 40, wherein the recommendation and the advertisement are presented at the same time.
42. The method of claim 41, wherein the recommendation and the advertisement are presented within a single web page.
43. A recommender system for recommending items of interest to users interacting with an online service provider, comprising:
means for defining a pay-for-placement influence policy applicable to a third party advertiser;
means for providing a biasing policy based on the influence policy to influence recommendations made by the recommendation system,
wherein the biasing policy is representative of preferences of the third party advertiser, and biases the recommender system to recommend items associated with the third party advertiser;
means for responding to a request for a recommendation from a user; and
means for implementing the biasing policy when presenting recommendations to the user in response to the request.
44. The recommender system of claim 43, further comprising:
means for presenting advertising content to the user based on the biasing policy when presenting recommendations to the user in response to the request.
45. A computer-readable medium having stored thereon, computer-executable instructions that, if executed by a computing device, cause the computing device to perform a method comprising:
defining a pay-for-placement influence policy applicable to a third party advertiser;
establishing a biasing policy based on the influence policy for influencing recommendations made by the recommender system to a user of the recommender system,
wherein the biasing policy is representative of preferences of the third party advertiser, and biases the recommender system to recommend items associated with the third party advertiser;
responding to a request for a recommendation; and
applying the biasing policy to the recommender system when presenting a policy influenced recommendation in response to the request.
46. A method for influencing a recommender system, comprising:
defining a pay-for-placement influence policy applicable to a third party advertiser;
specifying third party advertiser preferences from the influence policy;
implementing the third party advertiser preferences to bias the recommender system to items associated with the third party advertiser;
responding to a recommendation request from a user;
generating a policy influenced recommendation for the user by enforcing the influence policy;
presenting the policy influenced recommendation to the user; and
evaluating performance of the influence policy.
Patent Citations (99)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US4870579(A) | 1989-09-01 | Hey | Applicant |
| US4996642(A) | 1991-02-01 | Hey | Applicant |
| US5459306(A) | 1995-10-01 | Stein et al. | Applicant |
| US5717865(A) | 1998-02-01 | Stratmann | Applicant |
| US5884282(A) | 1999-03-01 | Robinson | Applicant |
| US5901246(A) | 1999-05-01 | Hoffberg et al. | Applicant |
| US5903454(A) | 1999-05-01 | Hoffberg et al. | Applicant |
| US5918014(A) | 1999-06-01 | Robinson | Applicant |
| US5974398(A) | 1999-10-01 | Hanson et al. | Applicant |
| US6064980(A) | 2000-05-01 | Jacobi et al. | Applicant |
| US6266649(B1) | 2001-07-01 | Linden et al. | Applicant |
| US6314420(B1) | 2001-11-01 | Lang et al. | Applicant |
| US6389372(B1) | 2002-05-01 | Glance et al. | Applicant |
| US6418424(B1) | 2002-07-01 | Hoffberg et al. | Applicant |
| US6484123(B2) | 2002-11-01 | Srivastava | Applicant |
| US6526440(B1) | 2003-02-01 | Bharat | Applicant |
| US6600898(B1) | 2003-07-01 | De Bonet et al. | Applicant |
| US6609096(B1) | 2003-08-01 | De Bonet et al. | Applicant |
| US6687696(B2) | 2004-02-01 | Hofmann et al. | Applicant |
| US6697800(B1) | 2004-02-01 | Jannink et al. | Applicant |
| US6735572(B2) | 2004-05-01 | Landesmann | Examiner |
| US6766525(B1) | 2004-07-01 | Lee et al. | Applicant |
| US6775664(B2) | 2004-08-01 | Lang et al. | Applicant |
| US6782370(B1) | 2004-08-01 | Stack | Applicant |
| US6801909(B2) | 2004-10-01 | Delgado et al. | Applicant |
| US6853982(B2) | 2005-02-01 | Smith et al. | Examiner |
| US6912505(B2) | 2005-06-01 | Linden et al. | Applicant |
| US7043531(B1) | 2006-05-01 | Seibel et al. | Examiner |
| US7113917(B2) | 2006-09-01 | Jacobi et al. | Applicant |
| US7155401(B1) | 2006-12-01 | Cragun et al. | Applicant |
| US7356490(B1) | 2008-04-01 | Jacobi et al. | Examiner |
| US2002/0042733(A1) | 2002-04-01 | Lesandrini et al. | Applicant |
| US2002/0078056(A1) | 2002-06-01 | Hunt et al. | Applicant |
| US2002/0097265(A1) | 2002-07-01 | Kurapati et al. | Applicant |
| US2002/0123988(A1) | 2002-09-01 | Dean et al. | Applicant |
| US2002/0133417(A1) | 2002-09-01 | Hanks et al. | Applicant |
| US2002/0154157(A1) | 2002-10-01 | Sherr et al. | Applicant |
| US2002/0174429(A1) | 2002-11-01 | Gutta et al. | Applicant |
| US2002/0180805(A1) | 2002-12-01 | Chickering et al. | Applicant |
| US2002/0184139(A1) | 2002-12-01 | Chickering et al. | Applicant |
| US2002/0194079(A1) | 2002-12-01 | Kimble | Applicant |
| US2003/0033292(A1) | 2003-02-01 | Meisel et al. | Applicant |
| US2003/0036944(A1) | 2003-02-01 | Lesandrini et al. | Applicant |
| US2003/0046156(A1) | 2003-03-01 | Cromer et al. | Applicant |
| US2003/0050805(A1) | 2003-03-01 | Gouyet et al. | Applicant |
| US2003/0050815(A1) | 2003-03-01 | Seigel et al. | Examiner |
| US2003/0055816(A1) | 2003-03-01 | Paine et al. | Applicant |
| US2003/0055898(A1) | 2003-03-01 | Yeager et al. | Applicant |
| US2003/0074253(A1) | 2003-04-01 | Scheuring et al. | Applicant |
| US2003/0093329(A1) | 2003-05-01 | Gutta | Applicant |
| US2003/0093331(A1) | 2003-05-01 | Childs et al. | Applicant |
| US2003/0093338(A1) | 2003-05-01 | Kim et al. | Applicant |
| US2003/0093793(A1) | 2003-05-01 | Gutta | Applicant |
| US2003/0097302(A1) | 2003-05-01 | Overhultz et al. | Applicant |
| US2003/0135490(A1) | 2003-07-01 | Barrett et al. | Applicant |
| US2003/0135513(A1) | 2003-07-01 | Quinn et al. | Applicant |
| US2003/0149614(A1) | 2003-08-01 | Andrus et al. | Applicant |
| US2003/0167195(A1) | 2003-09-01 | Fernandes et al. | Examiner |
| US2003/0195793(A1) | 2003-10-01 | Jain et al. | Examiner |
| US2003/0204449(A1) | 2003-10-01 | Kotas et al. | Applicant |
| US2003/0208399(A1) | 2003-11-01 | Basak et al. | Applicant |
| US2003/0208578(A1) | 2003-11-01 | Taraborelli et al. | Applicant |
| US2003/0212619(A1) | 2003-11-01 | Jain et al. | Applicant |
| US2003/0216942(A1) | 2003-11-01 | Hawks | Applicant |
| US2003/0233278(A1) | 2003-12-01 | Marshall | Applicant |
| US2004/0015386(A1) | 2004-01-01 | Abe et al. | Examiner |
| US2004/0030525(A1) | 2004-02-01 | Robinson et al. | Applicant |
| US2004/0034570(A1) | 2004-02-01 | Davis | Applicant |
| US2004/0054572(A1) | 2004-03-01 | Oldale et al. | Applicant |
| US2004/0068451(A1) | 2004-04-01 | Lenk et al. | Applicant |
| US2004/0073919(A1) | 2004-04-01 | Gutta et al. | Applicant |
| US2004/0076936(A1) | 2004-04-01 | Horvitz et al. | Applicant |
| US2004/0088303(A1) | 2004-05-01 | Elder et al. | Applicant |
| US2004/0088730(A1) | 2004-05-01 | Gopalan et al. | Applicant |
| US2004/0103092(A1) | 2004-05-01 | Tuzhilin et al. | Applicant |
| US2004/0116067(A1) | 2004-06-01 | Karaoguz et al. | Applicant |
| US2004/0122686(A1) | 2004-06-01 | Hill et al. | Examiner |
| US2004/0148275(A1) | 2004-07-01 | Achlioptas | Applicant |
| US2004/0167814(A1) | 2004-08-01 | Hawks | Applicant |
| US2004/0172267(A1) | 2004-09-01 | Patel et al. | Applicant |
| US2004/0215733(A1) | 2004-10-01 | Gondhalekar et al. | Applicant |
| US2004/0220826(A1) | 2004-11-01 | Hey | Applicant |
| US2004/0225577(A1) | 2004-11-01 | Robinson | Applicant |
| US2004/0230546(A1) | 2004-11-01 | Rogers | Applicant |
| US2004/0254857(A1) | 2004-12-01 | Onizuka | Applicant |
| US2005/0021499(A1) | 2005-01-01 | Bradley et al. | Applicant |
| US2005/0066350(A1) | 2005-03-01 | Meuleman | Applicant |
| US2005/0086300(A1) | 2005-04-01 | Yeager et al. | Applicant |
| US2005/0120006(A1) | 2005-06-01 | Nye | Applicant |
| US2005/0125309(A1) | 2005-06-01 | Song | Examiner |
| US2005/0203807(A1) | 2005-09-01 | Bezos et al. | Examiner |
| US2006/0100956(A1) | 2006-05-01 | Ryan et al. | Applicant |
| WO98/34189 | 1998-08-01 | Applicant | |
| WO1/46930 | 2001-06-01 | Applicant | |
| WO2/25937 | 2002-03-01 | Applicant | |
| WO2/37840 | 2002-05-01 | Applicant | |
| WO3/088107(A2) | 2003-10-01 | Applicant | |
| WO2004/052010 | 2004-06-01 | Applicant | |
| WO2004/053757 | 2004-06-01 | Applicant |
Non-Patent Literature (130)
- Solomon, Mary “Targeting Trendsetters: Cyberstudies capture the perceptions and insights of early adopters” Marketing Research Summer 1996, vol. 8, No. 2.Examiner
- Morgan, Fred, Jr. “Are Early Triers Heavy Users?” The Journal of Business, vol. 52, No. 3 (Jul. 1979), pp. 429-434.Examiner
- Greco, Alan J.; Fields, D. Michael, “Profiling Early Triers of Service Innovations: A Look at Interactive Home Vid . . . ”, The Journal of Services Marketing; Summer 1991; 5, 3; p. 19.Examiner
- Harold Zane Daniel, Jr., “The Role of MARKR-Entry Timing In The Evaluation of Manufacturing Technology Innovations”, dissertation, University of Connecticut, 1997.Examiner
- Patrick Y.K. Chau, Kai Lung Hui, “Identifying early adopters of new IT products: A case of Windows 95”, Information & Managegment 33 (1998) 225-230.Examiner
- http://www.amazon.com/gp/earlyadopter/browse.html/ref=cm—ea—pdp/102-4825329-5718538?categoryID=2&filterID=-1, Mar. 9, 2004, 1 page.Applicant
- Andrew Chen, “Forecasting Gross Revenues at the Movie Box Office,” ECON 482, Econometric Methods, Final Paper, Jun. 4, 2002, 20 pages.Applicant
- Tal Garber et al., “From Density to Destiny: Using Spatial Analysis for Early Prediction of New Product Success,” Marketing Science Institute Working Paper No. 02-106, Feb. 2002, 44 pages.Applicant
- Nicholas Kushmerick, “Robustness analyses of instance-based collaborative recommendation,” believed to be published in Proceedings of European Conference on Machine Learning (Helsinki), 2002, 12 pages.Applicant
- Shyong K. Lam et al., “Shilling Recommender Systems for Fun and Profit,” believed to be published in Proceedings International WWW Conference, New York, USA, 2004, pp. 393-402.Applicant
- Steve Lawrence, “Implicit feedback: Good may be better than best,” Workshop on Implicit Measures of User Interests and Preferences, Aug. 2003, 47 pages.Applicant
- Paolo Massa, et al.,“Trust-aware Collaborative Filtering for Recommender Systems,” Published in International Conference on Cooperative Information Systems (CoopIS) 2004, 17 pages.Applicant
- Greg Notess, “Freshness Issue and Complexities with Web Search Engines,” http://www.onlinemag.net/OL2001/net11—01.html, Nov. 2001, 5 pages.Applicant
- Douglas Oard, et al., “Protecting the Privacy of Observable Behavior in Distributed Recommender Systems,” Presented at the SIGIR Workshop on Implicit Methods in Toronto in Aug. 2003, 17 pages.Applicant
- Matthew Richardson, et al., “The Intelligent Surfer: Probabilistic Combination of Link and Content Information in PageRank,” believed to be published in Advances in Neural Information Processing Systems 14, Cambridge, MA: MIT Press, 2002, 8 pages.Applicant
- Andrew Weigend, “Analyzing Customer Behavior at Amazon.com,” KDD 2003, Washington, DC, 7 pages.Applicant
- Jie Wu, et al., “Semantic web graph implied by user preferred activities,” Workshop on Implicit Measures of User Interests and Preferences, Aug. 1, 2003, 14 pages.Applicant
- http://www.amazon.com/exec/obidos/tg/browse/-/3072321/ref=pd—rhf—c—1/104-4844530-6855147 by Amazon Mar. 2003.Applicant
- Adomavicius, Gediminas, et al., “Extending Recommender Systems: A Multidimensional Approach,” IJCAI-01 Workshop on Intelligent Techniques for Web Personalization (ITWP'2001), Seattle, Washington, Aug. 2001, 5 pgs.Applicant
- Arasu, A., et al., “PageRank Computation and the Structure of the Web: Experiments and Algorithms,” Technical Report, IBM Almaden Research Center, San Jose, CA, Nov. 2001, 5 pages.Applicant
- Baeza-Yates, Ricardo, et al., “Web Structure, Age and Page Quality,” http://www.dcs.bbk.ac.uk/webDyn2/proceedings/baeza—yates web structure.pdf, 12 pages.Applicant
- Barroso, Luiz Andre, et al., “Web Search for a Planet: The Google Cluster Architecture,” IEEE Micro vol. 23, Issue 2 (Mar. 2003) pp. 22-28.Applicant
- Brin, Sergey, et al., “The anatomy of a large-scale hypertextual web search engine,” believed to be published in Proceedings of the 7 International World Wide Web Conference, Brisbane, Australia, 1997, vol. 3, ACM Press, 23 pages.Applicant
- Brin, Sergey, et al., “The PageRank Citation Ranking: Bringing Order to the Web,” http://www.actonvision.com/GooglePageRank1.html, Jan. 28, 1998, 13 pages.Applicant
- Davison, Brian, “Overview: WWW Search Engines,” www.cse.lehigh.edu/˜heflin/courses/semweb/se-overview.pdf, 2003, 38 pages.Applicant
- Diaz, Fernando, et al., “Temporal Profiles of Queries,” Yahoo Research Labs Technical Report, Pasadena, CA, 2004, 9 pages.Applicant
- Emergent Music Website at: http://www.emergentmusic.com, accessed on Dec. 9, 2004, 16 pages.Applicant
- Flake, Dr. Gary, “Self-Organization of the Web and Identification of Communities,” press release from NEC Research Institute, Mar. 6, 2002, 1 page.Applicant
- Geyer-Schulz, Andreas, et al., “Comparing Two Recommender Algorithms with the Help of Recommendations by Peers,” In O.R. Zaiane, J. Srivastava, M. Spiliopoulou, and B. Masand, editors, WEBKDD 2002—Mining Web Data for Discovering Usage Patterns and Profiles 4th International Workshop, Edmonton, Canada, Jul. 2002, Revised Papers, Lecture Notes in Computer Science LNAI 2703, pp. 137-158. Springer-Verlag, 2003, 23 pages.Applicant
- Ghani, Rayhid, et al. “Using Text Mining to Infer Semantic Attributes for Retail Data Mining,” IEEE International Conference on Data Mining, Dec. 9-12, 2002. Maebashi, Japan, 8 pages.Applicant
- Goombah website at: http://goombah.emergentmusic.com, accessed on Dec. 9, 2004, 9 pages.Applicant
- Greco, Alan J. et al., “Profiling Early Triers of Service Innovations: A Look at Interactive Home Vid . . . ”, The Journal of Services Marketing; Summer 1991; 5, 3; p. 19.Applicant
- Guha, R., “Open Rating Systems,” Technical Report, Stanford Knowledge Systems Laboratory, Stanford University, CA, USA, 2003, 10 pages.Applicant
- Hart, David, “Researchers develop techniques for computing Googlestyle Web rankings up to five times faster,” http://www.eurekalert.org/pub—releases/2003-05/nsf-rdt051303.php, 3 pages.Applicant
- Haveliwala, Taher H., “TopicSensitive PageRank,” believed to be published in Proceedings of the Eleventh International World Wide Web Conference, Honolulu, Hawaii, 2002, 10 pages.Applicant
- Haveliwala, Taher H., et al., “An analytical comparison of approaches to personalizing PageRank,” Technical report, Stanford University, 2003, 4 pages.Applicant
- Haveliwala, Taher H., et al., “Computing PageRank using Power Extrapolation,” Technical report, Computer Science Department, Stanford University, 2003, 12 pages.Applicant
- Haveliwala, Taher H., et al., “The Second Eigenvalue of the Google Matrix,” Technical report, Computer Science Department, Stanford University, 2003, 8 pages.Applicant
- Heer, Jeffrey, “search (and destroy),” http://jheer.org/blog/archives/000006.html, Jul. 15, 2003, 5 pages.Applicant
- http://www.k-praxis.com/archives/000037.html, “Blogs and Google: The Future of Categorized Indexes,” May 10, 2003, 2 pages.Applicant
- Kamvar, Sepandar, et al., “Adaptive Methods for the Computation of PageRank,” Technical report, Stanford University, 2003, 15 pages.Applicant
- Kamvar, Sepandar, et al., “The Condition Number of the PageRank Problem,” Technical report, Stanford University, 2003, 4 pages.Applicant
- Kamvar, Sepandar, et al., “The EigenTrust Algorithm for Reputation Management in P2P Networks,” In Proceedings International WWW Conference, Budapest, Hungary, May 20-24, 2003, 12 pages.Applicant
- Kamvar, Sepandar, et al., “Exploiting the Block Structure of the Web for Computing PageRank,” Technical report, Stanford University, 2003, 13 pages.Applicant
- Kamvar, Sepandar, et al., “Extrapolation Methods for Accelerating PageRank Computations,” Technical report, Stanford University, 2003, 10 pages.Applicant
- Moe, Wendy M., et al., “Using Advance Purchase Orders to Forecast New Product Sales,” Marketing department, University of Texas—Austin, Jul. 2002, 33 pages.Applicant
- Montaner, Miguel, “Opinion-Based Filtering Through Trust,” technical paper, Institut d'Informatica I Aplicacions, Agents Research Laboratory, Universitat de Girona Campus Montilivi, 17071 Girona, Spain, 2002, 15 pages.Applicant
- Morgan, Fred, Jr. “Are Early Triers Heavy Users?” The Journal of Business, vol. 52, No. 3 (Jul. 1979), pp. 429-434.Applicant
- Olsen, Stefanie, “Are search engines confusing surfers?,” http://news.com.com/2100-1024-5090701.html, Oct. 13, 2003, 4 pages.Applicant
- Olsen, Stefanie, “AskJeeves denounces paid inclusion,” http://news.com.com/2100-1024—3-5168805.html?tag=nefd—top, Mar. 4, 2004, 4 pages.Applicant
- Olsen, Stefanie, “Searching for the personal touch,” http://news.com.com/2100-1024—3-5061873.html, Aug. 11, 2003, 3 pages.Applicant
- Orlowski, Andrew, “Google to fix blog noise problem,”http://www.theregister.co.uk/content/6/30621.html, Sep. 5, 2003, 4 pages.Applicant
- Pavlov, Dimitry Y., et al., “A Maximum Entropy Approach to Collaborative Filtering in Dynamic, Sparse, High-Dimensional Domains,” In Proceedings of the Sixteenth Annual Conference on Neural Information Processing Systems (NIPS-2002), 8 pages.Applicant
- Pavlov, Dimitry, et al., “Mixtures of Conditional Maximum Entropy Models,” Proceedings of the Twentieth International Conference on Machine Learning (ICLM-2003), Washington, DC, 2003, 8 pages.Applicant
- Ramakrishnan, Naren, et al., “Privacy Risks in Recommender Systems,” http://computer.org/internet/, IEEE Internet Computing, Nov.-Dec. 2001, pp. 54-62.Applicant
- Robinson, Gary, “KnowledgeKnife A White Paper,” Transpose, LLC, [email protected], 2001, 12 pages.Applicant
- Robinson, Gary, “SongSifter,” Disclosure from U.S. Appl. No. 60/345,548, filed Oct. 18, 2001, 14 pages.Applicant
- Solomon, Mary, “Targeting Trendsetters: Cyberstudies capture the perceptions and insights of early adopters” Marketing Research Summer 1996, vol. 8, No. 2.Applicant
- Tang, Tiffany C., et al.,“Scaling Down Candidate Sets Based on the Temporal Feature of Items for Improved Hybrid Recommendations,” Eighteenth International Joint Conference on Artificial Intelligence, Aug. 9-15, 2003—Acapulco, Mexico, 8 pages.Applicant
- http://www.teoma.com/, “Adding a New Dimension to Search: The Teoma Difference is Authority,” Mar. 4, 2004, 6 pages.Applicant
- Vucetic, Slobodan, et al., “A Regression-Based Approach for Scaling-Up Personalized Recommender Systems in E-Commerce,” Workshop on Web Mining for E-Commerce, at the Sixth ACM SIGKDD Int'l Conf. on Knowledge Discovery and Data Mining, Boston, MA, Aug. 2000, 9 pages.Applicant
- http://webword.com/weblog/001628.html by WebWord Mar. 16, 2003.Applicant
- Woelflein, Mike, “Database Heroes,” Mainebiz, Nov. 25, 2002, vol. 8, No. 24, Portland, Maine, 2 pgs.Applicant
- Yu, Kai, et al., “Instance Selection Techniques for Memory-Based Collaborative Filtering,” in: Proc. 2nd SIAM Int. Conf. on Data Mining (SDM '02), http://citeseer.ist.psu.edu/yu02instance.html, 16 pages.Applicant
- Yu, Kai, et al., “Probabilistic Memory-Based Collaborative Filtering,” IEEE Transactions on Knowledge and Data Engineering, found at: http://www.brauer.informatik.tumuenchen.de/˜trespvol/papers/tkde—vu.pdf, Oct. 2004, 13 pages.Applicant
- Zaiane, Osmar, “Database Management Systems,” University of Alberta, CMPUT 391: Information Retrieval and the Web, Winter 2003, 9 pages.Applicant
- Amazon Early Adopters List, http://web.archive.org/web/20030604040602/http://www.amazon.com/exec/obidos/tg/feature/-/423495, Jun. 4, 2003, 17 pages.Applicant
- Amazon Game Boy Advance Product, http://web.archive.org/web/20030629184310/www.amazon.com/exec/obidos/tg/detail/B000087H7T?v=glance, Jun. 29, 2003, 10 pages.Applicant
- Amazon Panasonic Product, http://web.archive.org/web/20030629210353/www.amazon.com/exec/obidos/tg/detail/B00009KXA3?v=glance, Jun. 29, 2003, 6 pages.Applicant
- Wansink, Brian, Sea Bum Park; “Methods and Measures that Profile Heavy Users” Journal of Advertising Research, v40, n4, p. 61 Jul. 2000.Applicant
- Non-Final Office Action dated Mar. 16, 2009, issued in U.S. Appl. No. 10/862,929, filed Jun. 7, 2004, 21 pgs.Applicant
- Non-Final Office Action dated Jan. 8, 2008, issued in U.S. Appl. No. 10/862,929, filed Jun. 7, 2004, 20 pgs.Applicant
- Non-Final Office Action dated Sep. 12, 2006, issued in U.S. Appl. No. 10/862,929, filed Jun. 7, 2004, 21 pgs.Applicant
- Final Office Action dated Jul. 22, 2008, issued in U.S. Appl. No. 10/862,929, filed Jun. 7, 2004, 20pgs.Applicant
- Final Office Action dated Apr. 2, 2007, issued in U.S. Appl. No. 10/862,929, filed Jun. 7, 2004, 19pgs.Applicant
- Non-Final Office Action dated Oct. 29, 2008, issued in U.S. Appl. No. 10/862,930, filed Jun. 7, 2004, 22 pgs.Applicant
- Final Office Action dated May 6, 2009, issued in U.S. Appl. No. 10/862,930, filed Jun. 7, 2004, 24pgs.Applicant
- Non-Final Office Action dated Nov. 12, 2008, issued in U.S. Appl. No. 10/862,931, filed Jun. 7, 2004, 17 pgs.Applicant
- Final Office Action dated May 5, 2009, issued in U.S. Appl. No. 10/862,931, filed Jun. 7, 2004, 28pgs.Applicant
- Non-Final Office Action dated Mar. 4, 2009, issued in U.S. Appl. No. 10/863,642, filed Jun. 7, 2004, 18 pgs.Applicant
- Non-Final Office Action dated Apr. 28, 2009 issued in U.S. Appl. No. 10/863,742, filed Jun. 7, 2004, 32 pgs.Applicant
- Non-Final Office Action dated Mar. 28, 2008 issued in U.S. Appl. No. 10/863,742, filed Jun. 7, 2004, 14 pgs.Applicant
- Non-Final Office Action dated Mar. 4, 2009 issued in U.S. Appl. No. 11/123,415, filed May 6, 2005, 22 pgs.Applicant
- Non-Final Office Action dated Jul. 12, 2007 issued in U.S. Appl. No. 11/23,415, filed May 6, 2005, 14 pgs.Applicant
- Dan Ariely et al.,“Which Intelligent Agents Are Smarter? An Analysis of Relative Performance of Collaborative and Individual Based Recommendation Agents,” unpublished manuscript, Massachusetts Institute of Technology, 2000, 54 pages.Applicant
- John Battelle, “An Open Source Search Engine,” http://searchenginewatch.com/searchday/print.php/34711—3071971, Mar. 21, 2004, 4 pages.Applicant
- H. Berger, et al., “An Accommodation Recommender System based on Associative Networks,” In A. J. Frew, editor, Proceedings of the 11th International Conference on Information Technologies in Tourism (ENTER 2004), Cairo, Egypt, Jan. 26-28, Springer-Verlag, 12 pages.Applicant
- Craig Boutilier et al., “Online queries for collaborative filtering,” believed to be published in Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics (AISTAT-2003), 2003, 8 pages.Applicant
- S. Chinmayee, “All search engines are biased,” http://www.ciol.com/content/news/2003/103081408.asp, Aug. 14, 2003, 3 pages.Applicant
- Dan Cosley et al., “REFEREE: An open framework for practical testing of recommender systems using ResearchIndex,” believed to be published in Very Large Database (VLDB '02) Conference, China, 12 pages.Applicant
- Dan Cosley et al., “Is Seeing Believing? How Recommender Interfaces Affect Users' Opinions,” believed to be published in Proceedings of CHI 2003, 8 pages.Applicant
- James Dana et al., “Revenue Sharing and Vertical Control in the Video Rental Industry,” (with Kathryn Spier), believed to be published in Journal of Industrial Economics, vol. XLIX, No. 3, Sep. 2001, 36 pages.Applicant
- Chrysanthos Dellarocas, “Mechanisms for coping with unfair ratings and discriminatory behavior in online reputation reporting systems,” Proceedings of the twenty-first international conference on Information systems, 2000, pp. 520-525.Applicant
- Ayhan Demiriz, “Analyzing service order data using association mining,” Technical Report, 5 pages.Applicant
- H. Fagrell, “IntraNews: A News Recommending Service for Corporate Intranets,” believed to be published in Proceedings of CSCWD 99, Compenge, France, 7 pages.Applicant
- Jeremy Goecks et al., “NuggetMine: Intelligent groupware for opportunistically sharing information nuggets,” believed to be published in Proceedings of IUI '02, ACM Press, 2002, pp. 87-94.Applicant
- Ken Goldberg et al., “Eigentaste: A Constant Time Collaborative Filtering Algorithm,” believed to be published in UCB Electronics Research Laboratory Technical Report M00/41, University of California, Berkeley, 2000, 11 pages.Applicant
- Stephan ten Hagen, et al., “Exploration/exploitation in adaptive recommender systems,” believed to be published in Proceedings of the third European Symposium on Intelligent Technologies, Hybrid Systems and their implementation on Smart Adaptive Systems, Finland, 2003, 8 pages.Applicant
- Anick Jesdanun, “Hooked on Search: The trusted name in Web trolling,” Associated Press, Mar. 21, 2004, 4 pages.Applicant
- Linda Knapp, “Search internet on your own terms,” http://seattletimes.nwsource.com/html/businesstechnology/2001883991—ptgett20.html, 3 pages.Applicant
- W. S. Lee, “Collaborative learning for recommender systems,” believed to be published in Proc. of The 18th Int'l Conf. on Machine Learning, 2001, 8 pages.Applicant
- Joseph F. McCarthy, “The Virtual World Gets Physical: Perspectives on Personalization,” IEEE Internet Computing 5(6): 48-53 (2001).Applicant
- S. E. Middleton et al., “Capturing knowledge of user preferences: ontologies in recommender systems,” believed to be published in In International Conference on Knowledge Capture (K-Cap '01), pp. 100-107. ACM Press. New York, 2001.Applicant
- Netflix Press Release, “Personalized Movie Recommendor Provides Netflix Visitors With Highly Accurate Film Recommendations Based on Their Individual Movie Taste History,” http://www.netflix.com/Static?id=1019, Jan. 25, 2000, 2 pages.Applicant
- The Nutch Organization, “About,” http://www.nutch.org/docs/en/03/21/2004, 1 page.Applicant
- The Nutch Organization, “FAQ,” http://www.nutch.org/docs/en/faq.html, Mar. 21, 2004,3 pages.Applicant
- The Nutch Organization, “Organization,” http://www.nutch.org/docs/en/org.html, Mar. 21, 2004, 2 pages.Applicant
- The Nutch Organization, “Status,” http://www.nutch.org/docs/en/status.html, Mar. 21, 2004, 1 page.Applicant
- Michael O'Mahony et al., “Collaborative recommendation: A robustness analysis,” believed to be published in ACM Transactions on Internet Technology, 2003, 33 pages.Applicant
- Duncan Pemberton et al., “GroupMark: A WWW Recommender System Combining Collaborative and Information Filtering,” Presented at UI4ALL 2000, Florence, Italy Oct. 25-26, 2000, 13 pages.Applicant
- Bob Price et al., “A Bayesian Approach to Imitation in Reinforcement Learning,” believed to be published in Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI-2003), 6 pages.Applicant
- A. M. Rashid et al., “Getting to know you: Learning new user preferences in recommender systems,” believed to be published in Proceeding of the International Conference on Intelligent User Interfaces, 127-134, 2002.Applicant
- Naren Ramakrishnan et al., “When being Weak is Brave: Privacy Issues in Recommender Systems,” Technical paper posted on the Computing Research Repository at http://xxx.lanl.gov/abs/cs.CG/0105028, May 18, 2001, 12 pages.Applicant
- Paul Resnick, et al., “Recommender systems,” introduction to special section of Communications of the ACM, Mar. 1997, vol. 40(3), 3 pages.Applicant
- J. Ben Schafer, et al., “E-Commerce Recommendation Applications,” Data Mining and Knowledge Discovery 5(1/2): 115-153 (2001).Applicant
- J.B. Schafer, “Chapter 3: a taxonomy for recommender systems,” thesis paper, 31 pages.Applicant
- Rashmi Sinha, http://www.info-arch.org/lists/sigia-1/0301/0524.html, Jan. 28, 2003, 3 pages.Applicant
- Sander Spek, “Personalisation,” Institute for Knowledge and Agent Technology, Feb. 2003, 5 pages.Applicant
- Kirsten Swearingen et al., “Interaction design for recommender systems,” (DIS2002), London, Jun. 25-28, 2002, 10 pages.Applicant
- Y.Z. Wei et al., “Market-Based Recommendations: Design, Simulation and Evaluation,” believed to be published in Proceedings of The Fifth International Workshop on Agent-Oriented Information Systems (AOIS-2003), Melbourne, Australia, 8 pages.Applicant
- David Winer, “Nutch, an opensource search engine,” http://davenet.scripting.com/2003/08/13/nutchAnOpenSourceSearchEngine, Aug. 13, 2003, 4 pages.Applicant
- Ka-Ping Yee et al., “Faceted Metadata for Image Search and Browsing,” believed to be published in Proceedings of the ACM Conference on Computer-Human Interaction, 2003, 8 pages.Applicant
- Jeffrey Zaslow, “If TiVo Thinks You Are Gay, Here's How to Set It Straight,” http://online.wsj.com/article—email/0,,SB1038261936872356908,00.html, Nov. 26, 2002, 4 pages.Applicant
- Chau, Patrick Y.K., and Hui, Kai Lung, “Identifying Early Adopters of new IT products: A case of Windows 95”, Information & Management 33 (1998) 225-230.Applicant
- Daniel Jr., Harold Zane, “The Role of MARKR-Entry Timing in the Evaluation of Manufacturing Technology Innovations”, dissertation, University of Connecticut, 1997, 316 pages.Applicant
- Final Office Action dated Aug. 24, 2009, issued in U.S. Appl. No. 10/862,929, filed Jun. 7, 2004, 14pgs.Applicant
- Notice of Allowance dated Nov. 2, 2009, issued in U.S. Appl. No. 10/862,929, filed Jun. 7, 2004, 14 pgs.Applicant
- Non-Final Office Action dated Nov. 10, 2009, issued in U.S. Appl. No. 10/862,930, filed Jun. 7, 2004, 29 pgs.Applicant
- Final Office Action dated Sep. 16, 2009, issued in U.S. Appl. No. 10/863,642, filed Jun. 7, 2004, 20 pgs.Applicant
- Notice of Allowance dated Aug. 20, 2009 issued in U.S. Appl. No. 11/123,415, filed May 6, 2005, 7 pgs.Applicant