Automatic personal playlist generation with implicit user feedback

[0001] 1. Field of the Invention

[0002] The invention relates to mobile media playback devices. More particularly, the invention relates to the generation of playlists of media content to playback on mobile media playback devices.

[0003] 2. Description of Related Art

[0004] Conventional mobile media playback devices allow users to download and playback media, such as music, videos, pictures and images. Exemplary mobile media playback devices include mobile terminals, personal digital assistants, digital cameras, digital video recorders and combination devices. Typical mobile audio players, for example, generally have a relatively small memory capacity that allow for the storage and playback of a limited number of songs. Users store a music library containing songs on a PC or other computer device having a relatively large memory and download a portion of the songs in the music library to the mobile audio player.

[0005] The manual selection of songs to add to a playlist can be time consuming and tedious. Attempts have been made to automate the playlist selection process. One method includes selecting a random group of songs from the music library. This method can result in the playlist including a large number of songs that are not liked by the user.

[0006] Another prior art method includes generating a playlist that includes songs most liked by the user. This approach can lead to degenerative playlists. The playlists can become dominated by the same songs played over and over again. Some systems rely on the use of metadata to compare attributes of new songs to the attributes of songs that a user has indicated as enjoying. Without the required metadata, such systems do not work.

[0007] These drawbacks are not unique to mobile audio players and also apply to other mobile media playback devices.

[0008] Therefore, there is a need in the art for playlist selection systems and methods that automatically generate lists that include media pieces that a user likes while minimizing repetition and keeping aspects of surprise within the lists.

[0009] One or more of the above-mentioned needs in the art are satisfied by the disclosed playlists selection systems and methods. An adaptive set of media pieces is selected based on activities of a user. A random set of media pieces is also selected. A playlist selection module creates a playlist that includes media pieces from the adaptive set and the random set in a ratio determined by a surprise factor provided by a user. The playlist may also begin with a sure set of media pieces that are known to be enjoyed by the user.

[0010] In a first embodiment, a method of generating a playlist is provided. The method includes assigning individual weights to a plurality of songs based on activities of a user and selecting a plurality of songs from the music library, wherein the probability of each song being selected corresponds to the weight assigned to the song.

[0011] In other embodiments of the invention, computer-executable instructions for implementing the disclosed methods are stored on computer-readable media.

[0019] FIG. 1 illustrates a mobile music system in accordance with an embodiment of the invention. A mobile music device 102 stores and plays songs to a user. Mobile music device 102 may be implemented with an MP3 player, mobile telephone, personal digital assistant or other portable hand-held electronic devices that are capable of storing and reproducing music. Mobile music device 102 includes a song memory 104 for storing songs. Song memory 104 may be implemented with a removable memory module or a stationary memory module.

[0020] Mobile music device 102 may be coupled to a computer device 108, such as a laptop or desktop computer. One skilled in the art will appreciate that computer device 108 may be implemented with several different devices that have processor capacity for generating a playlist and a memory storage. Computer device 108 may include a personal music library database 110 for storing songs. The capacity of personal music library database 110 is generally larger than the capacity of song memory 104. A playlist generation module 106 is used to select a subset of songs from personal music library database 110 to store in song memory 104. In one alternative embodiment of the invention, one or more of the functions of playlist generation module 106 are performed by a module (not shown) within mobile music device 102.

[0021] Computer device 108 may be coupled to a wide area network, such as the Internet 112. Of course numerous databases and music websites, such as music library database 114 are also coupled to the Internet 112. Music library database 114 may store songs that are transmitted to the music library database and/or song memory 104. In one embodiment of the invention, mobile music device 102 is coupled to music library database 114 via the Internet 112 and downloads content directly from the music library database 114. Music library database 114 may also include a playlist generation module for selecting songs to transmit to personal music library database 110 and/or song memory 104.

[0022] In one embodiment, computer device 108 may be implemented with a device that reproduces music. A user may record music with mobile music device 102. Mobile music device 102 may generate a playlist or playlist data and transmit the playlist or playlist data to computer device 108. This particular embodiment allows the user to use mobile music device 102 in order to generate a personal playlist while away from home and have the information transferred to a relatively stationary computer device 108.

[0023] FIG. 1A illustrates an exemplary mobile music device 120 in accordance with an embodiment of the invention. Mobile music device 120 includes a song memory 104. Song memory 104 is described above. A playlist module 122 may be included for recording and and/or maintaining a log of user behaviors that may be used to generate playlists. Exemplary behaviors include recording a song, skipping a song, replaying a song, etc. A CPU 124 may be included to control the overall operation of mobile music device 120.

[0024] Mobile music device 120 may include one or more components for communicating with external devices. A short-range transceiver 126 may be included for communicating with devices such as computer device 108. In one embodiment, short-range transceiver 126 uses the Bluetooth protocol. Ultra Wideband technology may also be used for transferring large files. It should be noted that also other wireless short-range technologies may be used for communicating with other devices. Mobile music device 120 may also include conventional components such as an audio output 128, a network transceiver 130, a display 132 and an antenna 134. Moreover, mobile music device 120 may also include a camera that allows a user to record media pieces in the form of pictures, images and video. Display 132 may be used to playback picture, image and video media pieces. A time module and calendar module may be included to provide inputs during the music selection process.

[0025] FIG. 1B shows an example of a wireless communication system 10 in which the systems and methods of the present invention may be advantageously employed. One or more network-enabled mobile devices 12, such as a personal digital assistant (PDA), digital camera, cellular phone, mobile terminal, or combinations thereof, is in communication with a server 14. Although not shown in FIG. 1B, server 14 may act as a file server for a network such as home network, some other Local Area Network (LAN), or a Wide Area Network (WAN). Server 14 may be a personal computer, a mainframe, a television set-top box, or other device capable of storing and accessing data. Mobile device 12 may communicate with server 14 in a variety of manners. For example, mobile device 12 may communicate with server 14 via wireless network 18. Wireless network 18 may be a third-generation (3G) cellular data communications network, a Global System for Mobile communications network (GSM), or other wireless communication network. Mobile device 12 may also have one or more ports allowing a wired connection to server 14 via, e.g., universal serial bus (USB) cable 15. Mobile device 12 may also be capable of short-range wireless connection 20 (e.g., a BLUETOOTH link) to server 14. A single mobile device 12 may be able to communicate with server 14 in multiple manners.

[0026] Server 14 may act as a repository for storing files received from mobile device 12 and from other sources. Server 14 may have, or be coupled to, a wireless interface 22 configured to transmit and/or receive communications (such as messages, files, or other data) with mobile network 18. Server 14 may alternatively (or also) have one or more other communication network connections. For example, server 14 may be linked (directly or via one or more intermediate networks) to the Internet, to a conventional wired telephone system, or to some other communication network.

[0027] In one embodiment, mobile device 12 has a wireless interface configured to send and/or receive digital wireless communications within wireless network 18. As part of wireless network 18, one or more base stations (not shown) may support digital communications with mobile device 12 while the mobile device is located within the administrative domain of wireless network 18. The base station of wireless network 18 that is in communication with mobile device 12 may be the same or a different base station that is in communication with server 14. Indeed, mobile device 12 and server 14 may each be in communication with different wireless networks (e.g., mobile device 12 could be roaming), which could in turn be interlinked via one or more intermediate wired or wireless networks. For simplicity, server 14 and mobile device 12 are shown within the same wireless network 18.

[0028] Mobile device 12 communicates with server 14 via wireless network 18 and is configured to transmit data (such as, e.g., music content) for remote storage on server 14. Mobile device 12 may also be configured to access data previously stored on server 14. In one embodiment, file transfers between mobile device 12 and server 14 may occur via Short Message Service (SMS) messages and/or Multimedia Messaging Service (MMS) messages transmitted via short message service center (SMSC) 24 and/or a multimedia messaging service center (MMSC) 26. Although shown as part of network 18, SMSC 24 and MMSC 26 may be part of another network or otherwise outside of network 18. Although shown as separate logical entities, SMSC 24 and MMSC 26 could be a single entity. Further, SMSC 24 and MMSC 26 may coordinate via signaling between themselves for improving the file transfer process. For example, because SMSC 24 and MMSC 26 may be store-and-forward systems, rather than real-time systems, a file requested via an SMS message from mobile device 12 may still reside on MMSC 26 based upon a previous request. As such, SMSC 24 may copy MMSC 26 on an SMS file request and, if applicable, MMSC 26 may notify the user of the previously stored file. Further, MMSC 26 may simply transfer the requested file based on its stored copy of the file. In other embodiments, MMSC 26 may act as a repository for files, and mobile device 12 may simply request transfer of files from MMSC 26.

[0029] FIG. 2 illustrates a relationship between sets of songs and a playlist in accordance with an embodiment of the invention. The basic relationship shown in FIG. 2 may be applied to other implementations that include other media pieces, such as songs and video clips. A playlist generation module 202 generates an adaptive set of songs 204, a random set of songs 206, a sure set of songs 208 and receives a surprise parameter 210 and produces a playlist 212. Playlist generation module 202 may also receive adaptive set of songs 204, random set of songs 206 and/or sure set of songs 208 from another source.

[0030] Adaptive set of songs 204 may be selected based on attributes that correspond to activities of a user or other factors such as the time of day and date. Such activities may include recording the songs, repeating the songs and not skipping the songs. Time information may be used, for example, to select a playlist based on whether it is early in the morning or in the afternoon. Date information may be used, for example, to select different music during holidays or weekends than would be selected during a work week.

[0031] Exemplary attributes and values that capture aspects of a user's behavior with respect to a song include:

• • [0032] 1. Recorded: boolean. Is the song recorded by the user (vs. automatically by the system)?
  • [0033] 2. Skipped: integer>=0. Approximate number of times the song has been skipped by the user during replay. A song may be considered skipped if the user presses a “next” button during the replay of the song and the song has been playing for less than a predetermined period of time, such as 1 minute. In one embodiment, a song is not considered skipped if a “previous” button was used to reach the song.
  • [0034] 3. Repeated: integer>=0. Number of songs replayed since the song was last repeated by the user. A song may be considered repeated when a user presses “previous” button to reach the song and listens to it more than, say, 30 seconds. A user might be looking for a song replayed some time ago, and during the search for the song he listens to the beginnings of other songs. When counting the number of songs replayed since last repeat, also skipped and repeated songs count as replayed.
  • [0035] 4. Age: integer>=0. Number of days since the song was recorded (by the user or by the system). In alternative embodiments different time frames may be used.
  • [0036] 5. Last_played: integer>=0. Number of songs replayed since the last replay of this song. One advantage of using the number of songs is that the choice adapts to the user's rate of listening to a playlist. Alternatively Last_played could be the number of days since the last replay of this song.
  • [0037] An attribute of a given song, such as repeated, may be denoted by song.repeated.

[0038] In one embodiment, adaptive set of songs 204 is more likely to include recently recorded songs than older recordings. Random set of songs 206 may include a set of songs that are not necessarily preferred by the user. Random set of songs 206 may be used to add variety to the ultimate playlist. Sure set of songs may include songs that are known to be liked by the user. The user may indicate which songs to include in sure set of songs 208. In an alternative embodiment, one or more activities of the user, such as recording a song and repeating a song, may be used in the selection of sure set of songs 208. Surprise parameter 210 may be supplied by the user and may determine the percentage of songs included in the playlist that are from random set of songs 206. Surprise parameter may have a value of 0% to 100% or may comprise other values such as high, medium and low.

[0039] In one embodiment of the invention, playlist 212 includes a first group of songs selected from sure set of songs 208. These songs may be placed at the beginning of the playlist to ensure a good user experience. Next, playlist 212 includes songs selected both from adaptive set of songs 204 and random set of songs 206 with a ratio determined by surprise parameter 210. Once playlist 212 is generated, the corresponding music files are downloaded from personal music library database 110, or alternatively from music library database 114 over the Internet 112.

[0040] Playlist 212 may be created separately for each station or genre that the user listens to. For example, a first genre of music may include jazz songs and the songs comprising playlist 212 may be selected from a particular jazz station. Time, place and other factors may influence the content of playlist 212.

[0041] FIG. 3 illustrates a method of generating a playlist in accordance with an embodiment of the invention. First, in step 302, individual weights are assigned to a plurality of songs based on activities of a user. The weights may be used to select an adaptive set of songs. A first weight w_1a(song) may be selected to favor recorded songs, such that:
w_1a(song)=1 if song.recorded=true
w_1a(song)=0 otherwise.

[0042] A second weight w_1b(song) may be selected to favor songs that the user has repeated over other songs. Recently repeated songs may also be favored over older songs. For example, w_1b(song) may be used to define how much more likely recently repeated songs are to select than songs that have never been repeated by defining w_1b(song) as follows:
w_1b(song)=1+2^{−song.repeated/h}_1b*max_weight_1b.

• • [0043] In one embodiment, the following values are used as default choices.
  • [0044] max_weight_1b=3
  • [0045] h_1b=20

[0046] For a song that has never been repeated, w_1b(song) may be defined to equal 1. When the last song that was replayed was also repeated, w_1b(song)=1+max_weight_1b. The weight (actually weight−1) halves every h_1b days and approaches 1 for songs last repeated a long time ago.

[0047] A third weight w_1c(song) may be selected to favor songs that have not been skipped by the user over songs that have been skipped. In one embodiment, songs that have never been skipped may have a have a w_1c(song) equal to one. Songs skipped and not recorded by the user may have a w_1c(song) equal to zero and songs recorded by the user may have a w_1c(song) that quickly decreases. For example:
If song.skipped=0 then w_1c(song)=1
If song.recorded=true then w_1c(song)=2^{−song.skipped/h1c}
If song.recorded=false and song.skipped>0 then w_1c(song)=0

• • [0048] In one embodiment, h_1c=0.5.

[0049] A fourth weight w_1d(song) may be selected to favor recently recorded songs. In one embodiment, w_1d(song) may be equal to one for sounds recorded on the day that the weight is used and w_1d(song) may be equal to 2^{−song.age/h1d} for songs recorded at other times. The value hid may be used to represent a half-life of the weight. For example, an h_1d=10 implies only very recent songs are favored and an h_1d=1000 implies older songs have almost equal weights. A suitable default value for h_1d is 100.

[0050] A fifth weight w_3a(song) may be selected to favor songs that have not been replayed recently over songs that have been replayed recently. For recently played songs w_3a(song) may be very small and w_3a(song) may approach 1 for songs that have not been played in a long time. For example:
w_3a(song)=1, if song has never been replayed
w_3a(song)=1/(1+steep^{−song.last—played+h3a}), if the song has been replayed
where “steep” is a parameter that governs how steep or sharp the division to recent and old songs is and h_3a is the half life and governs where the division takes place. A suitable default value for steep is 1.2 and for h_3a is 30.

[0051] In step 304, a plurality of songs are selected from the music library, wherein the probability of each song being selected corresponds to the weight assigned to the song. In one embodiment, the weight assigned to each song w_adapt(song) is equal to the product of the five weights described above. The probability of a given song being selected Pr_adapt(song) is proportional to its weight w_adapt(song). For example:
Pr_adapt(song)=w_adapt(song)/Σ_iw_adapt(song_i).

[0052] In step 306, a random group of songs is selected from the music library. In one embodiment, the random group of songs may be selected based on random weights assigned to each song and the random weight of each song w_rand(song) may be defined as follows:
w_rand(song)=w_1c(song)*w_3a(song).

[0053] In step 308, a surprise parameter may be received. The surprise parameter may be received from a user or selected by a playlist generation module. The surprise parameter may have a value of 0-100% and may represent the fraction of songs selected without regard to weights w1a(song), w1b(song) and w1d(song). A suitable default value for surprise factor is 20%.

[0054] Finally, in step 310, a playlist is generated that includes songs selected in step 304 and songs selected in step 306 with a ratio that corresponds to the surprise parameter. After the songs in the playlist are replayed, the songs may be reordered to provide variety.

[0055] As mentioned above, embodiments of the invention may use media pieces other than music or songs. In particular, the playlist selection methods disclosed herein may be used to generate playlists of images, pictures, video clips and other visual and/or audio pieces. FIG. 4 illustrates a mobile media playback system that is similar to the mobile music system shown in FIG. 1. A mobile media playback device 402 stores and plays back media to a user. For example, mobile media playback device 402 may be a personal digital assistant that plays back images to a user. Mobile music device 402 includes a media piece memory 404 for storing media pieces. Media piece memory 404 may be implemented with a removable memory module or a stationary memory module.

[0056] Mobile music device 402 may be coupled to a computer device 408 that includes a personal media piece library database 410 for storing media pieces and a playlist generation module 406. Computer device 408 may be similar to computer device 108 (shown in FIG. 1). Computer device 408 may be coupled to a wide area network, such as the Internet 412. Numerous databases and media websites, such as media piece library database 414 may also be coupled to the Internet 412. The operation of the elements shown in FIG. 4 is substantially similar to the operation of the elements shown in FIG. 4.

[0057] Aspects of the resent invention may also be applied to other embodiments that do not include mobile devices. For example, the playlist selection methods disclosed herein may be used to generate a playlist of images to display on a wall mounted display devices, such as a plasma television or liquid crystal television.

[0058] While the invention has been described with respect to specific examples including presently preferred modes of carrying out the invention, those skilled in the art will appreciate that there are numerous variations and permutations of the above described systems and techniques that fall within the spirit and scope of the invention as set forth in the appended claims.

[0059] The following code written in Perl is an exemplary algorithm that may be used by a playlist generation module to select a playlist.

• • [0060] Variables used (and not introduced in the algorithm):
    • [0061] music_library_size: total number of songs in the music library
    • [0062] recorded by_the_user: number of songs recorded by the user
    • [0063] first_songs: number of first songs to be selected as “sure”
    • [0064] surprise_ratio: surprise_ratio
    • [0065] last_played[ ]: array of last_played attributes of songs
  • [0066] For simplicity, in the algorithm below songs are identified by an index between 0 and music_library_size−1, and the first recorded_by_the_user songs are the ones recorded by the user.

[0067]

# 1. init playlist
my @playlist = ( ); # playlist to be generated
my $playlist_filled = 0; # number of songs in playlist so far
my $users_songs = 0; # number of user recorded songs in playlist so far
# 2. find out largest weights (normalization factors)
my $max_w_sure = 0;
my $max_w_adapt = 0;
my $max_w_rand = 0;
for ($s = 0; $s < $music_library_size; $s++) {
$max_w_sure = (w_sure($s) > $max_w_sure ? w_sure($s) : $max_w_sure);
$max_w_adapt = (w_adapt($s) > $max_w_adapt ? w_adapt($s) : $max_w_adapt);
$max_w_rand = (w_rand($s) > $max_w_rand ? w_rand($s) : $max_w_rand);
}
# 3. create a backup copy of last_played
my @last_played_copy;
for ($s = 0; $s < $music_library_size; $s++) {
$last_played_copy[$s] = $last_played[$s];
}
# 4. while playlist has room repeat
while (($playlist_filled < $playlist_size)
($playlist_filled < $music_library_size)) {
# 4.1. decide how to select a song and then search for one
my $random_song;
if ($playlist_filled < $first_songs) {
# Make a sure selection
# (Using rejection sampling: propose a random song (1), reject it
# stochastically (2) depending on the relative probability of the
# song. Repeat this until a proposed song is accepted.)
do {
$random_song = int(rand($recorded_by_the_user));    # (1)
} while (rand( ) > w_sure($random_song)/$max_w_sure);   # (2)
} elseif ((rand( ) > $surprise_ratio/100) &&
($users_songs < $recorded_by_the_user)) {
# Select a song recorded by the user
do {
$random_song = int(rand($recorded_by_the_user));
} while (rand( ) > w_adapt($random_song)/$max_w_adapt);
$users_songs++;
} else {
# Select a random song
do {
$random_song = $recorded_by_the_user +
int(rand($music_library_size − $recorded_by_the_user));
} while (rand( ) > w_rand($random_song)/$max_w_rand);
}
# 4.2. add the song to the playlist
$playlist[$playlist_filled++] = $random_song;
# 4.3. update last_played fields
for ($s = 0; $s < $music_library_size; $s++) {
$last_played[$s]++;
}
$last_played[$random_song] = 0;
# 4.4. find out new largest weights
my $max_w_sure = 0;
my $max_w_adapt = 0;
my $max_w_rand = 0;
for ($s = 0; $s < $music_library_size; $s++) {
$max_w_sure = (w_sure($s) > $max_w_sure ? w_sure($s) : $max_w_sure);
$max_w_adapt = (w_adapt($s) > $max_w_adapt ? w_adapt($s) : $max_w_adapt);
$max_w_rand = (w_rand($s) > $max_w_rand ? w_rand($s) : $max_w_rand);
}
}
# 5. restore last_played to its original value
for ($s = 0; $s < $music_library_size; $s++) {
$last_played[$s] = $last_played_copy[$s];
}