SOCIAL REMINDERS

[0001] This application claims priority from U.S. Provisional Ser. No. 62/057,996, filed on Sep. 30, 2014, entitled SOCIAL REMINDERS, which is hereby incorporated by reference in its entirety for all purposes.

[0002] The present disclosure relates generally to computer-based reminders, and more specifically to providing reminders based on social interactions between users.

[0003] Electronic devices can display reminder alerts that remind users to take certain actions at certain times. For example, a meeting reminder reminds a user to attend a scheduled meeting. A conventional reminder is created by specifying a date and/or time with a reminder message. The reminder may then be displayed at the specified date and/or time. Conventional reminders are not as helpful when it is infeasible for the user to specify a date and/or time ahead of time.

[0004] Techniques for providing (e.g., creating and outputting) reminders that are based on social interactions between users are described below. These types of reminders may be referred to as social reminders. Social reminders need not be created based on a pre-determined date and time; for example, a social reminder can be set to display when the user comes within range of another user (e.g., when the devices of both users come within a threshold range).

[0005] In some embodiments, a method for managing social reminders at an electronic device includes: receiving input representing user instruction to provide a reminder in the future, the instruction identifying an entity; after receiving the input, detecting an external electronic device, the external electronic device associated with the entity; in response to detecting the external electronic device, determining whether the external electronic device is within a threshold proximity; and in accordance with a determination that the external electronic device is within the threshold proximity, providing the reminder.

[0011] In the following description, reference is made to the accompanying drawings in which it is shown by way of illustration specific examples that can be practiced. It is to be understood that other examples can be used and structural changes can be made without departing from the scope of the various examples.

[0012] Attention is directed to techniques for providing reminders based on social interactions between users of electronic devices. These reminders may be referred to as social reminders. User instructions to create social reminder may be received through graphical user interface input and/or voice input. The instruction includes a social setting in which the electronic device should provide the reminder. The condition can be, for example, physical proximity with another person. For instance, an exemplary social reminder can be created in response to the instruction “create a reminder reminding me to ask Joe about the promised documents, the next time I see him.” In this way, electronic devices can provide reminders based on social context, without relying on the a priori input of a specific date and/or time at which the reminder should be displayed. Social reminders are particularly helpful when it is infeasible to predict when the user will next see Joe.

[0013] FIG. 1 illustrates exemplary system 100 for providing social reminders in some embodiments. Exemplary system 100 includes user device 102 (or multiple user devices 102) that can provide a user input interface or environment. User device 102 can include any of a variety of devices, such as a cellular telephone (e.g., smartphone), tablet computer, laptop computer, desktop computer, portable media player, wearable digital device (e.g., digital glasses, wristband, wristwatch, brooch, armbands, etc.), television, set top box (e.g., cable box, video player, video streaming device, etc.), gaming system, or the like. User device 102 can have display 112. Display 112 can be any of a variety of displays, and can also include a touchscreen, buttons, or other interactive elements. In some embodiments, display 112 is incorporated within user device 102 (e.g., as in a touchscreen, integrated display, etc.). In some embodiments, display 112 is external to—but communicatively coupled to—user device 102 (e.g., as in a television, external monitor, projector, etc.).

[0014] User device 102 can include or be communicatively coupled to human interface device (HID) 114, which can capture user-input (e.g., text input, touch input, voice input, etc.). In some embodiments, HID 114 is a keyboard. For example, HID 114 can be a virtual keyboard on a touchscreen capable of receiving text entry from a user (e.g., detecting character selections from touch). In another example, HID 114 is a virtual keyboard shown on a display (e.g., display 112), and a pointer or other indicator is used to indicate character selection (e.g., indicating character selection using a mouse, remote control, pointer, button, gesture, eye tracker, etc.). In some embodiments, HID 114 is a handwriting recognition device. For example, HID 114 can include a touch sensitive device capable of recognizing handwritten characters. In some embodiments, HID 114 is a voice input device, for example a microphone.

[0015] User device 102 can also include processor 104, which can receive text entry from a user (e.g., from HID 114) and interact with other elements of user device 102 as shown. In one example, processor 104 can be configured to perform any of the methods discussed herein, such as creating and outputting social reminders. In other examples, processor 104 can cause data (e.g., entered text, user data, etc.) to be transmitted to server system 118 through network 116. Network 116 can include any of a variety of networks, such as a cellular telephone network, WiFi network, wide area network, local area network, the Internet, or the like. Server system 118 can include a server, storage devices, databases, and the like and can be used in conjunction with processor 104 to perform any of the methods discussed herein. For example, processor 104 can cause an interface to be provided to a user for create a reminder and can transmit some or all of the user input to server system 118, and can cause reminders to be displayed on display 112.

[0016] In some embodiments, user device 102 includes storage device 106, memory 108, reminders database 110. Reminders database 110 may be stored on storage device 106. The functions or methods discussed herein can be performed by a system similar or identical to system 100. It should be appreciated that system 100 can include instructions stored in a non-transitory computer readable storage medium, such as memory 108 or storage device 106, and executed by processor 104. The instructions can also be stored and/or transported within any non-transitory computer readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a “non-transitory computer readable storage medium” can be any medium that can contain or store the program for use by or in connection with the instruction execution system, apparatus, or device. The non-transitory computer readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, a portable computer diskette (magnetic), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable optical disc such as CD, CD-R, CD-RW, DVD, DVD-R, or DVD-RW, or flash memory such as compact flash cards, secured digital cards, USB memory devices, memory sticks, and the like.

[0017] It should be understood that system 100 is not limited to the components and configuration of FIG. 1, but can include other or additional components in multiple configurations according to various embodiments. For example, user device 102 can include a variety of other mechanisms for receiving input from a user, such as an optical sensor, camera, gesture recognition sensor, proximity sensor, ambient light sensor, or the like. Additionally, the components of system 100 can be included within a single device, or can be distributed among multiple devices. For example, although FIG. 1 illustrates reminders database 110 as part of user device 102, it should be appreciated that, in other embodiments, one or more parts of database 110 can be stored remotely as part of a remote storage system such as server system 118.

[0018] Attention is now directed to an exemplary provision of social reminders, with reference to FIGS. 2A-2C. As shown FIG. 2A, electronic device 200 receives input 204, from user 202, instructing the device to create a reminder. In some embodiments, device 200 is user device 102 (FIG. 1). Notably, the user's instructions 204 to create a reminder do not provide a specific time or date for the reminder to occur; rather, the instructions identify a social context in which the reminder should be triggered: namely, the device is to provide a reminder when user 202 meets someone named “Joe.”

[0019] In the illustrated embodiment, user input 204 to electronic device 200 is provided verbally. U.S. Non-Provisional patent application Ser. No. 12/987,982, entitled “Intelligent Automated Assistant,” filed Jan. 10, 2011, describes techniques for accepting verbal commands. The application is hereby incorporated-by-reference in its entirety for all purposes. In some embodiments, other types of input modalities, such as touchscreen input, are accepted for purposes of managing social reminders.

[0020] Also, even though user instructions 204 refer to social interactions with a person named Joe (who may be a phonebook contact), the underlying computing processes carried out by device 200 may rely on (among other things) interactions with electronic device(s) that are associated with and Joe. More specifically, in some embodiments, device 200 considers Joe to be present when an electronic device associated with Joe is detected. Restated, the presence of Joe's electronic device(s) is taken as a proxy for Joe's presence. In some embodiments, device 200 considers Joe to be present when actual detection of Joe's presence is made, such as through voiceprint identification as received through a microphone of device 200.

[0021] Turning to FIG. 2B, user 202 encounters Joe (user 212) later in the day at the company cafeteria. The encounter may be a chance encounter, meaning that user 202 did not have prior knowledge of Joe's whereabouts. As device 200 (of user 202) and Joe's electronic device 210 come into proximity, device 200 is able to detect and identify electronic device 210 as being associated with Joe. This determination may be made in various ways. For example, device 200 may recognize Joe as a phonebook contact who is associated with a cellular telephone number and/or an e-mail address. Device 210 may transmit (e.g., broadcast or other communicate) an identification token to device 200 that permits device 200 to determine that device 210 belongs to the phonebook contact known as Joe. In some embodiments, external electronic device 210 is user device 102 (FIG. 1), and the relevant communication between devices 200 and 210 occurs over one or more communications modalities such as Bluetooth Low Energy and/or WiFi.

[0022] Turning to FIG. 2C, as device 200 detects Joe's device 210, electronic device 200 displays reminder 206. Reminder 206 reminds user 200 to ask Joe about an upcoming meeting, in accordance with user instructions 204 depicted in FIG. 2A. In this way, electronic device 200 provides user 202 with a relevant reminder, based on the social context of user 202 and Joe (user 212) being physically proximate.

[0023] Attention is now directed to another exemplary provision of social reminders, with reference to FIGS. 3A-3C. As shown in FIG. 3A, electronic device 300 receives input 304 from user 302 instructing the device to remind user 302 and his daughter about bringing gym clothes to a gym event. In some embodiments, device 300 is device 102 (FIG. 1). As shown in FIG. 3B, when user 302 encounters external electronic device 310—which is associated with his daughter (user 312)—electronic device 300 performs two actions: First, electronic device 300 displays social reminder 306 alerting user 302 to remind his daughter about bringing gym clothes. Second, electronic device 300 transmits instructions to his daughter's electronic device 310, instructing device 310 to create a reminder. As depicted in FIG. 3C, this additional reminder 314 is displayed by the daughter's electronic device 310 on Friday, alerting her to bring her gym clothes. In this way, electronic device 300 is able to provide user 302 with a relevant reminder based on the context of meeting his daughter. In addition, electronic device 300 enables the daughter's electronic device 310 to provide a relevant reminder on the day of the relevant event.

[0024] Attention is now directed to FIG. 4, which illustrates exemplary process 400 for providing social reminders. In some embodiments, process 400 is performed by electronic devices 200 and 300 to carry out the social reminder functionalities discussed above with reference to FIGS. 2-3.

[0025] At block 402 of process 400, input representing user instructions to provide a reminder in the future is received. The instructions identify a social context in which the reminder should be triggered. In some embodiments the instruction specifies a social context involving an entity. As used herein, an “entity” refers to a person or thing. In the above-described examples of FIGS. 2-3, the social context is physical proximity with a person, such as Joe (user 212). In another example, the instruction may be to “remind me to disconnect and take my AppleTV when I am near it.” In this instance, the entity is the AppleTV device. Optionally, in response to receiving the instruction, a record representing the reminder is stored into a database such as reminder database 110 (FIG. 1).

[0026] At block 404 of process 400, the electronic device that is performing process 400 detects an external electronic device associated with the entity specified at block 402. In some embodiments, the detection is made wirelessly via a direct communication link between the two devices, such as via a Bluetooth, Bluetooth Low Energy, ad hoc WiFi connection, and the like. In some embodiments, the detection is made over a shared network, such as a local area network having one or more WiFi access points. In some embodiments, the electronic device that is performing process 400 recognizes an external electronic device as being associated with a specified entity based on contacts information. For instance, in the example of FIGS. 2A-2C, electronic device 200 may recognize “Joe” as a phonebook contact, possibly even a user-designated (e.g., favorite) contact. In that instance, the phonebook provides information such as telephone number and/or e-mail address usable to identify those devices of the contact. In some embodiments, association between an entity and a device is determined based on information being broadcasted by the device(s). For instance, a phone or network-enabled television set-top device may broadcast information usable to determine its identity, such as by broadcasting a universally unique identifier (UUID) or other suitable token.

[0027] At block 406, responsive to detecting the external electronic device, a determination is made as to whether the external electronic device is within a threshold proximity. As used herein, “threshold proximity” refers to physical proximity. In some embodiments, physical proximity is determined based on distance between devices. Distance can be measured in various ways. In some examples, distance is measured by having the external electronic device emit sound at a certain loudness (e.g., sound pressure level, or SPL), and detecting how loudly (e.g., in terms of SPL) the emitted sound is received at the location of the electronic device, in order to determine the distance based on attenuation of the sound. In some examples, distance is measured by comparing GPS output as determined by two relevant devices. In some examples, distance is measured by obtaining location information from a location service, such as a location beacon and comparing the obtained location information against that obtained by other devices. Distances of one to twenty feet is generally useful as thresholds for deciding whether two people are physically proximate for purposes of social reminders.

[0028] In some embodiments, physical proximity is determined based on the ability of devices to communicate via limited-range communication modalities. For example, the Bluetooth Low Energy (BTLE) communication technology is commonly accepted as having a range of about 30 inches in some operational settings, and the Near Field Communication (NFC) technology is commonly accepted as having a range of about 5 inches in some operational settings. Two devices that are in communication via the BTLE or NFC protocols can be considered to be proximate for purposes of social reminders based on these understood ranges.

[0029] In some embodiments, physical proximity between two devices is determined by comparing environmental conditions as perceived by the devices. For example, both the electronic device (that is performing process 400) and the external device can record their respective microphone inputs, and the recordings are then compared to determine if a common sound source is within the microphone range of both devices. In another example, both the electronic device (that is performing process 400) and an external device can capture camera input, and the captured visual information are then compared to determine if a common object is present in the field of view of both devices. In these examples, the presence of common element(s) in the environment can be interpreted as indicators of physical proximity between the devices.

[0030] At block 408, in accordance with a determination that the external electronic device is within the threshold proximity, the reminder is provided. In some embodiments, providing the reminder involves displaying a visual message alerting the user to the reminder, such as the display of reminder 206 (FIG. 2C). In some embodiments, providing the reminder involves audibly informing the user of the reminder using a speaker of the electronic device. In some embodiments, providing the reminder involves providing a haptic output. Alternatively, in accordance with a determination that the external electronic device is not within the threshold proximity, processing returns to block 406 where the device awaits a more appropriate time for providing the reminder.

[0031] Optionally, at block 408, an instruction is sent, from the electronic device that is performing process 400 to an external electronic device, instructing the external device to create a reminder. In some embodiments, this additional reminder is related to the initial reminder (displayed at block 408) in that the two reminders at least partially overlap in content. For example, additional reminder 314 shown in FIG. 3C is a follow-on reminder to reminder 306 shown in FIG. 3B, and both reminders stemmed from user input 304 (FIG. 3A).

[0032] Attention is now directed to additional, e.g., optional, aspects of social reminders. As discussed above, a social reminder may be created when a user identifies an entity (among other things) by name. The examples of FIGS. 2A-2C are directed to this approach in that user 202 identifies user 212 as the subject of reminder 206 by the proper name of “Joe.” In addition to this approach, in some embodiments, an entity that is associated with a reminder can be identified by profile information instead of by name. As used herein, “profile information” is information describing an entity other than the proper name (e.g., proper noun) of the entity. For example, profile information can indicate whether a user is a soccer fan. Put another way, it is possible to create a social reminder whereby an electronic device reminds its user “to refrain from mentioning the current score when I next encounter a soccer fan”.

[0033] In some embodiments, as part of detecting the presence of an external electronic device, at least two communication modalities are utilized. Consider, for instance, that in order to reduce power consumption, an electronic device may forego detection using a second (subsequent) communication modality until a first (initial) communication modality has detected a relevant electronic device. In some examples, an electronic device uses GPS information to initially determine if it is near an external electronic device. When the GPS information indicates proximity, the electronic device additionally powers up a NFC wireless communication module requesting further communication with the external electronic device. Restated, the NFC detection modality may remain (partially if not entirely) powered down until GPS information indicates that a likelihood that a nearby device is within NFC communication range.

[0034] In some embodiments, the first communication modality is predetermined. For example, when social reminders are provided using a cellular phone, the first communication modality may be a cellular-based modality because cellular communication circuitry is very likely to be consistently powered-on by the cellular phone. In another example the first communication modality can be Bluetooth or Bluetooth Low Energy as these protocols are generally accepted as having low power requirements. In some embodiments, the first communication modality is a changeable default modality. For example, when social reminders are provided using a smartphone having multiple communication capacities, the first communication modality may be cellular- or WiFi-based depending on whether the smartphone is in the airplane mode.

[0035] In some embodiments, the first and second communication modalities utilize application data objects to determining whether an external electronic device is likely nearby. The application data objects can be calendar data objects, for example, meaning that the electronic device can parse calendar entries to determine if a user of the electronic device is scheduled to meet with a particular contact at the current time. A finding that the users are scheduled to meet can be interpreted as increasing the likelihood of the two users being nearby. Use of the other data objects, such as emails and text messages, are also possible.

[0036] In some embodiments, the first and second communication modalities include a modality that relies upon comparing the Service Set Identifier (SSID) of a network connected to the electronic device against the SSID of a network connected to an external electronic device to determine if the two devices are connected to the same wireless network and thus likely proximate to one another.

[0037] In some embodiments, the first and second communication modalities include a modality that relies upon comparing the cell identification of a cellular phone network connected to the electronic device against the cell identification of a cellular phone network connected to an external electronic device to determine if the two devices are in the same cell and thus likely proximate to one another.

[0038] In some embodiments, the first and second communication modalities include a modality that relies on recorded sound. For example, an electronic device can record ambient sound using its microphone and determine if the recording contains an identifiable voice of a known contact. Voice identification (e.g., voiceprints) may be stored with or otherwise associated with phonebook contacts that are accessible to the electronic device. As another example, an external electronic device can emit an ultrasound signal that registers on the microphone input of an electronic device (if the two devices are sufficiently proximate) and be used by the electronic device to identify the external electronic device. Sound frequencies that are above generally-accepted human hearing thresholds at reasonable SPLs, such as frequencies higher than 15 kHz, can be used. Preferably, frequencies that are producible by mass-produced speakers and detectible by mass-produced microphones, yet outside of the human-hearing threshold at those producible SPLs, are employed (e.g., 15-20 kHz).

[0039] In some embodiments, the first and second communication modalities include a modality that relies on recorded images. For example, an electronic device can record images and/or video using its camera and determine whether the recorded imagery includes an identifiable face. Face recognition data, including data useful for SIFT-based image comparison technologies, may be associated with phonebook contacts accessible to the electronic device.

[0040] In some embodiments, the providing of a reminder includes sending a data object to the external electronic device. For example, a document or a contact vCard can be sent from the electronic device to the external electronic device. In some embodiments, the providing of a reminder includes activating a feature on the electronic device and/or the external electronic device. For example, both the electronic device and the external electronic device can begin to play the same song to support a music flash mob.

[0041] FIG. 5 shows a functional block diagram of exemplary electronic device 500 configured in accordance with the principles of the various described examples. The functional blocks of device 500 can be implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described examples, including those described with reference to process 400 of FIG. 4. As shown, device 500 includes display unit 502. Display unit 502 is configured to display interfaces for providing social reminders. User input interface 504 is configured to receive user input including instructions to create reminders. In some embodiments, user input interface 504 is integrated with display unit 502 to form a touchscreen for receiving user input via a virtual keyboard. In some embodiments, user input interface 504 includes a microphone for receiving verbal instructions from a user. Electronic device 500 further includes RF unit 506 which may have circuitry for communicating with external electronic devices using one or more communication modalities, such as cellular, WiFi, Bluetooth, Bluetooth Low Energy, NFC communications, and the like.

[0042] Display unit 502, user input interface 504, and RF unit 506 are coupled to processing unit 508 which includes an external device detection unit 510 and reminder unit 512. Processing unit 508 can be configured to receive input from a user such as spoken input described with reference to FIGS. 2-3. Upon receiving user instructions to create a reminder, reminder unit 512 may register into a storage location (such as reminders database 110 depicted in FIG. 1) a corresponding record of reminder. Reminder unit 512 is also configured to trigger an output of the reminder at the appropriate time, e.g., by displaying a visual reminder on display unit 502.

[0043] External device detection unit 510 is configured to aid in the determining when a reminder should be triggered. External device detection unit 510 may receive communication data from RF unit 506 indicating the presence of the nearby devices. External device detection unit 510 may be configured to determine whether an external device is physically proximate, using the techniques discussed above with reference to FIG. 4 (e.g., block 404 and 406). For example, external device detection unit 510 may be configured to power up a second communication modality (for obtaining a finer determination of the proximity a nearby device) only after a first communication modality has identified the relevant device as being likely nearby. Based on information obtained through one or more communication modalities, external device detection unit 510 can indicate to reminder unit 512 that a relevant external electronic device is physically proximate.

[0044] Based in information received from external device detection unit 510, reminder unit 512 may cause device 500 to provide an indication of the reminder (e.g., by displaying a visual reminder on display unit 502, by providing a haptic feedback, by providing audible output through a speaker (not shown) of device 500, so forth). In some embodiments, reminder unit 512 may also cause device 500 to send an instruction to the detected external electronic device to create a corresponding reminder, such as reminder 313 described with respect to FIGS. 3A-3C.

[0045] It is understood by persons of skill in the art that the functional blocks described in FIG. 5 can be combined or separated into sub-blocks to implement the principles of the various described examples. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

[0046] Although examples have been fully described with reference to the accompanying drawings, it is to be noted that various changes and modifications will become apparent to those skilled in the art (e.g., modifying any of the systems or processes discussed herein according to the concepts described in relation to any other system or process discussed herein). Such changes and modifications are to be understood as being included within the scope of the various examples as defined by the appended claims.