BULK INITIAL DOWNLOAD OF MOBILE DATABASES

[0001] The present application claims the benefit of U.S. Provisional Patent Application No. 61/425,554, filed Dec. 21, 2010 and entitled, “Bulk Initial Download of Mobile Databases”, which is incorporated herein by reference in its entirety.

[0002] The present application is also related to co-pending U.S. patent application Ser. No. 12/813,104, filed on Jun. 10, 2010 and entitled “Message Based Synchronization for Mobile Business Objects”, which is incorporated herein by reference in its entirety.

[0003] 1. Field of the Invention

[0004] The present invention relates generally to databases and, in particular, to database synchronization.

[0005] 2. Description of the Background Art

[0006] Relative to a server-class computer system, a mobile device has a relatively slow CPU and relatively slow input/output performance (e.g., for reading/writing database files on media cards). A mobile device also has a relatively small amount of memory (e.g., RAM), which limits the effectiveness of any kind of in-memory database cache, such as a row cache or page cache.

[0007] As a result, the time taken for a mobile device to synchronize an initial database from a server can be excessive. CPU and I/O utilization can reach capacity during this time, rendering the mobile device unresponsive for the duration of the initial download. Initial synchronization of a 25 Mb database from a server to a mobile device common in use in 2010 can take upwards of 48 hours, during which time the device is rendered barely usable for its normal functions, such as for placing or receiving telephone calls.

[0008] Accordingly, what is desired is a mechanism to synchronize an initial database without excessive resource utilization by the mobile device.

[0009] Embodiments of the invention include a method comprising receiving a subscription request from a client, impersonating the client to generate a temporary database file based on the subscription request, and transmitting the temporary database file to the client.

[0010] Embodiments of the invention also include a computer-readable storage device having computer-executable instructions stored thereon that, if executed by a computing device, cause the computing device to perform operations comprising receiving a subscription request from a client, impersonating the client to generate a temporary database file based on the subscription request, and transmitting the temporary database file to the client.

[0011] Embodiments of the invention also include a system comprising a memory configured to store modules comprising a receiving module configured to receive a subscription request from a client, an impersonating module configured to impersonate the client to generate a temporary database file based on the subscription request, and

[0012] a transmitting module configured to transmit the temporary database file to the client, and one or more processors configured to process the modules.

[0013] Embodiments of the invention also include a method comprising transmitting a subscription request to a server, receiving a page of a database file based on the subscription request, encrypting the page of the database file, and writing the encrypted page of the database file to a storage device.

[0014] Embodiments of the invention also include a computer-readable storage device having computer-executable instructions stored thereon that, if executed by a computing device, cause the computing device to perform operations comprising transmitting a subscription request to a server, receiving a page of a database file based on the subscription request, encrypting the page of the database file, and writing the encrypted page of the database file to a storage device.

[0015] Embodiments of the invention also include a system comprising a memory configured to store modules comprising a transmitting module configured to transmit a subscription request to a server, a receiving module configured to receive a page of a database file based on the subscription request, an encrypting module configured to encrypt the page of the database file, and a writing module configured to write the encrypted page of the database file to a storage device, and one or more processors configured to process the modules.

[0016] Further features and advantages of the invention, as well as the structure and operation of various embodiments of the invention, are described in detail below with reference to the accompanying drawings. It is noted that the invention is not limited to the specific embodiments described herein. Such embodiments are presented herein for illustrative purposes only. Additional embodiments will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein.

[0022] The present invention will now be described with reference to the accompanying drawings. In the drawings, generally, like reference numbers indicate identical or functionally similar elements. Additionally, generally, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears.

[0023] The following detailed description of the present invention refers to the accompanying drawings that illustrate exemplary embodiments consistent with this invention. Other embodiments are possible, and modifications can be made to the embodiments within the spirit and scope of the invention. Therefore, the detailed description is not meant to limit the invention. Rather, the scope of the invention is defined by the appended claims.

[0024] It would be apparent to one of skill in the art that the present invention, as described below, can be implemented in many different embodiments of software, hardware, firmware, and/or the entities illustrated in the figures. Any actual software code with the specialized control of hardware to implement the present invention is not limiting of the present invention. Thus, the operational behavior of the present invention will be described with the understanding that modifications and variations of the embodiments are possible, and within the scope and spirit of the present invention.

[0025] Reference to modules in this specification and the claims means any combination of hardware or software components for performing the indicated function. A module need not be a rigidly defined entity, such that several modules may overlap hardware and software components in functionality. For example, a software module may refer to a single line of code within a procedure, the procedure itself being a separate software module. One skilled in the relevant arts will understand that the functionality of modules may be defined in accordance with a number of stylistic or performance-optimizing techniques, for example.

[0026] FIG. 1 is an exemplary enterprise network 100, in accordance with an embodiment of the present invention. The enterprise network 100 includes a mobile device 102, in accordance with a further embodiment of the present invention. Mobile device 102 may include, by way of example and not limitation, mobile devices including Java-enabled phones with CLDC 1.1 Java mobile runtime environment such as the BLACKBERRY by RESEARCH IN MOTION of Waterloo, Ontario, Canada; the APPLE IPHONE and IPAD by APPLE COMPUTER, INC. of Cupertino, Calif.; Java-enabled phones with the GOOGLE ANDROID operating system by GOOGLE INC. of Mountain View, Calif.; and the WINDOWS PHONE by MICROSOFT CORPORATION of Redmond, Wash. One skilled in the relevant arts will recognize that techniques described herein as applicable to mobile device 102 may also generally be applied to non-mobile devices as well, such as, for example, a personal computer.

[0027] In accordance with an embodiment of the present invention, mobile device 102 has a client application 104 installed thereon. Client application 104 is able to interface with device inputs and outputs (“I/O”) 106, such as, for example, a monitor, keypad, or touchscreen display, in accordance with an embodiment of the present invention. Client application 104 is also able to interface with a local database 108, which stores a set of data intended for use by client application 104. In accordance with an embodiment of the present invention, local database 108 is encrypted by a device-specific encryption key.

[0028] Mobile device 102 is in communication with synchronization server 110, in accordance with an embodiment of the present invention. Additional mobile devices 112a-c are similarly in communication with synchronization server 110, in accordance with a further embodiment of the present invention. The various mobile devices can be connected to synchronization server 110 via any one or more communications channels, as would be understood by one skilled in the relevant art. For example, connectivity between mobile device 102 and synchronization server 110 may involve, in an exemplary embodiment, communication hops over both a cellular communication network and the Internet. The various communication hops may themselves be either public or private networks, and may include components located on the Internet as well as various private intranets.

[0029] Synchronization server 110 sits between the one or more mobile devices 102 and 112a-c and an Enterprise Information System (“EIS”) 114, in accordance with an embodiment of the present invention. Synchronization server 110 assists in capturing changes to relevant data made by the EIS 114 and providing the changes to mobile devices 102 and 112a-c. Synchronization server 110 also assists in capturing changes made by the mobile devices 102 and 112a-c and providing the changes to EIS 114. In this manner, data available to a mobile device 102 in local database 108 can be synchronized with data from the corresponding data store of EIS 114, the enterprise data system 116 (or “central database”). In accordance with an embodiment of the present invention, synchronization server 110 maintains a cache reflecting the data from enterprise data system 116.

[0030] EIS 114 is connected to synchronization server 110 in order to allow synchronization server 110 to provide the aforementioned data synchronization services, in accordance with an embodiment of the present invention. Communications between EIS 114 and synchronization server 110 can likewise be through any communication channels, as would be understood by one skilled in the relevant art. One skilled in the relevant arts will further understand that EIS 114 and synchronization server 110 may share a same physical server or distributed server as separate software components therein, or may even be compiled as a single combined application. Therefore, it is understood that synchronization server 110 and EIS 114 may be disposed in a number of different locations within enterprise network 100, and are shown as separate computing devices in FIG. 1 by way of example, and not limitation.

[0031] As previously noted, EIS 114 further includes, or is otherwise communicatively coupled to, an enterprise system 116, in accordance with an embodiment of the present invention. In accordance with a further embodiment of the present invention, data within local database 108 comprises a subset of data from enterprise data system 116. This data may be in for the form of a mobile business object (“MBO”), in accordance with an embodiment of the present invention. An MBO is a business object that can be synchronized between enterprise information system 114 and mobile device 102. The MBO can be persisted, by storage in local database 108, in order to allow for access by the mobile device 102 during periods without connectivity to EIS 114. A non-limiting example of MBOs is provided in U.S. patent application Ser. No. 12/503,573, filed on Jul. 15, 2009, Atty. Dkt. No. 1933.0720001, entitled “Metadata Driven Mobile Business Objects,” which is incorporated by reference herein in its entirety.

[0032] Client applications 104 running on mobile device 102 access (read and write) to the local database 108. The local database 108 is accessed by client application 104 via generated classes, where each class (corresponding to the aforementioned MBOs) represents one table, in accordance with an embodiment of the present invention.

[0033] Communications between client application 104 and synchronization server 110 may be handled in a number of different ways, as will be recognized by one skilled in the relevant art. In accordance with an embodiment of the present invention, a communications framework is embedded directly within client application 104 as provided by a code generator or by a developer. In accordance with an additional embodiment of the present invention, a helper application may be deployed on mobile device 102 to manage the communications.

[0034] The data on central database 116 is synchronized with the data in local database 108 through the use of a database synchronization system, such as synchronization server 110, in accordance with a non-limiting example embodiment of the present invention. For example, when client application 104 first creates a local database 108, it sends a subscription request to the server system 114. The server 114 responds by sending, for each database table, a set of rows that need to be inserted into the client's local database. This set of rows may or may not be the same set of rows that would be sent to a different mobile device 112a-c for a different user. The client application inserts these rows one at a time into local database 108, and maintains table indexes, until all the rows have been inserted.

[0035] The potentially numerous insertions that must be made by client application 104 into local database 108 can overwhelm the resources of mobile device 102. A number of solutions have been considered to alleviate this problem. One solution is prepared statement caching. Rather than the client application 104 asking the mobile database software managing local database 108 to parse a SQL “insert” statement every time a row is to be inserted, the parsed statement can be reused. This may potentially reduce the initial download time by 5-10%, for example.

[0036] Another approach is to use long transactions. Since most database systems incur a not insignificant cost for every committed transaction, multiple insert statements can be executed in a single transaction to reduce transaction commit costs as a proportion of the overall download time. Either the complete download can operate as a single transaction, or as a set of transactions each of which is for a batch of inserted rows. As an example, the use of long transactions might improve download performance by a factor of between 3 and 10, although during that shortened download period, the device can still become quite unresponsive.

[0037] An additional approach is to send preformatted rows. Typically, a mobile device database 108 stores each table row as a contiguous sequence of bytes, and multiple rows (as many as will fit) are grouped together into fixed length pages (e.g. 1024 bytes). If a SQL “insert” statement is executed with a set of prepared statement parameters, the corresponding sequence of bytes must be constructed from the set of parameter values. If the server knows the expected row representation of the mobile device database, the server can send each row as a preformatted sequence of bytes, reducing the CPU utilization of the mobile device. As an example, the use of preformatted rows might improve download performance by a factor of 5, although during the shortened download period, the device can still be quite unresponsive, because packing of rows into pages and index maintenance still consumes a lot of CPU time and results in many input/output operations to the storage media on which local database 108 is held.

[0038] Yet another approach is to send pregenerated databases to mobile device 102. The server 114 can be configured to occasionally generate a client database as a file, which is downloaded by each mobile device 102 and 112a-c when it first subscribes. Simple file download to a mobile device 102 can be quite fast, e.g. a few seconds per megabyte, depending on the speed of the network (or cable) connection between the device and the server. If a database contains a lot of reference data, such as a product catalog, this technique can be particularly effective. However when the set of rows is not the same for each device, then a pregenerated database will not be appropriate. Also if the client database must be encrypted with a device-specific encryption key, then the server will not be able to send to the device an appropriately encrypted database, unless the client first sends its encryption key to the server, which may be considered an unacceptable security risk.

[0039] FIG. 2 is a flowchart 200 illustrating steps by which a bulk download synchronization is accomplished, in accordance with an embodiment of the present invention. Flowchart 200 is described with continued reference to exemplary enterprise network 100 of FIG. 1. The method begins at step 202 and proceeds to step 204 where server 114 receives a subscription request from client application 104, running on the mobile device 102, indicating that it wishes to download its initial database to be stored at local database 108. In accordance with an embodiment of the present invention, this request is handled through synchronization server 110. In accordance with a further embodiment of the present invention, the initial database comprises a subset of the total data available in central database 116.

[0040] In accordance with an embodiment of the present invention, the client application 104 indicates in this request which operating system software and database system software it is using, as well as its desired database page size. The server 114 is expected to have compatible database system software installed for each database software variant that might be needed for any of the mobile devices 102 and 112a-c that might be expected to connect to it.

[0041] At step 206, server 114 determines which portion of the data in enterprise data system 116 will be transmitted to client application 104 for synchronization. The precise data may vary based on, by way of non-limiting example, the contents of the subscription request of step 204, or the mobile device 102 and 112a-c making the request.

[0042] In accordance with an embodiment of the present invention, the server 114 process checks if it has a bulk download program that can generate a suitable database for the client application 104, based on the client's indicated operating system and database system software used by local database 108. If the server process cannot find a suitable bulk download program, it invokes its code generator to generate one, by generating code for the server (a bulk download program) that will emulate the code that would normally be generated for use on a mobile device (i.e. the MBO classes). In accordance with another embodiment of the present invention, generation of a suitable database for client application 104 is accomplished by consulting metadata information regarding the structure of local database 108. One skilled in the relevant arts will further recognize that the suitable database can be generated by other mechanisms, including a combination of a bulk download program (either pre-configured or generated) and metadata.

[0043] At step 208, the server 114 process starts a task (thread or subprocess) which tuns the bulk download program, in accordance with an embodiment of the present invention. This task impersonates the original client, and makes a loopback connection to the server 114 to download the initial data and insert it into a temporary database file (unencrypted, or encrypted using a server-specific encryption key) on the server's 114 disk. This task can benefit from the comparably expansive resources of server 114 by, for example, utilizing a large database cache size (making effective use of server RAM to reduce disk I/O), and using long transactions (or a single transaction) for the rapid construction of the temporary database file.

[0044] At step 210, the server 114 process then starts another task to transmit the temporary database file over the network to the mobile device 102, in accordance with an embodiment of the present invention. The file can be sent in chunks, where each chunk is one or more pages according to the client's requested page size, in order to improve transmission speeds. In accordance with an embodiment of the present invention, to avoid clear-text interception of the temporary database file, each chunk is compressed and then encrypted (with a client/server-negotiated session key, for example such as would be used over a standard HTTPS network connection) as it is transferred over the network to the client application 104 on mobile device 102. The method then ends at step 212.

[0045] The manner by which database synchronization is requested and handled by mobile device 102 is now described. FIG. 3 is a flowchart 300 illustrating steps by which a client application 104 receives a bulk download, in accordance with an embodiment of the present invention. The method begins at step 302 and proceeds to step 304 where the client application 104 sends a subscription request to server 114 (possibly via synchronization server 110), in accordance with an embodiment of the present invention. This subscription request is analogous to the request received by server 114 at step 204 of FIG. 2.

[0046] At step 306, mobile device 102 receives the database file from server 114. The mobile device 102, upon receiving each chunk over the network, decrypts (with the client/server-negotiated session key) and then uncompresses the chunk. At step 308, for each page in the chunk, mobile device 102 encrypts that page using the client-specific database encryption key, and writes that page out to a file on the mobile device file system (e.g. media card). This behavior is termed “encrypt-on-write”. The method then ends at step 310.

[0047] When all pages have been received by the client 102, the temporary database file can be deleted from the server's 114 file system.

[0048] In accordance with an embodiment of the present invention, the encrypt-on-write scheme relies on the ability to correctly encrypt any given database page knowing (1) the page contents (a sequence of bytes whose length is the negotiated page size), (2) the encryption key, and (3) the page number. By way of non-limiting example, if the encryption algorithm used is AES/CBC, then the page number can be used to derive an initial vector, which is provided to the AES/CBC encryption library on the device together with the page contents and encryption key. Therefore, the client database software needs to support page-based encryption.

[0049] Client database encryption schemes where some pages are not encrypted, and the determination of whether or not a page is encrypted depends on information contained at arbitrary positions within the database file (such as page tables), will in general be incompatible with encrypt-on-write. The encryption algorithm used by the encrypt-on-write component of the system is generally provided by the client database software as a callable API, injected into the database software, or replicated in both the database software and the encrypt-on-write component.

[0050] One skilled in the relevant arts will appreciate that encrypt-on-write is not necessary in order to realize the benefits described herein. Notably, a number of these benefits can be realized even if the client is not capable of implementing encrypt-on-write. For example, the server can encrypt the temporary database file using the client's encryption key, such that the client can simply write the database file to its file system upon receipt without further encryption. However, this requires the client 104 entrusting server 114 with its encryption key. Moreover, this encryption would have to occur prior to any compression, unlike the compression/encryption of communications which will be decrypted/uncompressed by mobile device 102. The encrypted file would be relatively uncompressible over the network. As an example, the data stream transmitted over the network might be ten or more times larger than if the client database could support encrypt-on-write. It is therefore beneficial for the mobile device 102 to implement encrypt-on-write if able.

[0051] While the database file is being downloaded to the mobile device 102, callbacks are made into the client application 104 to allow it to indicate the download progress. When the complete database file is available, another callback is made into the client application 104 to notify it that initial data is available, and therefore the application 104 is ready for use.

[0052] Various aspects of the present invention can be implemented by software, firmware, hardware, or a combination thereof. FIG. 4 illustrates an example computer system 400 in which the present invention, or portions thereof, can be implemented as computer-readable code. For example, the methods illustrated by flowcharts 200 of FIGS. 2 and 300 of FIG. 3, can be implemented in system 400. Various embodiments of the invention are described in terms of this example computer system 400. After reading this description, it will become apparent to a person skilled in the relevant art how to implement the invention using other computer systems and/or computer architectures.

[0053] Computer system 400 includes one or more processors, such as processor 404. Processor 404 can be a special purpose or a general purpose processor. Processor 404 is connected to a communication infrastructure 406 (for example, a bus or network).

[0054] Computer system 400 also includes a main memory 408, preferably random access memory (RAM), and may also include a secondary memory 410. Secondary memory 410 may include, for example, a hard disk drive 412, a removable storage drive 414, and/or a memory stick. Removable storage drive 414 may comprise a floppy disk drive, a magnetic tape drive, an optical disk drive, a flash memory, or the like. The removable storage drive 414 reads from and/or writes to a removable storage unit 418 in a well known manner. Removable storage unit 418 may comprise a floppy disk, magnetic tape, optical disk, etc. that is read by and written to by removable storage drive 414. As will be appreciated by persons skilled in the relevant art(s), removable storage unit 418 includes a computer usable storage medium having stored therein computer software and/or data.

[0055] In alternative implementations, secondary memory 410 may include other similar means for allowing computer programs or other instructions to be loaded into computer system 400. Such means may include, for example, a removable storage unit 422 and an interface 420. Examples of such means may include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM, or PROM) and associated socket, and other removable storage units 422 and interfaces 420 that allow software and data to be transferred from the removable storage unit 422 to computer system 400.

[0056] Computer system 400 may also include a communications interface 424. Communications interface 424 allows software and data to be transferred between computer system 400 and external devices. Communications interface 424 may include a modem, a network interface (such as an Ethernet card), a communications port, a PCMCIA slot and card, or the like. Software and data transferred via communications interface 424 are in the form of signals that may be electronic, electromagnetic, optical, or other signals capable of being received by communications interface 424. These signals are provided to communications interface 424 via a communications path 426. Communications path 426 carries signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link or other communications channels.

[0057] In this document, the terms “computer program medium” and “computer usable medium” are used to generally refer to media such as removable storage unit 418, removable storage unit 422, and a hard disk installed in hard disk drive 412. Signals carried over communications path 426 can also embody the logic described herein. Computer program medium and computer usable medium can also refer to memories, such as main memory 408 and secondary memory 410, which can be memory semiconductors (e.g. DRAMs, etc.). These computer program products are means for providing software to computer system 400.

[0058] Computer programs (also called computer control logic) are stored in main memory 408 and/or secondary memory 410. Computer programs may also be received via communications interface 424. Such computer programs, when executed, enable computer system 400 to implement the present invention as discussed herein. In particular, the computer programs, when executed, enable processor 404 to implement the processes of the present invention, such as the steps in the methods illustrated by flowcharts 200 of FIGS. 2 and 300 of FIG. 3, discussed above. Accordingly, such computer programs represent controllers of the computer system 400. Where the invention is implemented using software, the software may be stored in a computer program product and loaded into computer system 400 using removable storage drive 414, interface 420, hard drive 412 or communications interface 424.

[0059] The invention is also directed to computer program products comprising software stored on any computer useable medium. Such software, when executed in one or more data processing device, causes a data processing device(s) to operate as described herein. Embodiments of the invention employ any computer useable or readable medium, known now or in the future. Examples of computer useable mediums include, but are not limited to, primary storage devices (e.g., any type of random access memory), secondary storage devices (e.g., hard drives, floppy disks, CD ROMS, ZIP disks, tapes, magnetic storage devices, optical storage devices, MEMS, nanotechnological storage device, etc.), and communication mediums (e.g., wired and wireless communications networks, local area networks, wide area networks, intranets, etc.).

[0060] While various embodiments of the present invention have been described above, it should be understood that they have been presented by way of example only, and not limitation. It will be understood by those skilled in the relevant art(s) that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined in the appended claims. It should be understood that the invention is not limited to these examples. The invention is applicable to any elements operating as described herein. Accordingly, the breadth and scope of the present invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.