System and method for flushing bean cache

[0001] This application claims priority to U.S. Provisional Patent Application No. 60/335,633, filed Oct. 25, 2001, entitled “SYSTEM AND METHOD FOR FLUSHING BEAN CACHE,” incorporated herein by reference.

[0002] The following applications are cross-referenced and incorporated herein by reference:

[0003] U.S. Provisional Application No. 60/305,986 entitled “DATA REPLICATION PROTOCOL,” by Dean Bernard Jacobs, Reto Kramer, and Ananthan Bala Srinvasan, filed Jul. 16, 2001.

[0004] U.S. Provisional Application No. 60/316,187 entitled “CLUSTER CACHING WITH CONCURRENCY CHECKING,” by Dean Bernard Jacobs and Rob Woollen, filed Aug. 30, 2001.

[0005] A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document of the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.

[0006] The invention relates generally to a system and method for storing data on a network.

[0007] When a data item is stored in a single database or data store that is accessible over a network, it is often the case that multiple servers or clients will require access to that data item. Traditionally, this requires a hit to the database each time the data item is accessed. Each hit to a database is relatively resource intensive and relatively inefficient.

[0008] One way of overcoming some of the efficiency and scalability problems is to store a local copy of the data item in cache memory. A server or client can then use that local copy if future access to the data item is needed. This process may be appropriate and efficient for data items that never change, but problems can arise when a data item is updated in the database.

[0009] If a data item in the database is updated, a copy of that data item stored in a local cache on the network will be different than the item in the database, as the cache will not automatically receive the update. The problem intensifies when there are local copies on multiple servers and/or clients on the network. Since each of these local copies is created at a different time, there can be multiple versions of the data item on the network. If a user tries to update or view the data item, the copy accessed by the user may not be current and correct.

[0010] Such a problem with data latency can cause serious problems for applications that require near real-time accuracy, such as web sites that offer “real time” stock prices. Such an application might utilize a database table having at least two columns, one column containing stock symbols, which can be used as primary keys for the table, and one column containing the current price of each stock. In such an application, most of the activity involves users accessing the site and reading the current stock values. There is typically also activity involving back-end applications or systems that come in periodically, such as once every minute, with updated stock prices. These back-end systems need read/write access to the database in order to update the data.

[0011] Most access to the system will be read only. For these read-only users, the system can cache data to provide faster access. The system can update the cached information periodically, such as every fifteen minutes. In such a “read-mostly” situation, however, it may be preferable to give a user the most recent data. A fifteen minute delay in providing accurate information may be undesirable for many applications. It is typically desirable to give users information that is as accurate as possible.

[0012] One way to ensure that users get accurate information, or at least information that is current with data stored in the database, is to pull the information from the database for each request instead of reading a cached copy. This can be very expensive for many applications, as a hit to a database is much more time and resource intensive than reading a value from memory.

[0013] For people updating the data in the database, it may be desirable to wrap as many updates as possible into a batch transaction in order to improve performance. Wrapping updates into a single transaction also ensures that either all the updates occur or none of the updates occur. Problems arise, however, in how to update cached copies for each item updated in a transaction.

[0014] A system and method are included for updating a copy of a data item stored in local cache on at least one server in a network cluster. Identification information is provided to a read/write bean stored on a server in the cluster. The identification information relates to any server in the cluster that contains a read-only bean and a copy of the data item in local cache. A read-only bean provides read access to the local copy of the data item. The original data item is stored in a network database, and is updated using the read/write bean. When the data item is updated by the read/write bean, an invalidate request can be sent or multicast from the server containing the read/write bean to the entire cluster, or can be sent to any server or read-only bean identified by the identification information having a local copy of the data item. Any local copy of the data item can then be dropped in response to the request. A current copy of the data item can be read from the database and stored in local cache.

[0021] In order to maintain consistency among items distributed on a network, a system in accordance with the present invention can take advantage of beans, or JavaBeans. A bean is basically a framework for components that can be added to a server to extend functionality. One embodiment utilizes two types of beans, “read-only” entity beans and “read/write” entity beans. An entity bean is a bean that is persistent, allows shared access, has primary keys, and can participate in relationships with other entity beans. Each entity bean can have an underlying table in a relational database, with each instance of the bean corresponding to a row in that table.

[0022] A read-only bean is a bean that can be cached on a server, such as an enterprise JavaBean that resides in a network cluster. The read-only bean can provide read access to any server in the cluster, as well as to any client inside or outside of the cluster. The read/write bean is transactional, residing on a server in the cluster and providing cluster servers with read/write access to a network database. The read-only bean deals with data in local cache on a cluster server. The read/write bean deals with information in a database.

[0023] One way to address the concurrency of information in the cache and in the database is to associate a timeout value with each read-only entity bean. For example, a read-only bean can be deployed with a default cycle of ten minutes. After each period of ten minutes passes, the read-only bean goes back to the database and reads the current value. This approach can work well for certain applications, such as those applications with values that change at a regular interval.

[0024] There may be applications, however, which have data that changes very infrequently. When this data changes, users may want to know about the change as soon as possible. Since the data does not change very often, it is tempting to set a long read cycle time in order to conserve resources. This can have the undesirable effect, however, of creating latency issues with the data, as the resultant delay in updating the data can be almost as long as the cycle time, depending on the point in the cycle at which the update occurs. For such applications, it is desirable that the data accessible by a read-only user is updated as soon as possible after the data is the database is updated.

[0025] One system in accordance with the present invention provides an interface, exposed by a read-only bean. The interface allows a user or application to tell the system to drop a cache, or “invalidate” a cache, when the user updates a data item or is aware of an update. This interface shall be referred to as a “CachingHome,” as an entity bean typically has a “home” or factory that creates it. CachingHome can have three methods on it, and be coded as follows:

[0026]

package weblogic.ejb;
public interface CachingHome {
	public void invalidate(Object pk) throws RemoteException;
	public void invalidate(Collection pks) throws RemoteException;
	public void invalidateAll( ) throws RemoteException;
}

[0027] The method invalidate (Object pk) lets a user invalidate data associated with a particular primary key in a database or data table. The method invalidate (Collection pks) lets a user invalidate data for a collection or group of keys. The method invalidate All ( ) allows a user to invalidate data for all keys in the database table. These invalidate methods allow a user to ensure that values are stored in local cache until a programmer, application, user, or system says otherwise.

[0028] One such method 300 is shown in the flowchart of FIG. 5. A copy of a data item is stored on at least one server in a network cluster 302. The data item can be updated in the database using a read/write bean on one of the cluster servers 304. An invalidate request can then be initiated using an interface of a read-only bean located on one of the servers, the request being sent to any server containing a local copy of the data item 306. Any copy of the data item can then be dropped from a server receiving the request 308.

[0029] In a system 100 with a network cluster 104, such as is shown in FIG. 1, it is possible that a copy of a value 122 stored in a database 108 is cached on each server 110, 112, 114 in the cluster 104. If a client 102 contacts server 110 through the network 106 and requests that server 110 invalidate a given key, such as by making the request “invalidate (Key),” it is easy for server 110 to drop its cached copy 116 of the value 122 or values associated with that key. A problem exists, however, in how to get servers 112 and 114 to drop their cached copies 118, 120 as well.

[0030] One embodiment allows server 110 to drop a copy 116 in local cache when it receives an invalidate request 124 from the client 102, as shown in FIG. 2. After dropping the copy 116 from local memory, server 110 can send a message 126 on multicast to the other servers 118, 120 or read-only beans in the cluster 104 to drop the copy of the value in local cache. Multicast is a technique for sending a packet of information or a message from one server, or source, to other servers without having any unnecessary packet duplication, as one packet is sent from the source and replicated across the network as needed. This approach allows each server to drop the cached value when the value in the database is updated.

[0031] FIG. 6 shows another system which includes multiple clients 402 and multiple databases 416 in communication with the network 404. A data item 418 is stored in one of the databases 416. A copy 420 of the data item is stored in a cluster server 406 of cluster 408. There are two copies 422 stored in cluster servers 422 of cluster 412, as well as a copy 424 on server 414, which is not contained in any server cluster. This system would work similar to the system of FIGS. 1 and 2, in that one of the servers 406, 410, 414 containing a copy of the data item 418 can drop a copy in local cache when it receives an invalidate request from a client, and can send a message on multicast to the other servers on the network to drop any copy of the value in local cache.

[0032] Another problem exists due to the fact that a multicast message is only sent once by the source and does not wait for confirmation of receipt by the other servers. A server in the cluster might not get an invalidate request, such as if it is temporarily offline. A system in accordance with the present invention can provide a more reliable multicast by tagging each such message or request with a version number or sequential number. In this way, a server receiving a request will know the version of the request, as well as the version of the last request it received, such that the server will know if it missed a message. Once a server determines that it has missed a message, it can request that the message be sent again so that it can update accordingly.

[0033] One problem with this approach, however, is that a server will not know it has missed an update until another update is sent. In certain applications such as an on-line store posting weekly specials for weeks 1, 2, and 3, it may be unacceptable to wait until the next update to get correct information. The store would not realize that it had missed the specials for week 2 until the update for week 3. The week 1 specials would have remained up during week 2, displaying the wrong information to any user accessing the system during that time. When the system realizes that it missed the week 2 update, it will already be week 3. The server will end up simply discarding the week 2 information without the information ever having been displayed to a user.

[0034] A system in accordance with the present invention can get around this problem by periodically “heartbeating” information to the servers in the cluster. A server heartbeats a packet of information or a message by sending the message periodically over the network or cluster. A heartbeat message can contain information such as the latest version number, previous version numbers, or the actual update information itself if the update information is small enough to be practical. If a server receives a heartbeat message containing the latest version number, and the server is not on that version of the data or did not receive the latest invalidate request, the server can request or pull the invalidate message from the server.

[0035] The initiating server that initially sent the invalidate request, which may also be the server sending the multicast and/or heartbeats, can store recent requests for a certain amount of time or can store a certain number of recent requests. If a cluster server requests an invalidate message that the initiating server is still storing, the initiating server can simply send the message to the cluster server, by a method such as a multicast or a point-to-point connection. If the initiating server no longer has the message, the initiating server can tell the cluster server to simply drop its entire cache, since it is not possible to tell the cluster server which keys have changed. The cluster server can read new and/or current information from the database. This can temporarily lessen performance, but the newly-cached information will at least be current with the information in the database.

[0036] In operation, a client or application can update a data item through a read/write entity bean. The update, or a transaction containing multiple updates, will commit to the database. An invalidate message can be sent to the servers in the cluster, with the message being triggered for example by the client or server updating the data item. The cluster servers can each drop any copy in local cache and can read in the new value from the database, either right away, later, or when necessary to serve a request. Normally it is not possible to read uncommitted data, so it may be preferable to use a two-step process where the data is committed first and then a message is multicast to the cluster.

[0037] One problem with the above approach is that it forces a client to initiate an invalidate request, which can involve a little more complexity for the client. There is also the possibility that the client could use the invalidate method incorrectly or make a mistake. It may therefore be preferable that the system can do it automatically.

[0038] A system in accordance with the present invention can address this problem by using an “invalidation target.” An invalidation target is based on the idea that the read-only and read/write beans point to the same data in the database, with the people reading the data using the read-only bean and the people updating the data using the read/write bean. The idea is to invalidate the read-only bean when the read/write bean is updated or modified.

[0039] When deploying an entity bean or enterprise JavaBean, there is typically a deployment descriptor used to store meta data about the actual entity bean. A deployment descriptor can be, for example, an XML document used to provide information about the services provided by the bean that should be made available to clients. The information can provide a wide variety of information to the clients, such as request routing information, as well as method and class details for a supporting Java class. A tag can be added to the deployment descriptor, referred to previously as an “invalidation target.” The invalidation target for a read/write bean can contain the identity of any associated read-only bean. The invalidation target can be used to automatically invalidate any associated read-only bean(s) when the read-only bean is updated.

[0040] In one embodiment, the invalidation target can be updated when a server requests information from a read/write bean or generates a read-only bean does the request. When the read/write bean forwards information from a database or data store to the requesting server, the read/write bean can also update the invalidation target. An XML file stored on the server containing the read/write bean can be updated to include the identity of the server requesting the information or creating the bean.

[0041] FIG. 3 again shows the system 100 of FIG. 1, except in this embodiment the system is shown taking advantage of an invalidation target. Whenever a read/write bean 128 is used to update a data item 122, the system can look to the invalidation target associated with that read/write bean 128 and send an invalidate request to each read-only bean 130 associated with that read/write bean. The invalidate request can be acted on directly by server 110, which contains both the read/write bean 128 and a read-only bean 130. The invalidate request can also be sent by any appropriate protocol to other servers containing a read-only bean 130, which would be in the invalidation target. In one approach, server 110 contacts server 112 directly by a point-to-point connection 134, telling the read-only bean 130 on server 112 to drop the cached copy 118 on server 112. In another approach, server 110 can send a multicast message 132 over the network 106 to any server which contains a read-only bean 130 within the scope of the invalidation target, such as server 112.

[0042] A subsequent call to an invalidated read-only bean can cause a method such as ejbLoad to be called, which can read current information from the database into cache. For example, a container-managed persistence (CMP) bean, an entity bean whose state is automatically synchronized with a database, can use an invalidation-target element in an XML file such as ejb-jar.xml to specify a read-only entity bean that should be invalidated when the read/write bean has been modified. The container in this example can automatically invalidate the invalidation-target, such as after the transaction is completed.

[0043] Such a method 200 is shown in the flowchart of FIG. 4. A copy of a data item is stored on at least one server in a network cluster 202. Identification information, or an invalidation target, is provided to a read/write bean on a server in the cluster, the information relating to any server in the cluster containing a read-only bean and a local copy of the data item 204. The data item can be updated in the database using a read/write bean 206. An invalidation request can be sent from the server containing the read/write bean to any server identified by the identification information 208. The copy of the data item can then be dropped by each server receiving the request 210.

[0044] In this way, customers or clients do not have to write any additional code to invalidate an item. In accordance with an embodiment of the present invention, only an invalidation target must be specified in order to keep the read/write and read-only beans coherent. The beans can coexist on the same server, with the read-only bean reading items from local cache and the read/write bean reading from, and writing to, the database.

[0045] In order to improve performance, a system in accordance with the present invention can instead wait until an entire transaction or series of updates is committed to the database or data table, instead of sending an individual message for each update. A server, such as the server initiating the update, can keep track of which keys were updated during the transaction and multicast a single message that includes information for all updated primary keys. Such batching of messages can improve the overall performance of such a system and can reduce the opportunity for error or inconsistencies.

[0046] One example of a system that can be used in accordance with the present invention contains a table of stock symbols, as well as information associated with each symbol, such as price and volume. A Java server page can be used to allow a user to request the current price of a stock. The Java server page can read the information from a read-only entity enterprise Java bean. A Java Message Service (JMS) queue can receive messages with updates to stock prices. An message-driven bean can de-queue these messages and update the associated CMP entity bean. When this modification occurs, the container can invalidate the associated read-only bean.

[0047] The foregoing description of preferred embodiments of the present invention has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations will be apparent to one of ordinary skill in the relevant arts. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, thereby enabling others skilled in the art to understand the invention for various embodiments and with various modifications that are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalence.