Method and apparatus for optimizing playback of media files over a data network

[0002] The invention relates to the transmission of multimedia files over a data network, and more specifically to optimizing the transmission of multimedia files in order to provide real-time playback.

[0003] As personal computers have become more sophisticated, the functions they perform have become more varied. Software now exists which allows computer users to view video clips or listen to sound recordings. Before the multimedia information may be played on a computer this information must first be stored in multimedia computer file which is recognized by the software. Some examples of formats for multimedia files are moving pictures expert group (MPEG) for video encoding and decoding, and musical digital interface (MIDI) for encoding and decoding audio data. These multimedia files are uploadable to a computer, and once loaded, are playable by the software resident on the machine.

[0004] Frequently, these multimedia files require a significant amount of memory, which may make them cumbersome for transmitting over data networks. One solution has been to use compression software for the transmission of these files. Prior to transmission, the file is converted to a compressed format. After the files is transmitted, the receiving computer decompresses the information and the file is ready to be played.

[0005] A desirable feature when downloading multimedia files may be to either view or listen to the information in real-time. Difficulties may be encountered when trying to perform this multimedia streaming function due to the quality of the connection established between apparatus transmitting the multimedia file and the apparatus which receives and plays it. As is well known, there are a variety of ways to access information through the Internet. Connection possibilities may range from a 14.4 k baud modem connection over a phone line with an Internet service provider (ISP), to a local area network (LAN) with an direct high-speed connection connection to the worldwide web. When making a direct comparison of the two examples recited, the LAN connection to the worldwide web has greater capacity to receive and transmit information.

[0006] The type of connection established for playing multimedia files has an effect on the quality of the presentation. If there are too many frames per second being transmitted for a particular type of connection, a certain amount of the data may be lost during transmission which affects the continuity as well as the resolution of the presentation. In situations where the data content of the multimedia files is too great for the type of connection established, it may not be feasible or possible to upgrade the connection.

[0007] Accordingly, a primary objective of the invention is to provide an apparatus and method providing adaptable multimedia streaming over a data network.

[0008] In addressing this objective the present inventors have recognized that although many compression schemes currently exist or multimedia streaming over a data network, none of these system provides the adaptability to allow a system user to select a particular transmission speed for the uploading of a multimedia file. Further, it has been recognized a number of factors, such as network congestion, may affect the quality of the transmission at different times.

[0009] The invention described herein is an adaptable communications system for streaming multimedia files and allowing their playback in real-time. The communications system may be incorporated into a network server which is part of a data network such as the worldwide web. The system user may access information on the server through a connection established on a network node. Through the connection established to the data network, the system user may download multimedia files which are stored on the network server. Technology currently exists for playing of some multimedia files during download. This is better known as multimedia streaming. One problem which may be encountered during streaming is that the configuration of the multimedia file may not be compatible with the connection established by the system user. If this connection is not compatible with the file format, the real-time playing of the file may lack a good quality continuity or resolution.

[0010] The network server may include a number of components, including a network interface, which provides for the two-way transmission of information between the network server and the data network. Connected to the network interface is a processor. This processor coordinates all internal functions for the network server. Also included in the network server is an interface monitor which monitors the transmission of information from the network server. A file compression module provides for the compression of files which are downloaded from the data network. A memory is provided which, under direction of the processor, is used to store the compressed files.

[0011] In one aspect of the invention, the network server first downloads multimedia files from a remote source. According to this process, these files may be downloaded from system user with network access, another network server, or any other device which provides this functionality that is connected to the data network. As the multimedia files are downloaded from a remote source, the systems incorporated in the network server perform two functions. The first function is to create multiple copies of the multimedia file in a variety of different formats. These different formats facilitate the transmission of the multimedia files to remote system users through a variety of different data network connections. For example, multimedia files may be generated for transmission especially through a 14.4 k baud modem connection, a 28.8 k baud modem connection, high-bandwidth line, such as an ADSL lines, as well as through a backbone connection. The creation of files in this manner takes into account the resolution and continuity which can be maintained during transmission over these connections.

[0012] Each multimedia file generated may include reference markers positioned at predetermined points in time during the playing of the multimedia file. For example, a reference mark may be placed at every minute mark in each file. The only requirement is that each multimedia file generated in the different formats has the reference markers placed at the same point along the time line. The second function which may be carried out by the network server with respect to the multimedia file is to compress each file and store it in memory. Upon storage, each file may be marked according to the criteria from which it was created.

[0013] The compressed files in the memory of the network server may be accessible by a system user through the data network. The different selections for multimedia files may be presented on a web page or other similar means. Upon accessing the network server and viewing the options, the system user may make a selection of both the multimedia file to be played as well as the connection speed through which it will be played. Upon receiving these instructions, the processor retrieves the appropriate file from the memory and begins to transmit the compressed filed over the data network. The system user may then play the file as it is downloaded.

[0014] As the selected multimedia file is being transmitted from the network server, the network interface monitors the streaming of the data. The transmission speed is directly related to the quality of the connection established by the system user. If the actual transmission speed does not match the transmission speed chosen by the system user, the processor for the network server uses this information to select a file from memory which better operates at the desired speed. In order to provide a seamless presentation, the processor, while outputting the original multimedia files, searches for the next reference mark. When the reference point is located, the processor then locates that same reference mark in the replacement file. When the playing of the multimedia files reaches the identified reference point, the processor will exchange the original file with the replacement file and resume transmission. If further noise or distortion disrupts the transmission of the replacement multimedia file, and the same process as described above may be performed again to find a multimedia file which is better suited for transmission over the connection.

[0015] Numerous modifications and additions will be apparent to those skilled in the art upon further consideration of the invention.

[0016] FIG. 1 discloses a portion of the data network in which a network server is incorporated.

[0017] FIG. 2 discloses an internal system diagram for the network server.

[0018] FIGS. 3aand bdisclose representations of portions of the multimedia files including reference marks.

[0019] FIG. 4 discloses a flow diagram which describes the downloading and creation of multimedia files.

[0020] FIG. 5 discloses a flow chart which describes the transmission of selected multimedia file from the network server.

[0021] Disclosed in FIG. 1 is a system diagram for a portion of a data network which incorporates the invention described herein. The data network may be the worldwide web or any other type of node-based network . In the portion of the data network shown in FIG. 1, network server 2 is in connection with other nodes in the network, specifically network node 6 and network node 4. System users may establish a connection to the data network through the various network nodes. The network nodes may be other network servers, some of which are controlled by Internet service providers which provide access to the worldwide web for remotely located system users. The network servers may also provide access to the worldwide web for system users connected to the server as part of a local area network (LAN).

[0022] User interfaces 8 and 10 may be personal computers which include software for playing multimedia files. It is well known in the art that video and audio information may be stored as a data file which, when loaded into the appropriate software, is playable over a user interface. These multimedia files are transferable over the data network between the various user interfaces, the network nodes, and the servers. These files can be stored on a server and downloaded and played by a system user. One mode of operation is for the system user to download the entire file onto a local storage device and then once this is complete, execute and play the file with multimedia software. Software now exists for real-time playing of the multimedia files during the download from a remote source. A problem which may be encountered during the real-time playing of multimedia software is that connections established between the user interfaces and the network nodes are of differing quality and the speeds and amounts of information which can be transmitted can vary greatly. For example, one user interface may be connected to the data network through a 14.4 k modem which has very limited capabilities for the transfer of information. Another user interface may be connected to the data network through an high-bandwidth line, such as an ADSL line which has a large capacity for the transfer of information. In either case, if the multimedia file is not in a format which is compatible with the type of connection established, the multimedia information will not play properly in real-time, or it will lack the best resolution possible for the type of connection established.

[0023] Described in FIG. 2 is an internal system diagram for the network server 2. This server may of type which are currently commercially available, and which include the capability to perform a variety of data processing functions. Communication between the data network and the internal components of the network server are controlled through network interface 20. In connection with the network interface is processor 24 which controls the internal functions for the network server. These functions include the receipt and transmission of information, the retrieval and storage of files in memory, as well as compression of multimedia files. In connection with the network interface 20, is interface monitor 22. This monitor provides feedback as to the actual transmission speed for electronic data transmitted from the network server.

[0024] Also in connection with processor 24, is file compression module 26. This file compression module compresses the multimedia file so that they may be transmitted over the data network in a more expedient manner. The file compression module may incorporate any of a number of file compression programs commercially available. Files created through the compression module may be stored in memory 28. The processor 24 may also retrieve any information stored in memory and transmit it over the network.

[0025] Before access may be provided to the various multimedia files, the network server must first receive these files from another source. As is typical in a data network, such as the worldwide web, multimedia files can be downloaded from a variety of sources including the various servers connected to the network. The format of the multimedia files downloaded may be of many different types. For example, there are many different formats for creating video files. The same can be also said for audio multimedia files. The processor incorporated in the network server includes the functionality to identify a variety of different multimedia formats. Once the format has been identified, the processor further includes the ability to generate copies of the multimedia files and amend the format of these copied files for transmission over a variety of different connection made to the network server. The procedure for downloading and copying the multimedia files is described in greater detail in the flow chart of FIG. 4.

[0026] To begin this process, a remote site which contains the multimedia file is accessed. The multimedia file is downloaded and the processor identifies which format the file has been created in. At this point, the processor begins the task of creating copies of the file in different transmission formats. Stored in memory are a listing of formats to convert the files to. In the example described herein, the formats may be 14.4 k baud modem, 28.8 k baud modem, and ADSL format. One skilled in the art would know that many additional formats are possible for transmission of multimedia files. The inclusion of just three formats is for descriptive purposes only, and is not meant as limitation on the scope of invention.

[0027] In the first step, the query is made as to whether a 14.4 k baud modem compatible file has been created. If the file hasn't been previously created, the processor will then make a copy of the multimedia file and then modify it such that it is compatible with the transmission medium. The file is then sent to the compression module where it is compressed, labeled and then stored in memory. At this point, the query is then made as to whether there is a 28.8 k baud modem compatible file in memory. If there is not, this file is created by the processor, compressed by the compression module, labeled, and then stored in memory. Finally, the query is made as to whether this multimedia file exists in an ADSL format. If it does not, the processor copies and converts the multimedia file to this format, the file is then transmitted to the compression module where it is compressed, labeled, and then stored in memory. Once files have been created in all the desired formats, the file creation process is complete.

[0028] In creating multimedia files that are compatible with different types of connections, the main formatting concern is the number of frames which may be transmitted during a particular time period. Because a connection such as a 14.4 k modem have limited ability to receive information, the files created for this connection will have a reduced frames per second rate of transmission. Although this affects the resolution of the media being played, it allows for playing of the information in real-time with better continuity. As the speed of the connection increases, files with a higher frames per second rate may be transmitted.

[0029] When the different multimedia files are created, one feature which may be incorporated therein is a reference marker positioned at particular points in time during the playback of the clips. For each file created from an original multimedia file, the reference markers are positioned at the same points. Shown in FIGS. 3aand 3bare representations of small portions of multimedia files stored in different formats. As an example, assume that the file in FIG. 3ais for a 14.4 k baud modem transmission and the file in 3bis for transmission over a high-bandwidth line, such as an ADSL line. For each of the multimedia files, a reference marker is placed at the one minute mark for real-time playback of the file. As one can see, there are many more multimedia frames between the zero minute mark and the one minute mark in the ADSL file of FIG. 3 than there are in the 14.4 k baud modem file of FIG. 3a.The main difference in the creation of the different formats is the resolution (band width) which is possible over the particular transmission medium.

[0030] Once all of the files have been created and stored in memory, they are available to a system user who has access to the network server for downloading. One procedure for downloading these multimedia files may include the incorporation of a web page into the network server which system users may access. Once the web page is accessed, the system user is able to view summaries of the available multimedia files. The system user may then select a file to download. Once a selection is made, the system user is given the option to choose the format of the file to be downloaded. In most situations, the system user is aware of the type of connection established between the computer and the data network and can best decide which file would work best for transmission.

[0031] The procedure for downloading the multimedia files is disclosed in FIG. 5. To begin the process, a query is made as to whether the system user who has accessed the web page containing the multimedia file choices has made a selection. If the file is not in a database or some other problem is encountered, an error message is returned to the system user. If the file is in memory, the system user is provided with a second page in which a selection can be made for the format of the file to be downloaded. Once the system user has selected the desired file, the processor retrieves this file from memory and begins the download process. As was described above, the system user has multimedia software incorporated into the user interface for receiving and playing the compressed multimedia file in real-time.

[0032] During the time that the multimedia files are being transmitted to the system user, the transmission is monitored by the interface monitor 22. With this interface monitor, information is received back from the user interface as to how many packets of information are being received at the user interface, how many packets are being lost, and how many packets are then recoverable at the user interface. If during the monitoring of this process the packet recovery falls below a certain standard, or the connection is so good that it is possible to increase the band width transmission, the user interface will inform the processor of the particular situation. The processor, using this transmission information, will select from memory a multimedia file in a format which is more suitable for the transmission connection established. Once the correct file is identified, the processor locates the next reference frame in the multimedia file which is currently being transmitted. Once this reference frame is located, the identical reference frame is located in the replacement file. Once the reference frame is reached during the transmission of the original file, the processor replaces the original file with the new file in such a manner that the actual playing of the multimedia file at the user interface is not disrupted. Due to this change in files being transmitted, the performance and playability of the multimedia file should be improved. The interface monitor 22 continues the monitoring of the transmission throughout the duration of the entire file. The replacement of one file for the other can be done as long as reference marks still exist in the remainder of the multimedia file to be played.

[0033] The foregoing description of the present invention has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit the invention to the form disclosed herein. Consequently, variations and modifications commensurate with the above teachings, and the skill or knowledge of the relevant art, are within the scope of the present invention. The embodiments described hereinabove are further intended to explain best modes known for practicing the invention and to enable others skilled in the art to utilize the invention in such, or other, embodiments and with various modifications required by the particular applications or uses of the present invention. It is intended that the appended claims be construed to include alternative embodiments to the extent permitted by the prior art.