Generation and use of compressed image data

[0003] This invention relates to the generation and use of compressed image data. In particular, it relates to a method and computer system for generating compressed image data, and to a method and image rendering device for rendering pixel data using the compressed image data.

[0004] The invention is particularly concerned with graphics data which is intended for display on a monitor associated with a computer system or some other control means including a processor means. For instance, the display may be the monitor of a computer system, the graphics data generated being stored on the memory of the computer system or on, for instance, a CD-ROM and being retrieved for rendering to the monitor of the computer system. Alternatively, the compressed image data for rendering to the monitor may be retrieved from a remote source. It is a characteristic of such systems that the image rendering device which displays the compressed image data has associated therewith a user interface which enables user interaction.

[0005] One objective for the local image rendering device is to allow pixel data to be rendered for display with the minimum amount of time.

[0006] An objective for forming the compressed image data from the image source is to represent the original image in a useful format which allows rendering time to be kept to a minimum yet results in a flexible and useful system. A particularly useful feature of flexibility is to allow the display system to receive compressed image data from a number of different image sources without significantly altering the rendering time for display of the pixel data on the local display. Another desirable feature is to reduce the amount of unnecessary pixel data which is generated, particularly in respect of objects which are invisible in the final display.

[0007] One known data structure according to which compressed image data is transmitted is a Binary Space Partition (BSP) tree according to which polygons defined in the image to be rendered are held in an index in which each polygon is identified as a list of a number of identifiers. According to convention, each polygon is identified by three identifiers such that they are mapped onto triangles, but this is not essential. The data structure includes a list which maps each identifer onto a point defined in Cartesian coordinates (X,Y,Z). Using this basic format, a BSP tree is established for an image by a process of allocating subdividing polygons. The BSP tree is a useful format for compressed image data, although it does have some limitations.

[0008] One of these is that, in rendering an image from compressed image data stored as a BSP tree, pixel data may be generated for polygons that are not in fact visible in the final display. This thus is a waste of processing time.

[0009] Moreover, some polygons are required to be split up during formation of the BSP tree so that effectively two polygons have to be created for a single polygon in the image.

[0010] According to one aspect of the present invention there is provided a method of generating compressed image data defining a 3-D image to be rendered in 2-D comprising:

[0011] defining in the image a plurality of hulls, each hull being defined as a set of polygons in a designated coordinate system;

[0012] defining for each of at least some of said hulls a portal, each portal being defined as a set of points in the designated coordinate system for that hull and a viewing transform for mapping the portal to another, target hull; and

[0013] storing a data structure which holds the defined hulls and portals in a manner describing their physical relationship in the image, the data structure representing the compressed image data.

[0014] By defining hulls and portals, a number of different advantages can be obtained.

[0015] The viewing transform for each portal that allows one hull to be matched to another allows hulls to be generated on independent coordinate systems which can be related by the viewing transform. Thus, it is possible to provide compressed image data from a number of different image sources each providing their own hulls and portals, provided that the viewing transforms allow for this mapping.

[0016] Another advantage is that there is much more flexibility about the arrangement of hulls within the image. The physical relationship of hulls can be changed relative to one another by altering the viewing transforms of the portals. Importantly, this allows for an easy technique for coping with moving objects within an image.

[0017] The use of portals also allows clipping to be effected when an image is rendered, for example by frustrum culling so that only that part of a hull which can be seen through the portal from the defined user viewpoint is rendered.

[0018] A portal maps the coordinate set of one hull into another, and a transformation is provided to achieve this. Thus, a transformation may map a target hull to a source hull. The definition of each portal may also include an inverse viewing transform. The inverse viewing transform maps the source hull to the target hull. The provision of such inverse transforms allows a viewpoint to move through a portal from a source hull into a target hull.

[0019] In the preferred embodiment the data structure comprises an index in which each polygon is stored as a group of identifiers, and a list in which each identifier denotes a point in the designated coordinate system. Typically, a Cartesian coordinate system could be used. A suitable form of data structure is a BSP tree. The data structure can also hold for each polygon, polygon properties such as colour, texture and normals.

[0020] In the described embodiment, each portal is defined as a convex quadrilateral by four points. This renders clipping techniques such as frustum culling particularly simple. However, portals of other shapes can be defined.

[0021] The method of the invention is particularly useful when applied to an image in which objects are moving. Moving objects are defined in the data structure as respective sub-hulls, each with an associated moving portal in the hull containing the sub-hull. Thus a nomad is a moving portal within a hull, the moving portal mapping into that hull a sub-hull.

[0022] The invention also provides a method of operating a computer system to generate compressed image data defining a 3-D image to be rendered in 2-D, comprising:

[0023] displaying a representation of the 3-D image on a display;

[0024] operating a user interface to define in the representation of the image a plurality of hulls and, for each of at least the sum of said hulls, a portal,

[0025] wherein the computer system defines each hull as a set of polygons in a designated coordinate system, and defines each portal as a set of points in the designated coordinate system for that hull and a viewing transform for mapping the portal to another, target hull; and

[0026] storing in a memory of the computer system a data structure which holds the defined hulls and portals in a manner describing their physical relationship in the image, the data structure representing the compressed image data.

[0027] The invention also provides a computer system for generating compressed image data defining a 3-D image to be rendered in 2-D, the data server comprising:

[0028] an interface for allowing a user to define in a representation of the 3-D image a plurality of hulls and, for each of at least some of said hulls, a portal;

[0029] a processor for defining each hull as a set of polygons in a designated coordinate system, and each portal has a set of points in the designated coordinate system for the hull with which it is associated and a viewing transform for mapping the portal to another, target hull; and

[0030] a memory holding a data structure which holds the defined hulls and portals in a manner describing their physical relationship in the image, the data structure representing the compressed image data.

[0031] According to another aspect of the present invention there is provided a method of rendering an image from compressed image data which is in the form of a data structure which holds data defining hulls and portals in a manner describing their physical relationship in the image, wherein each hull is defined as a set of polygons in a designated coordinate system and has a portal defined as a set of points in that designated coordinate system and a viewing transform for mapping that portal to another, target hull, the method comprising:

[0032] traversing the data structure according to a selected view point and selecting for rendering hulls and portals visible from that viewpoint;

[0033] rendering the visible hulls and portals in sequence by generating pixel data from the data held in the data structure for each hull and portal; and

[0034] when a portal is encountered, rendering the target hull according to the viewing transform associated with that portal.

[0035] This method allows for the use of compressed image data in the form defined in the first aspect of the invention so as to render images in a time effective manner. The view transforms associated with the portals are used to greatly simplify rendering of the hulls as pixel data.

[0036] This aspect of the invention also provides an image rendering device having a communications interface for receiving compressed image data which is in the form of a data structure which holds data defining hulls and portals in a manner describing their physical relationship in an image, wherein each hull is defined as a set of polygons in a designated coordinate system and has a portal defined as a set of point in that designated coordinate system and a viewing transform for mapping that portal to another, target hull;

[0037] a processor for traversing the data structure according to a selected viewpoint and selecting for rendering hulls and portals visible from that viewpoint;

[0038] pixel data generating means for generating pixel data from the data held in the data structure for each of the visible hulls and portals, thereby to render those hulls and portals in sequence in such a manner that, when a portal is encountered, the target hull is rendered according to the viewing transform associated with that portal.

[0039] For a better understanding of the present invention and to show how the same may be carried into effect, reference will now be made by way of example to the accompanying drawings in which:

[0040] FIG. 1 is a schematic block diagram of one environment in which the present invention can be used;

[0041] FIG. 2 illustrates a flow diagram of the general steps for implementing the hulls and portals system;

[0042] FIG. 3 illustrates the creation of a three dimensional scene using a 3-D Studio Max editor;

[0043] FIG. 4 illustrates in two dimensional form a simple geometrically correct scene;

[0044] FIG. 5 illustrates in three dimensional form the simple scene of FIG. 4;

[0045] FIG. 6 illustrates the flow diagram for the steps of creating a scene;

[0046] FIG. 7 illustrates a simple scene in a geometrically incorrect arrangement;

[0047] FIG. 8 illustrates a flow diagram of the steps for interpreting the scene;

[0048] FIG. 9 illustrates the division of a scene to create a BSP tree;

[0049] FIG. 10 illustrates the construction of a BSP tree for the scene of FIG. 9;

[0050] FIG. 11 illustrates the transformation of mapping portals;

[0051] FIG. 12 is a flow diagram for the steps in rendering a scene;

[0052] FIG. 13 illustrates the concept of frustum culling;

[0053] FIG. 14 illustrates further the concept of frustum culling;

[0054] FIG. 15 illustrates a special case scene of a hull within a hull;

[0055] FIG. 16 illustrates a special case of an irregular shaped portal;

[0056] FIG. 17 illustrates the special case of a moving object traversing between hulls;

[0057] FIG. 18 illustrates the construction of a BSP tree for the specific example of FIG. 4;

[0058] FIG. 19 illustrates the construction of a BSP tree for the specific example of FIG. 4; and

[0059] FIG. 20 illustrates the order in which the three-dimensional scene of FIG. 4 is drawn.

[0060] FIG. 1 shows an exemplification of an image visualisation device 2 which is connected to receive compressed information via communications link 4 from an image generation device 6. The image generation device 6 may be remote from the image visualisation device 2, or they may in fact be the same device. In the event that they are the same device, they can be considered as two devices being separated in time, in that the image generation device 6 must first generate an image before it can be displayed on the image visualisation device 2. The image need only be generated once, no matter how many times thereafter it needs to be visualised.

[0061] The image generator 6 comprises an image creation controller 10 which receives inputs from an image input 8. The image creation controller 10 outputs signals to an image display 14, and an image store 12. The image store 12 generates signals on the communication link 4.

[0062] The image visualisation device 2 includes a communication interface 18 for receiving the compressed information on the communication link 4 and transferring such to a display controller 22. The display controller 22 receives inputs from a display input 24, and generates outputs to a display generating means 20. The display generating means 20 outputs signals to a scene display 16. The image generation device 6 is typically a computer system, the image input 8 comprising a keyboard and the image display 14 comprising a monitor. The image creation controller 10 comprises the processor circuitry of the computer system, and the image store 12 storage means in the computer system such as memory.

[0063] The image visualisation device 2 may similarly be a computer system, and in one embodiment the same computer system as the image generation device 6, or may alternatively be an application specific block provided simply for the purpose of displaying the compressed information. The display input 24 of the image visualisation device 2 may be any kind of input means such as a keyboard, a mouse, or a joystick. The display controller 22 may be processor circuitry of a computer system, or dedicated processor circuitry associated with the application specific to the image visualisation device 2. The display generation means 20 is a pixel generating means, which generates pixels to the scene display 16 in dependence on data provided thereto by the display controller 22. The scene display 16 is a monitor.

[0064] The compressed information may be provided to image visualisation device 2 from a number of sources. For instance, the communication link 4 may provide the compressed information from a memory store such as a CD-ROM, the compressed information having been written to the CD-ROM by the image generation device 6.

[0065] The image generation device 6 is responsible for generating compressed information in the form of compressed image data from a three dimensional image source. This can be considered as comprising of two main phases: creation of a scene; and interpretation of a scene.

[0066] The mechanisms for implementing a hulls and portals system can be broken into three distinct stages. Hereinafter, these three stages will be described individually, with some examples and explanations of the terms and techniques used at each stage. Referring to FIG. 2, there is shown a flow chart illustrating the three distinct phases. The first stage 40 is to create a scene, the second stage 42 is to interpret the scene, and the third stage 44 is to visualise the scene. The stages of creating a scene and interpreting a scene are performed in the image generation device 6, and results in the generation of the compressed information. The first stage of visualising a scene is performed in the image visualisation device 2 based on the compressed information generated by the image generation device 6.

[0067] In one example, the image generation device 6 comprising a computer system employs a standard three dimensional (3-D) editing package such that a user can create, via the image input 8, 3-D scenes. In one particular application, a 3-D editing package known as “3-D Studio Max” is used. This enables a scene to be created having rooms and doorways/windows which link the rooms, together with objects in the rooms.

[0068] Referring to FIG. 3, there is illustrated one exemplification of the image display 14 whilst the 3-D Studio Max package is in use for generating a 3-D scene. In this particular example, a scene is being created having three rooms, the three rooms being interlinked by two doorways.

[0069] FIG. 3 shows a top view 50, left-side view 52, a front view 54, and a user (three dimensional) view 56 as seen on the image display 14 by the individual creating a 3-D scene using the image input 8. The scene comprises a first room 58, a second room 60 and a third room 62. The first room 58 and second room 60 are linked via a first doorway 64, and the second room 60 and third room 62 are linked by a second doorway 66. Thus, in this example, the first, second and third rooms are designated as first, second and third “hulls”, and the first and second doorways are designated as first and second “portals”.

[0070] A hull is merely a collection of “objects”. An empty room comprises walls, a floor and a ceiling. Thus defining a set of walls, a floor and a ceiling as objects and associating them with a hull name defines that hull. Portals are doorways or windows from one hull to another hull. Thus a portal in one hull must generally be associated with a portal in another hull, but in a special case may be a “mirror” portal providing a doorway or window back into the same hull.

[0071] For the purpose of further description of the invention, the scenes will be considered in two dimensions only. The simplified two dimensional form can be easily extended into three dimensional form, and readily enables understanding of the present invention.

[0072] Referring to FIG. 4, there is illustrated a scene comprising a first room 92, a second room 94, and having a doorway or window 96 between the first and second rooms. The first and second rooms are designated as first and second hulls, and the doorway is designated as a portal. For completeness, FIG. 5 shows the two-dimensional example of FIG. 4 in its three-dimensional form.

[0073] Hulls are given an arbitrary name. Thus in relation to FIG. 4 we can name the first hull 92 HULL#1 and the second hull 94 HULL#2. In real examples, each of the first and second hulls may comprise objects therein, in the illustrated examples the only objects contained in a hull are the walls, floor and ceiling. Any object belonging to a hull is defined as:

[0074] objectname[hull name]

[0075] Thus, if there are two nominal objects in the first hull, they can be designated as OBJECT#1 [HULL#1] and OBJECT#2 [HULL#1]. Thus, any number of objects can be designated as belonging to a given hull.

[0076] FIG. 6 illustrates a general flow chart for carrying out the steps of creating a scene. A first step 80 is to define any hulls in the scene. Thus, in the present example, two hulls are defined as HULL#1 and HULL#2. A second step 82 is to define any objects. Thus in the present example we have defined two objects OBJECT#1 and OBJECT#2.

[0077] A third step 84 is to define nomads. Nomads are a special case of objects in that they are moving objects. A nomad is actually defined as a hull, and to distinguish from “normal” fixed hulls they are also referred to a sub-hulls. Nomads are simply objects that can move independently of the hulls. They are defined as objects, but are not statically associated with any hull.

[0078] In a next step 86, the defined objects are associated with respective defined hulls. Thus OBJECT#1 and OBJECT#2 are defined as associated with HULL#1.

[0079] A next step 88 is to define any portals in a scene. In the present example described with reference to FIG. 4, a portals 96 needs to be defined. There are several ways in which a portal can be defined, and two ways in which such definition can be made are described hereinbelow with reference to FIG. 4 and also FIG. 7.

[0080] All objects are defined as a set of triangular polygons. Thus an object is made up of a plurality of triangular polygons. Each triangular polygon is defined by three coordinates. Thus, for example, if an object is of a rectangular shape, such rectangular shape is actually drawn by drawing two triangles. Thus in defining objects, a number of three coordinate polygons are listed which make up the shape of the object. In addition in defining the shape of objects, other information associated with the objects is also defined. For instance, the brightness of the objects in dependence on whether they are orientated towards a light source, texture coordinates associated with the objects etc. For a more detailed discussion of the various information defined in defining objects, reference should be made to the simple examples for creating the scenes of FIGS. 4 and 7 given at the end of this description with reference to Appendices A to J.

[0081] Portals are defined as four sided polygons by reference to four coordinates. As with objects, other information associated with the portals is defined.

[0082] Referring to FIG. 4 the arrangement shown therein is called a “geometrically correct” arrangement of connected hulls. In this geometrically correct arrangement, both the first 92 and second 94 hulls share a common coordinate system. That is, the first and second hulls 92 and 94 are both defined in the same coordinate system, and the portal 96 between the two hulls can be defined in this coordinate system such that if the portal 96 is defined in relation to the first hull 92 it will have the same coordinates as when it is defined in relation to the second hull 94.

[0083] The portal 96 shown in FIG. 4 is known as a “gap portal”. The portal 96 can be defined as a single four sided polygon that references two hulls. The naming convention for defining gap portals is as follows:

[0084] portalname[hull1name]<−>[hull2name]

[0085] This portal is defined in this way in relation to both of the first and second hulls. Thus, and referring to FIG. 4, the portal shown therein may be defined as:

[0086] portal#1[hull#1]<−>[hull#2]

[0087] A second arrangement of hulls is known as a “geometrically incorrect” arrangement and is shown in FIG. 7. It is so called because since the first 92 and second 94 hulls are defined in different, independent coordinate systems. Referring to FIG. 7, the first hull 92 has a portal 100 associated therewith, and the second hull 94 has a portal 98 associated therewith. When the scene is created, the portals 98 and 100 are intended to be merged such that the scene is seen as an arrangement similar to that of FIG. 5. Thus this geometrically incorrect arrangement requires each of the portals 98 and 100 to be mapped onto each other to enable the scene to be created. Each portal belongs to a particular hull, and is defined as a four sided polygon. Each portal belonging to a particular hull must also reference a target hull and a portal in that target hull to enable the scene to be created. Thus, there is provided a mapping between two unconnected portals, and the portals in this arrangement are termed “mapping portals”.

[0088] The naming convention for mapping portals is:

[0089] portalname[hullname]−>targetportalname[targethullname]

[0090] There are two of these definitions for a mapping portal, one associated with each hull. Thus in the definition of the first hull 92 there will include a definition of a mapping portal being portal#1[hull#1]−>portal#1[hull#2]. In the definition of the second hull there will be a definition of a mapping portal of portal#1[hull#2]−>portal#1[hull#1]. Thus, the step 88 of defining the portals will take one of two routes in dependence on whether the portals between hulls are in a geometrically correct or geometrically incorrect system.

[0091] It should be noted that under software control gap portals could be converted to mapping portals. This allows the coordinate system for each hull in a geometrically correct arrangement to be altered such that they become independent, even if joined by gap portals.

[0092] Referring to FIG. 5 again, the next step 90 in creating a scene description by creating an editor file. The editor file assembles together for any particular scene the appropriate hulls, portals and nomads associated with the scene. This editor file effectively describes the geometry of the hulls, portals and nomads as described hereinabove. An overall description file is also established which contains a graph linking all the hulls, portals and nomads in the editor file according to their definition in the 3-D editor.

[0093] At this stage, any inconsistencies in the design of the scene can be picked up. For example, hulls that cannot be reached, references to non-existent hulls, etc. Furthermore, polygon properties such as animated textures, colours, and polygon normals can be inserted into the files at this stage.

[0094] Once the 3-D editor, in this case 3-D Studio Max, has been used to create the editor file and the overall description file that defines the connection graph, they must be parsed into a suitable format for real time visualisation. This primarily involves taking the hulls, portals and nomads, and the overall description file, parsing them and physically creating the graph described in the overall description file and linking hulls together via portals by creating the relevant transforms to achieve this. A flow chart illustrating the steps involved in interpreting the scene is illustrated in FIG. 8.

[0095] As shown in FIG. 8, in interpreting a scene a first step 102 parses the scene description (editor files and overall description file), a second step 104 creates portal transforms, and a third step 106 creates data structures. These steps will now all be described in detail hereinafter. The interpretation process results in a data structure which is used to contain the collections of polygons that represent objects in the scene, and the position and geometry (i.e. sub-hull) of any nomad in the scene. The data structure thus created is stored in the image store 12 of the image generation device 6.

[0096] When visualising a three dimensional scene, some form of data structure is used to contain the collections of polygons that represent the objects in the scene. Precisely what form this data structure takes depends on the mechanism employed to remove hidden surfaces. Hidden surfaces are surfaces and/or parts of objects that are either outside the viewer's field of view, or are obscured by other objects nearer to the viewer. In the described embodiment, the polygons are contained in a tree structure known as a binary space partition (BSP) tree. This can order the polygons in the scene in such a way as to make it possible to draw them in a given order depending on the viewpoint, that will ensure that the polygons that are nearest the viewer are drawn last. Hence polygons that should be obscured in a scene are drawn before the polygons actually obscuring them, and overdrawn thereby. With reference now to FIGS. 9 and 10, the construction of a BSP tree to form the data structure as used in the present invention will be described.

[0097] Referring to FIG. 9athere is shown a scene containing five objects: A,{overscore (A)},B,C,D and E. Each object has a face direction denoted by an arrow extending perpendicular from the object. In the first step of creating the data structure, the scene shown in FIG. 9ais sub-divided by selecting an arbitrary object in the scene and drawing a line in the plane thereof. Thus, referring to FIG. 9b,a line is drawn in the plane of object E sub-dividing the scene into two halves.

[0098] Referring to FIG. 10, there is shown the construction of the BSP tree at various stages. Initially the BSP tree consists of a list together of all of the objects shown in the scene of FIG. 9a,as shown in FIG. 10a.After dividing the scene in the plane of object E as shown in FIG. 9b,an initial part of the BSP tree has been formed as shown in FIG. 10, with the object E at the top of the tree, and the tree having two branches from object E; a negative branch and a positive branch. As can be seen in FIG. 9b,the drawing of a line in the plane of object E intersects object C such that two parts of object C are created, which in FIG. 9bare denoted as C₁and C₂. Referring to FIG. 9b,it is seen that the object B and the object C₁are on the side of the line drawn in the plane of object E which object E is facing. Therefore objects B and C₁are considered to be on the positive side of object E. Conversely object C₁, A, D and A are on the opposite side of object E to the direction that E is facing, and can be considered as being on the negative side of object E. Thus FIG. 10bshows that the BSP three branches in a negative direction to objects A, {overscore (A)}, C₂and D, and in a positive direction to objects B and C₁.

[0099] As shown in FIG. 9c,two further lines are drawn in the scene, one in the plane of object C and one in the plane of object A, both to intersect with the line in the plane of object E. In the same way as described hereinbefore, this further splits up the scene such that object B is on the positive side of object C₁, no objects being on the negative side of C₁. Object D is on the negative side of A. Object E does need to be considered since it forms a part of a higher level of the BSP tree, and specifically the root of the tree. Object C₂is on the positive side of object A. Thus, referring to FIG. 10c,the BSP tree structure can be further enhanced.

[0100] Finally, and referring to FIG. 9d,additional lines are drawn through the planes of objects B and D. There are no objects on the negative side of B, and all objects on the positive side of B have already been accounted for in the BSP tree. There are no objects on the positive side of D, and all objects on the negative side of D have already been accounted for in the BSP tree. Thus, the BSP tree is complete.

[0101] Reference is now further made to FIG. 9d,wherein a viewpoint V is identified. Given a selected viewpoint V, and the BSP tree of FIG. 10, it is possible to determine how the polygons should be drawn on the display in order to ensure that they are drawn in the correct order. The plane of object E is at the root of the BSP tree of FIG. 10, it is first determined whether viewpoint V is on the positive side or the negative side of the plane of object E. The viewpoint V is on the negative side of the plane of E, and therefore the positive side of the tree is traversed from E to C1. Again, as V is on the negative side of the plane of C1 the tree is traversed down the positive side thereof to object B. There are no further branches after object B, therefore this is the first polygon to be drawn. Traversing back up the tree, C1 is the next node and as there are no negative polygons C1 is drawn and the tree is traversed back to the root E, E being the next object drawn. The negative side of the tree is then traversed to reach object A. Viewpoint V is in the positive side of the A plane, and therefore the negative side of the tree is traversed down to object D, object D drawn, and then the tree traversed back up to A which is then drawn. The tree is then traversed back to C2 which is drawn, and that completes the reconstruction of the scene. Thus, the BSP tree is used to enable the scene to be drawn in the order B, C1, E, D, A, C2.

[0102] {overscore (A)} is never entered into the list because it faces the wrong way. Had the view direction been on the negative side of the A plane, then A would not have been listed and {overscore (A)} would.

[0103] The ordering is independent of the direction of the viewpoint. Thus, if the viewpoint V was pointing away from all the polygons, then they would still be put out as rendering candidates in the order mentioned, but the reliance would be on the rest of the graphics pipeline to clip them out of the eventual view.

[0104] Portals are inserted into the chosen data structure representing the scene, and are effectively treated as object polygons themselves. In other words, a doorway (portal) into another room (hull) is treated as a polygon and is drawn in the correct order for the containing room. When it is nominated for drawing by the system (whether it is using a BSP tree structure or some other mechanism), the hull through the portal is rendered fully, before continuing to draw the rest of the objects (polygons) for the original hull.

[0105] Referring to FIG. 8, the step 104 of creating portal transforms is relevant only to mapping portals as described hereinabove. As described earlier, each mapping portal is defined in association with a viewing transform that defines mapping between respective hulls.

[0106] Referring to FIG. 11 consider two portals 110 and 112 defined in association with two respective hulls (not shown) defined in their own coordinate systems. Portal 110 has four coordinates A,B,C,D defining a polygon and portal 112 has four coordinates a,b,c,d defining a polygon.

[0107] The polygon needs to be defined such that A maps to a, B maps to b, C maps to c and D maps to d. This mapping is handled by the mechanism employed during the creation of portals. The preferable procedure is to create one portal, copy it, rotate through 180 degrees, scale and move to the desired destination. In this way, the definition of the coordinates maps correctly when the portal is exported ready for the next stage of the process.

[0108] Now that the two coordinate system are defined (this can be done since each portal defines a plane, and a normal can be determined due to the ordering of the points), a transformation to move from one coordinate system to the other can be created in step 104 of FIG. 8. An example of the creation of the transformation for FIG. 11 follows.

[0109] Given the definition of the two portals by coordinate sets {A,B,C,D} and {a,b,c,d}, the following basis vectors for their respective coordinate spaces can be defined.

[0110] Basis set {U,V,W} where U is the unit vector along the direction of AB, V is the unit vector along the direction of AD and W is the normal to these two vectors created from the created from the cross product of U and V (that is AB×AD).

[0111] Basis set {u,v,w} created similarly from {a,b,c,d}.

[0112] The transformation from one space to another (say from {U,V,W} to {u,v,w}) is then defined by MO where MO=Inv(L)N where L is the matrix created from the U,V,W basis set, and N the matrix created from the u,v,w basis set.

[0113] Since U,V and W are unit vectors, Inv(L) is in fact Transpose(L).

[0114] Thus MO is in fact Transpose(L)N.

[0115] Taking into account any translation, a 4×4 homogeneous transformation from one coordinate space to another can be created. The translation is question can be calculated by comparing the absolute positions of two equivalent points in the two portals (say A and a).

[0116] The difference is then converted into a 4×4 transpose matrix r, and the results composed to form the full transformation M (where M=rMO).

[0117] The process can be repeated to form the transformation from coordinate space {u,v,w} to {U,V,W}, or the inverse transformation can be defined from the one already calculated.

[0118] Once all the portal transforms have been defined as described, and the portals inserted in the correct place in the containing hulls data structures, they are ready to be interpreted in a real time visualisation system. The thus created data structures, transforms, editor files, and overall description file are then stored in the image store 12 of the image generation device 6 as indicated by step 107 in FIG. 8.

[0119] The compressed information is retrieved from the image store 12 and transferred to the image visualisation device 2 via communications link 4. At the image visualisation device 2, a visualisation of the scene in real time takes place. The definition of the original image in terms of hulls and portals allows this stage to be made as fast as possible by traversing each scene, including passage through portals, in a simple fashion. A short description of the processes involved in rendering a scene to the scene display 16 follows.

[0120] The data structure constituting the compressed information on communication link 4 is received at the communication interface 18 and read into the display controller 22 within the image visualisation device 2. Referring to FIG. 12, there are illustrated the steps performed by the image visualisation device 2 in rendering the image to the scene display 16.

[0121] The initial scene graph is read into the image visualisation device 2 in step 120. The data structure is traversed according to a defined user viewpoint. As discussed hereinabove, this involves traversing the BSP tree. Initially, the viewpoint may be provided by default by the scene graph. However, a user may alter the viewpoint using the display input 24, and thus the graph is traversed in dependence on the viewpoint whether it is determined by an initial setup or by user interaction.

[0122] As discussed hereinabove, polygons are discarded if they are facing away from the viewpoint. In addition, polygons are discarded if they are outside the field of view for the viewpoint, as will be discussed in further detail hereinbelow.

[0123] The subset of polygons that remain are then transformed to the coordinate space of the viewpoint in step 126. This is a standard technique in graphics and basically involves creating a transformation matrix that moves the viewpoint to the origin, and includes a scaling factor based upon the distance from the viewpoint (perspective correction), such that objects that are further away from the viewpoint appear smaller. This matrix is then applied to all the polygons (objects) in the scene.

[0124] In the next step 128, the polygons are normalised to screen space. This usually simply involves scaling the coordinate system to a nominated viewport (for instance 0 to 640 in the x-axis and 0 to 480 in the y-axis). The polygons are then ready to be drawn in a two dimensional screen buffer, namely the display generating means 20 of the image visualisation device 2.

[0125] The above-described steps for image rendering represent known technology, except that the received data structure has a novel format in its definition of hulls and portals as described hereinabove with relation to the steps 40 and 42 of creating and interpreting a scene. There is, however, now a significant difference in the generation of pixel data from the polygons as a result of the definition of hulls and portals. This distinction arises in step 122 when the data structure is traversed. Some of the polygons in the scene graph actually represent portals into other hulls (for instance doorways into an adjacent room). When a polygon portal is encountered, the transformation associated with the portal is applied to the viewing transformation described above, so that the hull associated with the portal is then drawn using this new viewing transform. This process can be recursive, simply composing more and more viewing transforms to the basic transform as the hull-portal data structure is traversed.

[0126] Care should be taken that cycles do not occur. This can be done either by preventing traversal of the same hull twice, or by placing an upper limit on the number of hulls that can be traversed. The advantage of the second option is that facing mirrors can be implemented.

[0127] The introduction of portals means that extra culling techniques can be imposed to reduce the number of unnecessary polygons that are drawn. Many polygons will be completely obscured, but still drawn by the renderer since it cannot know that they are overdrawn by nearer polygons. By using hulls and portals, extra clipping can be performed to reduce this. The simplest of these techniques is called frustum culling.

[0128] The concept of frustum culling using hulls and portals will now be described first with reference to FIG. 13. FIG. 13ashows a scene comprising a first hull 150 and a second hull 152 separated by a wall 154. Portals 156 and 158 provide “windows” between the first and second hulls. A viewer is at a point 160 in the first hull 150. The scenario shown in FIG. 13ais a common scenario found in many graphics implementations. The viewer at the point 160 in hull 1 looks into hull 2 through the portals 156 and 158. Referring to FIG. 13bit can be seen that when the viewer looks into hull 2 it sees two frustums 162 and 164 defined by the viewer's field of view into the second hull.

[0129] Thus the scene is rendered in the normal way by the image visualisation device 2 using the BSP tree as described hereinabove. However, when the portals are due to be rendered, the adjoining hull is drawn instead. That is, when the portals 156 and 158 are to be drawn, rather than drawing those portals the second hull 152 is drawn. The second hull is drawn by calculating a viewing frustum relative to the portals 156 and 158 and the viewer at point 160, and projecting them into the same hull. This means that a small subset of the second hull 152 is rendered instead of all of it. Therefore the subsets 162 and 164 are rendered, and not the whole of the hull 152.

[0130] A second common scenario is illustrated in FIG. 14. This is an extension of the scenario of FIG. 13. In the scenario of FIG. 14, a third hull 166 is visible from the first hull 150 through the second hull 152. This scenario can be resolved by the following steps:

[0131] trace a frustum through the portal being considered;

[0132] if the frustum intersects another portal, then determine the intersection points; and

[0133] when rendering the hull, when the portal is signalled as a candidate for rendering, trace a new frustum from those points into the next hull.

[0134] In order to effect this algorithm, a proper frustum polygon intersection function needs to be added to determine if a portal actually has intersected a frustum. Thus, referring to FIG. 14, it can be seen that the frustum 162 intersects with the portal 170 of the third hull 166. The intersection points of the frustum 162 with the portal 170 are defined by the extremities of the portal 170. When rendering the hull, when the portal 170 is signalled as a candidate for rendering, a new frustum is traced from this intersection, i.e. the extremities of the portal 170 in the case of FIG. 14, into the next hull. This new frustum is indicated as frustum 168 in the hull 166 in FIG. 14.

[0135] Referring to FIG. 15, a third scenario is illustrated: a hull within a hull. This scenario is useful for several situations, considered a city-scape, with buildings, each with a series of rooms. The buildings themselves are hulls, each containing a series of rooms which are also hulls. This means that the rooms can each be encapsulated as a hull with portals leading between them.

[0136] Referring to FIG. 15 there is shown a first hull 170 and a second hull 172 wholly contained within the first hull 170. A viewer at the point 174 within the first hull 170 is looking into the second hull 172 through portals 182 and 184, the respective viewing frustums being 180 and 178 in the second hull 172. The second hull 172 additionally has a “candidate” portal which leads back to the first hull 170. Following the rules laid out in the second scenario described above, the tertiary hull, i.e. the first hull 170, would normally be rendered. However, in this case rendering of a “tertiary” hull through the candidate portal 184 can be stopped very easily by enforcing a check to ensure that the candidate hull has not already been rendered. This is done simply by allocating each hull a unique key, which is a check when a portal intersection is detected.

[0137] As has been described hereinabove, each portal is defined as a four sided polygon having four coplanar coordinates. However, this does not exclude the possibility of having irregular shaped portals. Referring to FIG. 16, there is shown a wall 190 having an irregular shaped portal 192 therein. Such a portal shape is achieved by ensuring that for any BSP node, the portals objects are drawn first. Thus, referring to FIG. 16, a portal 194 is first drawn. A frustum can then be generated and projected as described herein above, and a secondary hull rendered, and then the irregular shapes 192 of the hull drawn on top over the portal. This imposes the constraint that the wall containing a portal must be ordered very carefully. The portal must be rendered before any of the surrounding wall. This could be ensured in one of two ways:

[0138] i) careful construction of the BSP tree to make sure that the portal is drawn first; or

[0139] ii) position the portal behind the wall so that it is always drawn first since it is physically behind the wall.

[0140] This would prevent further complication of the BSP tree construction.

[0141] One of the main problems for a BSP based system is that of moving objects. In order to be correct, any object moving in a scene must be re-merged with the BSP tree, cutting polygons according to all the candidate planes in the BSP tree. The use of portals simplifies this process greatly. It is not onerous to insert a single polygon into a BSP tree, particularly if a “centroid” based approach is used since sub-division does not occur and the polygon can be filtered down the tree very quickly. If this polygon is a portal, then instead of drawing a polygon the hull that is linked to the portal can be traversed, and a sub-tree inserted into the scene. The “centroid” approach means identifying the central point of a moving object (i.e. a nomad), and using the only central point to identify the location of the object when rendering the scene.

[0142] Nomads (or moving hulls) are themselves BSP tree “hulls”. They are inserted into the BSP tree using a single central point, the “centroid” approach described hereinabove. The single central point is put in at a leaf the BSP tree that already exists. Referring back to FIG. 10 (d) the single central point could be inserted beneath any of D, C₂or B on the positive or negative side thereof, or on the negative side of C₁. The central point will have a transform associated therewith to map the portal associated with the nomad, which in turn maps to the sub-hull of the nomad.

[0143] This centroid approach means there is scope for error since occlusions may be incorrect for parts of the model. Consider the scenario illustrated in FIG. 17. This illustrates how it can be possible to insert an object into a BSP tree and then view it in such a way as to be rendered in the incorrect order. FIG. 17 shows a nomad 200 moving from a portion of a first hull 208 into a portion of a second hull 210. The nomad 200 is moving through a portal 212, defining a doorway, which creates two sections of a wall 202 and 204. The viewpoint is a point 206 in the second hull 210.

[0144] The centroid 214 of the nomad 200 is clearly on the other side of the portal 212 from the viewpoint 206. This means that when the image is rendered the whole of the nomad 200 will be drawn before any of the wall. Clearly, in order to render the image correctly from the viewpoint 206, the wall 202 should be drawn prior to the nomad 200, and then the wall 204 drawn.

[0145] The solution to this problem is to create a half space clip plane 213. The half space clip plane 213 is a plane arbitrarily drawn through the plane of the portal 212 and in the portal space, which splits the two walls 202 and 204 into two spaces 203 and 205. Ideally but not necessarily, the half space clip plane 213 is perpendicular to the plane of the portal 212 and splits the portal into two halves. This artificial plane is added when the 3-D model is created. The drawing of the clip plane 213 results in the portal 212 being associated with one of the halves 203 or 205. The clip plane 213 may be drawn, therefore, in line with the edges of the portal 212. In FIG. 17, the artificial plane 213 associates the portal 212 with the space 203. Thus the nomad 200 is in the space 203.

[0146] The problem of rendering the image in the correct order is then solved by not ignoring any intersections with potential half clip planes. By checking the bounding sphere of a given nomad whilst inserting it into a BSP tree, it is possible to determine when a potential problem arises. A bounding sphere of a nomad is a sphere drawn around the object completely encompassing such using the centroid of the object as the central point of the sphere.

[0147] In FIG. 17, it can be seen that the bounding sphere of the nomad 200 intrudes into the clip plane of both of the walls 202 and 204. Once this intrusion has been determined, it is possible to calculate within each of the two half spaces 203 or 205 whether the nomad or the respective wall 202 or 204 is closer to the viewpoint 206. In the example shown, the half space 203 will be drawn before the half space 250 because it is further away from the viewpoint 206. Within the half space 203, the wall 202 is drawn before the nomad 200 because it is further away from the view point. As a general case, each time a nomad moves, intrusion into the clip plane of any object is tested for.

[0148] In the example described above, the creation and interpretation of the scene are carried out at the image generator and are performed initially and once only. Once a scene is ready for visualisation it can be visualised as many times as is necessary in the image rendering device. For a typical application, this will be every time a new frame is generated. For real time visualisation systems, this can be of the order of 30 frames per second. While in the described example, portals are created at the image generator, it is also possible to create portals dynamically by using the user interface in the image rendering device, together with the necessary viewing transforms.

[0149] Two simple specific examples will now be described for creating a simple data structure and editor files for storing in the image store 12 of the image generation device.

[0150] A first simple case is described with reference to the scene shown in FIG. 4 comprising two hulls in a common coordinate system and a single gap portal. A second simple case is described with reference to the scene shown in FIG. 7 comprising two hulls in independent coordinate systems, each having a respective mapping portal. As has been described hereinabove, when recreated the scene for both these simple examples looks like the scene shown in FIG. 5.

[0151] Referring to the first simple example of FIG. 4, the two hulls 92 and 94 are joined by a single portal 96. These are modelled in the 3-D editor as being geometrically correct, i.e. the hulls are modelled connected as shown in FIG. 4. Individual hulls and portals are designated by naming the separate objects within the scene according to a defined grammar, as discussed hereinabove. The two hulls 92 and 94 are named arbitrarily as “room1” and “room2” respectively. The portal 96 is named simply as “doorway”. The objects to be defined in the hulls are, in this simple example, merely the walls. Thus the names of the objects in this scene are:

[0152] walls[room1]

[0153] walls[room2]

[0154] doorway[room1]<->[room2]

[0155] To understand how this labelling works, it can be seen that the first hull consists of objects named walls belonging to the hull named room1. Similarly the second hull consists of objects named walls belonging to the hull named room2. The portal linking these two hulls together is an object named doorway linking hulls named room1 and room2.

[0156] It can be seen from this description that hulls themselves are not objects, but rather a hull is defined by the objects it contains. For instance the hull could be defined as floor, ceiling and walls each named as floor[room1], ceiling[room1], walls[room1]. In the simple case presented herein, the only objects given for any hull are the walls, which are taken to implicitly include the ceiling and floor.

[0157] Referring now to the second simple example of FIG. 7, the two hulls 92 and 94 are modelled in the 3-D editor as being geometrically incorrect, i.e. the hulls are modelled independently and are not physically joined. Each hull has a mapping portal associated therewith, and to ensure recreation of a scene as shown in FIG. 5 each portal of each hull must be mapped onto the other. When a scene is ultimately rendered to a viewer it will be similar to the scene of FIG. 5.

[0158] Using the 3-D Studio Max editor to create the 3-D scenes, as in the present simple example, the names that can be given to objects must follow a stricter convention. This allows the exporter to determine what hulls various objects belong to, etc. The quick grammar is defined as follows:

[0159] nomad:=name

[0160] hullconstituent:=name[name]

[0161] mappingportal:=name[name]→name[name]

[0162] gapportal:=name[name]<->[name]

[0163] Appendix A includes a definition of terms of the grammar. The raw output that is created by the exporter for the example of

[0164] FIG. 4 is shown in Appendix B. Appendix B shows the first file of this raw output, known as a “fit” file (fundamental interconnectedness of all things). The first section of this file is the hull list, which lists the hulls that the environment contains. The next section lists the nomads (“sub-hulls”), and in this particular environment there are no nomads. The next section lists the portals. In this simple example only a single gap portal is defined with links rooms 1 and 2 through the portal called “doorway”. The “clip” and “mirror” fields are set zero to indicate that the portal is not to be clipped and is not a mirror.

[0165] The final section defines start points. In this case there is one start point defined in room1.

[0166] Each hull, nomad and portal has an associated file that defines them. The file defining room1 is shown in Appendix C. The description file of Appendix C is based on virtual reality modelling language (VRML) version 1.0. This format has been modified to allow for extra features such as better texturing definition. Note that anything following a hash symbol is a comment and has no semantic relevance. Each of the sections of the description file of Appendix C are identified and discussed in Appendix D. The associated file defining room2 will be very similar to Appendix C. Appendix E shows the associated file for the doorway. This file defines the coordinates representing the portal. It is defined in the same coordinate system as the parent hull. It is again in the same format as the hull file. Once the doorway is defined in this way, the intermediate level file definition for this simple case is complete.

[0167] Appendix F shows the fit file for the second simple example of FIG. 7. This is identical to the previous example shown in Appendix B, except for the portal section. In this case there are two mapping portal files. Appendix G shows the two portal files created.

[0168] The final part of the process is to pass these intermediate files into a binary file which represents the BSP tree structure used by the real time renderer, which also includes information to link the portals and hulls. The process of creating the BSP tree data is two-folded. Firstly the binary format BSP tree files are compiled from the intermediate level files. Secondly these files are used in the real time renderer. In the following description these two stages are combined.

[0169] Returning to the simple case of FIG. 4, the portal in this section is a gap portal, and therefore the resulting matrix that is created as part of the portal is simply the identity matrix. This identity matrix is incorporated into the BSP tree that is created from the files. The output from the compilation stage of the BSP tree creation is given and annotated in Appendix H.

[0170] FIG. 18 shows an illustration of the two BSP trees created in Appendix H. The boxes 200 represent the nodes in the list, and the numbers in the boxes represent the plane equation of each node. The tree in FIG. 18 is for room 2. A tree for room 1 is given in FIG. 19. As can be seen, the structures of both FIGS. 18 and 19 is linear, i.e. there is not a great deal of branching in the structure. This is because of the nature of the chosen objects: two cuboid rooms linked with a single portal. If the structure was less regular and square, then there would be a lot more branching.

[0171] FIG. 20 illustrates how the second BSP tree of FIG. 19, relating to room 1, corresponding to an actual three-dimensional model. A to I is the order in which the objects are drawn determined by the BSP tree of FIG. 19. Essentially, the order is (from A to I): ceiling, floor, left wall, back wall, right wall, front wall, top of porch, right of porch, left of porch.

[0172] The portal that is in the porch linking the two BSP trees together is in itself a node in the BSP trees. It will have a plane equation of the form [0, −1,0,n]. This is not shown in the raw output from the compiler. It is inserted as the first node, and all other nodes in the tree will be in its positive sub-tree in this case.

[0173] Shown in Appendix I, is the traversal of the tree and rendering of the polygons contained within it.

[0174] Appendix J illustrates the case for the simple example of FIG. 7. The two BSP trees will look very similar to the BSP trees of FIGS. 18 and 19. However, the portal that is defined in each of the BSP trees is significantly different. It is represented in Appendix J.

[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]



[Inline image omitted. See PDF.]