Graphically representing data values

[00002] This application claims priority under 35 USC §119(e) to U.S. patent application Ser. No. 60/183,174, filed on Feb. 17, 2000, the entire contents of which are hereby incorporated by reference.

[00003] This invention relates to the graphic representation of data values.

[00004] Digital images used to represent and convey information to a computer user may take a variety of forms such as alphanumeric characters, graphs, and pictures. Digital images may be stored digitally, manipulated, and then rendered on video monitors or printers.

[00005] Digital images may have a variety of properties such as color, texture, brightness, contrast, and may be associated with nonvisual properties such as speed of motion or sound effects. The properties of an image may be defined when the image is created or modified and may be stored with the image.

[00006] Image properties are generally quantifiable. For example, the brightness of each pixel or other part of an image may be expressed as a quantity that may vary between 0% white (i.e., 100% black) and 100% white. Quantifiable image properties may be modified through mathematical translations of the data values that define the image properties of the parts of the image. In some systems, the user may alter the data values defining one or more image properties by using a keyboard or other input device. In other systems, a user may modify image properties through a graphical user interface (GUI). For example, a range of possible data values may be displayed graphically to the user and the user may select the desired new data value with a mouse or other pointing device.

[00007] One particular GUI used to alter image properties uses a control tab of the kind shown in FIG. 2. A square 200 representing an x-y coordinate system is displayed to the user. The range of base data values for the image property (image brightness is shown in FIG. 2) is displayed graphically along the bottom horizontal edge (the x-coordinate) 210 of the square, while the range of possible new data values is displayed graphically along the left vertical edge (the y-coordinate) 220 of the square. The diagonal line 230 from the bottom left to the top right corner of the square represents a base state in which each base data value is identical to the new data value (i.e., x=y), indicating that no mathematical transformation has yet been performed. Each point on the line represents all of the pixels (or other parts) of the image that have the brightness value indicated on the x-axis.

[00008] To alter the image property the user may select a point on the diagonal line (e.g., point A) and drag it to a new position within the square (e.g., point N). Software then generates a new curve 240 that passes through the endpoints of the diagonal line and the new position of the point. The shape of curve 240 depends on choices made by the software designer. The curve 240 defines a mathematical translation of old data values into new data values. Every point along curve 240 has an x-y coordinate, where the horizontal, x-coordinate represents the old data value and the vertical, y-coordinate represents a corresponding new data value. Thus, when the transformation defined by the new curve is applied, every pixel having an old brightness value of x will assume a new brightness value of y. In this way, the brightness characteristics of the whole image can be altered. For example, dark pixels can be made lighter.

[00009] The GUI shown in FIG. 2 can be confusing to a user. The diagonal line indicating the base state of no change has a constant slope that users may interpret as implying a constantly increasing change of pixel brightness rather than a lack of change, because rising lines in x-y coordinate systems usually represent constant change. Moving point A to destination N (as indicated by arrow 225) to create a new curve may give the impression that the old data value associated with point A (the x-coordinate of point A) has acquired the new data value associated with point N (the y-coordinate of point N). In fact, it is the old data value associated with point B, located directly below point N, that has acquired the new data value associated with point N, because the x-coordinates of points N and B are identical.

[00010] In general, in a first aspect, the invention features a method of graphically representing base data values and possible data values associated with a quantifiable image property including the steps of displaying to a user a line that represents a range of base data values, displaying to a user a two-dimensional coordinate space in which the two dimensions are respectively parallel to and normal to the line, and representing a range of possible data values by distances along the normal dimension of the coordinate space from corresponding base data values on the line.

[00011] Implementations of the invention may include the following features. The line may be displayed horizontally and the coordinate space may be skewed from the horizontal direction. The two-dimensional coordinate space may be a polygon, wherein two nonadjacent vertices of the polygon coincide with the ends of the horizontal line in the two-dimensional coordinate space. The polygon may be a parallelogram, wherein the parallelogram includes a first vertical side parallel to a second vertical side and a first skewed side parallel to a second skewed side. The possible data values represented by endpoints of the first vertical side and the second vertical side may include the minimum and maximum of the range of possible data values, wherein the possible data values represented by points on the first skewed side comprise the minimum of the range of possible data values, and wherein the possible data values represented by points on the second skewed side comprise the maximum of the range of possible data values. The data values may be used in a computer graphics display, and the quantifiable image property may include brightness or color.

[00012] In another aspect, the invention features a method of interactively transforming a graphical representation of data values associated with a quantifiable image property including the steps of displaying to a user a line that represents a range of base data values, displaying to a user a two-dimensional coordinate space in which the two dimensions are respectively parallel to and normal to the line, representing a possible range of data values by distances along the normal dimension of the coordinate space from corresponding base data values on the line, and interactively providing a curve in the coordinate space, a change in each base data value corresponding to the distance along the normal dimension of the coordinate space from the curve to the point on the line representing the base data value.

[00013] Implementations of the invention may include the following features. Interactively providing a curve in the coordinate space may include selecting a point on the line, dragging the point to a new position in the coordinate space, and defining a curve in the coordinate space through the ends of the line and through the new position of the point. The tangent to the curve at the new position of the point may be parallel to the line. Interactively transforming a graphical representation of data values associated with a quantifiable image property may further include displaying to a user the base data value of the selected point, and displaying to a user the new data value of the point as it is dragged. One or more nodes in the coordinate space may be defined and a curve in the coordinate space through the ends of the line and through the nodes may be defined. The tangent to the curve at each node may be parallel to the line.

[00014] In a further aspect, the invention features method of graphically representing old data values and new data values, the data values being associated with a quantifiable image property, including displaying to a user a line that represents a range of old data values, displaying to a user a two-dimensional coordinate space in which the two dimensions are respectively parallel to and normal to the line, representing new data values with a curve in the coordinate system, and displaying a line along the normal dimension of the coordinate system between an extremum of the curve and the line.

[00019] As shown in FIG. 3, a GUI display image 300 includes a square 302 that defines an x-y coordinate system used for modifying the data values associated with quantifiable image properties. The x-coordinate runs along the horizontal direction of square 302 and the y-coordinate runs along the vertical direction of the square, although (unlike a conventional rectilinear coordinate system) a given value of y occupies a vertical position that depends on its location along x-axis. The square 302 contains an inactive background region 310 and an active region 312 in the shape of a parallelogram that is, for example, lighter in color and therefore visually distinguishable from inactive background region 310. Two sides 303 of the parallelogram active region 312 are parallel to the vertical sides 304 of square 302, while two sides 305 of parallelogram active region 312 are skewed from the direction of the horizontal sides 306 of the square. A straight, horizontal line 315 in active region 312 runs parallel to the horizontal sides 306 of square 302, between the two vertices 317 that form the obtuse angles of parallelogram active region 312. (In this context, the term “line” refers a line segment, a straight segment of finite length.)

[00020] The straight, horizontal line 315 represents a range of base data values assigned to the image property to be modified (for example, the brightnesses of pixels in an image), and active region 312 represents the parameter space of data values that may be assigned when the base range of data values is modified. The x-coordinate 331 of a point on line 315 corresponds to a base data value within the range of base data values assigned to an image property to be modified. The y-coordinate in active region 312 directly above 333 or below 335 a point on line 315 corresponds to a possible new data value that may be assigned to the point when the image property is modified.

[00021] Horizontal line 315, representing base data values for an image property, combined with parallelogram active region 312, representing possible new data values for an image property, provides an intuitive graphical space for translating base data values into new data values. The ends of horizontal line 315 represent the extrema of a range of base data values. For example, in FIG. 3, where brightness is the image property to be modified, the ends of horizontal line 315 correspond to 0% and 100% brightness, and points on line 315 between the ends correspond to intermediate brightness values. Since the points at the ends of horizontal line 315 correspond to the extrema of the range of base data values, upon modification the point at one end of the line 315 may only acquire a larger value and the point at the other end of the line 315 may only acquire a smaller value. Thus, at one end of horizontal line 315 available space in active region 312 exists only above point 318, because the data value assigned to this point can only increase, while at the other end of line 315 space in active region 312 exists only below point 319, because the data value assigned to this point can only decrease. Graphically, this requires that parallelogram vertices 317 coincide with the ends of horizontal line 315.

[00022] Points on horizontal line 315 between the ends 317 of the line correspond to base data values intermediate between the minimum and maximum of the range of values. Since they are intermediate values, they may be either increased or decreased when modified and, therefore, space exists in active region 312 both above and below line 315 for every point on the line except for the endpoints 317. For example, in FIG. 3, the point 318 at the left end of line 315 represents a data value indicating minimum brightness, so active space only exists above point 318, while the point 319 of at the right end of line 315 represents a data value that indicating maximum brightness, so active space only exists below point 319. The point in the middle of line 315 represents 50% brightness and can increase to 100% brightness or decrease to 0% brightness, which is why the active space above it is equal to the active space below it.

[00023] A graphical bar 320 representing the range of data values currently assigned to the image property runs parallel to the bottom edge 306 of the square 302. For example, in FIG. 3, the image property to be changed is brightness, and graphical bar 320 displays the range of brightness from 0% brightness at the left end of graphical bar 320 to 100% brightness at the right end of graphical bar 320.

[00024] In addition to horizontal bar 320 in image 300, vertical graphical bars 330, representing the range of possible data values that may be assigned to the image property when it is modified, run parallel to the vertical sides 304 of square 302. Vertical graphical bars 330 do not run the entire length of the vertical sides 304 of the square. Rather, they run parallel to the vertical sides of parallelogram active region 312. Graphical bar 331 becomes brighter as it goes up because the brightness of point 318 can only increase, while graphical bar 332 becomes darker as it goes down because the brightness of point 319 can only decrease.

[00025] A nonorthogonal grid may be drawn inside active region 312. Vertical grid lines 352 parallel to the sides 304 of square 302 indicate lines of constant base data values. Skewed lines 354, parallel to the skewed sides of parallelogram active region 312 indicate lines of constant possible data values that the base data values may acquire through a modification of an image property. For example, in FIG. 3, line 353 at the bottom of parallelogram active region 312 indicates 0% brightness, line 355 at the top of parallelogram active region 312 indicates 100% brightness and line 354 midway between, and parallel to, lines 353 and 355 indicates 50% brightness.

[00026] A user may modify the data values associated with a quantifiable image property by altering the horizontal line 315. A user may select a starting point, e.g., point A, on horizontal line 315 with a mouse or other pointing device and drag the point to a new location in active region 312, e.g., point N. When the user stops dragging the starting point and releases it in active region 312, a new line 360 (which in general may be a curved segment of essentially any shape) is created and drawn though the ends 317 of line 315 and the new position, point N, of the point the user has dragged. In one implementation, the tangent to new line 360 at point N is parallel to horizontal line 315. New line 360 defines the modification of the data values associated with the quantifiable image property, in that the x-coordinate of a point on new line 360 corresponds to the old data value and the y-coordinate of the point on new line 360 corresponds to the new data value. Thus, for example, dragging starting point A on line 315 in FIG. 3 to point N on new line 360 causes grays of 25% brightness in the original image to acquire a value of approximately 66% brightness.

[00027] Further referring to FIG. 3, when a user drags a point on line 315 to a new location, e.g., point N, a vertical line 370 perpendicular to line 315, from point N to line 315 can be displayed to the user. Vertical line 370 serves to remind the user that it is the x-coordinate of point N, rather than the x-coordinate of starting point A, that corresponds to the old data value when the image property is modified. A dot 372 at the base of vertical line 370 on line 315 can be displayed to further reinforce this notion.

[00028] New line 360, passing through end points 317 of line 315 and point N, may be displayed to the user while the starting point is being dragged and before the starting point is released by the user. New line 360 will necessarily change as the starting point is dragged to different locations within active region 312 until the point is released. Additionally, the value of the old data value and the new data value it will acquire when the starting point is released may be displayed either through graphical aids 380 or through a numerical display 382. For example, in FIG. 3, both graphical aids 380 and numerical display 382 indicate that moving starting point A to point N results in changing grays of 25% brightness to grays of 66% brightness.

[00029] As shown in FIG. 4, data values corresponding to an image property may be modified with the use of an x-y coordinate system that includes a skewed active region and a horizontal line through the active region 400. The coordinate system, the active region and the horizontal line are displayed to a user 400. The user selects a point on the horizontal line 410, through the use of a mouse or other pointing device, which may include keyboard commands. The user then drags the selected point to a new location within the active region 420. While the selected point is dragged, a vertical line may be drawn from the point to the horizontal line. The user releases the selected point 430 at the desired destination point and a new line, passing through the ends of the horizontal line and through the destination point, is defined 440. Data values corresponding to the image property are redefined according to the shape of the new line 450.

[00030] Other embodiments are within the scope of the following claims.

[00031] For example, when an image property is modified by modifying the horizontal line and creating a new line, the new line need not be defined by only the endpoints of the horizontal line and a single point dragged by the user. More than one point on the horizontal line may be dragged to new positions and the new line may be defined to pass through all of the new positions of the points in addition to the endpoints of the horizontal line. The new line may also be defined by use of control points that the new line does not pass through. Furthermore, positions of points through which new line 360 passes need not be defined by dragging points from line 315, but may be defined by other selection methods, such as with a mouse or other such pointing device or by entering the new point's coordinates with, e.g., a keypad. The multiple new positions of points selected by the user may define the nodes of the new line that passes through them and define the shape of the new line.

[00032] The horizontal line that represents the range of base values need not be strictly horizontal. It can, for example, be vertical or at some angle to vertical or horizontal that is close enough to vertical or horizontal to provide the same benefits. In other arrangements, the line can be at any angle to horizontal or vertical if the skewed region showing the possible new values has a grid that is generally perpendicular to the line. The shape of the active region of possible new data values need not be a parallelogram.

[00033] The line 370 that shows the change in y-value for a base x-value is useful even with the known arrangement of FIG. 2.

[00034] A variety of mathematical or empirical functions may be used for drawing curve 360 to pass through the point N and the end points of line 315.

[00035] The relationship of (i) the normal distance of a point on new line 360 to line 315 to (ii) the change from old value to new value need not be linear. Thus, for example, the scale of the grid in the active region can be logarithmic in the normal direction.

[00036] The base values can be default values with the new line showing a current transformation from the default. The base values can also be values the resulted from a previously-performed transformation. Or, the base values can be selected in some other way.

[00037] The techniques described here may be implemented in hardware, software, or a combination of the two. The techniques may be implemented in computer programs executing on programmable computers that each include a processor, a storage medium readable by the processor, including volatile and nonvolatile memory and/or storage elements, and other suitable input and output devices.

[00038] FIG. 1 illustrates one such computer system 100, including a central processing unit (CPU) 110, random access memory (RAM) 120, read only memory (ROM) 122 and an input/output controller 130 coupled by a CPU bus 140. The input/output controller 130 may also be coupled by an input/output bus 198 to input devices such as a keyboard 160, a pointing device 170, e.g., a mouse, and output devices such as a display device 180, e.g., a video monitor. A computer program implementing a routine that manages the control box shown in FIG. 3 may be stored in RAM 120, ROM 122, or on a storage medium or device 190, e.g., CD-ROM, hard disk or magnetic diskette. The computer program may be readable by a general or special purpose programmable computer for configuring and operating the computer to perform the techniques described here.