Automatic Touch Screen Calibration Method

1. Field of the Invention

The present invention relates in general to a method of calibrating a touch screen used in conjunction with a display such as a cathode ray tube (CRT); and more particularly, to a method for automatically calibrating a CRT touch screen prior to each session during which the touch screen is used.

2. Description of the Related Art

The use of a touch screen overlaying a CRT screen is well known for inputting data into a computer. Such touch screens are substantially transparent and may include a net of wires or conductive plastic or utilize ultrasonic or photo-detectors at the edges of the screen. Since the purpose of such touch screens is to detect the location of a finger or other object disposed in proximity to an area on the CRT display, it is necessary for the computer receiving data from a touch screen to know the correspondence between the display and the touch screen. Therefore, it is necessary to calibrate the coordinates output by the touch screen with the coordinates of points onthe display.

A conventional method for establishing the correspondence between a display and a touch screen is described in U.S. Pat. No. 4,442,317 to Jandrell. Two techniques are described in Jandrell. The first technique utilizes a pair of scaling and offset amplifiers in each of the horizontal and vertical directions to adjust the output of the touch screen to match that of the display. The second technique is used in the case of non-uniform electric fields and is based on mapping between display positions and probe potentials. In both cases, it is necessary for manual adjustments to be performed, either to the scaling and offset amplifiers or to establish the mapping between the touch screen and the displayed data.

After a touch screen has been calibrated as described in Jandrell, the calibration operator can be expected to properly input data using the touch screen. However, studies have been performed which indicate that different people tend to touch the screen of a display at different locations when selecting the same area on the CRT. For example, someone seated below the screen may consistently touch the bottom of a displayed symbol, while someone standing above the screen may consistently touch a higher point. Similarly, one person may touch to the left and another to the right of a symbol. Therefore, it is desirable to make some allowance for individual variation in interaction with a touch screen overlaying a computer display.

One solution to the problem of individual variance is described in U.S. Pat. No. 4,396,977 to Slater et al. The solution in Slater et al. is to provide feedback to an operator to indicate the position detected as having previously been touched. A two step process is used in the case of critical inputs in which the first step involves selecting a command by touching one of the options displayed on the screen and the second step involves touching another area on the screen to execute the command selected. This method requires that the feedback routines and two step selection process be used in every program utilizing the touch screen input. In addition, there is no allowance for a severely misaligned touch screen which would require an operator to remember not to touch a desired symbol but to touch the next symbol to the right, for example. In other words, this method results in the training of an operator rather than the "training" or calibration of the equipment.

An object of the present invention is to provide a method for improving the accuracy of input on a touch screen overlaying a graphic display.

Another object of the present invention is to provide an automatic touch screen calibration method which is performed at the beginning of an operator session and does not require any specialized procedures in subsequent data input routines.

A further object of the present invention is to provide an automatic touch screen calibration method which adapts to the idiosyncracies of each individual operator.

Another object of the present invention is to provide an automatic touch screen calibration method which can be used with a variety of touch screens having different coordinate ranges and formats.

Yet another object of the present invention is to provide an automatic touch screen calibration method which detects severe misalignment of a touch screen and an associated display.

The above objects are provided by an automatic touch screen calibration method comprising the steps of displaying, prior to each interactive session, a first point on a display; obtaining first coordinates of a first touch on the touch screen at the first point; and generating a translation matrix for translating the coordinates of the first touch into display coordinates. Preferably, the method includes displaying second and third points on the display and obtaining second and third coordinates of second and third touches at the second and third points, respectively. When the first, second and third coordinates of all three points have been obtained, the translation matrix is preferably generated in dependence upon the first, second and third coordinates. In addition, the first, second and third points preferably define perpendicular axes and the automatic touch screen calibration method includes verifying that an angle defined by the first, second and third coordinates is substantially equal to 90.degree. and that the coordinates lie within predetermined ranges.

These objects, together with other objects and advantages which will be subsequently apparent, reside in the details of the construction and operation as more fully hereinafter described and claimed, reference being had to the accompanying drawings forming a part hereof, wherein like reference numerals refer to like parts throughout.

FIG. 1 is a block diagram of a computer system to which the present invention can be applied;

FIG. 2 is a representation of the output displayed during one step of an embodiment according to the present invention;

FIG. 3 is a flowchart of an embodiment of the present invention; and

FIG. 4 is a graphical representation of the correspondence between display screen and touch screen axes.

As the reliability of touch screen input devices improves, this method of input may be expected to be more widely used since no training is required to learn how to use such input devices. Thus, the computer user is not required to be a proficient typist or know how to use a "mouse" input device. In addition, while the terms CRT and display are used interchangeably in describing this embodiment, it is known that touch screens can also be used with plasma and rear-projection displays, etc. The present invention can be used to calibrate or map any type of touch screen overlay with virtually any type of display.

A typical device using a touch screen is illustrated in FIG. 1. In FIG. 1, a touch screen overlay 10 is mounted on a CRT 15 and both are connected to a processor 20 within a display module 25. The processor 20 within the display module 25 may be connected to a system data base 30 along with other display modules 35 via a bus 40. Other devices (not shown) may also be connected to the bus 40, such as a mainframe computer, input/output devices or process control equipment. The computer system illustrated in FIG. 1 may be used for applications as diverse as seat reservations on an airplane or in a theatre and process control in a power generating or automated manufacturing plant.

An embodiment of the present invention begins the process of calibrating a touch screen 10 with a display 15 to allow for an individual's input idiosyncracies by displaying a screen format such as that illustrated in FIG. 2. As illustrated in FIG. 2, three points are displayed on the CRT 15 and a message is displayed on the screen to "PLEASE TOUCH THE BRIGHT REGION". In FIG. 2, block 45 is illustrated as currently being bright and blocks 50 and 55 are illustrated as currently being dark.

As mentioned above, many different types of graphic display terminals 15 and touch screens 10 are known in the art. Anyone who has written a program for a graphic display terminal, with or without the use of packaged graphics software, can be expected to easily display three rectangles on the screen and a short message, as illustrated in FIG. 2. To give a concrete example, a graphics processor (included in processor 20 of FIG. 1) can be used to communicate with application software using the NAPLPS standard described in ANSI standard X3.110-1983 published by the American National Standards Institute, Inc. The preferred embodiment utilizes at least two, and preferably three, levels of subroutines to separate those routines which are unique to the NAPLPS standard and those which could be used with any graphics and touch screen software. The subroutines which are unique to NAPLPS in the preferred embodiment include a subroutine which transmits NAPLPS commands, a routine which selects a texture for a defined area of the screen, a routine which selects a texture pattern, and a routine which defines the boundaries of the rectangle. Other graphics packages might require different types of routines, such as a series of line segments to outline a rectangle or the specification of a single pixel to define a rectangle.

In the preferred embodiment, one or more higher levels of subroutines call the routines which are unique to the NAPLPS standard. Preferably, general drawing routines are available which are designed to be used with different types of graphics terminals and software packages. Thus, the subroutines described above as being unique to NAPLPS include the communication protocol and character values which are required to cause the NAPLPS processor to display corresponding figures such as rectangles on the CRT 15, while the general figure drawing routines call the NAPLPS routines and pass codes which indicate, e.g., what size rectangle to draw or what texture to use. Therefore, another graphic display terminal could be used and only the NAPLPS routines would have to be replaced.

A flowchart of an embodiment of a method according to the present invention, which calls the general figure drawing routines, is illustrated in FIG. 3. After initialization in block 60, the screen illustrated in FIG. 2 is created by commands indicated by blocks 65 and 70. When the operator responds to the prompt by touching block 45, the processor 20 can obtain the coordinates of the touch, as indicated by block 75, from the touch screen 10 overlaying the CRT 15.

In block 80, the coordinates are checked to determine whether the touch occurred within an expected touch area. If the touch is outside of the expected touch area, a suitable error is displayed by block 85 on the CRT 15 and the operator is given a new opportunity to touch the screen or perform manual calibration. Thus, if the cause of the error message was operator error, such error can be corrected and if the cause of the error was gross misalignment of the touch screen 10 and CRT 15, the touch screen can be manually calibrated to correct for the error.

Assuming the first touch is within an expected touch area, the first block 45 will be darkened and the second block 50 will become the bright region, as indicated by block 90. When the operator touches the second block 50, the coordinates of the second touch are obtained and checked for reasonablness in block 95 using the check described above for block 80. This procedure is repeated, darkening block 50 and brightening block 55 to prompt and then obtain the coordinates for a third touch in block 100 and the third coordinates are checked for reasonableness in block 105.

After the three coordinates have been obtained, the angle defined by these three coordinates is preferably calculated (block 110) to determine whether the angle is substantially 90 degrees (block 115), e.g., 75.degree. to 105.degree.. FIG. 4 is a diagram illustrating the relationship between the axes X and Y of the display 15 and the axes X' and Y' of the touch screen 10. It is assumed that the CRT 15 displays the blocks 45, 50 and 55 at a 90.degree. angle. However, the touch screen 10 supplies the coordinates of an operator's touch at blocks 45, 50 and 55 in terms of the coordinates of the X' and Y' axes. These coordinates will be referred to as (X0, Y0) for block 50, (Xl, Yl) for block 45 and (X2, Y2) for block 55. Using these coordinates, the angle .theta..sub.2 between the X' and Y' axes can be found according to the following equations in which D1 is the distance between blocks 45 and 50, and D2 is the distance between blocks 50 and 55, both in terms of the scale used by the touch screen 10. ##EQU1## If the cos .theta..sub.2 is close to zero, then .theta..sub.2 is approximately 90.degree..

If the assumption is made that the X' axis of the touch screen 10 substantially overlies the X axis of the CRT 15 and the Y' axis of the touch screen 10 substantially overlies the Y axis of the CRT 15, then it is sufficient to have the operator touch only a single point. Such a procedure may be possible when the touch screen is installed by the manufacturer of the graphic display unit or the X' and Y' axes are visible on the touch screen 10 and are aligned with the X and Y axes on the CRT 15 when the touch screen 10 is mounted on the CRT 15.

In such a case, it is possible to determine scaling factors in the X and Y directions by having an operator touch a point located a substantial distance from both the X and Y axes. By dividing the X and Y coordinates detected by the touch screen with the X and Y graphic display coordinates, respectively, X and Y coefficients, X.sub.coef and Y.sub.coef, which form two elements of a simplified coordinate translation matrix can be generated as indicated by formulas (4) and (5), where X.sub.gd and Y.sub.gd are graphic display coordinates and Y.sub.ts and Y.sub.ts are touch screen coordinates:

As a further improvement on this simplified embodiment, the operator may be prompted to touch the origin of the X and Y axes (point 50 in FIG. 4) to generate touch screen coordinates of X0 and Y0 corresponding to X and Y intercepts. Assuming the graphic display coordinates of this point are (0, 0), the following matrix equation results from using a coordinate translation matrix generated from the coordinates input via the touch screen 10, when two points are touched on the touch screen: ##EQU2##

The preferred embodiment, however, allows for greater variety in misalignment of the touch screen 10 with the CRT 15. In the preferred embodiment, after checking that the coordinates for each of the points is reasonable and the axes X' and Y' are at an angle of approximately 90.degree., the preferred method according to the present invention calculates a coordinate translation matrix for translating touch screen coordinates into display coordinates using the coordinates of blocks 45, 50 and 55. This step is indicated as block 120 in FIG. 3 and will be described with reference to FIG. 4. As illustrated in FIG. 4, an angle .theta..sub.1 is formed between the X' axis of the touch screen 10 and the X axis of the CRT 15. The angle .theta..sub.1 is defined by equations (6) and (7):

The coordinate translation matrix generated in block 120 preferably stores the relationship between the touch screen coordinates and the display coordinates as a set of ten terms: calib(0) to calib(2) are used for generating an X display coordinate in which the distance from the point 45 to the point 50 is assumed to equal 1; calib(3) to calib(5) are used to generate a Y coordinate that assumes the distance between the point 50 and 55 equals 1; X and Y scale factors are stored in calib(6) and calib(7), respectively, which can be multiplied times the X and Y coordinates generated by calib(0) to calib(5); and calib(8) and calib(9) in which the X and Y coordinates of point 50 are stored, respectively, which allow for a difference between the assumed coordinates (0, 0) and the actual coordinates of point 50.

The values of the coordinate translation matrix generated in block 120 are set forth below in equations 8-17.

Once the coordinate translation matrix has been generated in block 120, subsequent application routines can use the coordinate translation matrix to generate normalized X and Y coordinates as indicated below. ##EQU3## Then, the actual display coordinates can be generated according to the following formulas (16) and (17).

Following the generation of the coordinate translation matrix, a message can be displayed to the operator indicating that the calibration is completed (block 125) and then the touch screen 10 and display 15 may be used in an interactive session as indicated by block 130. If the angle .theta..sub.2 is not reasonably close to 90.degree., a suitable error message can be displayed as indicated at block 135.

Thus, according to the present invention, an operator is required only to touch three areas on the touch screen 10 and the correspondence between the touch screen 1 and CRT 15 will be established for the manner in which that operator touches the screen 10. In addition, if the touch screen 10 and CRT 15 are severely misaligned, such misalignment will be detected and appropriate error messages can be displayed to inform the operator how to correct the problem. Furthermore, the invention is not limited to any particular touch screen or graphics display terminal or software and once implemented can be relatively easily adapted to different hardware and software packages.

The many features and advantages of the present invention are apparent from the detailed specification, and thus it is intended by the appended claims to cover all such features and advantages of the system which fall within the true spirit and scope of the invention. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the invention to the exact construction and operation illustrated and described, accordingly, all suitable modifications and equivalents may be resorted to, falling within the scope and spirit of the invention.