Coordinate input apparatus and its control method

[0001] The present invention relates to a coordinate input apparatus and its control method and, more particularly, to a coordinate input apparatus and its control method which is used to control a connected computer or to write characters, graphics, and the like by inputting coordinates in an input region using a pointing device.

[0002] Conventionally, as an apparatus of this type, touch panels of various methods have been proposed or are commercially available. Such touch panels are prevalently used since they allow easy operation of a personal computer (PC) on the screen without using any special tool or the like.

[0003] Various methods such as a method using a resistor membrane or an ultrasonic wave, and the like are available. As an apparatus using light, as disclosed in Japanese Patent Laid-Open Nos. 2001-84107, 2003-186616, and the like, there are provided apparatuses which detect the angle of a region which is shielded by a finger or the like in the input region, and detect the coordinates of the light-shielded position, i.e., input position in the arrangement in which a recurrent reflector is arranged around a coordinate input region, light coming from photo transmission means is reflected by the recurrent reflector, and a light amount distribution is detected by photo reception means.

[0004] These apparatuses aim at providing a coordinate input apparatus that allow a plurality of operators to make inputs. For example, Japanese Patent Laid-Open No. 2001-84107 describes the following contents. That is, an apparatus has angle detectors each of which cyclically detects a point designated on an input region by pointing means as an angle, and comprises an angle memory for storing an angle detected in the previous cycle, an angle selector for, when the number of detected angles in the current detection is larger than that in the previous detection, selecting an angle closest to the angle stored in the angle memory from the angles detected in the current detection, and a coordinate calculator for detecting coordinates on the basis of the selected angle. When the number of currently detected angles is larger by one than the number of previous detected angles for each angle detector, coordinates input by another pointing means are detected from the angle which is not selected by the angle selector. This apparatus also has identifier assignment means which can assign an identifier of the pointing means to coordinates to be output. When coordinates are to be output to a computer or the like, two coordinate positions designated by two pointing means can be identified from each other.

[0005] Japanese Patent Laid-Open No. 2003-186616 describes the following contents. That is, when two predetermined objects are simultaneously inserted onto an information input region, four two-dimensional position coordinate values are calculated except for a special case. The angles of light beams coming from photo transmission means, which are shielded or reflected by the predetermined objects that pointed the information input region, are calculated, and two two-dimensional position coordinate values actually pointed by the two predetermined objects are determined from the four calculated two-dimensional position coordinate values on the basis of the calculated angles. Furthermore, the arrangement has means for assigning different identification numbers to the two-dimensional position coordinate values calculated first. It is determined to which of the previously calculated two-dimensional position coordinate values each of the two two-dimensional position coordinate values which are kept calculated is closer, and the identification number of the previously calculated two-dimensional position coordinate value which is determined to be closer is kept assigned.

[0006] In the aforementioned apparatus, the pointing means is a finger or a pen-shaped pointing device. The coordinate input apparatus of this type is configured to control an application by pointing (touching) the screen. The pen which forms the above apparatus cannot transmit any commands for controlling a computer by itself, and has many restrictions upon operation of an application on the computer, thus causing problems in terms of functions. Hence, such pointing means is not always convenient.

[0007] On the other hand, upon operation of an application on the computer, functions corresponding to left clicking, right clicking, and the like as mouse functions are indispensable. An apparatus that allows various operations depending on how to touch so as to implement these functions is known. For example, this apparatus implements functions by so-called gesture input operations or the like by means of touch intervals and input traces. However, it is troublesome for the operator if he or she must learn gesture commands, and cannot operate intuitively. Hence, the operability is not so good as in the above case.

[0008] In order to allow inputs by a plurality of operators, attributes corresponding in number to input pointing means must be appended to coordinate values to be output to the computer side when the coordinate values are output. More specifically, in order to display the coordinate values input by each pointing means as a continuous input trace for that pointing means, the computer must recognize the pointing device that input the coordinate values.

[0009] As the apparatus corresponding to the above description, each of Japanese Patent Laid-Open Nos. 2001-84107 and 2003-186616 described above discloses the arrangement of a coordinate input apparatus which allows inputs by a plurality of operators. As identification numbers to be assigned to coordinate values input by the plurality of operators, different identification numbers are assigned in turn to those which are input earlier, and the same identification numbers are kept assigned to coordinate values which have closer distances to immediately preceding coordinate values. The identification number is kept assigned to the coordinate values of each pointing device until all inputs disappear.

[0010] However, with this arrangement, while the continuity of the trace after input can be assured, the attribute of the coordinate value input first cannot be determined.

[0011] Therefore, it is impossible to assign the attribute of the pen itself to the coordinate values to be output. A trace can be drawn during continuous input. However, since the identification number is cleared once the touch operation is completed, the functions of an application cannot be fully used. For example, problems are posed in operations when the operator wants to change a mode by pressing once an icon used to change a color, thickness, graphic, or the like as an operation for changing the line type or shape of drawing during input. That is, inputs using a plurality of pens are allowed, but combinations of pointing devices used to input and commands cannot be determined. Hence, processes corresponding to the attributes of pointing devices cannot be done.

[0012] Therefore, in order to use an application on the computer by a plurality of operators with high operability, unique attributes of pointing devices of the operators must be assigned to a plurality of input coordinate values.

[0013] In one aspect of the present invention, a coordinate input apparatus includes a photo transmission/reception unit having a phototransmitter which projects light along a plane of an input region, and a photoreceptor which detects incoming light; a reflector which is arranged on a rim portion of the input region, and configured to reflect incident light from the phototransmitter in the photo transmission/reception unit toward the photoreceptor in that photo transmission/reception unit; a pointing device used to make a pointing operation for pointing an arbitrary position in the input region; a detection unit configured to detect a shade formed by the pointing operation of the pointing device using the photo transmission/reception unit; and an output unit configured to calculate a coordinate value of the position on the basis of a state of the detected shade, and outputs information of the coordinate value, wherein the pointing device includes a transmission unit configured to transmit a control signal including information of a unique identifier based on the pointing operation, and the output unit extracts the information of the identifier from the control signal transmitted based on the pointing operation associated with the shade, and appends the information of the coordinate value when that information is output.

[0014] The above and other objects and features of the present invention will appear more fully hereinafter from a consideration of the following description taken in connection with the accompanying drawings wherein one example is illustrated by way of example.

[0044] Preferred embodiments of the present invention will be described in detail in accordance with the accompanying drawings. The present invention is not limited by the disclosure of the embodiments and all combinations of the features described in the embodiments are not always indispensable to solving means of the present invention.

(Schematic Arrangement of Coordinate Input Apparatus)

[0045] A schematic arrangement of an information processing system according to this embodiment will be explained first using FIGS. 1A and 1B. FIG. 1A is a perspective view showing the outer appearance of an image display/input unit of the information processing system of this embodiment, and FIG. 1B is a block diagram showing the hardware arrangement of this information processing system.

[0046] This information processing system has an arrangement including a projector 7 as a display apparatus for displaying an image, a coordinate input apparatus 8 arranged on an image display screen 7a of this projector 7, and a host computer 6 serving as a control apparatus (host) which is connected to the projector 7 and coordinate input apparatus 8, and controls them. The host computer 6 is implemented by, e.g., a personal computer (PC). Therefore, the projector 7 and coordinate input apparatus 8 can be connected to the host computer 6 via interfaces such as USB or the like, respectively.

[0047] The arrangement of the coordinate input apparatus 8 will be described below. Referring to FIG. 1B, reference numerals 1L and 1R denote sensor units each having a phototransmitter and photoreceptor. The sensor units 1L and 1R are set at positions spaced by a predetermined distance. The sensor units 1L and 1R are connected to a control/arithmetic unit 2 for making control and arithmetic operations, receives control signals from the control/arithmetic unit 2, and transmits detected signals to the control/arithmetic unit 2. Reference numeral 3 denotes a reflector having a recurrent reflection plane which reflects incoming light to the arrival direction, as shown in FIG. 2. The reflector 3 recurrently reflects light beams, which are projected from the left and right sensor units along the plane of an input region 5 within the range that covers the input region 5 (e.g., the range of about 90°), toward the sensor units. The reflected light is one-dimensionally detected by the photoreceptor of each sensor unit, which comprises a focusing optical system, line CCD, and the like, and its light amount distribution is sent to the control/arithmetic unit 2.

[0048] The input region 5 can be used as an interactive input device when it is formed on a display screen of a display apparatus such as a PDP, rear projector, LCD panel, or the like. In this system, when a predetermined pointing device 4 is pressed against the input region 5 and is moved, a trace 4a can be input. Also, when an instruction input is made to various kinds of information such as displayed icons and the like, various applications can be controlled.

[0049] With this arrangement, when an input instruction is made on the input region by the pointing device 4, a finger, or the like, light projected by each phototransmitter is shielded, and no reflected light of recurrent reflection is obtained. Hence, no light amount is obtained from only the input instruction position.

[0050] The control/arithmetic unit 2 detects a shade range of the input instruction portion on the basis of light amount changes of the left and right sensor units 1L and 1R, specifies a detection point within the range, and calculates respective angles. Then, the control/arithmetic unit 2 calculates a coordinate position on the input region 5 on the basis of the calculated angles, the distance between the sensor units, and the like, and outputs the coordinate value to the host computer 6.

[0051] In this way, the operations of the host computer 6 (e.g., drawing a line, operating icons, and the like on the screen) can be made by a finger or the like.

[0052] Respective units of the coordinate input apparatus 8 will be described in detail below.

(Detailed Description of Sensor Unit)

[0053] FIGS. 3A and 3B are views showing an example of the arrangement of the phototransmitter in each of the sensor units 1L and 1R. FIG. 3A is a top view of the phototransmitter (in a vertical direction to the input plane), and FIG. 3B is a side view of phototransmitter (in a horizontal direction to the input plane). Reference numeral 31 denotes an infrared LED which emits infrared light. The emitted light emanates in the range of about 90° via a projection lens 32. In this direction, light emitted by the infrared LED 31 is projected as a light beam which is limited in the up-and-down direction, and is mainly projected toward the reflector 3.

[0054] FIG. 4 shows the photoreceptor in each of the sensor units 1L and 1R when viewed from the vertical direction to the input plane. The photoreceptor has a one-dimensional line CCD 41, lenses 42 and 43 as a collection optical system, a stop 44 that limits the incident direction of incident light, and an infrared filter 45 which prevents extra light such as visible light or the like from entering.

[0055] Light emitted by the phototransmitter is reflected by the recurrent reflector, and passes through the infrared filter 45 and stop 44. Then, the light within the range of about 90° of the input plane is focused by the focusing lenses 42 and 43 on pixels depending on the incident angles on the detection plane of the CCD 41, thus representing light amount distributions for respective angles. That is, each pixel number expresses angle information.

[0056] FIG. 5 shows the arrangement of each of the sensor units 1L and 1R when the input plane is viewed from the horizontal direction. As shown in FIG. 5, the phototransmitter and photoreceptor are stacked. The distance between the optical axes of the phototransmitter and photoreceptor can be set within a sufficiently detectable range on the basis of the angle characteristics of the recurrent reflector.

(About Reflector)

[0057] The recurrent reflector 3 shown in FIG. 1 has reflection characteristics with respect to the incident angle. As can be seen from the graph showing the characteristics of the reflected light amount with respect to the incident angle shown in FIG. 6, when the recurrent reflector 3 is formed flat, the obtained reflected light amount decreases from a position where the angle from the reflector exceeds 45°, and when a shield exists, the change cannot be sufficiently detected.

[0058] The reflected light amount is determined by the light amount distribution (the illumination intensity and distance), the reflectance of the reflector (the incident angle, the width of the reflector), and imaging system illuminance (the cosine law). When the light amount is insufficient, the illumination intensity may be increased. However, if the reflection distribution is not uniform, when light of an intense portion is received, the CCD as light-receiving means may be saturated at that portion; the illumination intensity has an upper limit. Put differently, an increase in incident light amount to a low light amount portion can be expected by setting the reflection distribution of the reflector to be as uniform as possible.

[0059] In order to uniformize in the angle direction, a member to which the recurrent reflector 3 is to be adhered adopts a form defined by arranging triangular prisms, as shown in FIG. 7, and the recurrent reflector 3 is set on such member. With this arrangement, the angle characteristics can be improved. Note that the angle of each triangular prism can be determined from the reflection characteristics of the recurrent reflector, and its pitch is preferably set to be equal to or lower than the detection resolution of the CCD.

(Description of Control/Arithmetic Unit)

[0060] The control/arithmetic unit 2 and sensor units 1L and 1R shown in FIG. 1B exchange CCD control signals, CCD clock signals, CCD output signals, and LED drive signals.

[0061] FIG. 8 is a block diagram showing the arrangement of the control/arithmetic unit 2. The CCD control signals are output from an arithmetic control circuit (CPU) 83 which comprises a one-chip microcomputer or the like, so as to set the CCD shutter timings and to attain data output control and the like. The CCD clock signals are sent from a clock generation circuit 87 to the sensor units 1L and 1R, and are also input to the arithmetic control circuit 83 to make various kinds of control in synchronism with the CCDs.

[0062] The LED drive signals are supplied from the arithmetic control circuit 83 to the infrared LEDs of the sensor units 1L and 1R via LED drive circuits 84L and 84R.

[0063] Detection signals from the CCDs of the photoreceptors of the sensor units 1L and 1R are input to A/D converters 81L and 81R, and are converted into digital values under the control of the arithmetic control circuit 83. The converted digital values are stored in a memory (e.g., a RAM) 82, and are used in angle calculations. After a coordinate value is calculated from the calculated angles, the coordinate value data is output to the host computer 6 via line 88.

(Description of Light Amount Distribution Detection)

[0064] FIG. 9 is a timing chart of respective control signals associated with LED light emission.

[0065] Signals SH, ICGL, and ICGR are control signals for CCD control, and the CCD shutter release time is determined at the intervals of SH. The signals ICGL and ICGR are respectively gate signals to the left and right sensor units 1L and 1R and are used to transfer charges in internal photoelectric conversion units of the CCDs to read units. Signals LEDL and LEDR are respectively drive signals of the left and right LEDs. In order to turn on one LED in the first period of SH, the drive signal LEDL is supplied to the LED via the LED drive circuit, and the other LED is driven in the next period. After the drive operations of both the LEDs are completed, CCD signals are read out from the left and right sensor units.

[0066] As the signals to be read out, when no input is made, a light amount distribution shown in FIG. 10 is obtained as the output from each sensor unit. Of course, this light amount distribution is not always obtained by every systems, and changes depending on the characteristics of the recurrent reflector 3, the characteristics of the LEDs, and aging (contamination on the reflecting surface). In FIG. 10, a level A corresponds to a maximum light amount, and a level B corresponds to a minimum light amount. That is, in a state of no reflected light, the obtained level is near B, and the level shifts toward A with increasing reflected light amount. In this way, data output from the CCDs are sequentially A/D-converted, and are fetched by the CPU as digital data.

[0067] FIG. 11 shows an example of the output when an input is made by a finger or the like, i.e., when reflected light is shielded. Since reflected light of a portion C is shielded by a finger or the like, the light amount of only that portion lowers. Detection is made from such change in light amount distribution. More specifically, an input-less initial state shown in FIG. 10 is pre-stored, and whether or not a change shown in FIG. 11 takes place is detected by a difference from the initial state in each sample period. If the change is detected, an arithmetic operation for determining the input angle to have that portion as an input point is made.

(Description of Angle Calculation)

[0068] Upon executing angle calculations, a shade range must be detected first. As described above, since the light amount distribution is not always constant due to aging and the like, it is preferably stored at, e.g., the launch timing of the system. In this way, even when the recurrent reflection plane is contaminated with dust or the like, it can be used except for a case wherein the recurrent reflection plane cannot reflect any light. Data of one sensor will be described below. The other sensor makes the same process.

[0069] In an input-less state after power ON, the CCD output is A/D-converted without illumination from the phototransmitter, and this digital data is stored as Bas_data[N] in the memory. This data includes variations of a CCD bias and the like, and has a level near B in FIG. 10. Note that N is a pixel number, and pixel numbers corresponding to an effective input range are used.

[0070] Next, the light amount distribution in the illumination state from the phototransmitter is stored. The data to be stored is indicated by the solid line in FIG. 10 and is represented by Ref_data[N]. Using these data, it is determined first if an input is made and if a shade range is detected. Let Norm_data[N] be data in a given sample period.

[0071] In order to specify the shade range, the presence/absence of an input is determined based on the absolute change amount of data. This is to prevent any detection errors due to noise or the like, and to reliably detect a change of a predetermined amount. The absolute change amount is calculated at each pixel by:
Norm_data_—a[N]=Norm_data[N]−Ref_data[N]  (1)
where Norm_data_a[N] is the absolute change amount at each pixel.
The calculated absolute change amount is compared with a threshold Vtha which is determined in advance.

[0072] Since this process calculates a difference and compares it with the threshold, and does not spend a long processing time, the presence/absence of an input can be made at high speed. When a predetermined number or more of pixels exceed Vtha first are detected, the presence of an input is detected.

[0073] In order to allow detection with higher precision, a change ratio is calculated to determine an input point.

[0074] In FIG. 12, reference numeral 121 denotes a recurrent reflection plane of the reflector 3. Assuming that region A has low reflectance due to contamination or the like, the reflected light amount of region A decreases in the distribution of Ref_data[N], as shown in FIG. 13A. In this state, when a pointing device such as a finger or the like is inserted and covers nearly half the recurrent reflector, as shown in FIG. 12, the reflected light amount is nearly halved, and a distribution Norm_data[N] indicated by the bold line in FIG. 13B is observed.

[0075] If equation (1) is applied to this state, a result shown in FIG. 14A is obtained. In FIG. 14A, the ordinate plots a difference voltage from the initial state. When the threshold is applied to this data, the result may fall outside the original input range. Of course, if the threshold is lowered, an input can be detected to some extent, but it may be influenced by noise or the like.

[0076] Hence, the change ratio is calculated. Since the reflected light amounts of both regions A and B are half the initial amounts, the ratio is calculated by:
Norm_data_—r[N]=Norm_data_—a[N]/(Bas_data[N]−Ref_data[N])  (2)

[0077] This calculation result is, as shown in FIG. 14B, and is expressed by the change ratio. Hence, even when reflectances are different, data can be equally handled, and detection with high precision is attained.

[0078] A threshold Vthr is applied to this data, and an angle is calculated using a central pixel as an input pixel on the basis of the pixel numbers corresponding to the leading and trailing edges of the data. FIG. 14B is illustrated for the sake of convenience: such leading edge does not appear in practice, and pixels exhibit different levels.

[0079] FIG. 15 shows a detection example after the ratio calculations. Assuming that detection is made using the threshold Vthr, the leading edge of the shade area exceeds the threshold at the N_r-th pixel. Furthermore, the trailing edge becomes lower than Vthr at the N_f-th pixel. In this state, a central pixel N_p may be calculated by:
N_p=N_r+(N_f−N_r)/2  (3)
As a result, the pixel interval has a minimum resolution. Hence, in order to detect more precisely, a virtual pixel number that intersects the threshold is calculated using the level of each individual pixel and that of a pixel immediately before that pixel.

[0080] Let L_r be the level of N_r, and L_r−1 be the level of the N_r−1th pixel. Also, let L_f be the level of N_f, and L_f−1 be the level of the N_f−1-th pixel. Then, respective virtual pixel numbers N_rv and N_fv can be calculated by:
N_rv=N_r−1+(Vthr−L_r−1)/(L_r−L_r−1)  (4)
N_fv=N_f−1+(Vthr−L_f−1)/(L_f−L_f−1)  (5)

[0081] A virtual central pixel N_pv is expressed by:
N_pv=N_rv+(N_fv−N_rv)/2  (6)

[0082] By calculating the virtual pixel numbers from the pixel numbers and their levels, detection with higher resolution can be attained.

[0083] In order to calculate an actual coordinate value from the obtained central pixel number, the central pixel number must be converted into angle information. In actual coordinate calculations to be described later, it is convenient to calculate a tangent value at a given angle in place of the angle itself. Conversion from the pixel number into tan θ uses table lookup or a conversion formula.

[0084] FIG. 16 shows a graph formed by plotting tan θ values corresponding to the pixel numbers. An approximate expression is calculated based on this data, and the pixel number—tan θ conversion is made using this approximate expression. If a high-order polynomial is used as the conversion formula, higher precision can be assured. However, the conversion formula can be determined in consideration of the computation performance, precision specification, and the like.

[0085] When a 5th-order polynomial is used, six coefficients are required. Hence, this data can be stored in a nonvolatile memory or the like upon delivery or the like.

[0086] Let L5, L4, L3, L2, L1, and L0 be coefficients of the 5th-order polynomial. Then, tan θ is given by:
tan θ=((((L5*Npr+L4)*Npr+L3)*Npr+L2)*Npr+L1)*Npr+L0  (7)

[0087] When the same process is applied to respective sensors, their angle data can be determined. Of course, the above example calculates tan θ. Alternatively, the angle itself may be calculated, and tan θ may then be calculated based on the angle.

(Description of Coordinate Calculation Method)

[0088] Coordinates are calculated from the obtained angle data.

[0089] FIG. 17 shows the positional relationship with the screen coordinates. The sensor units are attached to the left and right ends of the lower side of the input range, and the distance between these sensor units is represented by Ds.

[0090] The center of the screen is an origin position of the screen, and P0 is an intersection between the angles 0 of the respective sensor units. Let θL and θR be the angles of the sensor units, and tan θL and tan θR are calculated using the above polynomial. At this time, the x- and y-coordinates of a point P are expressed by:
x=Ds/2*(tan θL+tan θR)/(1+(tan θL*tan θR))  (8)
y=−Ds/2*(tan θR−tan θL−(2*tan θL*tan θR))/(1+(tan θL*tan θR))+P0Y  (9)

[0091] FIG. 18 is a flowchart showing the steps from data acquisition until coordinate calculations.

[0092] After the power supply is turned on in step S101, various initialization processes including port settings, timer settings, and the like of the arithmetic control circuit 83 and the like are executed in step S102. Step S103 is preparation for unwanted charge removal, which is done only upon launching. A photoelectric conversion element such as a CCD or the like often accumulates unwanted charges when it is inactive. When such data is used as reference data intact, detection cannot be made, or detection errors are caused. To avoid such problems, a data read process is done a plurality of times first without illumination. In step S103, the number of read times is set. In step S104, data is read out a predetermined number of times without illumination, thus removing unwanted charges.

[0093] Step S105 is a decision statement to check if the predetermined number of times is repeated.

[0094] Step S106 is a process for fetching data without illumination as reference data, which corresponds to Bas_data. The fetched data is stored in the memory (step S107), and is used in calculations. Data Ref_data, which is another reference data and corresponds to the initial light amount distribution with illumination, is fetched (step S108), and is also stored in the memory (step S109). The steps described so far correspond to the initial setting operations upon power ON, and a normal fetch operation starts from the next step.

[0095] The light amount distribution is fetched, as described above, in step S110, and the presence/absence of a shade portion is determined based on the difference value from Ref_data in step S111. If it is determined in step S112 that no shade portion is present, the flow returns to step S110 to repeat the fetch operation.

[0096] At this time, if this repetition period is set to be about 10 [msec], sampling is made 100 times/sec.

[0097] If it is determined in step S112 that a shade area is present, the ratio is calculated by the process given by equation (2) in step S113. The leading and trailing edges are determined using the threshold with respect to the obtained ratio, and a central value is calculated using equations (4), (5), and (6) (step S114). From the calculated central value, tan θ values are calculated using the approximate polynomial (step S115), and x- and y-coordinates are calculated using equations (8) and (9) based on the tan θ values of the left and right sensor units (step S116).

[0098] It is checked in step S117 if a touch has been made. In this step, a proximity input state like a state in which a cursor is moved without pressing any mouse button, or a touch-down state as a state in which a left button has been pressed is determined. In practice, if the maximum value of the previously obtained ratio exceeds a predetermined value (e.g., 0.5), “down” is determined; otherwise, the proximity input state is determined. According to this result, a down flag is set (step S118) or reset (step S119).

[0099] Since the coordinate value and down state are determined, these data are transmitted to the host computer 6 (step S120). In the host computer 6, a driver interprets the received data, and moves a cursor, changes a mouse button state, or the like with reference to the coordinate value, flag, and the like.

[0100] Upon completion of the process in step S120, the flow returns to the operation in step S110, and the above processes are repeated until power OFF.

(Description of Coordinate Input Pen)

[0101] In the coordinate input apparatus of this embodiment, a finger input operation is allowed. Also, when input operations are made using a pointing device such as a pen or the like, operations corresponding to various buttons of a mouse can be intuitively made. A coordinate input pen (to be referred to as “pointing device” or “pen” hereinafter) 4 of this embodiment will be described below using FIG. 19.

[0102] The pointing device 4 of this embodiment comprises a pen tip switch (SW) 141 which is enabled when the pen tip portion as a writing tool is pressed, a plurality of pen side switches (SW) 142 provided to a housing of the pointing device 4, an infrared LED 143, and a drive circuit. In this embodiment, the plurality of pen side switches (SW) 142 includes two side switches 142a and 142b as illustrated in FIG. 19. When one of these switches is enabled, a signal is transmitted from the pointing device 4 at predetermined periods. More specifically, the drive circuit 145 emits optical signals serving as timing signals and command signals at predetermined periods.

[0103] This optical signal is received by a control signal detection circuit 86 (see FIG. 8). The control signal detection circuit 86 determines the enabled one of the switches of the pointing device 4 on the basis of the received optical signal. At the same time, the sensor units 1L and 1R begin to exchange CCD control signals, CCD clock signals, and LED drive signals. More specifically, the pointing device 4 superposes a signal indicating switch information on the optical signal emitted as a timing signal (in addition, an identification code or the like used to identify the coordinate input pen can be superposed). As a method of transmitting such information, a leader field which includes a consecutive pulse train, and a header field which follows the leader field and includes a code (vendor ID or the like) are output first, and a transmission data sequence including control signals and the like (e.g., a pen switch signal, etc.) is sequentially output in accordance with the order and format which are defined in advance. This method is well known (e.g., a remote controller utilizing infrared rays), and a detailed description thereof will be omitted.

[0104] As another method, the enabled switch can be identified based on the above information detected by changing the predetermined period of the coordinate input apparatus of this type that makes coordinate detection at the predetermined periods. If the coordinate input apparatus has a specification that can make coordinate detection at a maximum of 100 points/sec, i.e., at 10-msec intervals, settings are made to make coordinate calculations at, e.g., 100 points/sec when the pen tip SW 141 is enabled, and to make coordinate calculations at 80 points/sec when a given pen side SW 142 is enabled. In other words, the pointing device 4 emits signals at these periods. Hence, the control signal detection circuit 86 can detect the enabled switch by monitoring the periods.

[0105] A detailed arrangement of the pointing device 4 will be described later.

[0106] When the tip of the pointing device 4 is pressed against the input region 5, the pen tip switch 141 is turned on, and a state in which the operator is about to make a coordinate input so as to input handwriting (“pen down” state) is set. When the operator enables the pen tip switch twice within a predetermined period of time, the double-clicking action of a mouse is recognized by monitoring the reception times and interval of signals, coordinate calculation timings, or the like with reference to the coordinate sampling rate of the coordinate input apparatus.

(Description of Up/Down of Coordinate Input Pen)

[0107] The pen up/down process will be described below with reference to the flowchart of FIG. 20.

[0108] The state of the pen tip switch 141 is checked in step S402. The ON state of the pen tip switch 141 is a state in which the coordinate input pen is located on the input region 5, and the operator is about to make coordinate input so as to input handwriting (“pen down” state), and the handwriting displayed on the screen is faithfully reproduced in correspondence with the writing action by the operator. Also, for example, when the operator enables the pen tip switch twice within a predetermined period of time, the double-clicking action of a mouse is recognized by monitoring the reception times and interval of signals, coordinate calculation timings, or the like with reference to the coordinate sampling rate of the coordinate input apparatus.

[0109] On the other hand, if it is determined in step S402 that the pen tip switch 141 is not enabled (OFF state). The states of the pen side switches are checked in steps S403 to S405. In this embodiment, the plurality of pen side switches 142 includes two pen side switches 142a and 142b as described above, and operates as a “pen up” state if either of these switches is ON or a “pen down” state if both of these switches are ON.

[0110] In this manner, if the pen tip switch 141 is OFF and at least one of the side switches 142a and 142b is enabled (ON state), the operator is about to control the screen while suspending the pointing device 4 from the input region 5, i.e., is about to move a cursor to a desired position or to input handwriting at the suspended position (this input state will be referred to as “proximity input” hereinafter). More specifically, in the pen up state, i.e., when one of the side switches 142a and 142b is enabled, the operator can move the cursor displayed on the screen. In the pen down state, i.e., when both of these switches are enabled, the operator can display a moving trace of the pointing device as handwriting in a mode to be described later. Since the pointing device 4 is configured to simultaneously output the pen state (pen up/pen down) as information so as to implement such functions when the coordinate input apparatus calculates coordinates, control software, application software, or the like stored in the host computer 6 can implement a desired action on the basis of the information.

[0111] On the other hand, the state in which one of the switches of the pointing device 4 is enabled corresponds to a state wherein coordinate calculations are always made at the predetermined periods, i.e., signals are emitted from the pointing device 4 at predetermined periods. Therefore, whether coordinates are consecutively input from the coordinate value detected first or the consecutive coordinate input is interrupted can be determined by checking if optical signals as timing signals can be detected at predetermined periods. That is, by monitoring the generation timings of start signals (which are generated at 0.01-sec intervals if the coordinate sampling rate is 100 times/sec) of the control signal detection circuit 86, the consecutive input state can be determined.

[0112] Alternatively, as means for determining a consecutive input, a predetermined period of time may be set, and whether a signal (e.g., the optical signal as the start signal in this embodiment) or a coordinate value is detected within the predetermined period of time may be determined.

(Description about Detection of Input Coordinates by a Plurality of Pointing Devices)

[0113] In the above description, the number of shades detected by the CCD of each sensor unit is one (only one input coordinate value). A case will be explained below wherein a plurality of input coordinate values are calculated and are output to the host computer 6 when a plurality of operators make input using their pointing devices.

[0114] In this case, when a plurality of coordinate values as many as the number of pointing devices are to be output to the host computer 6, the coordinate values must be output after the attribute information of each pointing device is appended to the corresponding coordinate value. That is, in order to display coordinate values input by the pointing devices as consecutive input trances for respective pointing devices, the computer must recognize the pointing devices used to input the coordinate values. Hence, this embodiment adopts an arrangement in which optical signals emitted by the pointing devices are detected by the sensor units in addition to the control signal detection circuit 86.

[0115] FIGS. 21A to 21H are timing charts showing control signals and CCD detection signals when the plurality of pointing devices are used.

[0116] Assume that two pointing devices are used to input, and are defined as pens 1 and 2, respectively. Signals of FIGS. 21A and 21B are pen control signals transmitted by pens 1 and 2 as the pointing devices. The pen control signals are data sequences of infrared rays. In these data sequences, a and a′ of the former half indicate switch signals, and b and b′ of the latter half indicate ID signals.

[0117] Signals of FIGS. 21C and 21D are generated by the arithmetic control circuit 83 in accordance with the pen control signals of FIGS. 21A and 21B, and are switched to Hi/Low in accordance with the pen control signals detected at predetermined periods.

[0118] Signals of FIGS. 21E and 21F are control signals for CCD control, and are as described above. Hence, a description of these signals will be omitted.

[0119] As described above, when two shades h and i are detected in a CCD detection waveform of FIG. 21H detected by the normal fetch operation, it is determined that there are a plurality of inputs. Note that FIGS. 21A to 21H show only the CCD detection waveforms of one sensor unit. However, when the number of shades detected by either CCD is at least two, it is determined that there are a plurality of inputs.

[0120] When it is determined that there are a plurality of inputs, the CCD control signal is transmitted in synchronism with a timing e which is synchronized with the leading edge of the first pulse of the next pen control signal to be detected. An optical signal corresponding to a pulse width d is detected by the CCD during a shutter interval g. At this time, the illumination LED of the sensor unit is turned off by a control signal of FIG. 21G so as not to influence the dynamic range of the CCD. The detected waveform becomes, e.g., j and k of the CCD detection waveform of FIG. 21H, and if the number of pixels falls within a predetermined range by comparing a pixel number 1 corresponding to a shade h which was detected and stored previously with the currently detected pixel number m, it is determined that the shade h is input by pen 2 as the pointing device.

[0121] Note that the shutter interval g is varied as needed by the width of shades h and i previously detected by the CCD so that detection can be made at an appropriate level so as not to influence the dynamic range of the CCD. That is, when the previously detected shade width is broad, the shutter interval is controlled to be short; when it is narrow, the shutter interval is controlled to be long.

[0122] A description about association between the shades and ID signals of the pens will be given using examples of the coordinates on the input region 5 and waveforms detected by the CCD with reference to FIGS. 22A and 22B and FIG. 23.

[0123] FIG. 22A shows a case wherein inputs are made at positions A and B indicated by full dots on the input region 5. In this case, rough CCD detection waveforms are observed as (1)(a)(L) and (1)(b)(L) of FIG. 23 by the sensor unit 1L, and are observed as (1)(a)(R) and (1)(b)(R) of FIG. 23 by the sensor unit 1R. At this time, as described above, since a pixel number NL1 calculated for a shade QL1 and a pixel number NL2 corresponding to detection light (control signal) PL1 of the pen fall within a predetermined value range, it is determined that QL1 corresponds to PL1. Likewise, it is determined that a shade QR1 corresponds to PR1. Therefore, assuming that the number of pointing devices allowed to input is two, since one of the two shades is determined, it is determined that another shade is input by the other pointing device.

[0124] Meanwhile, when coordinates are calculated using the shades QL1, QL2, QR1, and QR2 in (1) of FIG. 23, coordinates A, B, A′, and B′ in FIG. 22A are calculated, i.e., true coordinates A and B, and false coordinates A′ and B′ are calculated, and true/false determination is required to calculate correct coordinates. However, when combinations of shades corresponding to the pointing devices are determined in advance, and coordinates are calculated using the CCD pixel numbers calculated based on the shades of the respective combinations, coordinates A and B can be uniquely determined.

[0125] When coordinates A and B nearly line up when viewed from the sensor unit 1L, as shown in FIG. 22B, coordinates B located at the back side viewed from the sensor unit side cannot be seen from that sensor unit due to the influence of the pen whose input position is located at the front side viewed from the sensor unit side. At this time, the CCD detection waveforms are detected as (2)(a) in FIG. 23, and in the waveform detected by the sensor unit 1L, only one shade is detected like (2)(a)(L) in FIG. 23. On the other hand, since the waveform detected by the sensor unit 1R includes two shades like (2)(a)(R) in FIG. 23, it is determined that there are a plurality of inputs, and correspondence with the pen control signals is determined. In this case, since the CCD of the sensor unit 1L detects only a pen signal corresponding to coordinates A, it is determined that QL1 corresponds to detected PL1. At this time, if the sensor unit 1R detects PR1, it is determined that QR1 corresponds to PR1.

[0126] Note that coordinates B are undesirably calculated as a point of B′ in FIG. 22B if nothing is done. In case of FIG. 22B, the location of B within a range C is unknown if nothing is done. Hence, coordinates B are calculated based on a correction value k which is derived from the relationship between the distance from the sensor unit 1R to coordinates B and the shade width QR2. This correction value k is calculated in advance on the basis of the relationship between the distance, which is calculated from the sensor unit and input coordinates, and the shade width detected by the CCD, is stored in the memory as table information, and is called from the arithmetic control circuit as needed when it is used. When the shades overlap each other in this manner, a third sensor may be arranged in addition to the method of correcting using the relationship between the distance and range. That is, when the number of shades detected by one sensor is 2, and that detected by another sensor is 1, the sensors may be switched to detect inputs from a non-overlapping direction. Even in such case, coordinates can be calculated with higher precision.

[0127] FIG. 24 is a flowchart showing the process for associating the shades and pen IDs in consideration of use of the plurality of pointing devices. The aforementioned process for determining a plurality of inputs is executed in accordance with the flowchart shown in FIG. 24. This routine is called from step S114 of the flowchart shown in FIG. 18, and is executed as follows.

[0128] In step S202, the number of shades is detected on the basis of the number of central pixel numbers between the previously calculated leading and trailing edges. If at least one sensor detects the two shades, it is determined that a plurality of inputs are made. Therefore, it is determined in step S203 that a plurality of inputs are made, a data fetch process without illumination is executed at a timing synchronized with the next pen control signal in step S204. The detected data is stored in the memory in step S205, and a process for determining the correspondence between the shades and detection light beams based on the control signals of the pointing devices is executed in step S206.

[0129] In step S207, a pen ID information read process is executed. In step S208, a process for associating the shades and pen IDs is executed on the basis of the relationship between the shades and control signals of the pointing devices processed in step S206. The determined values are returned in step S209. If it is determined in step S203 that each of the two sensors detects one shade, the flow advances to step S207, and the process for returning the shade and pen ID is executed in step S209 as in the above case.

[0130] In the arrangement of the above description, the pointing device 4 transmits a control signal only when its SW is pressed. When ID information is assigned in a proximity input mode, ID signals are transmitted for respective samples even when the SW of the pointing device 4 is not operated, and the shade is associated with the control signal light, thus realizing the proximity input. In this case, when an ID signal is detected by detecting the control signal, but no shade is detected, the host computer 6 outputs nothing. When both the shade and SW signal are detected, the pen SW signal is checked to determine the proximity input mode, and coordinate information is transmitted to the host computer while assigning the ID signal to the coordinate value. In this way, the process with the pen ID information can also be made in the proximity input mode.

[0131] For example, when the wait time process is prolonged since light beams for a plurality of pen control signals are superposed, and a problem is posed with the transmission periods of coordinates to the computer, the attribute of the pointing device may be checked for the first coordinate point corresponding to the detected input, and the subsequent coordinates may be transmitted by assigning information indicating continuity of the input coordinates to the coordinate information. The continuity of the input coordinate values can be determined on the basis of the distance between the previous and current coordinate values, vector information, and the like.

[0132] In the above description, whether or not the waveform detected by the CCD includes a shade is determined by calculating the leading and trailing edges of the detected waveform to obtain a nearly central pixel number. However, when the dynamic range of the CCD is considered, if an input that exceeds the maximum allowable voltage value Vmax of the CCD is made, its influence may reach the subsequent detected waveforms, as shown in FIG. 27. Therefore, the shade may be associated with the trailing edge information before the CCD is saturated. In this case, NL1 determined based on the leading edge information is calculated for the shade, NL2 determined based on the trailing edge information is calculated for light of the pen control signal, and they are compared with each other, thus associating the shade and pen.

[0133] In the above description, the case has been explained wherein the number of input pointing devices is two. Even when the number of pointing devices is three or more, the shades and pointing devices can be associated with each other by detecting light beams of the control signals of the pointing devices in turn. Furthermore, inputs using a finger and pen together are allowed. For example, an ID signal may be assigned to only a coordinate value, which is calculated based on a shade with which pen control signal light is associated. On the other hand, a shade with which no pen control signal light is associated may be determined as an input by the finger, and a flag indicating the finger input may be assigned to the coordinate value.

[0134] If the same processing method is used, for example, in a specification that permits use of only pens, and inhibits finger inputs, a process for determining a shade with which no pen signal is associated as a detection error, and inhibiting its coordinate value from being output can be executed. For example, even when a sleeve or hand touches the screen and is detected as a shade when an input is made using the pen, coordinates input by only the pen can be output, thus providing a convenient coordinate input apparatus.

[0135] With the above arrangement, various processes corresponding to the attributes of pointing devices can be made by, e.g., assigning color information to be displayed on the computer to each pointing device in advance. In addition, the aforementioned processes can be attained with low cost without increasing the number of parts, since an optical signal as a pen control signal is detected by the CCD for coordinate detection (shade detection). Furthermore, since a plurality of coordinate values can be transmitted to the computer while assigning the attributes of the pointing devices which can be uniquely determined, a coordinate input apparatus which allows a plurality of operators to use an application on the computer with high operability without limiting the functions can be provided.

(Description of Case with Undetectable Region)

[0136] An optical signal emitted by the infrared LED 143 (FIG. 19) which forms the pointing device 4 is preferably detected on the entire input region 5 by the control signal detection circuit 86. In some cases, an optical signal of the control signal cannot be detected on the entire inputtable region by the sensor unit depending on the positional relationship between the position (height from the input plane) of a phototransmitter member formed by the infrared LED 143 in the pointing device 4.

[0137] A case will be explained below using FIGS. 25A and 25B and FIG. 23 wherein a region that cannot be detected by the sensor unit 1L exists.

[0138] In FIGS. 25A and 25B, assume that a region D is an undetectable region of the sensor unit 1L. Even in this case, the ID information of the pointing device can be assigned to the coordinate value by executing the correspondence determination process between the shade and optical signal of the pointing device as long as the two sensor units do not have an overlapping undetectable region.

[0139] As shown in FIG. 25A, when both of a plurality of input points A and B exist in the undetectable region D, the waveform detected by the CCD is as shown in (3) in FIG. 23. Shades in (3)(a) in FIG. 23 can be determined in the same manner as in the above example. Furthermore, a shade width is also detected. The shade width can be easily calculated from leading edge information and trailing edge information, and whether or not an input exists in the undetectable region D can be determined using this shade width information.

[0140] Assuming that it is determined that shade widths ΔNL1 and ΔNL2 exist in the undetectable region since they exceed a predetermined width, it can be determined that the undetectable region is present near the sensor unit 1L. In this case, the CCD read process of an optical signal for the control signal of the pointing device is skipped, and an operation for reading a CCD detection signal based on a pen control signal is executed while turning off illumination only on the unit 1R side. In this manner, the read time can be shortened, thus speeding up the process.

[0141] Since a shade QR1 is associated with control signal light PR1, if ΔNL1>ΔNL2, the shade QR1 is closer to the unit 1L. Hence, the unit 1R can associate the shade QR1 with a shallower angle with a shade QL1, i.e., it associates the shade QL1 with the pen ID of the control signal light PR1.

[0142] When the input positions nearly line up within the control signal light undetectable region D when viewed from the unit 1R, as shown in FIG. 25B, it is determined based on shade widths in (4)(a) in FIG. 23 that they exist in the undetectable region. Since control signal light PR1 detected on the unit 1R side is an input point closer to the unit 1R, it is determined that a shade with a shallower angle corresponds to PR1 when viewed from the unit 1L side. Therefore, it is determined that a shade QL1 in (4)(a) in FIG. 23 corresponds to PR1.

[0143] In this way, since the shade and ID information of the pointing device can be associated with each other, ID information of the pointing device can be assigned to each of a plurality of coordinate values, and can be transmitted to the host computer 6.

[0144] In a coordinate input apparatus of this type using light, a phenomenon that projected light is reflected by the pointing device and directly returns to the sensor unit (direct reflection) may occur. Such phenomenon occurs in a surface state in which the paint of the pointing device is formed of a material with high reflectance, and a CCD detection waveform shown in FIG. 26 is obtained. In this case, when the leading and trailing edges of a shade in (a) of FIG. 26 are merely detected, it is determined as if two shades QL1 and QL2 existed. However, by associating with the detection light of the pen control signal, since only one detection light of the pen control signal is detected, as shown in (b) of FIG. 26, it is determined that the shade includes light directly reflected by the pen. Therefore, even when direct reflection of the pointing device occurs, detection errors can be prevented.

[0145] Also, when the number of pointing devices that can be used is limited, e.g., when two pointing devices are permitted, if the number of shades is three, it is determined that direct reflection is included. Even in this case, by associating the shades with the detection light of the pen control signals, a shade caused by direct reflection can be determined.

[0146] As many apparently widely different embodiments of the present invention can be made without departing from the spirit and scope thereof, it is to be understood that the invention is not limited to the specific embodiments thereof except as defined in the appended claims.

[0147] This application claims priority from Japanese Patent Application No. 2004-207607 filed on Jul. 14, 2004, the entire contents of which are hereby incorporated by reference herein.