Techniques for calculating surface breakpoints for secondary safety verifications in vehicle controls systems

[0001] The present application generally relates to vehicle control systems and, more particularly, to techniques for calculating surface breakpoints for secondary safety verifications in vehicle control systems.

[0002] Some vehicle control systems utilize complex models (e.g., neural network based models) to generate outputs. In order to satisfy functional safety requirements, the accuracy of this primary output must be checked or verified. Due to processing limitations, this verification is typically a comparison of the output to another secondary output generated using simpler mathematical calculations. When the secondary output confirms that the primary output is of the correct scale, the primary output is verified. One method for verifying more complex model primary outputs is using calibrated look-up tables or multi-dimensional surfaces (2D surfaces, 3D surfaces, etc.). Due to storage limitations, these tables/surfaces are broken down by identifying “breakpoints,” which is typically performed by an experienced human calibrator and takes significant time and resources. Accordingly, while such conventional control system safety verification techniques do work well for their intended purpose, there exists an opportunity for improvement in the relevant art.

[0003] According to one example aspect of the invention, a calibration system for a multi-dimensional surface for functional safety verification of a control system of a vehicle is presented. In one exemplary implementation, the calibration system comprises a memory configured to store operation data relative to the control system of the vehicle, the operation data representing a multi-dimensional surface comprising a plurality of data points and a computer system configured to identify a plurality of breakpoints for representing the multi-dimensional surface based on a maximum allowable number of breakpoints, instantaneous data point slopes, and minimum/maximum breakpoint spacing constraints, and generate a calibrated look-up table for the control system, the calibrated look-up table including the plurality of breakpoints, wherein the calibrated look-up table is configured to be utilized for functional safety verification of an output of the control system.

[0004] In some implementations, the computer system is further configured to output the calibrated look-up table to the control system, wherein the control system is configured to utilize the calibrated look-up table to perform functional safety verification of the output of the control system. In some implementations, the computer system is further configured to identify the plurality of breakpoints by (i) determining a minimum/maximum step and a slope deviation for identifying a breakpoint. In some implementations, the computer system is further configured to identify the plurality of breakpoints by (ii) determining an instantaneous slope at a first data point and creating a line with the determined instantaneous slope through the first data point. In some implementations, the computer system is further configured to identify the plurality of breakpoints by (iii) stepping to a second data point and comparing a Y-value of the second data point to a Y-value of the created line.

[0005] In some implementations, the computer system is further configured to identify the plurality of breakpoints by (iv) identifying the first data point as a breakpoint when the difference between the Y-values of the second data point and the created line is greater than the slope deviation. In some implementations, the computer system is further configured to identify the plurality of breakpoints by continuing identifying breakpoints until the maximum number of breakpoints have been identified In some implementations, the multi-dimensional surface is a two-dimensional (2D) surface. In some implementations, the multi-dimensional surface is a three-dimensional (3D) surface, and wherein the computer system is further configured to divide the 3D surface into a plurality of 2D surfaces. In some implementations, the computer system is further configured to identify the plurality of breakpoints for each of the plurality of 2D surfaces and determine a plurality of breakpoints for the 3D surface by calculating a weighted sum of an average and a maximum of the plurality of breakpoints for the plurality of 2D surfaces, respectively.

[0006] According to another example aspect of the invention, a calibration method for a multi-dimensional surface for functional safety verification of a control system of a vehicle is presented. In one exemplary implementation, the calibration method comprises accessing, by a computer system, a memory configured to store operation data relative to the control system of the vehicle, the operation data representing a multi-dimensional surface comprising a plurality of data points, identifying, by the computer system, a plurality of breakpoints for representing the multi-dimensional surface based on a maximum allowable number of breakpoints, instantaneous data point slopes, and minimum/maximum breakpoint spacing constraints, and generating, by the computer system, a calibrated look-up table for the control system, the calibrated look-up table including the plurality of breakpoints, wherein the calibrated look-up table is configured to be utilized for functional safety verification of an output of the control system.

[0007] In some implementations, the method further comprises outputting, by the computer system, the calibrated look-up table to the control system, wherein the control system is configured to utilize the calibrated look-up table to perform functional safety verification of the output of the control system. In some implementations, the method further comprises identifying the plurality of breakpoints by (i) determining, by the computer system, a minimum/maximum step and a slope deviation for identifying a breakpoint. In some implementations, the method further comprises identifying the plurality of breakpoints by (ii) determining, by the computer system, an instantaneous slope at a first data point and creating, by the computer system, a line with the determined instantaneous slope through the first data point. In some implementations, the method further comprises identifying the plurality of breakpoints by (iii) stepping, by the computer system, to a second data point and comparing, by the computer system, a Y-value of the second data point to a Y-value of the created line.

[0008] In some implementations, the method further comprises identifying the plurality of breakpoints by (iv) identifying, by the computer system, the first data point as a breakpoint when the difference between the Y-values of the second data point and the created line is greater than the slope deviation In some implementations, the method further comprises identifying, by the computer system, the plurality of breakpoints by continuing identifying breakpoints until the maximum number of breakpoints have been identified. In some implementations, the multi-dimensional surface is a 2D surface. In some implementations, the multi-dimensional surface is a 3D surface, and wherein the method further comprises dividing, by the computer system, the 3D surface into a plurality of 2D surfaces In some implementations, the method further comprises Identifying, by the computer system, the plurality of breakpoints for each of the plurality of 2D surfaces and determining, by the computer system, a plurality of breakpoints for the 3D surface by calculating a weighted sum of an average and a maximum of the plurality of breakpoints for the plurality of 2D surfaces, respectively.

[0009] Further areas of applicability of the teachings of the present application will become apparent from the detailed description, claims and the drawings provided hereinafter, wherein like reference numerals refer to like features throughout the several views of the drawings. It should be understood that the detailed description, including disclosed embodiments and drawings referenced therein, are merely exemplary in nature intended for purposes of illustration only and are not intended to limit the scope of the present disclosure, its application or uses. Thus, variations that do not depart from the gist of the present application are intended to be within the scope of the present application.

[0014] As previously discussed, one method for verifying more complex (e.g., neural network based) model outputs is using calibrated look-up tables or multi-dimensional surfaces (2D surfaces, 3D surfaces, etc.). Due to storage limitations, these tables/surfaces are broken down by identifying “breakpoints,” which is typically performed by an experienced human calibrator and takes significant time and resources. Thus, an opportunity exists for improvement in the relevant art. Accordingly, calibration techniques for surface breakpoints for secondary safety verifications in vehicle control systems are presented herein. These techniques utilize a memory storing vehicle operation data representing a multi-dimensional surface and a computer system configured to identify a plurality of breakpoints for representing the multi-dimensional surface. This identification is based on, for example, a maximum number of breakpoints, instantaneous slopes of the plurality of data points, and minimum/maximum breakpoint spacing constraints. The calibrated look-up table for the multi-dimensional surface could then be output, such as to the vehicle control system or another vehicle controller for usage in secondary safety verification of a more complex vehicle control system output (e.g., a neural network based output).

[0015] Referring now to FIG. 1, a functional block diagram of an example calibration system 100 for a surface breakpoint look-up table according to the principles of the present application is illustrated. The calibration system 100 primarily includes a computer system 104 and a memory 108. The memory 108 is configured to store operation data (e.g., test data) relative to a control system 154 of a vehicle 150. The operation data represents a multi-dimensional (e.g., 2D or 3D) surface comprising a plurality of data points. The computer system 104 is configured to analyze the operation data to identify a plurality of breakpoints for representing the multi-dimensional surface. This identification could be based on, for example, a maximum allowable number of breakpoints, instantaneous data point slopes, and minimum/maximum breakpoint spacing constraints. The specific process of identifying the breakpoints is described in greater detail below. After the plurality of breakpoints (e.g., the maximum allowable number of breakpoints) have been identified, the computer system 108 generates and outputs a calibrated look-up table including the plurality of breakpoints.

[0016] This calibrated look-up table could then be used by another system to determine values of the corresponding multi-dimensional surface, such as via linear extrapolation (i.e., between neighboring breakpoints). In one implementation, the computer system 108 is configured to output the calibrated look-up table to the control system 154 (or another controller) of the vehicle 100. The control system 154 is then able to use the calibrated look-up table to determine secondary outputs for comparison to primary outputs generated using a more complex model (e.g., a neural network based model). One example of a neural network based model that is utilized by the control system 154 of the vehicle 100 is an airflow model for calculating airflow into an internal combustion engine 158 and the resulting engine torque from the combustion of the air and liquid fuel (gasoline, diesel, etc.). Another example of a neural network based model that is utilized by the control system 154 of the vehicle is a spark timing model for calculating a maximum brake torque (MBT) spark timing, based also on engine speed (revolutions per minute, or RPM) and airflow (manifold absolute pressure, or MAP).

[0017] Referring now to FIGS. 2A-2C, plots 200, 230, 260 of two example 2D surfaces 210, 240 and one example 3D surface 270 and example breakpoints 220, 250, 280 respectively, identified using the techniques according to some implementations of the present application are illustrated. In plot 200 of FIG. 2A, the 2D surface 210 corresponds to the polynomial 0.5(x−5)²+10 and there are nine (9) identified breakpoints 220. In plot 230 of FIG. 2B, the 2D surface 240 corresponds to the polynomial 5(1−[1/√x]) and there are again nine (9) identified breakpoints 250. As can be clearly seen, the breakpoints 220 and 250 match/fit the corresponding surfaces 210 and 240 very well. In plot 260 of FIG. 2C, the 3D surface 270 relates MBT spark to MAP and RPM and there are 9×9 (81) identified breakpoints 280. The process for identifying the breakpoints 270 involves dividing the 3D surface 270 into a plurality of 2D surfaces (e.g., nine 2D surfaces), but this process is described in greater detail below. While tougher to visualize compared to the 2D examples of FIGS. 2A-2B, the breakpoints 280 do match/fit the corresponding 3D surface 270 very well.

[0018] Referring now to FIG. 3A, a flow diagram of an example 2D surface breakpoint identification method 300 according to the principles of the present application is illustrated. The method 300 begins at 304 where the computer system 104 collects the operation data (i.e., the plurality of data points), which could be already stored at the memory 108. At 308, the computer system 104 obtains or fits the data to a 2D surface (e.g., a polynomial curve). At 312, the computer system 104 determines minimum/maximum step sizes and slope deviation for breakpoint identification. A maximum allowable number of breakpoints could also be determined. These constraints could be predetermined or user-specified. At 316, the computer system 104 takes the instantaneous slope at a first point and creates a line having the determined slope therethrough. At 320, the computer system 104 steps to a next point (e.g., a minimum step) and compares a Y-value of this next point to the Y-value of the sloped line. At 324, the computer system 104 determines whether a difference between these Y-values (ΔY-Value) exceeds the slop deviation. When false, a breakpoint is not identified and the method 300 returns to 320. When true, the method 300 proceeds to 328 where a breakpoint is identified at the first point and the instantaneous slope at the current point is determined and a line is created having the determined slope therethrough.

[0019] At 332, the computer system 104 determines whether all points (i.e., the maximum allowable number of breakpoints) have been found. When true, the method 300 proceeds to 336 where the identified breakpoints are exported (e.g., to a calibrated look-up table) and the method 300 ends. When false, the method 300 continues to 340. At 340, the computer system 104 steps to a next point and compares the Y-value at the next point to the Y-value of the sloped line. At 344, the computer system 104 determines whether an end of the data has been reached. When false, the method 300 continues to 348. When true, the method 300 proceeds to 352. At 348, the computer system 104 determines whether the ΔY-value exceeds the slope deviation. When false, the method 300 returns to 340. When true, the method 300 returns to 328. At 352, the computer system 104 determines whether a maximum number of iterations (iteration limits) has been reached. When true, the method 300 proceeds to 336. When false, the method 300 continues to 356. At 356, the computer system 104 lowers or eases the requirements for breakpoint identification (min/max step, slope deviation, etc.) and the method/algorithm 300 resets and returns to 312.

[0020] Referring now to FIG. 3B, a flow diagram of an example 3D surface breakpoint identification method 370 according to the principles of the present application is illustrated. The method 370 begins at 374 where the computer system 104 collects the operation data (i.e., the plurality of data points), which could be already stored at the memory 108. At 378, the computer system 104 obtains or fits the data to a 3D surface. At 382, the computer system 104 divides or splits the 3D surface into a plurality of 2D surfaces (e.g., spaced apart by acceptable spacing and collectively representing the 3D surface). The method 370 then splits into parallel processes including a first process 386a where the computer system 104 executes the 2D surface breakpoint identification method 300 on the Y-axis and a second process 386b where the computer system 104 executes the 2D surface breakpoint identification method 300 on the X-axis. While shown as parallel processes, it will be appreciated that these processes 386a and 386b could be performed sequentially. After both 386a and 386b, the method 370 proceeds to 390. At 390, the computer system 104 takes a weighted sum of an average and a minimum of the identified breakpoints from 386a and 386b. The method 300 then proceeds to 394 where the breakpoints are exported (e.g., to a calibrated look-up table) and the method 370 ends.

[0021] It will be appreciated that the terms “controller” and “control system” as used herein refer to any suitable control device or set of multiple control devices that is/are configured to perform at least a portion of the techniques of the present application. Non-limiting examples include an application-specific integrated circuit (ASIC), one or more processors and a non-transitory memory having instructions stored thereon that, when executed by the one or more processors, cause the controller to perform a set of operations corresponding to at least a portion of the techniques of the present application. The one or more processors could be either a single processor or two or more processors operating in a parallel or distributed architecture.

[0022] It should also be understood that the mixing and matching of features, elements, methodologies and/or functions between various examples may be expressly contemplated herein so that one skilled in the art would appreciate from the present teachings that features, elements and/or functions of one example may be incorporated into another example as appropriate, unless described otherwise above.