Picture retaking method and apparatus, movable platform, system, and storage medium

[0001] This application is a continuation of International Application No. PCT/CN2021/106597, filed Jul. 15, 2021, the entire content of which is incorporated herein by reference.

[0002] The present disclosure relates to the movable platform technology and, more particularly, to a picture retaking method, a picture retaking apparatus, a movable platform, a system, and a storage medium.

[0003] A movable platform such as an unmanned aerial vehicle (UAV) photographs according to a certain sequence/distance when performing a survey operation. Due to apparatus failure or other factors, the movable platform misses shooting. Thus, a retaking operation needs to be performed. Currently, the missing position is determined manually, which involves a large amount of work, and the accuracy of the missing position identification is low. Moreover, for the missing positions, a flight path is planned manually to perform retaking, or the retaking is performed manually, which is time and labor consuming. Moreover, it is hard to ensure the retaken picture is effective and reliable.

[0004] Therefore, the convenience and reliability of the picture retaking need to be improved.

[0005] In accordance with the disclosure, there is provided a control system, including one or more processors and one or more memories. The one or more memories store one or more computer programs that, when executed by the one or more processors, cause the one or more processors to obtain operation information of a movable platform performing an operation in a target area, determine at least one operation abnormal area of the movable platform performing the operation in the target area according to the operation information, determine a corresponding supplement operation path according to the at least one operation abnormal area, and control the movable platform to perform a supplement operation based on the supplement operation path.

[0006] Also in accordance with the disclosure, there is provided a control method. The method includes obtaining operation information of a movable platform performing an operation in a target area, determining at least one operation abnormal area of the movable platform performing the operation in the target area according to the operation information, determining a corresponding supplement operation path according to the at least one operation abnormal area, and controlling the movable platform to perform a supplement operation based on the supplement operation path.

[0029] The technical solution of embodiments of the present disclosure is described in detail in connection with the accompanying drawings. Described embodiments are some embodiments of the present disclosure, not all embodiments. Based on embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts are within the scope of the present disclosure.

[0030] The flowchart in the accompanying drawings is illustrative, does not necessarily include all contents and operations/processes, and is not necessarily executed according to a described sequence. For example, some operations/processes can be divided, grouped, or partially combined. Thus, the actual execution sequence can be changed as needed.

[0031] The terminology used in the present disclosure is merely for the purpose of describing specific embodiments only and is not intended to limit the present disclosure. The singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context indicates otherwise. The term “and/or” refers to any one or more of the listed items and all possible combinations thereof.

[0032] The term “and/or” used in the specification and appended claims refers to one or more combinations and all combinations of associated items and includes these combinations.

[0033] Embodiments of the present disclosure are described in detail in connection with the accompanying drawings. When there is no conflict, embodiments and features of embodiments can be grouped with each other.

[0034] A movable platform, such as an unmanned aerial vehicle (UAV), can photograph according to a certain sequence/distance during a survey operation. Due to an apparatus failure or other factors, operation abnormality, such as missing shooting, can occur. An area in which operation abnormality occurs is also referred to as an operation abnormal area. For example,

[0035] FIG. 1 is a schematic diagram showing corresponding shooting points when a survey area operation is single-direction shooting consistent with an embodiment of the present disclosure. A blank area in the figure is an operation abnormal area. In the example shown in FIG. 1, shooting is not performed normally in the blank area (operation abnormal area) due to apparatus failure or other factors. For example, the shooting can be missed. The area where the shooting is missed can also be referred to as a missed area.

[0036] In some other embodiments, the operation abnormality in an operation abnormal area can include abnormal operation of a payload of the movable platform. The payload can include a spray device, a spreading device, or a position/attitude sensor, etc. An abnormal operation can include abnormal spraying, abnormal spreading, abnormal positioning, or abnormal attitude detection. When the payload is the camera device, the abnormal operation can include poor shooting quality, such as overexposure, or insufficient shooting clarity.

[0037] FIG. 2 is a schematic diagram of a corresponding shooting point when a survey area operation is multi-directional shooting consistent with an embodiment of the present disclosure.

[0038] Therefore, a retaking operation needs to be performed. Currently, a missing position is usually determined manually, which is labor-intensive. The accuracy of recognizing the missing position may not be high. For missing positions, the retaking can be performed by manually planning a path, or the retaking can be performed manually, which is labor and time consuming. Moreover, the picture cannot be retaken effectively and reliably.

[0039] To address the above issues, embodiments of the present disclosure provide a picture retaking method, a picture retaking apparatus, a movable platform, a system, and a storage medium, which are configured to improve the convenience and reliability of picture retaking.

[0040] FIG. 3 is a schematic structural diagram of a movable platform system consistent with an embodiment of the present disclosure. As shown in FIG. 3, the movable platform system 1000 includes a movable platform 100 and a camera apparatus 200. If the movable platform 100 misses shooting while performing the operation in the survey area, a corresponding retaking path is determined through the movable platform 100. Based on the retaking path, the camera apparatus 200 can be controlled to perform the picture retaking operation to obtain a retaken picture.

[0041] The movable platform 100 can include, but is not limited to, an unmanned aerial vehicle (UAV), such as a rotary-wing aerial vehicle, including a single-rotor aerial vehicle, dual-rotor aerial vehicle, tri-rotor aerial vehicle, quad-rotor aerial vehicle, hexa-rotor aerial vehicle, octa-rotor aerial vehicle, deci-rotor aerial vehicle, dodeca-rotor aerial vehicle, etc. The movable platform 100 can also include other types of UAVs or movable apparatuses, such as fixed-wing UAVs, which are not limited in the present disclosure.

[0042] The camera apparatus 200 can include, but is not limited to, a camera, a camcorder, etc. The camera apparatus 200 can also include other types of apparatuses having a shooting function, such as a visual sensor, which is not limited in the present disclosure.

[0043] For example, the movable platform 100 includes a power apparatus 110 and a picture retaking apparatus 120. The power apparatus 110 can be configured to provide power to the movable platform 100. The picture retaking apparatus 120 can include, but is not limited to, a flight controller, and can be configured to generate a corresponding retaking path corresponding to the operation of the movable platform 100 in the survey area and control the camera apparatus 200 to perform picture retaking based on the retaking path to improve the convenience and reliability of picture retaking.

[0044] In some embodiments, the power apparatus 110 can include one or more electronic speed controllers (referred to as ESCs), one or more propellers, and one or more motors corresponding to the one or more propellers. A motor can be connected between an ESC and a propeller. The ESC can be configured to provide drive current to the motor to control a rotation speed of the motor. The motor can be configured to drive the propeller to rotate to provide the power for the flight of the movable platform 100. This power can enable the movable platform 100 to move in one or more degrees of freedom. In some embodiments, the movable platform 100 can rotate around one or more axes. The motor can be a direct current (DC) motor or an alternating current (AC) motor. Furthermore, the motor can be a brushless motor or a brushed motor.

[0045] In some embodiments, as shown in FIG. 4, the movable platform system 1000 further includes a ground station 300. The ground station 300 establishes wireless communication with the movable platform 100. Through interactions among the ground station 300, the movable platform 100, and the camera apparatus 200, a survey area operation and a corresponding retaking operation can be implemented.

[0046] For example, the movable platform 100 can be a UAV, the picture retaking apparatus 120 can be a flight controller of the UAV, and the camera apparatus 200 can be a camera. As shown in FIG. 5, during the operation, the ground station uploads information about a planned operation flight path, shooting positions, a shooting sequence, etc., to the flight controller. Then, the flight controller sends the shooting sequence to the camera. When the camera performs shooting, the camera returns a shooting sequence number to the flight controller and requests for picture shooting information about a position, an attitude, a positioning status, etc. The flight controller records the picture shooting information about the corresponding shooting sequence number, the position, the attitude, the positioning status, etc., in a flight system. Then, the flight controller sends the picture-shooting information to the camera, and the camera writes the picture-shooting information into a picture corresponding to the shooting.

[0047] Then, the ground station can request the shooting sequence number from the flight controller. After receiving the request, the flight controller returns the shooting sequence number and confirms that communication is normal. The ground station can again request the picture shooting information about the position, the attitude, the positioning status, etc., corresponding to the shooting sequence number. The flight controller returns the picture shooting information. Then, the ground station can access to the picture shooting information about the shooting sequence number, the position, the attitude, the positioning status, etc.

[0048] If the UAV misses shooting during the survey area operation, the flight controller can obtain the picture shooting information, automatically generate a corresponding retaking path according to the picture shooting information, and control the camera to perform a picture retaking operation based on the retaking path.

[0049] Names of the various components in the movable platform system 1000 are merely for identification purposes and do not limit embodiments of the present disclosure.

[0050] The picture retaking method of embodiments of the present disclosure is described in detail based on the movable platform system 1000 and the movable platform 100 below. The movable platform system 1000 and the movable platform 100 in FIG. 1 and FIG. 2 are merely used to describe the picture retaking method of embodiments of the present disclosure and do not limit an application scene of the picture retaking method of embodiments of the present disclosure.

[0051] FIG. 6 is a schematic flowchart of a picture retaking method consistent with an embodiment of the present disclosure. The method can be applied in the movable platform above or another apparatus including a picture retaking apparatus. The application scene of the method is not limited in the present disclosure. The convenience and the reliability of the picture retaking can be improved based on the picture retaking method. The picture retaking method is described in detail by applying the picture retaking method to the movable platform.

[0052] As shown in FIG. 6, the picture retaking method includes processes S101 to S104.

[0053] At S101, the picture shooting information when the movable platform performs the survey area operation is obtained.

[0054] The picture shooting information can include at least one of a picture shooting sequence number, a position, an attitude, or a location status. The attitude can include but is not limited to a pitch attitude, a roll attitude, and a yaw attitude. The positioning status can include but is not limited to a Real-Time Kinematic (RTK) status.

[0055] In some embodiments, the movable platform can obtain the corresponding picture shooting information in real-time while performing operation in the survey area. In some other embodiments, the movable platform can obtain the picture shooting information corresponding to the operation after completing the operation in the survey area.

[0056] In some embodiments, the picture shooting information can also be obtained by the ground station or the camera apparatus. For example, when the movable platform performs the operation in the survey area, the ground station can communicate with the movable platform and the camera apparatus in real-time. First, the ground station can upload information about the planned operation flight path, the shooting positions, the shooting sequence, etc., to the movable platform. Then, the movable platform can send the shooting sequence to the camera apparatus such as the camera. When the camera apparatus performs shooting, the camera apparatus can return the shooting sequence number to the movable platform and request the picture shooting information about the position, the attitude, and the positioning status. The movable platform can record the picture shooting information about the shooting sequence number, the position, the attitude, and the positioning status in the flight system and then send the picture-shooting information to the camera apparatus. The camera apparatus can write the picture-shooting information into the picture corresponding to the shooting.

[0057] Then, the ground station can request the shooting sequence number from the movable platform. The movable platform can receive the request, then return the shooting sequence number, and confirm that the communication is normal. The ground station can then send another request to the movable platform for the picture shooting information about the position, the attitude, and the positioning status corresponding to the shooting sequence number. The movable platform can return the picture shooting information. Then, the ground station can obtain the picture shooting information about the shooting sequence number, the position, the attitude, and the positioning status.

[0058] In some embodiments, a list can be maintained in the ground station during a mission. The list can include picture shooting information about shooting sequence numbers, positions, attitudes, and positioning statuses for the most recent shooting points.

[0059] In some embodiments, for a certain operation performed by the movable platform in a survey area, a corresponding folder can be generated in the camera apparatus. The folder can include a text file storing the picture shooting information about the shooting sequence number, the position, the attitude, and the positioning status corresponding to the operation. After the operation, the ground station can access the file to obtain the picture shooting information.

[0060] At S102, at least one missed area is determined according to the picture shooting information when the movable platform performs the operation in the survey area.

[0061] During actual operation, shooting can be missed, and shooting can be even missed at one or more positions. To perform picture retaking, one or more missed areas can be determined according to the picture shooting information when the movable platform performs the operation in the survey area. Compared to manually determining the missed areas, the missed areas can be identified with higher accuracy.

[0062] In some embodiments, the movable platform can determine at least one corresponding missed area according to the picture shooting information corresponding to the shooting points where shooting has been performed in real-time during the operation in the survey area. That is, during the operation, the missed area can be determined in real-time once the shooting is missed.

[0063] In some other embodiments, after the operation in the survey area, the movable platform can determine the at least one missed area according to the picture shooting information corresponding to all shooting points of the operation in the survey area. That is, during the operation, the missing of the shooting may not be considered. After the operation, all missed areas corresponding to the operation can be determined.

[0064] In some embodiments, as shown in FIG. 7, process S102 includes sub-processes S1021 and S1022.

[0065] At S1021, a distance between two neighboring shooting points is calculated according to the picture shooting information.

[0066] In some embodiments, during mission planning, the operation can be set to correspond to a specific shooting sequence in different flight segments. The flight segments can be separated by points with specific positions. Each shooting sequence can be a single-direction shooting sequence or a multi-direction shooting sequence. For example, as shown in FIG. 8, different flight segments correspond to different shooting sequences. The boundaries of the flight segments are separation points of the shooting sequences. During mission planning, separation points of different shooting sequences of the flight segments can be recorded. For example, FIG. 8 shows point 1, point 2, and point 3 marked with squares. Point 1 is a separation point for a single-direction shooting sequence and a three-direction shooting sequence. Point 2 is a separation point for the three-direction shooting sequence and a four-direction shooting sequence. The symbols “↑” indicates that a shooting direction is “front,” “↓” indicates that the shooting direction is “rear,”←“—” indicates that the shooting direction is “left,” “→” indicates that the shooting direction is “right,” and “X” indicates that the shooting direction is “orthogonal.”

[0067] If the survey area operation is single-direction shooting, the distance between two neighboring shooting points can be directly calculated according to the picture shooting information corresponding to the shooting point, such as the position corresponding to the shooting point. To facilitate distinguishing, the distance between the two neighboring shooting points in the single direction shooting can be referred to as a first distance.

[0068] If the survey area operation is multi-directional shooting, a same shooting sequence can include a plurality of directions. A distance between two neighboring shooting points with the same direction of neighboring shooting sequences can be calculated, and a distance between two neighboring shooting points with different directions within the same shooting sequence can be calculated according to the picture shooting information corresponding to the shooting points. To facilitate distinguishing and description, the distance between the two neighboring points with the same direction of the two neighboring shooting sequences can be referred to as a second distance. The distance between the two neighboring shooting points with different directions within the same shooting sequence can be referred to as a third distance.

[0069] At S1022, if the distance between the two neighboring shooting points is greater than a distance threshold, an area between the two neighboring shooting points is determined to be a missed area.

[0070] In a normal shooting condition, distances between every two neighboring shooting points can be similar, e.g., around a certain distance threshold. If the distance between two neighboring shooting points is greater than the distance threshold, the area between the two neighboring shooting points can be determined to be the missed area. Thus, the shooting is missed.

[0071] The distance threshold can be a predetermined distance value or can be determined according to the scene where the movable platform performs the operation.

[0072] In some embodiments, as shown in FIG. 9, processes S105 and S106 are included before process S1022. Process S1022 further includes a sub-process S10221.

[0073] At S105, scene information is obtained for the operation in the survey area.

[0074] This scene information can include one or more of flight height (H), heading overlapping rate (P), or shooting field of view FOV. During the operation performed in the survey area, the movable platform can obtain the scene information of flight height H, heading overlapping rate P, and shooting field of view FOV corresponding to the operation.

[0075] At S106, according to the scene information, a first distance threshold is determined.

[0076] First distance threshold D1 can represent a theoretical distance between shooting points in the single-direction operation. First distance threshold D1 can be determined according to the scene information corresponding to the operation in the survey area.

[0077] In some embodiments, a predetermined distance formula corresponding to the first distance threshold can be predetermined. One or more of flight height H, heading overlapping rate P, or shooting field of view FOV corresponding to the operation performed by the movable platform in the survey area can be substituted into the formula to calculate first distance threshold D1.

[0078] For example, the predetermined distance formula is D=2*H*tan(FOV/2)*(1−P). After flight height H, heading overlapping rate P, and shooting field of view FOV of the movable platform when performing the operation in the survey area are obtained, flight height H, heading overlapping rate P, and shooting field of view FOV can be substituted into the formula to calculate first distance threshold D1.

[0079] At S10221, if the first distance between the two neighboring shooting points is greater than the first distance threshold, the area between the two neighboring shooting points is determined to be the missed area.

[0080] For the single-direction shooting sequence, after the first distance between the two neighboring shooting points is calculated according to the picture shooting information corresponding to the shooting point, the first distance can be compared to the first distance threshold D1. If the first distance between the two neighboring shooting points is greater than the first distance threshold D1, the area between the two neighboring shooting points can be determined to be the missed area.

[0081] In some embodiments, as shown in FIG. 10, process S105, process S107, and process S108. Process S1022 further includes sub-process S10222.

[0082] At S105, the scene information for the survey area operation is obtained.

[0083] Obtaining the scene information is described above, which is not repeated here.

[0084] At S107, according to the scene information, the second distance threshold between the two shooting points with the same direction in the neighboring shooting sequences is obtained.

[0085] Second distance threshold D2 can represent the theoretical distance between the two shooting points with the same direction in the neighboring shooting sequences under the multi-directional shooting operation. Second distance threshold D2 can be determined according to the scene information corresponding to the survey area operation. Second distance threshold D2 can be the same as or different from first distance threshold D1.

[0086] In some embodiments, the calculation method for second distance threshold D2 is the same as the calculation method for first distance threshold D1. Second distance threshold D2 between the two shooting points with the same direction in the neighboring shooting sequences can be calculated according to formula D=2*H*tan(FOV/2)*(1−P).

[0087] At S108, according to the second distance threshold, the third distance threshold between the two neighboring shooting points with different directions within the same shooting sequence is determined.

[0088] If the same shooting sequence of the survey area operation corresponds to N shooting directions (angles), according to second distance threshold D2 between the two shooting points with the same direction in the neighboring shooting sequences and the N shooting directions, third distance threshold d between the two neighboring shooting points with different directions within the same shooting sequence can be determined as d=D2/N.

[0089] For example, as shown in FIG. 11, D2 represents the second distance threshold between the two neighboring shooting points with the same direction (e.g., angle 1) of the neighboring shooting sequences (e.g., sequence 1 and sequence 2). d represents the third distance threshold between the two neighboring shooting points with different directions (e.g., angle 1 and angle 2) within the same shooting sequence (e.g., sequence 1).

[0090] At S10222, if the second distance between the two points with the same direction in the neighboring shooting sequences is greater than the second distance threshold, the area between the two shooting points with the same direction in the neighboring shooting sequences is determined as the missed area, or if the third distance between the two neighboring shooting points with different directions within the same shooting sequence is greater than the third distance threshold, the area between the two neighboring shooting points with different directions within the same shooting sequence is determined to be the missed area.

[0091] For the multi-direction shooting sequence, after the second distance between the two neighboring shooting points with the same direction of the two neighboring sequences and the third distance between the two neighboring shooting points with different directions within the same shooting sequence are calculated, the second distance can be compared to second distance threshold D2, and the third distance can be compared to third distance threshold d. If the second distance between the two shooting points with the same direction in the neighboring shooting sequences is greater than second distance threshold D2, the area between the two shooting points with the same direction in the neighboring shooting sequences can be determined to be the missed area. If the third distance between the two neighboring shooting points with different directions within the same shooting sequence is greater than third distance threshold d, the area between the two neighboring shooting points with different directions within the same shooting sequence can be determined to be the missed area.

[0092] At S103, the corresponding retaking path is determined according to the at least one missed area.

[0093] Since one or more missed areas correspond to the survey area operation of the movable platform, a retaking path corresponding to the single missed area or a retaking path corresponding to the plurality of missed areas can be determined for the situation of the single missed area and the plurality of missed areas.

[0094] In some embodiments, as shown in FIG. 12, process S103 includes sub-process S1031 and sub-process S1032.

[0095] At S1031, if there is only a single missed area, a starting shooting point and an ending shooting point for retaking are determined according to the single missed area.

[0096] For example, if the missed area corresponds to the single-direction shooting sequence, according to the last shooting point before missing the shooting, a position with first distance threshold D1 after the position of the last shooting point before missing the shooting can be used as the starting shooting point for retaking. According to a first shooting point when the shooting continues, a position with first distance threshold D1 before the position of the first shooting point when the shooting continues can be used as the ending shooting point for retaking.

[0097] If the missed area corresponds to a multi-directional shooting sequence, according to the last shooting point before missing the shooting, the position with third distance threshold d after the last shooting point before missing the shooting can be used as the starting shooting point for retaking. According to the first shooting point when the shooting continues, the position with third distance threshold d before the position of the first shooting point when the shooting continues can be used as the ending shooting point for retaking.

[0098] At S1032, the retaking path is determined according to the starting shooting point and the ending shooting point.

[0099] In some embodiments, for the single-direction shooting sequence, according to the starting shooting point and the ending shooting point for retaking and first distance threshold D1 between the two neighboring shooting points, shooting points for retaking can be determined in the missed area and can include the starting shooting point and the ending shooting point. According to the shooting points for retaking, the retaking path corresponding to the missed area can be determined.

[0100] For the multi-direction capture sequence, according to the shooting sequence included in the missed area, the starting shooting point and the ending shooting point for retaking, second distance threshold D2 between the two shooting points with the same direction in the neighboring shooting sequences, and third distance threshold d between the two neighboring shooting points with different directions within the same shooting sequence, the shooting points for retaking can be determined in the missed area. The retaking path corresponding to the missed area can be determined according to the shooting points for retaking.

[0101] In some embodiments, as shown in FIG. 13, process S103 includes sub-processes S1033 and S1034.

[0102] At S1033, if a plurality of missed areas are included, an execution sequence of the plurality of missed areas is determined.

[0103] When a plurality of missed areas are included, the missed areas can have a corresponding execution sequence when retaking is performed. In some embodiments, the execution sequence of the plurality of missed areas can be determined according to an execution sequence of a shooting path of the survey area operation. The execution sequence of the plurality of missed areas can be consistent with the execution sequence of the shooting path of the survey area operation. For example, as shown in FIG. 14, according to an initially planned flight execution direction, the direction of the execution sequence for retaking the plurality of missed areas is the same as the initially planned flight execution direction. The method for determining the execution sequence of the plurality of missed areas can be simple, logically clear, and less likely to have errors. However, since the direction of the execution sequence is consistent with the initially planned flight execution direction, the retaking path may not be optimal.

[0104] In some embodiments, as shown in FIG. 15, process S1033 includes sub-processes S10331 to S10333.

[0105] At S10331, a center position of each missed area and a starting position of the movable platform are obtained.

[0106] After the missed areas are determined, the center position of each of the missed areas can be determined. Thus, the center positions of the missed areas can be obtained. the starting position of the movable platform can be obtained when performing retaking, such as the flight starting position of the movable platform.

[0107] At S10332, according to the center positions of the plurality of missed areas and the starting position of the movable platform, a shortest execution path is determined.

[0108] In some embodiments, the starting position of the movable platform can be used as a starting point. According to the center positions of the plurality of missed areas and the starting position of the movable platform, the shortest execution path traversing all the positions can be determined using a distance algorithm.

[0109] At S10333, according to the shortest execution path, the execution sequence of the plurality of missed areas is determined.

[0110] After the shortest execution path is determined. The execution sequence of the missed areas can be determined according to the shortest execution path. For example, as shown in FIG. 16, the missed area is the same as the missed area in FIG. 14. However, the direction of the execution sequence of the missed areas determined according to the shortest execution path is opposite to the initially planned flight execution direction. The method for determining the execution sequence of the plurality of missed areas can have a short path and high execution efficiency. However, since the direction of the execution sequence of the missed areas is different from the initially planned flight execution direction, some shooting sequences may need to be performed in reverse.

[0111] In some embodiments, in addition to determining the shortest execution path based on the center positions of the missed areas, the shortest execution path can be determined according to other logic. For example, based on two endpoint positions of each missed area, an endpoint of all endpoints can be selected, which can be referred to as a first endpoint. Another endpoint corresponding to the first endpoint in the missed area can be referred to as a second endpoint. An endpoint close to the second endpoint can be determined from other endpoints except the first endpoint and the second endpoint and can be used as a third endpoint. Another endpoint corresponding to the third endpoint in the missed area can be referred to as a fourth endpoint. An endpoint closest to the fourth endpoint can be determined from other endpoints except the first endpoint, the second endpoint, the third endpoint, and the fourth endpoint and can be referred to as a fifth endpoint. Another endpoint corresponding to the fifth endpoint in the missed area can be referred to as a sixth endpoint, and so on until a last endpoint of the last missed area can be determined. According to the first endpoint, the second endpoint, the third endpoint, . . . , and a last endpoint, a corresponding execution path can be determined. With a traversing method, each endpoint can be used as the first endpoint in sequence. The corresponding execution path can be determined according to the above method. After the traversing, a plurality of execution paths can be obtained. the path lengths of the plurality of execution paths can be different.

[0112] For example, N missed areas can correspond to 2N endpoints. The 2N endpoints can be traversed in a traversing method. Assume that the first endpoint that is currently traversed is endpoint a, end point a can be used as the starting point of the current missed area. Another endpoint corresponding to endpoint a in the missed area is endpoint b. Endpoint c closest to endpoint b can be determined from 2N−2 endpoints except endpoint a and endpoint b and can be used as the starting point of the next missed area, and so on until all the endpoints are traversed to obtain 2N execution paths.

[0113] According to the path lengths of the execution paths, the shortest execution path can be determined. According to the determined shortest execution path, the execution sequence of the plurality of missed areas can be determined.

[0114] At S1034, according to the plurality of missed areas and the execution sequence of the plurality of missed areas, the retaking path can be determined.

[0115] For example, if the missed areas include missed area 1, missed area 2, and missed area 3, the corresponding execution sequence can be missed area 1 first, followed by missed area 3, and then missed area 2. Thus, the corresponding retaking path can be determined as missed area 1→missed area 3→missed area 2.

[0116] In some embodiments, as shown in FIG. 17, process S1034 includes sub-processes S10341 and S10342.

[0117] At S10341, a first shooting point and a second shooting point are determined for each of the missed areas.

[0118] Since each missed area includes one or more shooting sequences. If only one shooting sequence is included, the shooting sequence can be a single-direction shooting sequence or a multi-direction shooting sequence. If the plurality of shooting sequences are included, each shooting sequence of the plurality of shooting sequences can be a single-direction shooting sequence or a multi-direction shooting sequence. Thus, for each missed area, according to the shooting sequence corresponding to each shooting sequence, the first shooting point and the second shooting point of each missed area can be determined. The first shooting point can refer to a first point for retaking in the missed area. The second shooting point can refer to an endpoint for retaking in the missed area.

[0119] In some embodiments, determining the first shooting point and the second shooting point for each missed area can include determining the last shooting point before missing the shooting corresponding to each missed area and the beginning shooting point when the shooting continues, and determining the first shooting point and the second shooting point for each missed area according to the last shooting point before missing the shooting and the beginning shooting point when the shooting continues.

[0120] For example, if point A is the last shooting point before missing the shooting, and point B is the beginning shooting point when the shooting continues, the first shooting point can be determined according to point A, and the second shooting point can be determined according to point B.

[0121] In some embodiments, determining the first shooting point for each missed area can include determining a shooting sequence corresponding to the last shooting point before missing the shooting and determining the first shooting point of each missed area according to the shooting sequence corresponding to the last shooting point before missing the shooting.

[0122] For example, as shown in FIG. 18, point A is the last shooting point before missing the shooting and is in a single-direction shooting sequence, and point B is the beginning shooting point after the shooting recovers and is in a four-direction shooting sequence. The first shooting point is determined according to point A in the single-direction shooting sequence, and the second shooting point is determined according to point B in the four-direction shooting sequence.

[0123] In some embodiments, determining the first shooting point for each missed area can include if the shooting sequence of the last shooting point before missing the shooting corresponds to the single-direction shooting, determining a first candidate shooting point. The distance between the first candidate shooting point and the last shooting point before missing the shooting can be a second distance threshold. The second distance threshold can be a distance threshold between two shooting points with the same direction of neighboring shooting sequences. Determining the first shooting point for each missed area can further include determining the first candidate shooting point as the first shooting point.

[0124] For example, as shown in FIG. 18, if point A is in the single-direction shooting sequence. Second distance threshold D2 is added along the flight direction to point A. The corresponding first candidate shooting point is determined, and the first candidate shooting point is determined as the first shooting point.

[0125] The first shooting point can be located between point A and separation point 1 or may not be located between point A and separation point 1. If the first shooting point is not between point A and separation point 1, point A can be the last point of the single-direction shooting sequence. The retaking can directly begin from a next area (i.e., a three-direction shooting sequence). That is, the first shooting point can be in the three-direction shooting sequence. If the first shooting point is between point A and separation point 1, the first shooting point is still in the single-direction shooting sequence.

[0126] Point B can be the beginning shooting point when the shooting recovers after missing the shooting and can be in the four-direction shooting sequence. At a position having distance D2 to point B along a direction opposite to the flight direction, the second shooting point can be determined. Similarly, if the second shooting point is not between separation point 2 and point B, the ending shooting point for retaking in the missed area can be in the three-direction shooting sequence. If the second shooting point is between separation point 2 and separation point 1, the ending shooting point for retaking in the missed area can be in the three-direction shooting sequence. If the second shooting point is between separation point 1 and the first shooting point, the ending shooting point for retaking in the missed area can be in the single-direction shooting sequence. That is, the entire missing part can be in the single-direction shooting sequence.

[0127] For example, if point A is in the multi-direction shooting sequence, third distance threshold d can be added along the flight direction from point A, a corresponding second candidate shooting point can be determined, and the second candidate shooting point can be determined as the first shooting point.

[0128] At S10342, according to the first shooting point and the second shooting point of each missed area and the execution sequence of the plurality of missed areas, the retaking path is determined.

[0129] According to the first shooting point and the second shooting point for each missed area and the shooting sequence corresponding to each missed area, missing shooting points corresponding to each missed area can be completed. According to the complete shooting points of each missed area and the execution sequence corresponding to the missed areas, the retaking path can be determined.

[0130] For example, if the missed area starts from a multi-directional shooting sequence, shooting points corresponding to N-n shooting directions (angles) within the multi-directional shooting sequence can be completed first. N can be the total number of shooting directions (angles) within the entire shooting sequence, and n can be the number of shooting points within the sequence that are not missed. After completing N-n shooting points within the multi-direction sequence, other shooting points can be added according to distance D2 between sequences and distance d between different directions within the same sequence. Thus, the shooting points corresponding to the missing shooting directions in the multi-direction sequence in the missed area can be completed.

[0131] For example, as shown in FIG. 19, if the first shooting point is point A′, and the second shooting point is point B′, shooting points are completed starting from point A′ to separation point 1 with a distance D1 according to the single-direction shooting sequence. Shooting points are completed between separation point 1 and separation point 2 with distance D2 between sequences according to a three-direction shooting sequence and distance d=D2/3 between different directions within the same sequence. Shooting points are completed between separation point 2 and point B′ with distance D2 between sequences according to a four-direction shooting sequence and distance d=D2/4 between different directions within the same sequence.

[0132] By automatically generating the retaking sequence and retaking path connecting with the original flight path seamlessly, the retaking path can be generated efficiently and can be reliable.

[0133] At S104, based on the retaking path, the camera apparatus is controlled to perform a picture retaking operation to obtain a picture corresponding to at least one missed area.

[0134] After obtaining the retaking path corresponding to the missed area, the camera apparatus can be controlled to perform the picture retaking operation for the shooting points according to the retaking path to obtain pictures corresponding to the missing shooting points.

[0135] In some embodiments, as shown in FIG. 20, the method further includes process S109 after process S104.

[0136] At S109, the pictures obtained through the retaking operation are associated with the pictures shot during the survey area operation and saved.

[0137] To further enhance the user experience, a specific storage path can be set for the pictures obtained by the picture retaking operation. The pictures obtained through the picture retaking operation can be associated with the pictures shot during the survey area operation for storage. Thus, the retaking operation can be associated with the survey area operation performed earlier.

[0138] In some embodiments, as shown in FIG. 21, process S109 includes sub-processes S1091 and S1092.

[0139] At S1091, according to the first folder generated corresponding to the survey area operation, a second folder associated with the first folder is generated.

[0140] For example, if the first folder corresponding to the survey area operation can be yymmddhhmm_N by default or ABCDEF defined by the user. The first folder can store the pictures shot during the survey area operation. According to the folder name of the first folder, the second folder logically associated with the first folder can be generated with the name yymmddhhmm_N_sup or ABCDEF_sup.

[0141] At S1092, the pictures obtained through the retaking operation are stored in the second folder.

[0142] For example, the pictures obtained through the retaking operation can be stored within the second folder named yymmddhhmm_N_sup or ABCDEF_sup.

[0143] In some embodiments, a plurality of sub-folders that are logically associated can be generated under the second folder. The pictures obtained through the retaking operation can be dividedly stored in the sub-folders.

[0144] For example, if three segments are divided for flight path retaking, the sub-folders generated under the second folder can be named with yymmddhhmm_N_sup_n (n=1, 2, 3) or ABCDEF_sup_n (n=1, 2, 3). The pictures retaken for the segments of the flight path can be saved to the sub-folders to ensure the retaken pictures are associated with the pictures shot during the survey area operation.

[0145] In some embodiments, associating the pictures obtained through the retaking operation with the pictures shot during the survey area operation for storage can include according to the shooting points corresponding to the pictures obtained through the retaking operation, inserting the pictures obtained through the retaking operation into the saved pictures shot during the survey area operation.

[0146] Different from using the associated folder to save the pictures obtained through the retaking operation, if the user needs to sort and store all the pictures according to certain correct sequence numbers, the sequence numbers corresponding to the pictures can be determined according to the shooting points corresponding to the pictures obtained according to the retaking operation. According to the sequence of the sequence numbers, the pictures obtained through the retaking operation can be inserted into the pictures shot during the survey area operation.

[0147] It's worth noting that in addition to the two methods of associating and saving the pictures with the pictures captured during the survey area operations that were listed above, other methods of associating and saving the recaptured pictures with the original survey area pictures can also be employed. There are no specific restrictions in this regard.

[0148] Except for the two methods of associating the pictures obtained through the retaking operation with the pictures shot during the survey area operation for storage, other methods can also be used to associate the pictures obtained through the retaking operation with the pictures shot during the survey area operation for storage, which is not limited here.

[0149] By associating the pictures obtained through the retaking operation with the pictures shot during the survey area operation for storage, the retaken pictures can be associated with the pictures obtained during the survey area operation. Thus, the user can conveniently obtain all the pictures, which improves the user experience.

[0150] In the described embodiments, the picture shooting information can be obtained when the movable platform performs the survey area operation. According to the picture shooting information, at least one missed area can be determined when the movable platform performs the survey area operation. Then, according to the at least one missed area, the corresponding retaking path can be determined. Based on the retaking path, the camera apparatus can be controlled to perform the picture retaking operation to obtain the pictures corresponding to the at least one missed area. Compared to the method of manually determining the missing positions and planning the flight path for retaking, the pictures can be retaken conveniently and reliably.

[0151] FIG. 22 is a schematic block diagram of a picture retaking apparatus 200 consistent with an embodiment of the present disclosure.

[0152] As shown in FIG. 22, the picture retaking apparatus 200 includes a processor 211 and a memory 212. The processor 211 and memory 212 are interconnected via a bus. The bus, for example, can be an Inter-integrated Circuit (I2C) bus.

[0153] In some embodiments, the processor 211 can include a microcontroller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP), etc.

[0154] The memory 212 can include a flash memory chip, a read-only memory (ROM) disk, an optical disk, a USB drive, or a portable hard drive. The memory 212 can store various computer programs for execution by the processor 211.

[0155] The processor can be configured to execute the computer programs stored in the memory to obtain the picture shooting information when the movable platform performs the survey area operation, determine at least one missed area according to the picture shooting information when the movable platform performs the survey area operation, determine the corresponding retaking path according to the at least one missed area, and control the camera apparatus to perform the picture retaking operation based on the retaking path to obtain the pictures corresponding to the at least one missed area.

[0156] In some embodiments, the picture shooting information can include at least one of the shooting sequence number, the position, the attitude, or the location status.

[0157] In some embodiments, when determining the at least one missed area in real-time when the movable platform performs the survey area operation according to the picture shooting picture, the processor can be configured to determine the at least one missed area according to the picture shooting information corresponding to the shooting points where the shooting has been performed when the movable platform performs the survey area operation.

[0158] In some embodiments, when determining the at least one missed area when the movable platform performs the survey area operation according to the picture shooting picture, the processor can be configured to after the movable platform performs the survey area operation, determine the at least one missed area according to the picture shooting information corresponding to all the shooting points of the survey area operation.

[0159] In some embodiments, when determining the at least one missed area when the movable platform performs the survey area operation according to the picture shooting picture, the processor can be configured to calculate the distance between the two neighboring shooting points according to the picture shooting information, and if the distance between the two neighboring shooting points is greater than the distance threshold, determine the area between the two neighboring shooting points to be the missed area.

[0160] In some embodiments, when calculating the distance between the two neighboring shooting points according to the picture shooting information, the processor can be configured to if the survey area operation is single-direction shooting, calculate the first distance between the two neighboring shooting points according to the picture shooting information.

[0161] In some embodiments, before determining the area between the two neighboring shooting points to be the missed area in response to the distance between the two neighboring shooting points being greater than the distance threshold, the processor can be configured to obtain the scene information for the survey area operation and determine the first distance threshold according to the scene information. When determining the area between the two neighboring shooting points to be the missed area in response to the distance between the two neighboring shooting points being greater than the distance threshold, the processor can be configured to if the first distance between the two neighboring shooting points being greater than the first distance threshold, determine the area between the two neighboring shooting points to be the missed area.

[0162] In some embodiments, the scene information can include one or more of the flight altitude, the heading overlapping rate, or the shooting field of view.

[0163] When determining the first distance threshold according to the scene information, the processor can be configured to substitute the flight altitude, the heading overlapping rate, or the shooting field of view parameters into the predetermined distance calculation formula to calculate the first distance threshold.

[0164] In some embodiments, when calculating the distance between the two neighboring shooting points according to the picture shooting information, the processor can be configured to if the survey area operation is multi-direction shooting, calculate the second distance between the two shooting points with the same direction in the neighboring shooting sequences according to the picture shooting information and the third distance between the two neighboring shooting points with different directions within the same shooting sequence.

[0165] In some embodiments, before determining the area between the two neighboring shooting points to be the missed area in response to the distance between the two neighboring shooting points being greater than the distance threshold, the processor can be configured to obtain the scene information for the survey area operation, determine the second distance threshold between the two shooting points with the same direction of the neighboring shooting sequences according to the scene information, and determine the third distance threshold between the two neighboring shooting points with different directions within the same shooting sequence according to the second distance threshold. When determining the area between the two neighboring shooting points to be the missed area in response to the distance between the two neighboring shooting points being greater than the distance threshold, the processor can be configured to if the second distance between the two shooting points with the same direction of the neighboring shooting sequences is greater than the second distance threshold, determine the area between the two shooting points with the same direction of the neighboring shooting sequences to be the missed area, or if the third distance between the two neighboring shooting points with different directions within the same shooting sequence, determine the area between the two neighboring shooting points with different directions within the same shooting sequence to be the missed area.

[0166] In some embodiments, when determining the corresponding retaking path according to the at least one missed area, the processor can be configured to if there is only a single missed area, determine the starting shooting point and the ending shooting point for retaking according to the single missed area, and determine the retaking path according to the starting shooting point and the ending shooting point.

[0167] In some embodiments, when determining the corresponding retaking path according to the at least one missed area, the processor can be configured to if there are a plurality of missed areas, determine the execution sequence of the plurality of missed areas, and determine the retaking path according to the plurality of missed areas and the execution sequence of the plurality of missed areas.

[0168] In some embodiments, when determining the execution sequence of the plurality of missed areas, the processor can be configured to determine the execution sequence of the plurality of missed areas according to the execution sequence of the shooting path of the survey area operation.

[0169] In some embodiments, when determining the execution sequence of the plurality of missed areas, the processor can be configured to obtain the center position of each missed area and the starting position of the movable platform, determine the shortest execution path according to the center position of each missed area and the starting position of the movable platform, and determine the execution sequence of the plurality of missed areas according to the shortest execution path.

[0170] In some embodiments, determining the retaking path according to the plurality of missed areas and the execution sequence of the plurality of missed areas, the processor can be configured to determine the first shooting point and the second shooting point for each missed area, and determine the retaking path according to the first shooting point and the second shooting point for each missed area and the execution sequence of the plurality of missed areas.

[0171] In some embodiments, when determining the first shooting point and the second shooting point for each missed area, the processor can be configured to determine the last shooting point before missing the shooting corresponding to each missed area and the beginning shooting point when the shooting continues, and determine the first shooting point and the second shooting point for each missed area according to the last shooting point before missing the shooting and the beginning shooting point when the shooting continues.

[0172] In some embodiments, when determining the first shooting point of each missed area, the processor can be configured to determine the shooting sequence corresponding to the last shooting point before missing the shooting, and determine the first shooting point of each missed area according to the shooting sequence corresponding to the last shooting point before missing the shooting.

[0173] In some embodiments, determining the first shooting point of each missed area, the processor can be configured to if the shooting sequence of the last shooting point corresponds to the single-direction shooting, determine the first candidate shooting point and determine the first candidate shooting point to be the first shooting point. The distance between the first candidate shooting point and the last shooting point before missing the shooting can be the second distance threshold. The second distance threshold can be the distance threshold between two shooting points with the same direction of neighboring shooting sequences.

[0174] In some embodiments, when determining the first shooting point of each missed area, the processor can be configured to if the shooting sequence of the last shooting point before missing the shooting corresponds to the multi-direction shooting, determine the second candidate shooting point, and determine the second candidate shooting point as the first shooting point. The distance between the second candidate shooting point and the last shooting point before missing the shooting can be the third distance threshold. The third distance threshold can be the distance threshold between the two neighboring shooting points with different directions within the same shooting sequence.

[0175] In some embodiments, the processor can be configured to, after controlling the camera apparatus to perform the picture retaking operation according to the retaking path to obtain the picture corresponding to the at least one missed area, associate the picture obtained through the retaking operation with the picture shot during the survey area operation for storage.

[0176] In some embodiments, when associating the pictures obtained through the retaking operation with the pictures shot during the survey area operation for storage, the processor can be configured to generate the second folder associated with the first folder according to the first holder generated corresponding to the survey area operation and save the pictures obtained through the retaking operation into the second folder.

[0177] In some embodiments, when associating the pictures obtained through the retaking operation with the pictures shot during the survey area operation for storage, the processor can be configured to insert the pictures obtained through the retaking operation into the pictures shot during the survey area operation according to the shooting points corresponding to the pictures obtained through the retaking operation.

[0178] Embodiments of the present disclosure further provide a computer-readable storage medium. The computer-readable storage medium can store a computer program. The computer program can include program instructions. The processor can be configured to execute the program instructions to implement processes of the picture retaking method of embodiments of the present disclosure.

[0179] The computer-readable storage medium can be an internal storage unit of the movable platform or picture retaking apparatus, for example, a hard drive or memory of the movable platform or the picture retaking apparatus. The computer-readable storage medium can an external storage apparatus of the movable platform or the picture retaking apparatus, for example, a plugin hard drive, a smart media card (SMC), a secure digital (SD) card, or a flash card.

[0180] The described embodiments are some embodiments of the present disclosure. The scope of the present disclosure is not limited to this. Those skilled in the art can think of equivalent modifications or replacements within the scope of the present disclosure. These modifications and replacements are within the scope of the present disclosure. Thus, the scope of the present disclosure is subject to the scope of the claims.