Representations of foot features

[0001] A human foot has various features that can be identified for various purposes. For example, features of a particular foot can be used for building footwear that is customized for the particular foot. As another example, features of a foot can be identified for performing a medical diagnosis.

[0011] Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements. The figures are not necessarily to scale, and the size of some parts may be exaggerated to more clearly illustrate the example shown. Moreover, the drawings provide examples and/or implementations consistent with the description; however, the description is not limited to the examples and/or implementations provided in the drawings.

[0012] In the present disclosure, use of the term “a,” “an,” or “the” is intended to include the plural forms as well, unless the context clearly indicates otherwise. Also, the term “includes,” “including,” “comprises,” “comprising,” “have,” or “having” when used in this disclosure specifies the presence of the stated elements, but do not preclude the presence or addition of other elements.

[0013] In some examples, the identification of features of a foot can be based on manual analysis by a human, such as a doctor, a foot specialist, or another person. The identification of features of the foot may be based on a manual assessment of an actual foot of a user, or alternatively, can be based on a manual analysis of a three-dimensional (3D) image of the foot that has been acquired by an image scanner.

[0014] Such manual analysis of a foot to identify features can be time-consuming and may produce inaccurate results. For example, if the identification of the features of a foot is being performed as part of building footwear (e.g., customized orthotics, a customized shoe, etc.) for a user, inaccurate feature identification of a foot can lead to the footwear not properly fitting the foot of the user. Also, manual analysis of a foot to identify features may involve experts with special knowledge, such as doctors or foot specialists. Having to involve experts in identifying features for feet of users may slow down production of footwear, since the analyses performed by the experts can become a bottleneck in the footwear production process.

[0015] In accordance with some implementations of the present disclosure, an automated engine is provided to analyze a 3D representation of a foot for generating an output representation of features of the foot. The features of the foot include heads of certain metatarsal bones (e.g., the first and fifth metatarsal bones) of the foot, portions of a heel of the foot, and a portion of an arch of the foot, as examples.

[0016] A “feature” of a foot represented in the output representation can refer to any part of the foot of interest for a target purpose. The target purpose may include any of the following: building customized footwear (e.g., customized orthotics such as insoles, customized shoes, etc.) for a user, performing a medical diagnosis of an issue experienced by a patient, performing data analytics or machine learning, and so forth.

[0017] FIG. 1 is a diagram of an example arrangement that includes a foot features representation generation engine 102 according to some examples of the present disclosure. As used here, an “engine” can refer to a hardware processing circuit, which can include any or some combination of a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or another hardware processing circuit. Alternatively, an “engine” can refer to a combination of a hardware processing circuit and machine-readable instructions (software and/or firmware) executable on the hardware processing circuit.

[0018] The foot features representation generation engine 102 receives an input 3D representation 104 of a foot. Although reference is made to “foot” in the singular sense, it is noted that techniques or mechanisms according to some implementations of the present disclosure can process input representations of multiple feet and can produce output representations of features of multiple feet.

[0019] The input 3D representation 104 of the foot can be produced by a 3D scanner 106. The 3D scanner 106 can include an image sensor (e.g., a camera, a 3D depth sensor, etc.) that can acquire a 3D image of a foot. Alternatively or additionally, the 3D scanner 106 can include a different type of scanner, such as a pressure scanner that can reconstruct a 3D representation of a foot from detected pressure points when a foot is engaged with the pressure scanner. The pressure scanner can produce a representation of different pressures corresponding to different parts of the foot.

[0020] Based on the input 3D representation 104 of the foot, the foot features representation generation engine 102 produces an output representation 108 of foot features, such as heads of metatarsal bones and heels. In further examples, the foot features can further include a portion of an arch. The metatarsal bones of the foot are the bones at the forefoot that connect the cuneiform bones and cuboid bone of the foot to phalanges of the toes. There are five metatarsal bones, referred to as the first through fifth metatarsal bones. The first metatarsal bone is on the medial side of the foot (the side of the big toe), and the fifth metatarsal bone is on the lateral side of the foot (the side of the small toe). The heads of the metatarsal bones are connected to the cuneiform bones and cuboid bone of the foot.

[0021] The heel of the foot is the back part of the heel behind the arch and below the ankle. The heel includes a heel bone (or more formally, the calcaneus bone) at the posterior (rear) side of the foot. Portions of the heel bone of interest include a back heel (which is the most posterior portion of the heel bone), a medial heel (which is the portion of the heel bone on the medial side of the foot), and a lateral heel (which is the portion of the heel bone on the lateral side of the foot).

[0022] The arch of a foot is a region of the foot along the bottom of the foot between the ball of the foot and the heel of the foot. The ball of the foot is where the metatarsal bones are located. Generally, the arch is curved upwardly from the bottom of the foot.

[0023] As shown in FIG. 1, the foot features representation generation engine 102 identifies various points along a bottom surface corresponding to a sole (or plantar region) of a represented foot 110 (based on the input 3D representation 104 received by the foot features representation generation engine 102).

[0024] As used here, a “point” represents a location of an anatomical structure of the foot, such as heads of metatarsal bones, a portion of a heel, a portion of an arch, and so forth. For example, a point can be expressed in a position in 3D space (e.g., X, Y, Z coordinates).

[0025] In some examples, the points of the represented foot 110 identified by the foot features representation generation engine 102 include points 112 and 114 corresponding to heads of metatarsal bones of the represented foot 110. More specifically, the point 112 represents a location of the head of the first metatarsal bone, and the point 114 represents a location of the head of the fifth metatarsal bone.

[0026] Further points identified by the foot features representation generation engine 102 include points 116, 118, and 120 corresponding to various portions of the heel of the represented foot 110. The point 116 represents the location of the back heel of the represented foot 110, the point 118 represents the location of the medial heel of the represented foot 110, and the point 120 represents the location of the lateral heel of the represented foot 110.

[0027] Another point identified by the foot features representation generation engine 102 is a point 122 corresponding to the highest part of the arch 124 along the sole of the represented foot 110. The arch 124 of the represented foot 110 has an upward curve (see also FIG. 1A, which is not to the same scale as FIG. 1). The highest part of the arch 124 is defined in the Z axis, as measured from the bottom of the represented foot 110.

[0028] The foot features representation generation engine 102 is able to first identify the points 112 and 114 corresponding to the heads of the first and fifth metatarsal bones. Based on the locations of the points 112 and 114, the foot features representation generation engine 102 is able to determine locations of the points 116, 118, and 120 corresponding to the back heel, the medial heel, and the lateral heel of the represented foot 110. Based on the location of the point 118, the foot features representation generation engine 102 is able to determine the location of the point 122 that is the highest point of the arch 124 of the represented foot 110.

[0029] From the identified points 112, 114, 116, 118, 120, and 122, the foot features representation generation engine 102 generates the output representation 108 of foot features. The foot features included in the output generation can include a collection of the points 112, 114, 116, 118, 120, and 122, where the “collection” of the foregoing points can include a subset (less than all) of the points or all of the foregoing points.

[0030] FIG. 2 is a flow diagram of an example process 200 for generating an output representation (e.g., 108) of foot features, which can be performed by the foot features representation generation engine 102. Although FIG. 2 shows a specific sequence of tasks, it is noted that in other examples, the tasks can be performed in a different order, and some of the tasks can be omitted and/or replaced with other tasks.

[0031] The process 200 receives (at 202) an input 3D representation (e.g., 104) of a foot acquired by a 3D scanner (e.g., 106).

[0032] The process 200 identifies (at 204) a ground plane for the foot represented in the input 3D representation of the foot. The ground plane is a planar surface that provides a plane of reference for further processing by the foot features representation generation engine 102. It is noted that the input 3D representation of the foot may have an arbitrary orientation that depends on the orientation of the foot relative to a 3D scanner (e.g., 106) when the input 3D representation was acquired.

[0033] The ground plane can be a plane at Z=0 in the X-Y plane. The X-Y plane is a plane parallel to the X and Y axes, and Z=0 refers to a coordinate of 0 (or any other reference position) along the Z axis. In other examples, other ground planes can be defined. An example ground plane 302 is shown in FIG. 3, which is a rear view of the represented foot 110.

[0034] In the ensuing discussion, it is assumed that the ground plane 302 is at Z=0. In other examples, the ground plane 302 can be at Z=r, where r is any reference coordinate in the Z axis.

[0035] The process 200 settles (at 206) the represented foot 110 onto the ground plane 302. Settling the represented foot 110 onto the ground plane 302 refers to re-orienting (translating and/or rotating) the represented foot 110 such that the lowest portions of the sole of the represented foot 110 rest on the ground plane 302.

[0036] Once the represented 3D foot has been settled onto the ground plane 302, it is assumed that the heads of the first and fifth metatarsal bones should be on the ground plane, as well as the bottom of the heel. For example, FIG. 3 shows that the bottom 304 of the heel is resting on the ground plane 302.

[0037] The process 200 applies (at 208) Z-axis truncation to identify a portion of the plantar region of the foot. The portion of the plantar region that is identified satisfies a specified criterion, e.g., Z≤c, where c is a predefined constant value that can be tuned (e.g., adjusted by a human or another entity such as a program or machine based on empirical data or other information). FIG. 3 shows a dashed line 306 that represents c, which is a height above the ground plane 302 (which is at Z=0, for example).

[0038] In some examples, c can be set to 1 millimeter or another value.

[0039] The Z-axis truncation includes selecting (at 208-1) vertices at the bottom surface of the represented foot 110 where Z≤c (i.e., the Z coordinates of these selected vertices is less than or equal to c). A “vertex” refers a surface location on the bottom surface of the represented foot 110. If a vertex at the bottom surface of the represented foot 110 is at a height (along the Z axis) that is equal to or less than c, then the vertex is selected by the Z-axis truncation.

[0040] The Z-axis truncation includes projecting (at 208-2) the selected vertices onto the ground plane 302. Projecting a selected vertex onto the ground plane 302 refers to moving the selected vertex downwardly along the Z axis until the selected vertex is at Z=0. The selected vertices are included in a plantar profile 402 (with a dot pattern) in FIG. 4, where any selected vertex in the plantar profile 402 has been projected to Z=0.

[0041] The process 200 performs (at 210) metatarsal head location identification to extract the locations of the heads of the first and fifth metatarsal bones. The metatarsal head location identification determines (at 210-1) a length of the foot (referred to as “foot length 404”) based on the selected vertices in the plantar profile 402. The foot length 404 is the distance between the most posterior (rearmost) location 404-1 of the plantar profile 402 and the most anterior (frontmost) location 404-2 of the plantar profile 402.

[0042] The metatarsal head location identification identifies (at 210-2) a front region 406 of the represented foot 110, where the front region 406 is defined between a first position that is P % of the foot length 404 as measured from the most posterior location 404-1, and a second position that is Q % of the foot length 404 as measured from the most posterior location 404-1. In some examples, P is 50, and Q is 85. In other examples, P and Q can have other values, which can be derived based on empirical data generated from analyses of human feet, for example.

[0043] The metatarsal head location identification identifies (at 210-3) the widest portion (represented by a width line 408 in FIG. 4) of the front region 406. The metatarsal head location identification determines (at 210-4) a width W of the width line 408.

[0044] The metatarsal head location identification identifies (at 210-5) metatarsal head points (e.g., the points 112 and 114) along the width line 408 at locations W/a and W/b, respectively, where a equals 6 and b equals 12, in some examples. In other examples, the parameters a and b can be set to different values, which can be based on empirical data generated from analyses of human feet, for example.

[0045] In some examples, the value of W is less than each of a and b, so that W/a and W/b are fractional values less than 1. W/a is thus a fraction of the width line 408 as measured from the rightmost point of the width line 408, and W/b is another fraction of the width line 408 as measured from the rightmost point of the width line 408.

[0046] The points 112 and 114 represent the respective locations of the heads of the first and fifth metatarsal bones. It is noted that the head of the first metatarsal bone is more anterior along the Y axis (farther from the rear of the foot) as compared to the head of the fifth metatarsal bone.

[0047] The process 200 performs (at 212) heel identification to identify the locations of the back heel, the medial heel, and the lateral heel. The heel identification defines (at 212-1) a bisection line (409 in FIG. 4) that extends along the Y axis and is located half way between the points 112 and 114 (i.e., the bisection line 409 intersects a midpoint between the points 112 and 114).

[0048] The heel identification identifies (at 212-2) the intersection between the bisection line 409 and a posterior line 410 (along the X axis) that includes the most posterior location 404-1 of the plantar profile 402. The intersection point is represented by a circle 412 in FIG. 4. Note that the intersection point 412 is on the ground plane 302 (FIG. 3).

[0049] The heel identification identifies (at 212-3) a location of the back heel (e.g., the point 116) as a location that is a specified height H1 along the Z axis above the intersection point 412. In some examples, H1 is ⅜ inches or a different value, which can be selected based on an analysis of empirical data generated from analyses of human feet, for example.

[0050] The heel identification defines (at 212-4) a line 414 that is a specified distance Y1 along the Y axis from the posterior line 410. The specified distance Y1 can be 1 inch or a different value, which can be selected based on an analysis of empirical data generated from analyses of human feet, for example. As another example, the specified distance Y1 can be expressed as a percentage of the length of the represented foot 110.

[0051] The heel identification identifies (at 212-5) the point 118 corresponding to the medial heel (on the medial side of the represented foot 110), and identifies (at 212-6) the point 120 corresponding to the lateral heel (on the lateral side of the represented foot 110). Each of the points 118 and 120 are at the Y1 distance from the posterior line 410, and at the H1 height above the ground plane 302 (as depicted in FIG. 3).

[0052] Although FIG. 3 depicts an example in which the points 118 and 120 are at the same height as the point 116 corresponding to the back heel, it is noted that in other examples, the point 118 and/or the point 120 can be at a different height from the point 116.

[0053] The process 200 performs (at 214) an arch identification based on relationships between the point 112 representing the location of the head of the first metatarsal bone and the point 118 representing the location of the medial heel. The arch identification identifies (at 214-1) a projected point 130 on the bottom surface of the represented foot 110 that is X-axis aligned with the medial heel point 118 and that is on a line 132 that is parallel to the bisection line 409 of the represented foot 110 (as depicted in FIG. 1). The projected point 130 being X-axis aligned with the medial heel point 118 refers to the projected point 130 being shifted along the X axis to the lateral side of the represented foot 110 until the projected point 130 reaches the line 132.

[0054] The arch identification identifies (at 214-2) points 140 and 142 along the line 132, where the point 140 is at a location that is R % of the length of the line 132 as measured from the point 112, and the point 142 is at a location that is T % of the length of the line 132 as measured from the point 112. In some examples, R is 20 and T is 80; in other examples, other values of R and T can be used (which can be derived based on empirical data generated from analyses of human feet, for example). A region between the points 140 and 142 is a region where the highest part of the arch 124 is expected to be present.

[0055] The arch identification identifies (at 214-3) a connection line 134 connecting the point 112 and the point 118. The arch identification projects (at 214-4) vertical lines along the Z axis (e.g., vertical line 136) from the connection line 134. Each vertical line (e.g., 136) starts at the connection line 134 and ends when the vertical line reaches the surface of the arch 124 (which curves upwardly from the bottom of the represented foot 110). A comparison of the projected vertical lines can be used to determine which location of the arch 124 is at the highest elevation point (e.g., the longest vertical line of the projected vertical lines would correspond to the highest elevation point). Based on the projected vertical lines, the arch identification identifies (at 214-5) the highest elevation point of the arch 124. This highest elevation point is the location of the point 122.

[0056] Once the points 112, 114, 116, 118, 120, and 122 are identified, the process 200 generates (at 216) an output representation of features of the foot. The features can include information that includes coordinates of the points 112, 114, 116, 118, 120, and 122 (or a subset of these points). The output representation of the features of the foot can be in graphical form, in text form, or in a different form.

[0057] FIG. 5 is a block diagram of a non-transitory machine-readable or computer-readable storage medium 500 storing machine-readable instructions that upon execution cause a system to perform various tasks. The system can include a computer or multiple computers. The machine-readable instructions can be part of the foot features representation generation engine 102 of FIG. 1, for example.

[0058] The machine-readable instructions include input foot representation reception instructions 502 to receive an input 3D representation of a foot. The input 3D representation of the foot can include a representation of a single foot or both feet of a person.

[0059] The machine-readable instructions include input 3D representation analysis instructions 504 to analyze the input 3D representation of the foot to identify first points corresponding to metatarsal bones (e.g., first and fifth metatarsal bones) of the foot. Each first point can represent a location of the head of the respective metatarsal bone.

[0060] The machine-readable instructions include heel point identification instructions 506 to, based on locations of the first points corresponding to the metatarsal bones, identify a second point (e.g., 116, 118, or 120 in FIG. 1) corresponding to a heel of the foot.

[0061] The machine-readable instructions include foot feature representation generation instructions 508 to generate, based on the first points and the second point, a representation of features of the foot.

[0062] In further examples, the machine-readable instructions can identify the second point corresponding to the heel based on a line (e.g., the bisection line 409 of FIG. 4) that intersects a midpoint between the first points (e.g., the points 112 and 114 in FIG. 1).

[0063] In further examples, the machine-readable instructions can, in a specified front region (e.g., 406 in FIG. 4) of a sole of the foot, identify a width of a widest portion of the specified front region (e.g., along width line 408 of FIG. 4), and determine positions of the first points along the width.

[0064] In further examples, the second point corresponds to a back heel of the foot (e.g., 116 in FIG. 1), and the machine-readable instructions can identify, based on a location of the second point, further points corresponding to a medial heel and a lateral heel of the foot (e.g., 118 and 120 in FIG. 1).

[0065] In further examples, the machine-readable instructions can identify the further points based on determining locations of the further points that are a specified distance (e.g., the Y1 distance of FIG. 4) away from the second point corresponding to the back heel.

[0066] In further examples, the machine-readable instructions can re-orient (translate and/or rotate) the representation of the foot in the input 3D representation such that lowest portions of a sole of the foot are on a ground plane (e.g., 302 in FIG. 3). The first points are on the ground plane.

[0067] In further examples, the machine-readable instructions can, based on the locations of the first points and a location of the second point, identify an arch point (e.g., 122 in FIG. 1) on an arch of the foot.

[0068] In further examples, the second point corresponds to a back heel of the foot, and the machine-readable instructions can identify, based on a location of the second point, a medial heel point (e.g., 118 in FIG. 1) corresponding to a medial heel of the foot, where the arch point is identified based on the medial heel point.

[0069] In further examples, a given point of the first points corresponds to a first metatarsal bone. The machine-readable instructions can identify a region (e.g., the region between points 140 and 142 in FIG. 2) between the given point and a medial heel point along a length axis of the foot, and identify, within the identified region, a highest elevation point of a sole of the foot, where the arch point includes the highest elevation point.

[0070] In further examples, the region between the first point and the medial heel point along the length axis of the foot includes identifying a projected point (e.g., 130 in FIG. 1) that is aligned with the medial heel point along a width axis (e.g., X axis) of the foot, and a line (e.g., 132) is defined between the projected point and the first point. A central portion (e.g., the portion between points 140 and 142) of the line is identified, where the region identified above is in the central portion.

[0071] FIG. 6 is a flow diagram of a process 600 for building a footwear for a foot, according to some examples. The process 600 can be performed by the foot features representation generation engine 102 according to some examples.

[0072] The process 600 includes receiving (at 602) an input 3D representation of a foot acquired by a 3D scanner (e.g., 106 in FIG. 1).

[0073] The process 600 includes determining (at 604) a plantar surface of the foot in the input 3D representation. For example, the plantar surface can include selected vertices included in the plantar profile 402 of FIG. 4.

[0074] The process 600 includes identifying (at 606) first and second points (e.g., 112 and 114 in FIG. 1) on the plantar surface, the first and second points corresponding to heads of first and fifth metatarsal bones of the foot.

[0075] The process 600 includes identifying (at 608), based on first and second points, a heel point (e.g., 116 in FIG. 1) corresponding to a heel of the foot.

[0076] The process 600 includes identifying (at 610), based on the first point and the heel point, an arch point (e.g., 122 in FIG. 1) corresponding to an arch of the foot.

[0077] The process 600 includes generating (at 612) a representation of features of the foot, the features including the first and second points, the heel point, and the arch point.

[0078] FIG. 7 is a block diagram of a system 700 according to some examples. The system 700 can include a computer or a collection of multiple computers.

[0079] The system 700 includes a hardware processor 702 (or multiple hardware processors). A hardware processor can include a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or another hardware processing circuit.

[0080] The system 700 further includes a storage medium 704 storing machine-readable instructions executable on the hardware processor 702 to perform various tasks. Machine-readable instructions executable on a hardware processor can refer to the instructions executable on a single hardware processor or the instructions executable on multiple hardware processors.

[0081] The machine-readable instructions in the storage medium 704 include input foot representation reception instructions 706 to receive an input 3D representation of a foot.

[0082] The machine-readable instructions in the storage medium 704 include plantar surface determination instructions 708 to determine, based on the input 3D representation, a plantar surface of the foot.

[0083] The machine-readable instructions in the storage medium 704 include metatarsal point identification instructions 710 to identify first points representing locations of heads of metatarsal bones of the foot.

[0084] The machine-readable instructions in the storage medium 704 include second point identification instructions 712 to, based on locations of the first points, identify a second point representing a location of a portion of a heel of the foot, where the first points and the second point are on the plantar surface.

[0085] The machine-readable instructions in the storage medium 704 include output representation generation instructions 712 to generate, based on the first points and the second point, a representation of features of the foot.

[0086] A storage medium (e.g., 500 in FIG. 5 or 704 in FIG. 7) can include any or some combination of the following: a semiconductor memory device such as a dynamic or static random access memory (a DRAM or SRAM), an erasable and programmable read-only memory (EPROM), an electrically erasable and programmable read-only memory (EEPROM) and flash memory or other type of non-volatile memory device; a magnetic disk such as a fixed, floppy and removable disk; another magnetic medium including tape; an optical medium such as a compact disk (CD) or a digital video disk (DVD); or another type of storage device. Note that the instructions discussed above can be provided on one computer-readable or machine-readable storage medium, or alternatively, can be provided on multiple computer-readable or machine-readable storage media distributed in a large system having possibly plural nodes. Such computer-readable or machine-readable storage medium or media is (are) considered to be part of an article (or article of manufacture). An article or article of manufacture can refer to any manufactured single component or multiple components. The storage medium or media can be located either in the machine running the machine-readable instructions, or located at a remote site from which machine-readable instructions can be downloaded over a network for execution.

[0087] In the foregoing description, numerous details are set forth to provide an understanding of the subject disclosed herein. However, implementations may be practiced without some of these details. Other implementations may include modifications and variations from the details discussed above. It is intended that the appended claims cover such modifications and variations.