3D Fetal Face Reconstruction from Ultrasound Imaging

Ant

onia Alomar

1 a

, Araceli Morales

1 b

, Kilian Vellv

2 c

, Antonio R. Porras

3,4,5 d

Fatima Crispi

2 e

, Marius George Linguraru

3,6 f

, Gemma Piella

1 g

and Federico Sukno

1 h

Department of Information and Communications Technologies, Universitat Pompeu Fabra, Barcelona, Spain

Fetal Medicine Research Center (BCNatal), Hospital Cl

ınic and Hospital Sant Joan de D

eu, Universitat de Barcelona,

Barcelona, Spain

Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital, Washington, D.C., U.S.A.

Department of Biostatistics and Informatics, Colorado School of Public Health,

University of Colorado Anschutz Medical Campus, Aurora, CO, U.S.A.

Department of Pediatrics, School of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO, U.S.A.

Departments of Radiology and Pediatrics, School of Medicine and Health Sciences, George Washington University,

Washington, D.C., U.S.A.

Keywords:

Craniofacial Morphology, 3D Morphable Model, Facial Dysmorphology, Fetal Reconstruction, Prenatal

Diagnosis.

Abstract:

The fetal face contains essential information in the evaluation of congenital malformations and the fetal brain

function, as its development is driven by genetic factors at early stages of embryogenesis. Three-dimensional

ultrasound (3DUS) can provide information about the facial morphology of the fetus, but its use for prenatal

diagnosis is challenging due to imaging noise, fetal movements, limited ﬁeld-of-view, low soft-tissue contrast,

and occlusions. In this paper, we propose a fetal face reconstruction algorithm from 3DUS images based on

a novel statistical morphable model of newborn faces, the BabyFM. We test the feasibility of using newborn

statistics to accurately reconstruct fetal faces by ﬁtting the regularized morphable model to the noisy 3DUS

images. The algorithm is capable of reconstructing the whole facial morphology of babies from one or several

ultrasound scans to handle adverse conditions (e.g. missing parts, noisy data), and it has the potential to aid

in-utero diagnosis for conditions that involve facial dysmorphology.

1 INTRODUCTION

Craniofacial malformations that occur because of ab-

normal development comprise over one third of all

congenital (i.e., birth) defects (Mossey and Catilla,

2001). These anomalies comprise a wide range of

heterogeneous conditions and often have a multifac-

torial origin, including genetic and environmental fac-

tors (S

orop Florea et al., 2018). These malforma-

tions can impact swallowing, breathing, hearing, vi-

sion, speech, and cognitive development (on Gov-

ernment Affairs, 2020; EvansAnne et al., 2018), and

https://orcid.org/0000-0003-3658-5832

https://orcid.org/0000-0003-4930-6142

https://orcid.org/0000-0002-2376-7664

https://orcid.org/0000-0001-5989-2953

https://orcid.org/0000-0002-7422-5240

https://orcid.org/0000-0001-6175-8665

https://orcid.org/0000-0001-5236-5819

https://orcid.org/0000-0002-2029-1576

they impose a large psychosocial, healthcare, and eco-

nomic burden.

Early diagnosis is often crucial for the effective

treatment of functional and developmental aspects

(Learned-Miller et al., 2006; Tu et al., 2018; Tu et al.,

2019). However, not all syndromes are easily identi-

ﬁed, some of them having subtle physical manifesta-

tions; careful clinical assessment may be necessary to

distinguish an isolated abnormality from an atypical

or mildly manifested syndrome. Moreover, the iden-

tiﬁcation of the speciﬁc syndrome is important for the

overall care of the patient (EvansAnne et al., 2018). In

this sense, the analysis of facial morphology can pro-

vide relevant information and serve as a pre-screening

tool, facilitating the early detection of developmen-

tal disorders (Menezes et al., 2016; Merz and Welter,

2005). Efforts are being made to shift from diagno-

sis at birth, or during the ﬁrst years of life, to prenatal

diagnosis, which facilitates parents’ counselling and

careful planning of delivery and postnatal treatment

(Pooh and Kurjak, 2011). However, prenatal diag-

Alomar, A., Morales, A., Vellvé, K., Porras, A., Crispi, F., Linguraru, M., Piella, G. and Sukno, F.

3D Fetal Face Reconstruction from Ultrasound Imaging.

DOI: 10.5220/0010340306150624

In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 4: VISAPP, pages

615-624

ISBN: 978-989-758-488-6

615

nosis of fetal syndromes is not easy, mainly because

of the wide range of morphological features involved

and the challenging nature of medical images.

Ultrasound is the primary imaging modality for

fetal assessment. It is a noisy image modality, but

it has the advantage of being widely available, cost-

effective, non-ionizing, and it allows real-time exam-

ination. Three-dimensional ultrasound (3DUS) facil-

itates the evaluation of anatomical structures from the

facial surface and can therefore aid diagnosis (An-

dresen et al., 2012; Werner et al., 2016; Merz and

Abramowicz, 2012). A detailed 3D model of the fetus

face could thus play a crucial role in prenatal diagno-

sis.

Little research has been done in 3D face recon-

struction from fetal images, mainly due to the limita-

tions of prenatal imaging itself. In (Dall’Asta et al.,

2017), it was presented a statistical shape model con-

structed from 20 3DUS scans that were manually seg-

mented and aligned, and statistically signiﬁcant dif-

ferences in face shape were found between normal

and abnormal fetuses. There are some works on gen-

erating physical fetal models (although not always

face speciﬁc) from 3DUS, magnetic resonance imag-

ing, and computer tomography (Werner et al., 2010;

Werner et al., 2015; Menezes et al., 2016; Speranza

et al., 2017). However, they involve slice-by-slice

manual segmentation and post-processing with pro-

prietary software.

In this paper, we explore the feasibility of recon-

structing the facial morphology before birth by ana-

lyzing 3DUS images of the fetus from routine scan-

ning with the help of a recently proposed statisti-

cal model constructed from 3D scans of babies and

newborns: the Baby Face Model (BabyFM) (Morales

et al., 2020). Differently from previous works, we do

not build our model directly from the noisy fetal im-

ages, but employ statistics from newborns to constrain

the geometric reconstruction of the fetal face. In this

way, we circumvent the difﬁculties associated with

building accurate models from the noisy 3DUS im-

ages. Tests on a small set of fetal scans show promis-

ing results in both qualitative and quantitative terms,

even in adverse conditions (e.g. missing parts, noisy

data).

2 MATERIALS

2.1 3D Baby Face Morphable Model

A 3D morphable model (3DMM) is a tool for repre-

senting 3D shapes and textures. In the context of face

analysis, the idea is to learn a general 3D face model

that is able to encode the statistics of facial shape. A

crucial aspect to consider when using a 3DMM is that

the statistics encoded in the model must match those

of the target population, e.g., in terms of ethnicity,

gender and, especially important for this work, age.

The latter has been an important obstacle for the ap-

plication of pre-existing 3DMMs to fetal data, since

all available 3DMMs were built from adults and, al-

though sometimes they also included children, none

of them included babies. However, very recently,

Morales et al. (Morales et al., 2020) have published

the Baby Face Model (BabyFM), which constitutes

the ﬁrst 3DMM built exclusively from babies, includ-

ing an important proportion of data from newborns.

The BabyFM was built from 45 3D scans of baby

faces (mean age 8.42 ±6.45 months). Several ethnici-

ties were included: Caucasian (47%), African Ameri-

can (24%), Hispanic (20%), and Asian (9%). Also,

the data were roughly gender-balanced: 56% male

and 44% female. The BabyFM covers the facial re-

gion that is delimeted by the chin the forehead and

the ears, all included. Additionally, the vertice indices

for 23 anatomical landamrks (Figure 1) are provided,

which are used to initialise the 3D facial reconstruc-

tion (see Section 3.1.1).

Figure 1: Anatomical landmarks. Illustration of the 23

anatomical landmarks considered in this project on the

mean baby morphable model face. Landmark abbrevia-

tions: enL/R = endocanthion Left/Right; n = nasion; exL/R

= exocanthion Left/Right; aL/R = alare Left/Right; acL/R

= alar crest Left/Right; prn = pronasale; sn = subnasale;

chL/R = cheilion Left/Right; cphL/R = crista philtrum

Left/Right; ls = labiale superius; li = labiale inferius; sl =

sublabiale; pg = pogonion; tL/R = tragion Left/Right; and

oiL/R = otobasion inferius Left/Right.

2.2 Test Database

To evaluate our methods, 10 3DUS scans from 4 fe-

tuses were collected, i.e., there were multiple 3DUS

images for each of fetus, corresponding to different

VISAPP 2021 - 16th International Conference on Computer Vision Theory and Applications

616

viewing directions. These fetuses had no relation to

any of the babies used for the construction of the

BabyFM. The 3DUS scans were obtained at Hospi-

tal Cl

ınic and Hospital Sant Joan de D

eu, Barcelona,

according to its Ethical Research Committee and the

current legislation. Images were acquired using a

General Electric Voluson E6 (General Electric, IL,

USA) US machine with a low-frequency probe (4-8

MHz).

Three-dimensional meshes were extracted using

just a threshold segmentation. The meshes contained

not only the face but also other parts of the fetus’

body, placenta, and noise. Using the Landmark soft-

ware 3.6

, we positioned a subset from the 23 tar-

geted landmarks in the BabyFM (see Figure 1) on

each fetal scan, according to their visibility. The iden-

tiﬁcation of these anatomical landmarks in the fetal

scan is challenging because of the occlusions (e.g.,

the baby may be positioned with the hand on the face)

and the noisy nature of the data, and therefore not all

of them could be positioned for each fetus.

Additionally to the 3DUS scans, for each of the

babies we had three 2D postnatal photographs taken

from different viewpoints by the parents with their

mobile phones. These images were used to obtain a

2D-3D reconstruction of the baby face to which we

could quantitatively compare with the reconstruction

obtained from the US scans (see Section 3.2). This

simple setup was aimed to avoid having to scan new-

born babies with special equipment.

3 METHODS

Our data processing pipeline consists of two main

stages (Figure 2): 3DUS ﬁtting and multiple image

ﬁtting.

3.1 3D Ultrasound Fitting

First, the 3D reconstruction of the fetal face is ob-

tained from the 3DUS images by ﬁtting the BabyFM

to it, i.e., ﬁnding the shape parameters in the 3DMM

that best reproduce the face observed in the US. At

this stage, the BabyFM works as a statistical regular-

izer allowing a better robustness to noise and other

artifacts.

3.1.1 Landmark-based Fitting

A ﬁrst estimation of the 3D fetal face is obtained

considering only the landmarks positioned in the US

https://landmark2.software.informer.com/download/

mesh. For this, an iterative procedure consisting of (1)

landmark alignment and (2) shape parameter calcula-

tion is performed. In the landmark alignment stage,

Procrustes analysis is used to ﬁnd a similarity trans-

formation to ﬁt the mean face shape of the BabyFM to

the US landmarks. Then, the shape parameters α that

best deﬁne the fetal face in the US scan are estimated

by ﬁrst solving the normal equation:

α = (Φ

)

−1

(x − ¯x), (1)

where Φ

is the reduced shape basis matrix (i.e., the

rows of the eigenvector matrix that correspond to the

landmarks), and then regularizing to ensure obtain-

ing plausible faces. The shape parameters are as-

sumed to follow a multivariate Gaussian distribution.

Therefore, we constrain the shape parameter vectors

to lie within a hyper-ellipsoid in the parameter space,

the size of which is determined by the variances (the

eigenvalues) of the data.

The two-stage landmark-based ﬁtting is iterated

20 times to ensure convergence. Finally, the mean

approximation error (

E) between the landmarks of the

ﬁtted morphable face model and those of the US mesh

is calculated as follows:

E =

∑

j=1

||l

us, j

− T (l

model, j

)||,

(2)

where l

∈ R

is the j-th landmark, T is the transfor-

mation that maps the BabyFM mean to the US, and

m is the number of anatomical landmarks that were

positioned in the US mesh.

3.1.2 Iterative Closest Point with Statistical

Constraints

The fetal face reconstruction obtained from the

landmark-based ﬁtting is reﬁned using an iterative

closest point (ICP) algorithm. In every iteration, the

ICP algorithm ﬁts the face reconstruction to the 3DUS

mesh and then recovers the model’s shape parameter

α, analogously to the landmark-based ﬁtting (i.e., by

Eq. 1 followed by regularization) but now using the

whole surface, i.e., all rows of the shape basis Φ in-

stead of just Φ

. To increase the robustness to input

artifacts, the point-matching was applied under geo-

metric and uniqueness constraints (to minimize the

impact of outliers and ensure one-to-one mapping).

The ICP with statistical constraints was repeated by

alternating between the correspondence mapping and

the model’s parameter update followed by statistical

regularization, until the error difference between con-

secutive iterations was below a predeﬁned threshold.

In this work, this convergence threshold was set to

−2

mm.

3D Fetal Face Reconstruction from Ultrasound Imaging

617

Figure 2: Proposed pipeline: 3DUS ﬁtting to obtain the fetal face and multiple image ﬁtting to obtain the baby face after birth.

3.2 Multiple Image Fitting

In order to quantitatively validate the fetal shapes esti-

mated from the 3DUS scans, we reconstruct the new-

born 3D face from a set of three 2D images (frontal,

left, and right pose) taken shortly after birth. The

BabyFM is used here to estimate the facial 3D ge-

ometry of the newborn. Once the 3D geometry is ob-

tained, facial texture can also be added to obtain a

photo-realistic 3D reconstruction.

We address the 3D-from-2D reconstruction prob-

lem using sparse geometric features (edges and land-

marks). Our approach is based on the algorithm pro-

posed in (Bas et al., 2016), but using multiple images

rather than only a single image. We start by position-

ing the anatomical landmarks in the different images

using a 2D landmarker and obtaining the edges by ap-

plying the Canny edge detector.

Then, an initialization of the 3D face is obtained

using only the landmarks. The landmark ﬁtting is then

reﬁned in an iterative closest point manner by ﬁnding

the closest image edge to each model contour vertex.

The model edge vertices can then be considered as

landmarks with known 2D position, for which opti-

mal pose and shape estimates can be easily computed

under the assumption of a scaled orthographic projec-

tion. In particular, we obtain the optimal pose and

shape parameter by minimizing an objective function

that include landmark, edge, and prior terms:

E(α, R, t, s) = w

lmk

(α, R, t, s)+

+ w

Edge

(α, R, t, s) + w

Prior

(α), (3)

where α is the shape parameters vector and R, t, and s

the pose parameters (rotation, translation, and scale)

assuming a scaled orthographic projection. The pa-

rameters w

, w

and w

correspond to the weights

given to each error term, the sum of the three weights

should be equal to one. The used values to perform

the reconstructions were 0.25, 0.25, 0.50 respectively.

The landmark term penalizes differences between the

actual landmarks positions on the images and the ones

obtained by projecting the 3D model landmarks. The

edge term compares the edges detected on the image

with those induced by the model due to occluding

boundaries. The prior term acts as a regularizer of

the shape parameters based on the statistics encoded

in the BabyFM.

4 RESULTS

4.1 US Fitting

We applied our reconstruction pipeline to each of the

fetuses scans. Figure 3 shows the US images ob-

tained from the Voluson system (GE Healthcare), the

US meshes obtained after the threshold segmentation,

and the 3D reconstruction that we obtained. As can

be observed in Figure 3, the input data is quite chal-

lenging. For example, in most of the 3DUS images,

the ears are not present or are extremely noisy. Never-

theless, our method is able to estimate an approximate

ear shape by exploiting the geometric correlations en-

VISAPP 2021 - 16th International Conference on Computer Vision Theory and Applications

618

Figure 3: US images (top), US meshes (middle), and their corresponding reconstructions (bottom). From left to right: case 1,

case 2, case 3, and case 4.

coded in the BabyFM. Figures 4 displays the proﬁle

view of cases 1 and 2 to show the ears.

Figure 4: Example of reconstructions (left: case 1, right:

case 2) in proﬁle view.

4.1.1 Single US Fitting vs Multiple US Fitting

As we have multiple 3DUS scans from different views

for each individual, we checked whether the recon-

structions obtained independently for the same fetus

were similar. Also, we investigated whether the si-

multaneous use of multiple views improves the ob-

tained results. For this, the algorithm was adapted to

merge the correspondences of each 3DUS view and

then calculate a unique vector of shape parameters for

the multiple scans.

Figure 5-6 show examples of reconstructions ob-

tained independently for the same individual (cases 1

and 2, respectively) from two different 3DUS views.

In Figure 5, we can appreciate that the fetus has plump

lips (African ethnicity) and the model is not able to

full reconstruct them accurately, but it attempts to

compensate this fact by opening the mouth. The used

BabyFM was built including only a 24% of African

American babies, which might explain why it is not

able to perfectly reconstruct the lips. Nevertheless,

the nose, eyes, and pose are well estimated, and the re-

constructions are convincingly similar to the original

renderings. More importantly, there are no substan-

tial differences between the reconstructions obtained

form the two 3DUS scans of this same fetus; the sec-

ond one has wider cheeks, but the nose, mouth, and

eyes are similar. In Figure 6 (which corresponds to

a case of fetal growth restriction (Peleg et al., 1998;

Albu et al., 2014)), the obtained reconstructions cap-

ture the skinny face. On the other hand, the second

US has part of the mouth and chin occluded by an

arm, which makes the reconstruction more challeng-

ing.

Next, we reconstruct the face considering simulta-

neously the multiple US scans for each fetus. This can

3D Fetal Face Reconstruction from Ultrasound Imaging

619

Figure 5: Reconstructions from different views for case 1.

Figure 6: Reconstructions from different views for case 2.

be especially useful to obtain an accurate complete

reconstruction even when each scan provides only a

partial view of the facial anatomy. Figure 7 illustrates

this by reconstructing the face for case 3 from 4 dif-

ferent views.

4.2 Post-natal Fitting from Multiple

Images

Since we wish to use the post-natal reconstructions

to validate the geometries reconstructed from the

3DUS scans, it is important to assess the accuracy

that we can expect in the post-natal reconstruction.

To do so, we reconstructed the baby faces of an ad-

ditional dataset, which contained three 2D images

(frontal, right and left pose), as well as the 3D scan

for each of the three babies from the Children’s Na-

tional Hospital, Washington D.C. and one newborn

from the Hospital Cl

ınic and Hospital Sant Joan de

eu, Barcelona. To evaluate the quality of the recon-

structed 3D face, we compute the geometric error for

each reconstruction, by ﬁrst applying a transforma-

tion to align it with the ground-truth, which in this

case is the 3D scan, and then computing its point-to-

point distance.

The color error maps of the three babies in Figure

8 show that the 2D-3D ﬁtting performed obtained ac-

ceptable results, as the mean error of the reconstruc-

tions is around 4 mm. In the three cases, the large

errors in the forehead are caused by the lack of land-

marks in this region and the black hat that the babies

wear, as it interferes with the image edge detection

resulting in wrong correspondences. However, no-

tice that the error in the rest of the face, which cor-

responds to the main facial features, is considerably

smaller. This, together with the fact of using uncali-

brated input images, suggests that the accuracy of the

reconstructions is satisfactory.

Once we have obtained the 3D reconstruction

from multiple 2D images, we add texture to it to have

more photo-realistic results to show to the parents. As

expected, the textured meshes improve the visual re-

semblance with the images (bottom row of Figure 8).

4.3 Pre- and Post-natal Comparison

The reconstructions obtained by ﬁtting the BabyFM

to the 3DUS scan and to the set of 2D images after

birth are compared quantitatively, using color error

maps, as a proxy to evaluate the performance of the

fetal face reconstruction. The color error maps are

obtained computing the geometric error between the

prenatal and the postnatal reconstructions.

We compared the reconstructions from the 3DUS

ﬁtting and the multiple image ﬁtting for case 3, for

which we had both the postnatal 2D images and the

US scans, even if the difference between them was 16

weeks. Figure 9 shows the 2D frontal image of the

baby, the baby face reconstruction using the multiple

image ﬁtting, the US fetal face obtained, and the color

map error between the prenatal and the postnatal re-

construction. Although no age correction was applied

to the 3DUS reconstruction, we can appreciate sufﬁ-

cient resemblance to support the statement these are

two instances of the same baby, taken a few months

apart from each other.

In quantitative terms, the overall mean reconstruc-

tion error is 1.73 mm. Main facial features such as

eyes, nose, and mouth present the smallest reconstruc-

VISAPP 2021 - 16th International Conference on Computer Vision Theory and Applications

620

Figure 7: US reconstruction from multiples views for case 3. Top: US images. Bottom: single reconstruction shown from

different views.

tion error (see Figure 9), whereas the forehead region

-where no landmarks and few edges are available-

shows the largest reconstruction error. It is expected

that after applying some age correction mechanism,

the resemblance will be even stronger.

5 DISCUSSION

In this study, our aim is to obtain accurate fetal face

reconstructions from 3DUS images in the prenatal

stage in order to assess craniofacial morphology as

early as possible. In spite of the growing interest in

the early assessment of craniofacial morphology, the

3D renderings provided by the standard clinical view

software have some limitations, e.g., two images from

different views seems to be two different babies, or

the facial features cannot be distinguished due to the

smoothing or the large amount of noise. For this rea-

son, clinicians usually have to manually remove the

noise to assess the baby’s morphology. Manual seg-

mentation to remove the noise in the US is very time

consuming, even for expert clinicians. The presented

algorithm can help them to visualize the fetal face

with more quality and without performing any manual

segmentation. Therefore, the subjectivity introduced

by manual segmentation is avoided.

Our algorithm overcomes the limitation of noisy

data of the fetal face reconstruction algorithms pro-

posed so far, as no restrictions to the US data are ap-

plied. In (Bonacina et al., 2016) and (Dall’Asta et al.,

2017) studies, a strict inclusion criteria was applied

to the US data, as their procedure is really suscep-

tible to the quality of the input data. As a conse-

quence, all the cases with missing facial parts, large

amounts of noise, unsatisfactory deﬁnition of the fa-

cial borders or obstructed vision caused by the uterine

wall/intervening limbs/cord loops, were excluded for

those studies, limiting the applicability of their algo-

rithms. In contrast, our approach exploits the facial

geometry statistics encoded in the BabyFM, which

allows us to reconstruct the 3D fetal face even with

large amounts of noise, occlusions or missing parts.

Also, it allows reconstructing a unique facial geome-

try from multiple US scans, so that the speciﬁc details

available from each different view can be adequately

combined to produce an accurate reconstruction of the

whole facial geometry.

The reliability and accuracy of the reconstructions

depend on the quality of the US and the fetal pose

and expression. Higher levels of noise and/or oc-

clusions inevitably result in less accurate reconstruc-

tions. Nevertheless, our algorithm demonstrates that

reconstructions of the same individual from different

US scans are very similar and the faces differ in small

details. A large quantitative analysis was not possi-

ble, but the proposed 2D-3D ﬁtting tests performed

for those cases with ground-truth, show promising re-

sults that could be used as proxy ground-truth for the

fetal reconstructions. Qualitative assessment of the

results were largely convincing, highlighting the po-

tential of the proposed technique to aid prenatal diag-

nosis.

The immediate next steps in our work will be

to address a larger quantitative validation of the

reconstruction algorithm, especially in terms of com-

parisons between the reconstructions before and after

3D Fetal Face Reconstruction from Ultrasound Imaging

621

Figure 8: 3D reconstruction from 2D images. Top: 2D images (only the frontal view is shown). Middle: Error reconstruction

(using the frontal, right, and left views) to the ground truth scans. Bottom: 3D reconstruction with texture.

Figure 9: Comparison between prenatal and postnatal reconstructions for case 3. From left to right: frontal 2D image, Us

reconstruction, 3D from 2D reconstruction, and error between reconstructions.

birth. The US scans analyzed so far, were acquired

between weeks 20 and 30 of gestation and the post-

natal images were taken once the baby was already a

few weeks old. Thus, the time difference between the

US and the photos used to get the 3D reconstructions

was approximately about 20 weeks. This difference

might have a clear impact in the face morphology of

the baby. For this reason, we are also investigating an

age correction mechanism to be applied to the mesh

extracted from the 3D fetal faces to see if we can de-

scribe a realistic age progression of the face of

the baby.

After performing the quantitative analysis inves-

tigating whether the estimated morphology is signif-

icantly different between patients with intrauterine

grow restriction and controls is also an interesting line

of continuation of this work.

VISAPP 2021 - 16th International Conference on Computer Vision Theory and Applications

622

6 CONCLUSIONS

To conclude, in this paper we proposed a fetal face re-

construction algorithm from 3DUS images. The ap-

proach differs from the existing ones proposed in the

literature, as it is based on the ﬁtting of a deformable

BabyFM to 3DUS to remove the noise and to recover

the whole baby facial morphology. It was demon-

strated that our algorithm is able to reconstruct the

whole facial morphology of the babies under differ-

ent conditions (large amounts of noise, missing parts

or multiple US scans), obtaining promising results.

In the future, the presented technique could aid in

the prenatal assessment and in-utero diagnosis of syn-

dromes and diseases in which facial dysmorphology

is an indicator of early craniofacial abnormalities.

ACKNOWLEDGEMENTS

This work is partly supported by the Spanish Ministry

of Economy and Competitiveness under project grant

TIN2017-90124-P, and the Maria de Maeztu Units of

Excellence Programme (MDM-2015-0502).

REFERENCES

Albu, A., Horhoianu, I., Dumitrascu, M., and Horhoianu,

V. (2014). Growth assessment in diagnosis of fetal

growth restriction. review. J Med Life, 7(2):150–154.

Andresen, C., Matias, A., and Merz, E. (2012). Fetal Face:

The Whole Picture. Ultraschall in Med, 33:431–440.

Bas, A., Smith, W., Bolkart, T., and Wuhrer, S. (2016). Fit-

ting a 3D Morphable Model to Edges: A Compari-

son Between Hard and Soft Correspondences. Asian

Conference on Computer Vision Workshop on Facial

Informatics, Taipei (Taiwan), pages 377–391.

Bonacina, L., Froio, A., Conti, D., Marcolin, F., and

Vezzetti, E. (2016). Automatic 3D foetal face model

extraction from ultrasonography through histogram

processing. Journal of Medical Ultrasound, 24:124–

149.

Dall’Asta, A., Schievano, S., and Bruse, J. (2017). Quan-

titative analysis of fetal facial morphology using 3D

ultrasound and statistical shape modeling: a feasibil-

ity study. Am J Obstet Gynecol, 217(76):1–8.

EvansAnne, K. N., HingMichael, V., and Cunningham, L.

(2018). 100 - Craniofacial Malformations. Avery’s

Diseases of the Newborn, 10:1417–1437.

Learned-Miller, E., Lu, Q., Paisley, A., Trainer, P., Blanz,

V., Dedden, K., and Miller, R. (2006). Detecting

acromegaly: screening for disease with a morphable

model. In International Conference on Medical Im-

age Computing and Computer-Assisted Intervention

(MICCAI), page 495–503.

Menezes, G. A., J

unior, E. A., Lopes, J., S. Belmonte, G. T.,

and Werner, H. (2016). Prenatal diagnosis and phys-

icalmodel reconstruction of agnathia–otocephaly with

limb deformities (absent ulna, ﬁbula and digits) fol-

lowing maternalexposure to oxymetazoline in the ﬁrst

trimester. Journal of Obstetrics and Gynaecology Re-

search, 42(8):1016–1020.

Merz, E. and Abramowicz, J. (2012). 3D/4D ultrasound in

prenatal diagnosis: is it time for routine use? Clin

Obstet Gynecol, 55(1):336–351.

Merz, E. and Welter, C. (2005). 2D and 3D Ultrasound in

the evaluation of normal and abnormal fetal anatomy

in the second and third trimesters in a level III center.

Ultraschall Med., 1:1–16.

Morales, A., Porras, A. R., Tu, L., Linguraru, M. G.,

Piella, G., and Sukno, F. M. (2020). Spectral Corre-

spondence Framework for Building a 3D Baby Face

Model. In 15th IEEE International Conference on

Automatic Face and Gesture Recognition, pages 507–

514.

Mossey, P. A. and Catilla, E. E. (2001). Global registry

and database on craniofacial anomalies. WHO Reg-

istry Meeting on Craniofacial Anomalies.

on Government Affairs, A. C. (2020). Craniofacial Anoma-

lies. AAOMS.

orop Florea, M., Dragus

in, R.-C., Marinas

, C., S

orop, V.-

B., P

atru, C. L., Zoril

a, L. G., Neamt

u, C., Vedut

A., Iliescu, D. G., and Cernea, N. (2018). Congenital

Abnormalities of the Fetal Face. IntechOpen.

Peleg, D., Kennedy, C. M., and Hunter, S. K. (1998). In-

trauterine growth restriction: Identiﬁcation and man-

agement. Am Fam Physician, 58(2):453–460.

Pooh, R. and Kurjak, A. (2011). 3D/4D sonography moved

prenatal diagnosis of fetal anomalies from the second

to the ﬁrst trimester of pregnancy. J Matern Fetal

Neonatal Med, 25(5):433–455.

Speranza, D., Citro, D., Padula, F., Motyl, B., Marcolin, F.,

Cal

ı, M., and Martorelli, M. (2017). Additive Man-

ufacturing Techniques for the Reconstruction of 3D

Fetal Faces. Applied Bionics and Biomechanics.

Tu, L., Porras, A., Morales, A., Perez, D., Piella, G., Su-

kno, F., and Linguraru, M. (2019). Three-Dimensional

Face Reconstruction from Uncalibrated Photographs:

Application to Early Detection of Genetic Syndromes.

In: Uncertainty for Safe Utilization of Machine

Learning in Medical Imaging and Clinical Image-

Based Procedures. Lecture Notes in Computer Sci-

ence, 11840:182–189.

Tu, L., Porras, A. R., Boyle, A., and Linguraru, M. G.

(2018). Analysis of 3D Facial Dysmorphology in

Genetic Syndromes from Unconstrained 2D pho-

tographs. In International Conference on Medical Im-

age Computing and Computer - Assisted Intervention

(MICCAI), page 347–355.

Werner, H., Lopes, J., Tonni, G., and J

unior, E. A.

(2015). Physicalmodel from 3D ultrasound and

magnetic resonance imaging scan data reconstruc-

tion of lumbosacral myelomeningocelein a fetus with

chiari ii malformation. Child’s Nervous System,

31(4):511–513.

3D Fetal Face Reconstruction from Ultrasound Imaging

623

Werner, H., Santos, J., Belmonte, S., Ribeiro, G., Daltro, P.,

Gasparetto, E., and Marchiori, E. (2016). Applicabil-

ity of three-dimensional imaging techniques in fetal

medicine. Radiol Bras, 49(5):281–287.

Werner, H., Santos, J. R. L. D., and Fontes, R. (2010). Addi-

tive manufacturing models of fetuses built from three-

dimensional ultrasound, magnetic resonance imaging

and computed tomography scan data. Ultrasound in

Obstetrics & Gynecology, 36(3):355–361.

VISAPP 2021 - 16th International Conference on Computer Vision Theory and Applications

624